Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 State of Arizona Department of Education Office of John Huppenthal Superintendent of Public Instruction Arizona FFY 2012 Annual Performance Report for Special Education Highly Effective Schools Division Exceptional Student Services 1535 West Jefferson, Phoenix, AZ 85007 http://www.azed.gov/ February 3, 2014 Revised April 30, 2014 Part B Arizona Annual Performance Report for FFY 2012 1 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Arizona FFY 2012 Annual Performance Report for Special Education Submitted to the Office of Special Education Programs United States Department of Education Highly Effective Schools Division Exceptional Student Services Angela Denning Deputy Associate Superintendent 602-542-4013 essdesk@azed.gov http://www.azed.gov/special-education/ February 3, 2014 Revised April 30, 2014 Part B Arizona Annual Performance Report for FFY 2012 2 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Table of Contents Overview of the Annual Performance Report Development ......................................................................... 4 Indicator 1: Graduation Rates ....................................................................................................................... 6 Indicator 2: Dropout Rates .......................................................................................................................... 18 Indicator 3: Assessments ............................................................................................................................ 29 Indicator 4A: Suspension and Expulsion .................................................................................................... 46 Indicator 4B: Suspension and Expulsion by Race or Ethnicity ................................................................... 51 Indicator 5: School Age LRE ....................................................................................................................... 57 Indicator 6: Preschool LRE ......................................................................................................................... 63 Indicator 7: Preschool Outcomes ................................................................................................................ 66 Indicator 8: Parent Involvement .................................................................................................................. 71 Indicator 9: Racial / Ethnic Disproportionality ............................................................................................. 77 Indicator 10: Racial / Ethnic Disproportionality by Disability ....................................................................... 84 Indicator 11: Evaluation Timelines .............................................................................................................. 92 Indicator 12: Early Childhood Transition ..................................................................................................... 99 Indicator 13: Secondary Transition ........................................................................................................... 106 Indicator 14: Post School Outcomes ......................................................................................................... 120 Indicator 15: Effective General Supervision .............................................................................................. 137 Indicator 18: Resolution Session Effectiveness ........................................................................................ 156 Indicator 19: Mediation Agreements ......................................................................................................... 159 Indicator 20: State Reported Data ............................................................................................................ 162 Attachments .............................................................................................................................................. 168 Part B Arizona Annual Performance Report for FFY 2012 3 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 The Part B Arizona Annual Performance Report for Special Education Federal Fiscal Year 2012 Overview of the Annual Performance Report Development In accordance with 20 U.S.C. § 1416 (b)(2)(C)(ii) and 34 CFR § 300.602, the State of Arizona must report annually to the United States Secretary of Education on Arizona’s performance under its Part B State Performance Plan (SPP). The annual report is the Part B Annual Performance Report (APR). The submission of the Part B APR, due February 3, 2014, reflects those requirements and the State’s progress toward the goals established in the State Performance Plan submitted to the United States Department of Education (USDOE) in December 2005. The February 3, 2014, APR gives actual target data and other responsive information for Indicators 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, and 19. The Annual Performance Report was developed by the staff at the Arizona Department of Education/Exceptional Student Services (ADE/ESS) and the Arizona Department of Education/Early Childhood Special Education (ADE/ECSE). A number of Arizona Department of Education staff members with specialization in different areas examined improvement activities, collected and analyzed the data, and drafted the reports for the 17 indicators. Members of the Special Education Advisory Panel (SEAP) and education personnel from the field reviewed data, annual targets, and improvement activities and offered suggestions. Descriptions of the data, including sources, sampling methodology, and validity and reliability, are located under each indicator. Information is included that replies to the Arizona Part B FFY 2011 SPP/APR Response Table from the Office of Special Education Programs (OSEP). One revision for FFY 2012 was made to Arizona’s FFY 2005–2012 State Performance Plan for Special Education. Annual Measureable Objectives (AMOs) for Indicator 3C were added on page 23. The document is available on the ADE/ESS Web site at http://www.azed.gov/specialeducation/resources/spp-apr/ under the list titled State Performance Plan. Stakeholder Involvement As data and other communications became available after the close of the 2012–2013 school year, the ADE/ESS staff reported to the Special Education Advisory Panel (SEAP). The SEAP members represent a broad group of stakeholders throughout Arizona. Groups represented on the panel include parents of children with disabilities, individuals with disabilities, teachers, early childhood education, charter schools, school districts, institutions of higher education that prepare special education and related services personnel, secure care facilities, and public agencies. The ADE/ESS responded to questions and comments from the SEAP members and considered the panel’s advice. In addition to reporting on the APR to the SEAP, ESS requested input from special education administrators through meetings of the regional organizations, small workshops, and large conferences. The ADE/ESS data management coordinator trained data managers and administrators on the data requirements and also requested input for improving the State’s data collection and reporting process. ESS program specialists spoke to administrators and teachers specifically about the 0% and 100% compliance indicators during on-site visits, seeking information for the revision of improvement activities to increase compliance. Part B Arizona Annual Performance Report for FFY 2012 4 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Public Reporting and Dissemination Arizona must report annually to the public on: (1) the State’s progress and/or slippage in meeting the measurable and rigorous targets in the SPP and, (2) the performance of each public education agency (PEA) in the State on the SPP targets. The annual performance report (APR) on the State’s progress and/or slippage is available on the ADE/ESS Web site at http://www.azed.gov/special-education/resources/spp-apr/ under the list titled Annual Performance Report, beginning on February 3, 2014. The title of the APR is Arizona FFY 2012 Annual Performance Report. The State Performance Plan (SPP) for FFY 2012 is available on the ADE/ESS Web site at http://www.azed.gov/special-education/resources/spp-apr/ under the list titled State Performance Plan, beginning on February 3, 2014. The title of the SPP is Arizona State Performance Plan FFY 2005–2012 Revised FFY 2012. The annual public reports will be available on the ADE/ESS Web site at http://www.azed.gov/specialeducation/resources/ under the list titled Public Reports School Year 2012–2013, within 120 days of the February 3, 2014, submission of the APR. These reports list the performance of each school district and charter school in Arizona on the SPP targets. The SPP and APR are disseminated to the public by hard copy, e-mail, and the ADE/ESS Web site. Each member of SEAP receives a copy of the SPP and the APR, as does Arizona’s Parent and Training Information Center. The ESS special education listserv, ESS and ECSE specialists, trainings, and conferences serve as the vehicles to notify parents, the PEAs, and the public of the availability of the SPP and APR. Special Education Monitoring Alerts, memoranda pertaining to specific topics including the SPP/APR, are sent to the field electronically on the ESS listserv and distributed by hard copy through the ESS specialists. Part B Arizona Annual Performance Report for FFY 2012 5 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 1: Graduation Rates Percent of youth with IEPs graduating from high school with a regular diploma. (20 U.S.C. 1416 (a)(3)(A)) Measurement: States must report using the adjusted cohort graduation rate required under the ESEA. Target Data for FFY 2012 FFY Measurable and Rigorous Target 2012 80% Arizona’s single statewide target graduation rate is 80%. Actual Target Data for FFY 2012 number of youth with IEPs who graduated in 4 years with a regular high school diploma number of youth with IEPs who entered high school 4 years earlier (adjusting for transfers, home schooled, and deceased youth) 4,816 7,361 Actual Target Data for FFY 2012 65% 4,816  7,361 = 0.6542  100 = 65% Arizona did not meet the target. Data Data Source The graduation data from Arizona’s 2012 (school year 2011 – 2012) cohort were reported by the public education agencies (PEAs) through the Student Accountability Information System (SAIS), a Web-based system for reporting all student-level details to the Arizona Department of Education. Data Description The graduation data were analyzed by the Arizona Department of Education’s Accountability Division/Research and Evaluation Section (ADE/R and E) and the Information Technology Division (IT). Part B Arizona Annual Performance Report for FFY 2012 6 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 The same graduation rate calculation was used, and it is the same data as that reported to the U.S. Department of Education under Title I of the Elementary and Secondary Education Act (ESEA). Target Data The target data are the same as the annual graduation rate targets under Title I of the ESEA and explained in Arizona’s Accountability Workbook. Arizona’s single, statewide target graduation rate is 80%. Valid and Reliable Data The graduation data were obtained from the ADE Accountability Division/Research and Evaluation Section (ADE/R and E), which follows an internal process to ensure that data are valid, reliable, and accurate. Conditions to Graduate Graduation Cohort Arizona uses a four-year cohort. Any student who receives a traditional high school diploma within the first four years of starting high school is considered a four-year graduate. A four-year rate is calculated by dividing the sum of all four-year graduates in a cohort by the sum of those who should have graduated and did not transfer to another qualified educational facility, left to be home schooled, or are deceased. Students who receive a diploma prior to September 1 of the school year following their fourth year are included as part of the graduation cohort. Conditions to Graduate with a Regular Diploma Conditions students without disabilities must meet in order to graduate with a regular high school diploma:  Complete their PEA’s requirements to receive a regular high school diploma (Arizona Revised Statutes § 15-701.01 (C) and Arizona Administrative Code R7-2-302); and  Achieve passing scores on Arizona’s Instrument to Measure Standards (Arizona Revised Statutes § 15-701.01 (A)). Conditions students with disabilities must meet in order to graduate with a regular high school diploma:  The local governing board of each school district is responsible for developing a course of study and graduation requirements for all students placed in special education programs (Arizona Administrative Code R7-2-302 (6)).  Students with disabilities do not have to achieve passing scores on Arizona’s Instrument to Measure Standards (AIMS) or Arizona’s Instrument to Measure Standards Alternate (AIMS A) to graduate with a regular high school diploma unless specifically required by the Individualized Education Program (IEP) team (Arizona Revised Statutes § 15-701.01 (B)). Explanation of Slippage That Occurred for FFY 2012 Arizona’s graduation target (80%) for FFY 2012 is the same as the annual graduation rate target for all Arizona students under Title 1 of the Elementary and Secondary Education Act (ESEA). The State did not meet this target and slipped from FFY 2011 (67%) to FFY 2012 (65%). In school year 2011–2012, the minimum subject area course credit requirements for high school graduation in Arizona increased to Part B Arizona Annual Performance Report for FFY 2012 7 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 include one additional credit in science and one additional credit in mathematics. Slippage in the graduation rate may be related to these increased academic requirements. Arizona maintains that quality transition planning for students from secondary to postsecondary education, training, and employment settings can positively impact graduation rates for students with disabilities. To this end, the ADE/ESS transition specialists provided training and technical assistance to 1,581 participants at 76 training sessions offered throughout the state; established 14 new teams through the Secondary Transition Mentoring Project (STMP); held a statewide conference dedicated to transition; and developed and disseminated information and materials through webinars and updates to the ADE/ESS secondary transition Web site. The ADE/ESS is committed to offering intensive capacity building grant opportunities to PEAs in need of assistance in transition planning and service provision. In FFY 2012, the Secondary Transition Mentoring Project (STMP), a two-year capacity building grant opportunity, provided funding to 28 PEAs, including 14 new teams (2012–2014 cohort) and 14 returning teams (2011–2013 cohort). The ADE/ESS, in collaboration with training facilitators from the University of Kansas Transition Coalition, provided a training series that included an introductory session and webinars, three two-day face-to-face trainings, a Web site for project participants containing all training materials and extensive resources, an online short course for Year 1 teams, and ongoing support and technical assistance. This capacity building training focused on developing strategies and products for use in participating PEAs to ensure 100% compliance on Indicator 13, using State and local data results from Indicators 1, 2, and 14 as tools to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. As part of the FFY 2012 STMP grant, teams reviewed their own PEA’s data in the following areas: graduation rates, dropout rates, post school outcomes data (if available), and Indicator 13 compliance. Throughout the two-year process, the STMP teams focused on identifying PEA needs and priorities and developed action plans to address the identified needs. If applicable, teams developed action plans for increasing graduation rates or reducing dropout rates for students with disabilities that included root cause analyses to determine potential barriers within their schools. Additionally, the STMP team trainings assisted local PEAs in creating sustainable community teams to engage community partners and facilitate positive post school outcomes for students with disabilities. Measures used to determine the effectiveness of STMP team trainings showed overall knowledge increases in transition compliance requirements (particularly in the use of transition assessments and understanding of courses of study), as well as progress in school- or district-wide goals toward 100% compliance on Indicator 13. Arizona’s Twelfth Annual Transition Conference offered two breakout sessions specifically targeting improving graduation rates and decreasing dropout rates of students with disabilities. Sessions were led by the director of the National Dropout Prevention Center for Students with Disabilities (NDPC-SD). Resources and facilitated discussions aimed at addressing graduation and dropout issues were provided. The ADE/ESS secondary transition Web site also was updated with additional resources and materials on dropout prevention. In addition to PEA training opportunities, internal and external stakeholder collaboration efforts were conducted to enhance the secondary transition planning process. During FFY 2012, intradepartmental collaboration meetings were held with the ADE sections of High School Renewal and Improvement (AZHSRI), Career and Technical Education (CTE) (which includes School/Career Counseling), Dropout Prevention, and Career and College Readiness. These collaborative meetings resulted in cross-training for conferences that were sponsored by different ADE sections and included the topic of secondary transition. Further, ADE/ESS transition and program specialists worked closely with PEAs to assist in data review, analysis, and training. Transition specialists analyzed data collected from on-site annual site visits to Part B Arizona Annual Performance Report for FFY 2012 8 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 PEAs (technical assistance visits conducted by ESS program specialists) and targeted staff development to those PEAs most in need of training with regard to secondary transition requirements. ESS program specialists also reviewed graduation rates with PEAs on an annual basis as part of a larger data review that included performance and compliance Indicator data, dispute resolution data, and annual technical assistance visit information. If a PEA had not met the State target for graduation rate, then the PEA may have been required to conduct a drill down exercise that is designed to explore root causes. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Revise, implement, and evaluate a comprehensive plan for training PEAs to increase compliance with postsecondary requirements related to Indicator 13 Sub-Activities (Objectives or Action Steps) a) On an annual basis, identify PEAs in Years 2 and 3 of the monitoring cycle through collaboration with ESS specialists b) On an annual basis, review, revise (if necessary), and implement the comprehensive training plan, emphasizing the eight required components of Indicator 13 Timeline Complete Activity completed 8/1/12. Projected 7/1/12– 6/30/13 The FFY 2012 PEA list was completed August 2013. The Annual Site Visit Log (ASVL) from school year 2011–2012 was used to identify PEAs most in need of training and technical assistance (TA) for Indicator 13. Activities completed from 7/1/12 to 6/30/13. Resources (Planned) ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/12– 6/30/13 ADE/ESS Transition Specialists The FFY 2012 review and revision of the Strategic Plan for Statewide Transition Planning was completed July 2012. Implementation of the Transition Strategic Plan was completed from July 2012 to June 2013. The FFY 2012 Transition Strategic Plan includes seven main components: 1. Provide training to targeted PEAs and in response to requests from non-targeted PEAs on Indicator 13; 2. Organize Arizona’s Twelfth Annual Transition Conference focusing on improving post school outcomes for students with disabilities by providing sessions on transition planning and dropout prevention; 3. Provide training to special education directors from across the state at the annual ADE/ESS Part B Arizona Annual Performance Report for FFY 2012 9 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Directors Institute; 4. Provide capacity building grants to PEAs to facilitate intra/interagency collaboration and build local capacity to improve post school outcomes through local interagency work, as well as provide intensive training and support for PEAs to achieve 100% compliance on Indicator 13; 5. Collaborate with national technical assistance centers and organizations including NSTTAC, NPSO, NDPC-SD, and the NASDSE IDEA Partnership Community of Practice on Transition and Transition Coalition; 6. Participate with other Arizona State agencies including Rehabilitation Services of Arizona/Vocational Rehabilitation (RSA/VR), Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN); 7. Collaborate with other ADE sections: High School Renewal and Improvement (AZHSRI), Career and Technical Education (CTE), Dropout Prevention, and School Counselors and ADE/ESS areas: Program Support, Assistive Technology, and Parent Information Network. All components of Arizona’s FFY 2012 Strategic Plan for Statewide Transition Planning were immediately implemented upon revision, from 8/1/12 to 6/30/13. Activities completed:  There were 1,581 participants from 519 targeted and nontargeted PEAs who received Part B Arizona Annual Performance Report for FFY 2012 10 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 secondary transition training, including Indicator 13 trainings from ADE/ESS transition specialists at 76 sites statewide. Trainings were delivered in regional or direct school locations.  Arizona’s Twelfth Annual Transition Conference was held in October 2012 and offered sessions focused on improving compliance with the eight components of Indicator 13. There were 903 participants who attended the conference, including education and agency professionals, youth, young adults, family members of youth with disabilities, and vendors/exhibitors.  Two sessions on secondary transition were provided by ADE/ESS transition specialists at the annual ADE/ESS Directors Institute (DI). Of the 716 in attendance at the DI, 59 participants attended sessions related to secondary transition.  There were 28 PEAs that participated in team trainings in Year 1 (14 PEAs in the 2012– 2014 cohort) or Year 2 (14 PEAs in the 2011–2013 cohort) of the Secondary Transition Mentoring Project (STMP) capacity building grant. Through a contract with the University of Kansas Transition Coalition and in collaboration with ADE/ESS, training was provided for teams to achieve and sustain 100% compliance on Indicator 13 using State and local data results from Indicators 1, 2, and 14 as tools to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. Team training included: three two-day face-toface training sessions, webinars, a Web site for project participants Part B Arizona Annual Performance Report for FFY 2012 11 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 containing all training materials and extensive resources, an online short course for Year 1 teams, and attendance at Arizona’s Twelfth Annual Transition Conference.  Collaboration with national technical assistance centers and organizations occurred throughout the year and included: participation in NPSO and NSTTAC Community of Practice calls; use of resources from the NSTTAC Web site; participation in the NASDSE IDEA Partnership, Community of Practice National Meeting; and attendance at the National Secondary Transition Planning Institute where OSEP, NPSO, NSTTAC, and NDPC-SD provided guidance. Additionally, ADE/ESS maintained ongoing collaboration with the University of Kansas Transition Coalition and the Mountain Plains Regional Resource Center.  At the State level, ADE/ESS collaborated with RSA/VR, DDD, DBHS, and OCSHCN, and met every other month with these agency stakeholders through the Arizona Community of Practice on Transition (AZCoPT). In FFY 2012, AZCoPT revised a presentation used statewide through RSA/VR and DBHS teleconferencing media to introduce participants to the supports/services available to school-aged and adult individuals with disabilities.  ADE/ESS collaboration meetings with the ADE areas of High School Renewal and Improvement (, Career and Technical Education, Dropout Prevention, and Career and College Readiness were scheduled and held throughout the year and resulted in crosstraining for conferences that were Part B Arizona Annual Performance Report for FFY 2012 12 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 sponsored by different ADE sections and included the topic of secondary transition.  Intra-ADE/ESS collaborative efforts included: monthly meetings with PINS (Parent Information Network Specialists) as fellow AZCoPT members, as well as involvement with PINS during quarterly Transition Conference Planning Committee meetings; at least quarterly meetings with ESS Program Support to discuss the use of the Annual Site Visit Log (ASVL); review/revision of the secondary transition section of the monitoring manual and needed secondary transition trainings for ESS program specialists and PEAs; and the development of collaborative presentations with the ADE/ESS assistive technology unit. c) On an annual basis, create and disseminate information through a variety of sources: annual statewide conference, monitoring alerts, Web site, and listserv announcements Activities completed from 8/1/12 through 6/30/13. 7/1/12– 6/30/13 ADE/ESS Transition Specialists  Two secondary transition presentations were offered at the ADE/ESS Directors Institute in August 2012 for approximately 59 participants.  There were102 sessions either directly or indirectly related to Indicator 13 offered at Arizona’s Twelfth Annual Transition Conference held in October 2012.  Between July 2012 and June 2013, the ADE/ESS Secondary Transition Web site was updated to include additional and/or revised Indicator 13 materials from NSTTAC, IDEA Partnership, and other secondary transition technical assistance centers. The Web address is http://www.azed.gov/specialeducation/special-projects /secondary-transition/. Part B Arizona Annual Performance Report for FFY 2012 13 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 d) On an annual Activities completed from 7/1/12 to basis, analyze pre6/30/13. and post-training data collected  Analysis of ASVL pre-training through the Annual data was completed in January Site Visit Log (ASVL) 2013 after all ESS program for each PEA to specialists were able to complete determine level of at least one PEA annual site visit compliance on all during fall 2012. The analysis eight required showed a 72.8% average for components of compliance with the eight items of Indicator 13 Indicator 13. 7/1/12– 6/30/13 ADE/ESS Transition Specialists ADE/ESS Program Specialists  Post-training data analysis of all PEAs that received a pre-training annual site visit and were trained in secondary transition during FFY 2012 showed an 85.8% average for compliance with the eight items of Indicator 13. Through pre- and post-training analysis, an increase of 13.3% in compliance for Indicator 13 was demonstrated after training and technical assistance was provided to PEAs by transition and education program specialists. 2) Provide a twoyear capacity building grant to PEAs to participate in the Secondary Transition Mentoring Project (STMP) Team Training a) On an annual basis, identify PEAs who meet eligibility requirements and extend invitations to them to participate in STMP trainings Activities completed from 1/1/12 to 7/30/13. 7/1/12– 6/30/13  Using Annual Site Visit (ASV) data for PEAs currently in Years 1, 2, or 3 of the monitoring cycle, 23 PEAs achieving less than 100% compliance on Indicator 13 were invited to apply for a noncompetitive Secondary Transition Mentoring Project (STMP) capacity building grant. ADE/ESS Transition Specialists ADE/ESS Program Specialists  Fourteen PEAs were accepted for participation in Year 1 of the STMP Team training in July 2012. b) On an annual basis, provide indepth and ongoing professional development on transition requirements and best practices Activities completed from 7/1/12 through 6/30/13. 7/1/12– 6/30/13 ADE/ESS Transition Specialists  STMP participants attended Arizona’s Twelfth Annual Transition Conference, which included a STMP team orientation and designated sessions. Part B Arizona Annual Performance Report for FFY 2012 14 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012  Training materials and activities were designed, created, and disseminated by ADE/ESS in collaboration with the University of Kansas Transition Coalition. Trainings focused on: identifying PEA barriers to meeting transition requirements; developing an action plan to eliminate barriers; creating IEPs that meet transition requirements; implementing training to build intra-PEA capacity to attain 100% compliance on secondary transition requirements; and determining improvements made and targeting areas still in need of improvement.  Team training for each cohort consisted of three two-day faceto-face trainings throughout the year. Additionally, Year 1 teams completed an online short course specially designed for STMP teams and focused on best practices in transition planning. c) On an annual Activity completed from 7/1/12 basis, analyze prethrough 6/30/13. and post-training data collected during  Measures to determine STMP trainings for effectiveness of STMP training each PEA that included the use of: participated to a) team process checklist; determine level of b) interrater reliability measures compliance on all for IEP file review; eight required c) training session evaluations; components of d) STMP team self-reported Indicator 13 progress toward completion of action plans; and e) feedback provided by ADE/ESS monitoring program specialists on compliance. 7/1/12– 6/30/13 ADE/ESS Transition Specialists Data from the measures are as follows: a) April 2013 trainings for both STMP cohorts included the completion of a 37-item “team process” checklist. Results from the checklist use indicated both Year 1 and Year Part B Arizona Annual Performance Report for FFY 2012 15 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 2 teams reported an average of 92.15% for the item “all team members are highly knowledgeable of transition compliance requirements” and an average of 80.8% for the item “STMP team collaboratively develops and implements an action plan that addresses the prioritized needs.” b) Training for both cohorts included instruction and practice activities using interrater reliability measures. c) All STMP training sessions during FFY 2012 included instruction, activities, and Indicator 13 file reviews. Based on a 5-point Likert scale (1 lowest and 5 highest), evaluation results indicated an average score of 4.27 across all activities for both cohorts. Additionally, all team members completed a pre-/postevaluation of their competency in meeting Indicator 13 requirements that asked the question, “How prepared do you feel?” Based on a 5-point Likert Scale (1 equaled unprepared and 5 equaled prepared) and using a Paired Samples T-Test, all scores indicated a significant increase in knowledge from the beginning to the end of the training for FFY 2012 for both cohorts (Years 1 and 2). The areas reporting the greatest increase in knowledge were transition assessments and documenting transition services and course of study that will support the student in reaching postsecondary goals. d) Review of each STMP team’s action plan indicated all teams set goals to build and sustain systems to ensure 100% compliance on Indicator 13 school- or district-wide. All teams reported progress in Part B Arizona Annual Performance Report for FFY 2012 16 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 meeting these goals. e) During the April 2013 training, Year 2 teams met with ADE/ESS program specialists to receive informal feedback on IEPs written during the second year of the STMP training program. Feedback indicated significant improvement in PEA knowledge and compliant practices. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 17 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 2: Dropout Rates Percent of youth with IEPs dropping out of high school. (20 U.S.C. 1416 (a)(3)(A)) Measurement: States must report using the dropout data used in the ESEA graduation rate calculation and follow the timeline established by the Department under the ESEA. Target Data for FFY 2012 FFY Measurable and Rigorous Target 2012 ≤ 4.9% Actual Target Data for FFY 2012 (using 2011–2012 data) number of youths with IEPs dropping out of grades 9–12 number of youths with IEPs in grades 9–12 2,125 36,200 Actual Target Data for FFY 2012 5.9% 2,125  36,200 = 0.587  100 = 5.9% Arizona did not meet the target. Data Data Source Arizona used the same data source and measurement that the State used for its FFY 2010 APR that was submitted on February 1, 2012. The dropout data were reported by the public education agencies (PEAs) through the Student Accountability Information System (SAIS), a Web-based system for reporting all student-level details to the Arizona Department of Education. Data Description The 2011–2012 data were analyzed by the Arizona Department of Education’s Accountability Division/Research and Evaluation Section (ADE/R and E). Valid and Reliable Data Part B Arizona Annual Performance Report for FFY 2012 18 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 The dropout data were obtained from the ADE Accountability Division/Research and Evaluation Section (ADE/R and E), which follows internal processes to ensure valid, reliable, and accurate data. Definition of Dropout and Methodology For purposes of calculating and reporting a dropout rate, Arizona used the annual event school dropout rate for students leaving a school in a single year determined in accordance with the National Center for Education Statistics (NCES) Common Core of Data. Consistent with this requirement, Arizona used the NCES definition of high school dropout, defined as an individual who: (1) was enrolled in school at some time during the previous school year; and (2) was not enrolled at the beginning of the current school year; and (3) has not graduated from high school or completed a State- or district-approved educational program; and (4) does not meet any of the following exclusionary conditions: (a) transfer to another public school district, private school, or State- or districtapproved educational program (including correctional or health facility programs); (b) temporary absence due to suspension or school-excused illness; or (c) death. The same definition and methodology for dropout rates apply to all students in Arizona. Explanation of Slippage That Occurred for FFY 2012 Arizona did not meet the FFY 2012 target of 4.9% and slipped from FFY 2011 (4.7%) to FFY 2012 (5.9%). In school year 2011–2012, the minimum subject area course credit requirements for high school graduation in Arizona increased to include one additional credit in science and one additional credit in mathematics. Slippage in the dropout rate may be related to these increased requirements. Arizona maintains that quality transition planning for students from secondary to postsecondary education, training, and employment settings can positively impact dropout rates for students with disabilities. To this end, the ADE/ESS transition specialists provided training and technical assistance to 1,581 participants at 76 training sessions offered throughout the state; established 14 new teams through the Secondary Transition Mentoring Project (STMP); held a statewide conference dedicated to transition; and developed and disseminated information and materials through webinars and updates to the ADE/ESS secondary transition Web site. The ADE/ESS is committed to offering intensive capacity building grant opportunities for PEAs in need of assistance in transition planning and service provision. In FFY 2012, the Secondary Transition Mentoring Project (STMP), a two-year capacity building grant opportunity, provided funding to 28 PEAs, including 14 new teams (2012–2014 cohort) and 14 returning teams (2011–2013 cohort). The ADE/ESS, in collaboration with training facilitators from the University of Kansas Transition Coalition, provided a training series that included an introductory session and webinars, three two-day face-to-face trainings, a Web site for project participants containing all training materials and extensive resources, an online short course for Year 1 teams, and ongoing support and technical assistance. This capacity building training focused on developing strategies and products for use in participating PEAs to ensure 100% compliance on Indicator 13, using State and local data results from Indicators 1, 2, and 14 as tools to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. As part of the FFY 2012 STMP grant, teams reviewed their own PEA’s data in the following areas: graduation rates, dropout rates, post school outcomes data (if available), and Indicator 13 compliance data. Throughout the two-year process, the STMP teams focused on identifying PEA needs and priorities and developed action plans to address the identified needs. If applicable, teams developed action plans for increasing graduation rates or reducing dropout rates for students with disabilities that included root Part B Arizona Annual Performance Report for FFY 2012 19 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 cause analyses to determine potential barriers within their schools. Additionally, the STMP team trainings assisted local PEAs in creating sustainable community teams to engage community partners and facilitate positive post school outcomes for students with disabilities. Measures used to determine the effectiveness of STMP team trainings showed overall knowledge increases in transition compliance requirements (particularly in the use of transition assessments and understanding of courses of study), as well as progress in school- or district-wide goals toward 100% compliance on Indicator 13. Arizona’s Twelfth Annual Transition Conference offered two breakout sessions specifically targeting improving graduation rates and decreasing dropout rates of students with disabilities. Sessions were led by the director of the National Dropout Prevention Center for Students with Disabilities (NDPC-SD). Resources and facilitated discussions aimed at addressing graduation and dropout issues were provided. The ADE/ESS secondary transition Web site also was updated with additional resources and materials on dropout prevention. In addition to PEA training opportunities, internal and external stakeholder collaboration efforts were conducted to enhance the secondary transition planning process. During FFY 2012, intradepartmental collaboration meetings were held with the ADE sections of High School Renewal and Improvement, Career and Technical Education (including School/Career Counseling), Dropout Prevention, and Career and College Readiness and resulted in cross-training for conferences that were sponsored by different ADE sections and included the topic of secondary transition. Further, ADE/ESS transition and program specialists worked closely with PEAs to assist in data review, analysis, and training. Transition specialists analyzed data collected from on-site PEA annual site visits (technical assistance visits conducted by ESS program specialists) and targeted staff development to those PEAs most in need of training with regard to secondary transition requirements. ESS program specialists also reviewed dropout rates with PEAs on an annual basis as part of a larger data review that included performance and compliance Indicator data, dispute resolution data, and annual technical assistance visit information. If a PEA had not met the State target for dropout rate, then the PEA may have been required to conduct a drill down exercise that is designed to explore root causes. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Revise, implement, and evaluate a comprehensive plan for training PEAs to increase compliance with postsecondary requirements related to Indicator 13 Sub-Activities (Objectives or Action Steps) a) On an annual basis, identify PEAs in Years 2 and 3 of the monitoring cycle through collaboration with ESS specialists b) On an annual basis, review, revise (if necessary), and implement the comprehensive training plan, emphasizing the eight required Timeline Complete Activity completed 8/1/12. Projected 7/1/12– 6/30/13 The FFY 2012 PEA list was completed August 2013. The Annual Site Visit Log (ASVL) from school year 2011–2012 was used to identify PEAs most in need of training and technical assistance (TA) for Indicator 13. Activities completed from 7/1/12 to 6/30/13. Resources (Planned) ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/12– 6/30/13 ADE/ESS Transition Specialists The FFY 2012 review and revision of the Strategic Plan for Statewide Transition Planning was completed July 2012. Implementation of the Transition Strategic Plan was Part B Arizona Annual Performance Report for FFY 2012 20 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 components of Indicator 13 completed from July 2012 to June 2013. The FFY 2012 Transition Strategic Plan includes seven main components: 1. Provide training to targeted PEAs and in response to requests from non-targeted PEAs on Indicator 13; 2. Organize Arizona’s Twelfth Annual Transition Conference focusing on improving post school outcomes for students with disabilities by providing sessions on transition planning and dropout prevention; 3. Provide training to special education directors from across the State at the annual ADE/ESS Directors Institute; 4. Provide capacity building grants to PEAs to facilitate intra/interagency collaboration and build local capacity to improve post school outcomes through local interagency work, as well as provide intensive training and support for PEAs to achieve 100% compliance on Indicator 13; 5. Collaborate with national technical assistance centers and organizations including NSTTAC, NPSO, NDPC-SD, and the NASDSE IDEA Partnership Community of Practice on Transition and Transition Coalition; 6. Participate with other Arizona State agencies including Rehabilitation Services of Arizona/Vocational Rehabilitation (RSA/VR), Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN); Part B Arizona Annual Performance Report for FFY 2012 21 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 7. Collaborate with other ADE sections: High School Renewal and Improvement (AZHSRI), Career and Technical Education (CTE), Dropout Prevention, and School Counselors and ADE/ESS areas: Program Support, Assistive Technology, and Parent Information Network. All components of Arizona’s FFY 2012 Strategic Plan for Statewide Transition Planning were immediately implemented upon revision, from 8/1/12 to 6/30/13. Activities completed:  There were 1,581 participants from 519 targeted and nontargeted PEAs who received secondary transition training, including Indicator 13 trainings from ADE/ESS transition specialists at 76 sites statewide. Trainings were delivered in regional or direct school locations.  Arizona’s Twelfth Annual Transition Conference was held in October 2012 and offered sessions focused on improving compliance with the eight components of Indicator 13. There were 903 participants who attended the conference, including education and agency professionals, youth, young adults, family members of youth with disabilities, and vendors/exhibitors.  Two sessions on secondary transition were provided by ADE/ESS transition specialists at the annual ADE/ESS Directors Institute (DI). Of the 716 in attendance at the DI, 59 participants attended sessions related to secondary transition.  There were 28 PEAs that Part B Arizona Annual Performance Report for FFY 2012 22 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 participated in team trainings for Year 1 (14 PEAs in the 2012– 2014 cohort) or Year 2 (14 PEAs in the 2011–2013 cohort) of the Secondary Transition Mentoring Project (STMP) capacity building grant. Through a contract with the University of Kansas Transition Coalition and in collaboration with ADE/ESS, training was provided to achieve and sustain 100% compliance on Indicator 13 using State and local data results from Indicators 1, 2, and 14 as a tool to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. Team training included: three two-day face-to-face training sessions, webinars, a Web site for project participants containing all training materials and extensive resources, an online short course for Year 1 teams, and attendance at Arizona’s Twelfth Annual Transition Conference.  Collaboration with national technical assistance centers and organizations occurred throughout the year and included: participation in NPSO and NSTTAC Community of Practice calls; use of resources from the NSTTAC Web site; participation in the NASDSE IDEA Partnership, Community of Practice National Meeting; and attendance at the National Secondary Transition Planning Institute where OSEP, NPSO, NSTTAC, and NDPC-SD provided guidance. Additionally, ADE/ESS maintained ongoing collaboration with the University of Kansas Transition Coalition and the Mountain Plains Regional Resource Center.  At the State level, ADE/ESS collaborated with RSA/VR, Division of Developmental Disabilities (DDD), Division of Part B Arizona Annual Performance Report for FFY 2012 23 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN), and met every other month with these agency stakeholders through the Arizona Community of Practice on Transition (AZCoPT). In FFY 2012, AZCoPT revised a presentation through RSA/VR and DBHS teleconferencing media that was used statewide to introduce participants to the supports/services available to school-aged and adult individuals with disabilities.  ADE/ESS collaboration meetings with the ADE areas of High School Renewal and Improvement, Career and Technical Education, Dropout Prevention, and Career and College Readiness were scheduled and held throughout the year and resulted in crosstraining for conferences that were sponsored by different ADE sections and that included the topic of secondary transition.  Intra-ADE/ESS collaborative efforts included: monthly meetings with PINS (Parent Information Network Specialists) as fellow AZCoPT members, as well as involvement with PINS during quarterly Transition Conference Planning Committee meetings; at least quarterly meetings with ESS Program Support to discuss the use of the Annual Site Visit Log (ASVL); review/revision of the secondary transition section of the monitoring manual and needed secondary transition trainings for ESS program specialists and PEAs; and the development of collaborative presentations with the ADE/ESS assistive technology unit. Part B Arizona Annual Performance Report for FFY 2012 24 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 c) On an annual basis, create and disseminate information through a variety of sources: annual statewide conference, monitoring alerts, Web site, and listserv announcements Activities completed from 8/1/12 through 6/30/13. 7/1/12– 6/30/13 ADE/ESS Transition Specialists 7/1/12– 6/30/13 ADE/ESS Transition Specialists  Two secondary transition presentations were offered at the ADE/ESS Directors Institute in August 2012 for approximately 59 participants.  102 sessions either directly or indirectly related to Indicator 13 were offered at Arizona’s Twelfth Annual Transition Conference held in October 2012.  Between July 2012 and June 2013, the ADE/ESS Secondary Transition Web site was updated to include additional and/or revised Indicator 13 materials from NSTTAC, IDEA Partnership, and other secondary transition technical assistance centers. The Web address is http://www.azed.gov/specialeducation/special-projects /secondary-transition/. d) On an annual Activities completed from 7/1/12 to basis, analyze pre6/30/13. and post-training data collected  Analysis of ASVL pre-training through the Annual data was completed in January Site Visit Log (ASVL) 2013 after all ESS program for each PEA to specialists were able to complete determine level of at least one PEA annual site visit compliance on all during fall 2012. The analysis eight required showed a 72.8% average for components of compliance with the eight items of Indicator 13 Indicator 13. ADE/ESS Program Specialists  Post-training data analysis of all PEAs that received a pre-training annual site visit and were trained in secondary transition during FFY 2012 showed an 85.8% average for compliance with the eight items of Indicator 13. Through pre- and post-training analysis, an increase of 13.3% in compliance for Indicator 13 was demonstrated after training and technical assistance was provided Part B Arizona Annual Performance Report for FFY 2012 25 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 to PEAs by transition and education program specialists. 2) Provide a twoyear capacity building grant to PEAs to participate in the Secondary Transition Mentoring Project (STMP) Team Training a) On an annual basis, identify PEAs who meet eligibility requirements and extend invitations to them to participate in STMP trainings Activities completed from 1/1/12 to 7/30/13. 7/1/12– 6/30/13  Using Annual Site Visit (ASV) data for PEAs currently in Years 1, 2, or 3 of the monitoring cycle, ESS invited 23 PEAs achieving less than 100% compliance on Indicator 13 to apply for a noncompetitive Secondary Transition Mentoring Project (STMP) capacity building grant. ADE/ESS Transition Specialists ADE/ESS Program Specialists  Fourteen PEAs were accepted for participation in Year 1 of the STMP team training in July 2012. b) On an annual basis, provide indepth and ongoing professional development on transition requirements and best practices Activities completed from 7/1/12 through 6/30/13. 7/1/12– 6/30/13 ADE/ESS Transition Specialists  STMP participants attended Arizona’s Twelfth Annual Transition Conference, which included a STMP team orientation and designated sessions.  Training materials and activities were designed, created, and disseminated by ADE/ESS in collaboration with the University of Kansas Transition Coalition. Trainings focused on: identifying PEA barriers to meeting transition requirements; developing an action plan to eliminate barriers; creating IEPs that meet transition requirements; implementing training to build intra-PEA capacity to attain 100% compliance on secondary transition requirements; and determining improvements made and targeting areas still in need of improvement.  Team training for each cohort consisted of three two-day faceto-face trainings throughout the year. Additionally, Year 1 teams completed an online short course specially designed for STMP Part B Arizona Annual Performance Report for FFY 2012 26 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 teams and focused on best practices in transition planning. c) On an annual Activity completed from 7/1/12 basis, analyze prethrough 6/30/13. and post-training data collected during  Measures to determine STMP trainings for effectiveness of STMP training each PEA that included the use of: participated to a) team process checklist; determine level of b) interrater reliability measures compliance on all for IEP file review; eight required c) training session evaluations; components of d) STMP team self-reported Indicator 13 progress toward completion of action plans; and e) feedback provided by ADE/ESS monitoring program specialists on compliance. 7/1/12– 6/30/13 ADE/ESS Transition Specialists Data from the measures are as follows: f) April 2013 trainings for both STMP cohorts included the completion of a 37-item “team process” checklist. Results from the checklist use indicated both Year 1 and Year 2 teams reported an average of 92.15% for the item “all team members are highly knowledgeable of transition compliance requirements” and an average of 80.8% for the item “STMP team collaboratively develops and implements an action plan that addresses the prioritized needs.” g) Training for both cohorts included instruction and practice activities using interrater reliability measures. h) All STMP training sessions during FFY 2012 included instruction, activities, and Indicator 13 file reviews. Based on a 5-point Likert scale (1 lowest and 5 highest), evaluation results indicated an average score of 4.27 across all activities for both cohorts. Additionally, all team members completed a pre-/post- Part B Arizona Annual Performance Report for FFY 2012 27 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 evaluation of their competency in meeting Indicator 13 requirements that asked the question, “How prepared do you feel?” Based on a 5-point Likert Scale (1 equaled unprepared and 5 equaled prepared) and using a Paired Samples T-Test, all scores indicated a significant increase in knowledge from the beginning to the end of the training for FFY 2012 for both cohorts (Years 1 and 2). The areas reporting the greatest increase in knowledge were transition assessments and documenting transition services and courses of study that will support the student in reaching postsecondary goals. i) Review of each STMP team’s action plan indicated all teams set goals to build and sustain systems to ensure 100% compliance on Indicator 13 school- or district-wide. All teams reported progress in meeting these goals. j) During the April 2013 training, Year 2 teams met with ADE/ESS program specialists to receive informal feedback on IEPs written during the second year of the STMP training program. Feedback indicated significant improvement in PEA knowledge and compliant practices. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 28 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 3: Assessments Participation and performance of children with IEPs on statewide assessments: A. Percent of the districts with a disability subgroup that meets the State’s minimum “n” size that meet the State’s AYP/AMO targets for the disability subgroup. B. Participation rate for children with IEPs. C. Proficiency rate for children with IEPs against grade level, modified and alternate academic achievement standards. (20 U.S.C. 1416 (a)(3)(A)) Measurement A. (choose either A.1 or A.2) A.1 AYP percent = [(# of districts with a disability subgroup that meets the State’s minimum “n” size that meet the State’s AYP targets for the disability subgroup) divided by the (total # of districts that have a disability subgroup that meets the State’s minimum “n” size)] times 100. A.2 AMO percent = [(# of districts with a disability subgroup that meets the State’s minimum “n” size that meet the State’s AMO targets for the disability subgroup) divided by the (total # of districts that have a disability subgroup that meets the State’s minimum “n” size)] times 100. B. Participation rate percent = [(# of children with IEPs participating in an assessment) divided by the (total # of children with IEPs enrolled during the testing window, calculated separately for reading and math)]. The participation rate is based on all children with IEPs, including both children with IEPs enrolled for a full academic year and those not enrolled for a full academic year. C. Proficiency rate percent = ([(# of children with IEPs scoring at or above proficient against grade level, modified and alternate academic achievement standards) divided by the (total # of children with IEPs who received a valid score and for whom a proficiency level was assigned, and, calculated separately for reading and math)]. The proficiency rate includes both children with IEPs enrolled for a full academic year and those not enrolled for a full academic year. Data Data Source The assessment data were from Arizona’s Instrument to Measure Standards (AIMS) and Arizona’s Instrument to Measure Standards Alternate (AIMS A). Data Description The assessment data were analyzed by the Arizona Department of Education’s Accountability Division/Research and Evaluation section (ADE/R and E) and the Information Technology division (IT). The Annual Measurable Objectives (AMO) data are the same data as the State’s data reported to the United States Department of Education (USDOE) under Title I of the Elementary and Secondary Education Act (ESEA). The participation and proficiency data are the same data as the State’s data Part B Arizona Annual Performance Report for FFY 2012 29 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 reported to the United States Department of Education in the Consolidated State Performance Report (CSPR) submitted December 20, 2013. Arizona’s alternate assessment, AIMS A, is based on alternate academic achievement standards. The AIMS A is not based on grade level academic achievement standards and is not based on modified academic achievement standards. The AIMS and AIMS A data were used for determining AMO and for reporting participation and performance. The grades tested for FFY 2012 were grades 3 through 8 and grade 10. The State uses four categories for the proficiency status:  Falls Far Below the Standard (F)  Approaches the Standard (A)  Meets the Standard (M)  Exceeds the Standard (E) Students who met the standard (M) or exceeded the standard (E) were counted as proficient. Valid and Reliable Data The assessment data were obtained from the ADE Accountability Division/Research and Evaluation section and the Information Technology division (IT), which follows internal processes to ensure valid, reliable, and accurate data. The ADE Standards and Assessment division/Assessment section ensures its assessments adhere to the Standards for Educational and Psychological Testing. Elementary and Secondary Education Act (ESEA) Flexibility The USDOE issued an extension to the conditional approval to Arizona’s request for Elementary and Secondary Education Act (ESEA) flexibility, also known as an ESEA waiver, on November 25, 2013. The conditional flexibility request is extended to the end of the 2013–2014 school year. Arizona’s goal for the flexibility request is to merge two different accountability systems, State and federal, into one seamless system that positions every student on track to college- and career-readiness. The ESEA waiver eliminates the requirement for Arizona to determine Adequate Yearly Progress (AYP). Arizona was given the flexibility to redefine proficiency targets, setting new Annual Measurable Objectives (AMOs) in mathematics and reading/language arts in order to achieve 100% proficiency for all students by school year 2019–2020. Indicator 3A.2 — Annual Measurable Objectives (AMOs) Target Data for FFY 2012 FFY Measurable and Rigorous Target FFY 2012 26.5% Indicator 3A.2 — Annual Measurable Objectives (AMOs) Actual Target Data for FFY 2012 Year Total Number of PEAs Number of PEAs That Met the Minimum “n” Size Number of PEAs That Met the Minimum “n” Size and Met AMO for FFY 2012 Percent of PEAs FFY 2012 (2012– 2013) 651 69 0 0% Part B Arizona Annual Performance Report for FFY 2012 30 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Arizona did not meet the target for Indicator 3A.2 for AMO. Indicator 3B — Target Data for Mathematics and Reading Participation for FFY 2012 FFY 2012 Measurable and Rigorous Targets Mathematics 95% Reading 95% Indicator 3B — Actual Target Data for Mathematics Participation for FFY 2012 Mathematics Assessment Participation for FFY 2012 Total Number of Children with Disabilities Participating Percentage of Children with Disabilities Participating Who Took the Specified Assessment a Children with IEPs enrolled 72,685 b Children with IEPs participating in regular assessment without accommodations 31,396 43.2% c Children with IEPs participating in regular assessment with accommodations 33,617 46.3% d Children with IEPs participating in alternate assessment against grade-level standards 0 0 e Children with IEPs participating in alternate assessment against modified standards 0 0 f Children with IEPs participating in alternate assessment based on alternate achievement standards 6,581 9.1% g Students with IEPs participating (b+c+d+e+f) 71,594 98.5% Children included in a but not included in the other counts above Children with IEPs who were not participants, were absent, or had invalid scores, consistent with Arizona’s Accountability Workbook and requirements. 1,091 1.5% Arizona exceeded the target for Indicator 3B for the mathematics participation rate. Part B Arizona Annual Performance Report for FFY 2012 31 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator 3B — Actual Target Data for Reading Participation for FFY 2012 Reading Assessment Participation for FFY 2012 Total Number of Children with Disabilities Participating Percentage of Children with Disabilities Participating Who Took the Specified Assessment a Children with IEPs enrolled 72,752 b Children with IEPs participating in regular assessment without accommodations 36,542 50.2% c Children with IEPs participating in regular assessment with accommodations 28,587 39.3% d Children with IEPs participating in alternate assessment against grade-level standards 0 0 0 0 6,581 9.0% 71,710 98.6% e f g Children with IEPs participating in alternate assessment against modified standards Children with IEPs participating in alternate assessment based on alternate achievement standards Students with IEPs participating (b+c+d+e+f) Children included in a but not included in the other counts above Children with IEPs who were not participants, were absent, or had invalid scores, consistent with Arizona’s Accountability Workbook and requirements. 1,042 1.4% Arizona exceeded the target for Indicator 3B for the reading participation rate. Mathematics and Reading Participation Data The mathematics and reading participation data are the same as the State’s data used for accountability reporting under Title I of the ESEA and were reported in Arizona’s Consolidated State Performance Report Part I, submitted December 20, 2013. Mathematics and reading participation rates are inclusive of all ESEA grades assessed in Arizona (grades 3 through 8 and grade 10) for students with IEPs and inclusive of all assessments (regular and alternate). The calculation includes all students with IEPs in all the grades assessed, including those enrolled for a full academic year and those not enrolled for a full academic year. The differences between the denominators for mathematics and reading can be attributed to the different assessment dates for the different grades and subjects. Mathematics and reading assessments were given to all students in grades 3 through 8 on April 15, 2013. The mathematics assessment was given to students in grade 10 on April 9, 2013. The reading assessment was given to students in grade 10 on February 26, 2013. Part B Arizona Annual Performance Report for FFY 2012 32 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator 3C — Target Data for Mathematics and Reading Proficiency for FFY 2012 Annual Measurable Objectives (AMOs) by Grade and Subject Grades Mathematics Reading 3 72% 80% 4 70% 79% 5 68% 82% 6 65% 84% 7 67% 85% 8 61% 76% 10 67% 81% The mathematics and reading AMOs are the same as the State’s AMOs as given in the State of Arizona ESEA Flexibility Request, dated July 13, 2012, which is considered the current Arizona Accountability Workbook. The mathematics and reading AMOs are the new AMOs established by the ADE that increase in equal increments annually and result in 100% proficiency no later than the end of school year 2019– 2020. Indicator 3C — Actual Target Data for Mathematics Proficiency for FFY 2012 FFY 2012 Grades Total enrolled Total number tested and enrolled for full academic year Total number children with IEPs scoring at or above proficient Total percent children with IEPs scoring at or above proficient Mathematics Assessment Proficiency 3 4 5 6 7 8 10 11,171 11,370 11,253 10,678 10,176 9,741 8,296 11,019 11,228 11,138 10,557 10,049 9,577 8,026 4,345 3,705 3,200 2,670 2,488 1,908 1,568 39.4% 33.0% 28.7% 25.3% 24.8% 19.9% 19.5% Arizona did not meet the target for Indicator 3C for mathematics proficiency in any of the assessed grades—3 through 8 and 10. Part B Arizona Annual Performance Report for FFY 2012 33 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator 3C — Actual Target Data for Reading Proficiency for FFY 2012 FFY 2012 Grades Total enrolled Total number tested and enrolled for full academic year Total number children with IEPs scoring at or above proficient Total percent children with IEPs scoring at or above proficient Reading Assessment Proficiency 3 4 5 6 7 8 10 11,170 11,368 11,253 10,678 10,176 9,741 8,366 11,014 11,223 11,139 10,557 10,050 9,575 8,152 4,489 4,777 4,600 4,183 4,861 2,825 3,480 40.7% 42.6% 41.3% 39.6% 48.4% 29.5% 42.7% Arizona did not meet the target for Indicator 3C for reading proficiency in any of the assessed grades—3 through 8 and 10. Mathematics and Reading Proficiency Data The mathematics and reading proficiency data are provided in the same format as the State’s data reported under Title I of the ESEA in Arizona’s Consolidated State Performance Report Part I, submitted December 20, 2013. Mathematics and reading proficiency rates are inclusive of all ESEA grades assessed in Arizona (grades 3 through 8 and grade 10) for students with IEPs and inclusive of all assessments (regular and alternate). The calculation includes all students with IEPs in all the grades assessed, including those enrolled for a full academic year and those not enrolled for a full academic year. The differences between the denominators for mathematics and reading can be attributed to the different assessment dates for the different grades and subjects. Mathematics and reading assessments were given to all students in grades 3 through 8 on April 15, 2013. The mathematics assessment was given to students in grade 10 on April 9, 2013. The reading assessment was given to students in grade 10 on February 26, 2013. Explanation of Progress and Slippage That Occurred for FFY 2012 The target was not met for Indicator 3A.2—AMO; Arizona saw slippage from FFY 2011 (2.9%) to FFY 2012 (0%). Arizona exceeded mathematics and reading targets for Indicator 3B—Participation. Both mathematics and reading participation remained the same as FFY 2011 (98.5% mathematics, 98.6% reading). Although Arizona did not meet the targets for Indicator 3C—proficiency in mathematics or reading, Arizona saw slight improvement in specific grade levels. In mathematics, there was an increase in proficiency for grades 6, 7, 8, and 10. The largest gain was a 1.5% increase in proficiency for grade 7. In reading, there was improvement in grades 3, 4, 5, 7, and 10. Grades 7 and 10 made the largest gains of Part B Arizona Annual Performance Report for FFY 2012 34 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 4.2% and 3.8%, respectively. Arizona will continue its efforts to increase student achievement in the two subject areas. In FFY 2012, the ADE/ESS supported PEAs in the area of mathematics, targeting a range of grades with the sponsorship of three initiatives—Special Education Using Mathematics for School Improvement Project (SUMS), Special Education Achieving Success in Mathematics (SEAS-Math), and Dimensions of Algebra. A reading program, Passages: Achieving Success in Reading, was implemented in middle schools and high schools. Both the math and reading initiatives provided research-based interventions, strategies and resources, and coaching to the school-based teams. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (Goal) 1) Increase opportunities for training in mathematics strategies to public education agency (PEA) special education personnel and distribute resource information in reading 2) Increase opportunities for training in reading strategies to public education agency Sub-Activities (Objectives or Action Steps) a) Conduct mathematics strategy trainings annually at the Directors Institute for special education personnel from school districts and charter schools Timeline Complete Activity completed August 2012. Projected Resources (Planned) 7/1/11– 6/30/13 CSPD Staff The 2012 Directors Institute offered special education directors/ teachers two sessions in mathematics These sessions offered strategies that could be implemented immediately. b) Compile mathematics strategy and resource information Activity completed from 7/1/11 to 6/30/12. Mathematics strategy and resource information was collected and is located at http://www.azed.gov/azcc rs/instructionaltoolbox/ 7/1/11– 6/30/13 CSPD Staff c) Disseminate mathematics strategy and resource information through the Arizona Promising Practices Web site, the ESS listserv, and ESS/CSPD trainings Activity completed 6/30/13 7/1/11– 6/30/13 CSPD Staff a) Represent ESS at the Response to Intervention (RTI) meetings with other ADE divisions Activity discontinued due to restructuring of personnel and reorganization of the Division. 7/1/10– 6/30/13 CSPD Staff New materials are being added on a regular basis. Part B Arizona Annual Performance Report for FFY 2012 35 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 (PEA) special education personnel and distribute resource information in reading b) Conduct reading strategy trainings annually at the Directors Institute for special education personnel from school districts and charter schools Activity completed August 2012. c) Compile reading strategy and resource information Activity completed from 7/1/12 to 6/30/13. 7/1/11– 6/30/13 CSPD Staff 7/1/11– 6/30/13 CSPD Staff 7/1/11– 6/30/13 CSPD Staff 9/1/11– 6/30/13 CSPD Staff The 2012 Directors Institute offered special education teachers two sessions on reading strategies for kindergarten through high school students. Reading strategy and resource information was collected and is located at http://www.azed.gov/azcc rs/instructionaltoolbox/ 3) By the end of FFY 2012, teams will increase their content knowledge of scientifically based strategies for mathematics instruction for grades K–2 students with IEPs through the Special Education Using Mathematics for School Improvement Project (SUMS) d) Disseminate reading strategy and resource information through the Arizona Promising Practices Web site, the ESS listserv, and ESS/CSPD trainings Activity completed in August 2012. a) Teams will learn and use a cyclical process of screening, content strategies, data analysis, and collaboration on student need and will implement studentspecific strategies. Activity completed from 7/1/12 to 6/30/13. Measurement will be team effectiveness data (each team will have a coach during the training sessions to assist with the team process) Each of the nine SUMS teams nine teams had coaches during the training sessions. Each team completed a survey rating the support received from the coaches. The five-point scale ranged from strongly disagree (1) to strongly agree (5). The overall rating for the effectiveness of the The 2012 Directors Institute offered special education teachers two sessions on reading strategies for kindergarten through high school students. All nine SUMS teams received training on the cyclical instruction process, and 100% of the nine teams completed instructional plans. Part B Arizona Annual Performance Report for FFY 2012 36 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 coaches was 4.2. 4) By the end of FFY 2012, teams will increase their content knowledge of b) Apply the 2010 Arizona mathematics standards using the mathematics processes that enable students with IEPs to become fluent in mathematics, as reflected in classroom observation protocol visits and various assessments Activity completed from 7/1/12 to 6/30/13. 9/1/11– 6/30/13 CSPD Staff c) Track and analyze data of students with IEPs over time using AIMS data and other assessment data provided by the schools AIMS data is not available for K-2 students yet because these students will take AIMS for the first time as third graders during school year 2013–2014. Other assessment data used by the group showed student growth ranged from 5% to 42%. 9/1/11– 6/30/13 CSPD Staff d) Analyze teacher pre- and postassessment data to determine continuity of learning and instruction Activity completed from 7/1/12 to 6/30/13. 9/1/11– 6/30/13 CSPD Staff a) Teams will learn and use a cyclical process of screening, content strategies, Activity completed from 7/1/12 to 6/30/13. 9/1/11– 6/30/13 CSPD Staff All of the SUMS teams received instruction in the Mathematics Processes. The Mathematics Student-Centered Observation Protocol was used to measure whether the strategies were being used in classrooms. The protocol was based on a two-point rating system with two being strategies totally observed and zero, strategies not observed. The schools started with an average of 0.9 and ended with an average of 1.3. Team members were given a pre-assessment and a post-assessment. The average score for the pre-assessment was 4.1 of 15 possible points. The average score for the post-assessment was 14.5 of 15 possible points. All of the 18 SEAS-Math Part B Arizona Annual Performance Report for FFY 2012 37 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 scientifically based strategies for mathematics instruction for students with IEPs in grades 3– 5 through the Special Education Achieving Success in Mathematics (SEASMath) data analysis, and collaboration on student need and will implement studentspecific strategies teams received training on the cyclical instruction process. 80% of the 18 teams completed instructional plans. Measurement will be team effectiveness data (each team will have a coach during the training sessions to assist with the team process) Each of the 18 teams had coaches during the training sessions. Each team completed a survey rating the support received from the coach. The five-point scale ranged from strongly disagree (1) to strongly agree (5). The overall rating for the effectiveness of the coaches was 4.2. b) Apply the 2008 Arizona mathematics standards and make connections to the 2010 Arizona mathematics standards, using the mathematics processes that enable students with IEPs to become more fluent in mathematics as reflected in the classroom observation protocol visits and various assessments Activity completed from 7/1/12 to 6/30/13. c) Track and analyze data of students with IEPs over time using AIMS data and other assessment data provided by the schools Activity completed from 7/1/12 to 6/30/13. 9/1/11– 6/30/13 CSPD Staff 9/1/11– 6/30/13 CSPD Staff All of the teams received instruction in the Arizona Common Core Standards for Mathematics. The Mathematics StudentCentered Observation Protocol was used to determine whether the strategies were used in the classrooms. The protocol was based on a two-point rating system with two being strategies totally observed and zero, strategies not observed. The schools started with an average of 0.7 and ended with an average increase of 1.4. The 18 SEAS-Math teams started the year with an average of 66.6% in AIMS math for grades three, four, and five and ended with an average of 68.5%. Some schools remained within one or Part B Arizona Annual Performance Report for FFY 2012 38 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 two percentage points on proficiency, and other schools had gains of five percent to ten percent for the first year of implementation. Ten of the teams are in their first year of implementation. d) Analyze teacher pre- and postassessment data to determine continuity of learning and instruction 5) By the end of FFY 2012, teams will increase their content knowledge of scientifically based strategies for mathematics instruction for students with IEPs in grades 7– 12 through Dimensions of Algebra a) Provide training in algebraic strategies for students with IEPs that will enable them to access mathematics from a concrete model to an abstract model b) Apply data analysis processes from various types of assessments to differentiate instruction for students with IEPs c) Provide a networking format for middle school and high school teachers Activity completed from 7/1/12 to 6/30/13. 9/1/11– 6/30/13 CSPD Staff All of the SEAS-Math teams completed the preassessment, resulting in an average score of four of 15 possible points The post-assessment average score was 12.7. . Activity completed from 7/1/12 to 6/30/13. 9/1/11– 6/30/13 CSPD Staff All three Dimensions of Algebra teams received instruction on scientificbased strategies. Strategies were taught starting with the concrete stage, then moving to the pictorial stage, and finally to the abstract stage through the use of written and verbal communication of mathematical ideas. Activity completed from 7/1/12 to 6/30/13. 9/1/11– 6/30/13 CSPD Staff 9/1/11– 6/30/13 CSPD Staff All of the Dimensions of Algebra teams administered algebra assessments to students with disabilities. The assessments were scored, and then student need was identified and instructional plans were written based on the data. This activity was not completed because the teams that remained as grant participants (three Part B Arizona Annual Performance Report for FFY 2012 39 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 6) Middle school teams participating in Passages: Achieving Success in Reading training will receive training that, when implemented, will result in an increased reading proficiency rate of eighth-grade students with IEPs to meet the transitional needs of students with IEPs as they move from middle school to high school d) Track and analyze data of students with IEPs over time using AIMS data and other assessment data provided by the schools schools), started their own email conversations. (The majority of the teams decided not to continue participation.) Implementation of the strategies began January 2013, and the involved schools have just begun tracking benchmark data based on implementation of the new strategies... Therefore, a comparison of AIMS data at this time would not reflect full implementation and cannot be made until the 2013–2014 school year. e) Analyze teacher pre- and postassessment data to determine continuity of learning and instruction Activity completed from 7/1/12 to 6/30/13. a) Conduct Passages training for special education teachers and reading specialists Activity completed from 10/1/12 to 3/31/13. 9/1/11– 6/30/13 CSPD Staff 9/1/11– 6/30/13 CSPD Staff 10/1/11– 6/30/13 CSPD Capacity Building Grants and CSPD staff Because of the reduction in the number of participating schools, the entire content and structure of the training changed; therefore, the post-assessment was no longer valid. One middle school completed Passages training. The school’s team consisted of a special educator, two language arts teachers, a reading interventionist, and the building principal. Additional training was held for middle school special educators from all district middle schools. Four trainings were held and professional development was provided on effective literacy intervention strategies for adolescents. CSPD Reading Specialist Passages Presenter Research on Improving Reading ( middle school) CSPD Capacity Building Coaches Part B Arizona Annual Performance Report for FFY 2012 40 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Provide training on assessing and diagnosing student needs to guide instruction/intervention Activity completed from 7/1/12 to 6/30/13. c) Provide instructional strategy training in the areas of word study, vocabulary, and comprehension Activity completed from 7/1/12 to 6/30/13. 10/1/11– 6/30/13 Same 10/1/11– 6/30/13 Same The Passages team was trained in the administration and interpretation of three assessments: MAZE, spelling assessment, and a writing prompt assessment to determine the range of students’ abilities and to determine the greatest areas of need. Participating teachers gathered student assessment data for MAZE, the spelling assessment, and the writing prompt in September 2012. Preassessment data was compared to postassessment data in April 2013. During the months between data collection, teachers were trained on strategies for reading intervention. The Passages team received training in the areas of oral presentation of words, Greek/Latin roots, identification of morphemes and their relationships to vocabulary, and using graphic organizers to support comprehension. Additional training was provided to middle school special educators from all district middle schools. Four trainings were held, and professional development was provided on effective literacy intervention Part B Arizona Annual Performance Report for FFY 2012 41 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 strategies for adolescents, including phonemic awareness, phonics, morphology, syllable types, vocabulary, fluency, and comprehension. 7) High school teams participating in Passages: Achieving Success in Reading training will receive training that, when implemented, will result in an increased proficiency rate of 10th grade students with IEPs d) Provide site-based technical assistance to participating schools to offer feedback and support in the implementation of new strategies Activity completed from 7/1/12 to 6/30/13. 10/1/11– 6/30/13 Same e) Collect progress monitoring student data throughout training to determine effectiveness of strategies Teachers collected preand post-assessment data and analyzed data to determine the effectiveness of reading strategies implemented during the year. 10/1/11– 6/30/13 Same f) Provide capacity building coaching support to each team to increase team effectiveness during training sessions a) Conduct Passages training for special education teachers and reading specialists Activity completed from 7/1/12 to 6/30/13. 10/1/11– 6/30/13 Same The team was provided a capacity building coach at each session. Activity completed from 7/1/12 to 6/30/13. 10/1/11– 6/30/13 CSPD Capacity Building Grants and CSPD Staff The middle school was visited on three occasions for observations in the classrooms and for meetings with the team and site administrator. Two high schools participated in Passages training. These two teams consisted of two special education teachers, five English/Language Arts teachers, one social studies teacher, and one chemistry teacher. An additional four half days of training were provided to high school special educators, remedial English teachers, and general education teachers from two participating high schools. Professional CSPD Reading Specialist Passages Presenter Research on Improving Reading (high school) CSPD Capacity Building Coaches Part B Arizona Annual Performance Report for FFY 2012 42 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Provide training on assessing and diagnosing student needs to guide instruction/intervention c) Provide instructional strategy training in the areas of word study, vocabulary, and comprehension development was provided on effective literacy intervention strategies for adolescents, including phonemic awareness, phonics, morphology, syllable types, vocabulary, fluency, and comprehension. Activity completed from 7/1/12 to 6/30/13. 10/1/11– 6/30/13 Same 10/1/11– 6/30/13 Same The Passages teams were trained in the administration and interpretation of three assessments: MAZE, spelling assessment, and a writing prompt assessment to determine the range of students’ abilities and to determine greatest areas of need. Participating teachers gathered student assessment data for MAZE, the spelling assessment, and the writing prompt in September 2012. Preassessment data was compared to postassessment data in April 2013. During the months between data collection, teachers were trained on strategies for reading intervention. Activity completed from 7/1/12 to 6/30/13. The Passages teams received training in the areas of oral presentation of words, Greek/Latin roots, identification of morphemes and their relationships to vocabulary, and using graphic organizers to support comprehension. Part B Arizona Annual Performance Report for FFY 2012 43 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Additional training was provided to high school special educators and remedial English teachers from both high schools. Four trainings were held and professional development was provided on effective literacy intervention strategies for adolescents, including phonemic awareness, phonics, morphology, syllable types, vocabulary, fluency, and comprehension. d) Provide site-based technical assistance to participating schools to offer feedback and support in the implementation of new strategies Activity completed from 7/1/12 to 6/30/13. e) Collect progress monitoring student data throughout training to determine effectiveness of strategies f) Provide capacity building coaching support to each team to increase team effectiveness during training sessions 10/1/11– 6/30/13 Same Activity completed 7/1/12 to 6/30/13 10/1/11– 6/30/13 Same Teachers collected preand post-assessment data and analyzed data to determine the effectiveness of reading strategies implemented during the year. Activity completed from 7/1/12 to 6/30/13. 10/1/11– 6/30/13 Same The participating high schools were visited on three occasions for observations in the classrooms and for meetings with the team. Each team was provided a capacity building coach at each session. Public Reporting Information The location (URL) of public reports of assessment results conforming to 34 CFR § 300.160 (f) is http://www.azed.gov/research-evaluation/aims-assessment-results/. The FFY 2012 Annual Performance Report (APR) gives information about the participation of students with IEPs. The APR is located on the ADE/ESS Web site at http://www.azed.gov/special-education /resources/spp-apr/ under the list titled Annual Performance Report. Part B Arizona Annual Performance Report for FFY 2012 44 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 45 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 4A: Suspension and Expulsion Percent of districts that have a significant discrepancy in the rate of suspensions and expulsions of greater than 10 days in a school year for children with IEPs. (20 U.S.C. 1416 (a)(3)(A); 1412 (a)(22)) Measurement Percent = [(# of districts that have a significant discrepancy in the rates of suspensions and expulsions for greater than 10 days in a school year of children with IEPs) divided by the (# of districts in the State)] times 100. Include State’s definition of “significant discrepancy.” Data Data Source The 2011–2012 data were reported by the PEAs through the Arizona Safety Accountability for Education (Az SAFE) application. The data are the same as the data reported under section 618, Table 5 (Report of Children with Disabilities Subject to Disciplinary Removal) for school year 2011–2012, which was due November 7, 2012. The October 1, 2011, child count data are the same as the State’s data reported under section 618, Table 1, Report of Children with Disabilities Receiving Special Education Under Part B of the Individuals With Disabilities Education Act. Note that the source of this data is from FFY 2011. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the Az SAFE data through the Annual Special Education Data Collection process, which uses internal edit checks. The State requires an assurance from the PEAs through their submission of a signed verification form attesting to the validity of the data. Definition of Significant Discrepancy and Methodology Arizona uses Statistical Analysis Software (SAS) to calculate rates of suspension and expulsion for children with IEPs. Arizona uses the state bar method to determine significant discrepancy. The State rate of suspensions/expulsions greater than 10 days for all students with IEPs is 0.82%. The State bar, 5.82%, is five percentage points greater than the State rate. A district or charter school has significant discrepancy when its suspension/expulsion rate greater than 10 days for students with IEPs is 5.82% or greater. There must be at least 50 students in the denominator of a suspension/expulsion rate for a district or charter school to be flagged as having significant discrepancy. The denominator represents the overall special education enrollment at the district or charter school. Part B Arizona Annual Performance Report for FFY 2012 46 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Using the minimum “n” size of 50 students for overall special education enrollment, Arizona excluded 8 PEAs from the calculation (excluded 8 from 599) and used the total number of PEAs (599) in the State in the denominator. Arizona compares the rates of suspensions and expulsions of greater than 10 days in a school year for students with IEPs among PEAs in the State. Target Data for FFY 2012 (using 2011–2012 data) FFY Measurable and Rigorous Target FFY 2012 (using 2011– 2012 data) 1.20% Indicator 4A — Actual Target Data for FFY 2012 (using 2011–2012 data) 0.3% Arizona met the target. PEAs with Significant Discrepancy in Rates for Suspension and Expulsion Year Total Number of PEAs* Number of PEAs That Have Significant Discrepancies Percent FFY 2012 (using 2011–2012 data) 599 2 0.3% *Arizona included the total number of PEAs in the denominator. Review of Policies, Procedures, and Practices for FFY 2012 (using 2011–2012 data) The State reviewed the PEAs’ suspension/expulsion data and identified two PEAs with a significant discrepancy. The PEAs reviewed their policies, procedures, and practices relating to the development and implementation of IEPs, use of positive behavioral interventions and supports, and procedural safeguards to determine if these contributed to the significant discrepancy. Arizona required the PEAs to have special education policies and procedures in compliance with all regulatory requirements prior to having Part B-IDEA Basic Entitlement Grant funds approved by the ADE/ESS. The PEAs were required to resubmit the discipline policies and procedures for review by ESS program specialists to determine if they were in alignment with the requirements of 34 CFR § 300.530 through § 300.536. The practices of the PEAs were reviewed by means of a self assessment. The PEAs conducted an assessment of their discipline practices, which consisted of a series of questions requiring narrative Part B Arizona Annual Performance Report for FFY 2012 47 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 responses and a review of student files using the State’s monitoring forms. ADE/ESS specialists conducted on-site visits and/or desk audits during the self assessments to validate the decisions made by the PEAs during the file reviews. Upon the completion of this review, Arizona determined that the two PEAs were in compliance with IDEA requirements that pertain to the development and implementation of IEPs, use of positive behavioral interventions and supports, and procedural safeguards. Arizona did not make any findings of noncompliance with Part B requirements as a result of the review required by 34 CFR § 300.170 (b). Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Conduct trainings related to the discipline process for students with disabilities Sub-Activities (Objectives or Action Steps) a) Conduct semiannual Principal Institutes in the three main geographical regions of the State b) Disseminate Special Education Handbook for Principals: A Quick Reference for Law Related Issues to participants at the Principals Institutes Timeline Complete Three Principals Institute: Special Education Legal Issues, were presented in:  Flagstaff on September 19-20, 2012  Phoenix on September 27 and 28, 2012  Tucson on Oct 2 and 3, 2012 The trainings were developed and presented by the Dispute Resolution director and the complaint investigator/corrective action compliance monitor. Activity completed from 7/1/12 to 6/30/13. Participants at the first session received:  Special Education Handbook for Principals: A Reference for Law Related Issues Part I  Principals Institute Power Point Part I  Special Education Acronyms  Glossary of Special Education Terms  Disability Classifications in Arizona Participants at the second session received:  Special Education Projected 7/1/11– 6/30/13 Resources (Planned) Comprehensi ve System of Personnel Development (CSPD) Staff Dispute Resolution Director and Staff 7/1/11– 6/30/13 CSPD Staff Dispute Resolution Director and Staff Part B Arizona Annual Performance Report for FFY 2012 48 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 2) Provide support for PEAs that are flagged as at risk for significant discrepancy a) On an annual basis, analyze data to flag PEAs that are at risk for significant discrepancy b) On an annual basis, notify PEAs that are flagged as at risk for significant discrepancy c) Provide assessment tools and resources to PEAs that are flagged as at risk to conduct root cause analyses 3) Provide support for PEAs that are flagged for significant discrepancy a) On an annual basis, notify PEAs that are flagged for significant discrepancy b) Provide technical assistance to PEA staff during the PEA’s review of Handbook for Principals: A Reference for Law Related Issues Part II  Dispute Resolution publication developed by Exceptional Student Services  Prior Written Notice form  What Circumstance Require a Prior Written Notice? handout  Annual Notification to Parents Regarding Confidentiality of Student Education Records form  Manifestation Determination form Activity completed 3/1/13. 7/1/11– 6/30/13 Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk suspension/expulsion rates. Activity completed from 4/22/13 to 6/30/13. 7/1/11– 6/30/13 ESS specialists reviewed suspension/expulsion data during on-site meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 4/22/13 to 6/30/13. 7/1/11– 6/30/13 ESS specialists gave self assessment tools and resources to PEAs that were identified as at-risk. Activity completed 4/22/13. ESS notified PEAs that were flagged for significant discrepancy. Activity completed from 4/22/13 to 6/30/13. ESS specialists made on- 7/1/11– 6/30/13 7/1/11– 6/30/13 Suspension/ Expulsion Data ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists Resource Tools and Lists ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists Part B Arizona Annual Performance Report for FFY 2012 49 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 policies, procedures, and practices site visits to work with PEA staff during the review of policies, procedures, and practices. Resource Tools and Lists Correction of FFY 2011 Findings of Noncompliance Arizona did not make any findings of noncompliance in FFY 2011 as a result of the review of policies, procedures, and practices required by 34 CFR 300.170 (b). Actions Taken if Noncompliance Not Corrected Arizona did not make any findings of noncompliance in FFY 2011. Verification of Correction (either timely or subsequent) Arizona did not make any findings of noncompliance in FFY 2011. Correction of Remaining FFY 2010 Findings of Noncompliance Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2009 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 50 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 4B: Suspension and Expulsion by Race or Ethnicity Percent of districts that have: (a) a significant discrepancy, by race or ethnicity, in the rate of suspensions and expulsions of greater than 10 days in a school year for children with IEPs; and (b) policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards. (20 U.S.C. 1416 (a)(3)(A); 1412 (a)(22)) Measurement Percent = [(# of districts that have: (a) a significant discrepancy, by race or ethnicity, in the rates of suspensions and expulsions of greater than 10 days in a school year of children with IEPs; and (b) policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards) divided by the (# of districts in the State)] times 100. Data Data Source The 2011–2012 data were reported by the PEAs through the Arizona Safety Accountability for Education (Az SAFE) application. The data are the same as the data reported under section 618, Table 5 (Report of Children with Disabilities Subject to Disciplinary Removal) for school year 2011–2012, which was due November 7, 2012. The October 1, 2011, child count data are the same as the State’s data reported under section 618, Table 1, Report of Children with Disabilities Receiving Special Education Under Part B of the Individuals With Disabilities Education Act. Note that the source of this data is from FFY 2011. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the Az SAFE data through the Annual Special Education Data Collection process, which uses internal edit checks. The State requires an assurance from the PEAs through their submission of a signed verification form attesting to the validity of the data. Definition of Significant Discrepancy and Methodology Arizona uses Statistical Analysis Software (SAS) to calculate rates of suspension and expulsion by race or ethnicity for children with IEPs. Arizona uses the state bar method to determine significant discrepancy. The State rate of suspensions/expulsions greater than 10 days for all students with IEPs is 0.82%. The State bar, 5.82%, is five percentage points greater than the State rate. Any district or charter school that suspends or expels 5.82% or more of its students with IEPs of a given race/ethnicity for more than 10 days is flagged for significant discrepancy. There must be at least 50 Part B Arizona Annual Performance Report for FFY 2012 51 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 students in the denominator of a suspension/expulsion rate for a district or charter school to be flagged as having significant discrepancy. The denominator represents the special education enrollment at the district or charter school for a given race/ethnicity. Using the minimum “n” size of 50 students for a given race/ethnicity enrollment, Arizona excluded 27 PEAs from the calculation (excluded 27 from 599) and used the total number of PEAs (599) in the State in the denominator. Arizona compares the rates of suspension and expulsion of greater than 10 days in a school year for students with IEPs among PEAs in the State. Target Data for FFY 2012 (using 2011–2012 data) FFY Measurable and Rigorous Target FFY 2012 (using 2011– 2012 data) 0.0% Indicator 4B — Actual Target Data for FFY 2012 (using 2011–2012 data) 0.0% Arizona met the target. 4B (a) PEAs with Significant Discrepancy, by Race or Ethnicity, in Rates of Suspension and Expulsion Year Total Number of PEAs* Number of PEAs That Have Significant Discrepancies by Race or Ethnicity Percent of PEAs FFY 2012 (using 2011–2012 data) 599 6 1.0% *Arizona included the total number of PEAs in the denominator. 4B (b) PEAs with Significant Discrepancy, by Race or Ethnicity, in Rates of Suspensions and Expulsions; and policies, procedures, or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards Part B Arizona Annual Performance Report for FFY 2012 52 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Year Total Number of PEAs* Number of PEAs That Have Significant Discrepancies, by Race or Ethnicity, and policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards FFY 2012 (using 2011–2012 data) 599 0 Percent of PEAs 0.0% *Arizona included the total number of PEAs in the denominator Review of Policies, Procedures, and Practices for FFY 2012 (using 2011–2012 data) The State reviewed the PEAs’ suspension/expulsion data by race or ethnicity and identified six PEAs with a significant discrepancy. These PEAs reviewed their policies, procedures, and practices relating to the development and implementation of IEPs, use of positive behavioral interventions and supports, and procedural safeguards to determine if these contributed to the significant discrepancy. Arizona required these PEAs to have special education policies and procedures in compliance with all regulatory requirements prior to having Part B-IDEA Basic Entitlement Grant funds approved by the ADE/ESS. These PEAs were required to resubmit the discipline policies and procedures for review by ESS program specialists to determine if they were in alignment with the requirements of 34 CFR § 300.530 through § 300.536. The practices of these PEAs were reviewed by means of a self assessment. The PEAs conducted an assessment of their discipline practices, which consisted of a series of questions requiring narrative responses and a review of student files using the State’s monitoring forms. ADE/ESS specialists conducted on-site visits and/or desk audits during the self assessments to validate the decisions made by the PEAs during the file reviews. Upon the completion of this review, Arizona determined that the six PEAs were in compliance with IDEA requirements that pertain to the development and implementation of IEPs, use of positive behavioral interventions and supports, and procedural safeguards. Arizona did not make any findings of noncompliance with Part B requirements as a result of the review required by 34 CFR § 300.170(b). Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Conduct trainings related to the discipline process Sub-Activities (Objectives or Action Steps) a) Conduct semiannual Principal Institutes in the Timeline Complete Three Principals Institute: Special Education Legal Issues, were presented in: Projected 7/1/11– 6/30/13 Resources (Planned) Comprehensi ve System of Personnel Part B Arizona Annual Performance Report for FFY 2012 53 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 for students with disabilities three main geographical regions of the State b) Disseminate Special Education Handbook for Principals: A Quick Reference for Law Related Issues to participants at the Principals Institutes  Flagstaff on September 19-20, 2012  Phoenix on September 27 and 28, 2012  Tucson on Oct 2 and 3, 2012 The trainings were developed and presented by the Dispute Resolution director and the complaint investigator/corrective action compliance monitor. Activity completed from 7/1/12 to 6/30/13. Participants at the first session received:  Special Education Handbook for Principals: A Reference for Law Related Issues Part I  Principals Institute Power Point Part I  Special Education Acronyms  Glossary of Special Education Terms  Disability Classifications in Arizona Participants at the second session received:  Special Education Handbook for Principals: A Reference for Law Related Issues Part II  Dispute Resolution publication developed by Exceptional Student Services  Prior Written Notice form  What Circumstance Require a Prior Written Notice? handout  Annual Notification to Parents Regarding Confidentiality of Student Education Records form  Manifestation Determination form Development (CSPD) Staff Dispute Resolution Director and Staff 7/1/11– 6/30/13 CSPD Staff Dispute Resolution Director and Staff Part B Arizona Annual Performance Report for FFY 2012 54 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 2) Provide support for PEAs that are flagged as at risk for significant discrepancy a) On an annual basis, analyze data to flag PEAs that are at risk for significant discrepancy b) On an annual basis, notify PEAs that are flagged as at risk for significant discrepancy c) Provide assessment tools and resources to PEAs that are flagged as at risk to conduct root cause analyses 3) Provide support for PEAs that are flagged for significant discrepancy a) On an annual basis, notify PEAs that are flagged for significant discrepancy b) Provide technical assistance to PEA staff during their review of policies, procedures, and practices Activity completed 2/1/13. 7/1/11– 6/30/13 Suspension/ Expulsion Data Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk suspension/expulsion rates by race/ethnicity. Activity completed from 4/22/13 to 6/30/13. 7/1/11– 6/30/13 ESS specialists reviewed suspension/expulsion data by race/ethnicity during onsite meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 4/22/13 to 6/30/13. ADE/ESS Directors and Program Specialists 7/1/11– 6/30/13 ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists ESS specialists gave self assessment tools and resources to PEAs that were identified as at risk. Activity completed 4/22/13. ESS notified PEAs that were flagged for significant discrepancy by race/ethnicity. Activity completed from 4/22/13 to 6/30/13. ESS specialists made onsite visits to work with PEA staff during the review of policies, procedures, and practices. 7/1/11– 6/30/13 7/1/11– 6/30/13 Resource Tools and Lists ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists Resource Tools and Lists Correction of FFY 2011 Findings of Noncompliance Arizona made one finding of noncompliance with Part B requirements as a result of the review required by 34 CFR § 300.170(b). Upon receipt of the finding of noncompliance, the PEA revised its discipline policies and procedures and corrected all noncompliance. The ADE/ESS program specialist verified that the PEA corrected all instances of noncompliance, including child specific. Based on subsequent file reviews of updated data, the specialist determined that the PEA implemented sustainable practices to meet the regulatory requirements. Part B Arizona Annual Performance Report for FFY 2012 55 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Actions Taken if Noncompliance Not Corrected All noncompliance was corrected. No further action necessary. Verification of Correction (either timely or subsequent) The ADE/ESS program specialist verified that the PEA timely corrected all instances of noncompliance, including child specific. Based on subsequent file reviews of updated data, the specialist determined that the PEA implemented sustainable practices to meet the regulatory requirements. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response Because the State reported less than 100% compliance for FFY 2011, the State must report on the status of correction of noncompliance reflected in the data the State reported for this Indicator. In the FFY 2012 APR for Indicator 4 and Indicator 15, Arizona reported on the status of the correction of noncompliance (FFY 2011 data). In reporting on correction of findings of noncompliance in the FFY 2012 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2011 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2012 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2012 APR for Indicator 4 and Indicator 15, Arizona reported that it verified each PEA with noncompliance (FFY 2011 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 56 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 5: School Age LRE Percent of children with IEPs aged 6 through 21 served: A. Inside the regular class 80% or more of the day; B. Inside the regular class less than 40% of the day; and C. In separate schools, residential facilities, or homebound/hospital placements. (20 U.S.C. 1416 (a)(3)(A)) Measurement A. Percent = [(# of children with IEPs served inside the regular class 80% or more of the day) divided by the (total # of students aged 6 through 21 with IEPs)] times 100. B. Percent = [(# of children with IEPs served inside the regular class less than 40% of the day) divided by the (total # of students aged 6 through 21 with IEPs)] times 100. C. Percent = [(# of children with IEPs served in separate schools, residential facilities, or homebound/hospital placements) divided by the (total # of students aged 6 through 21 with IEPs)] times 100. Indicator 5A, 5B, and 5C — Target Data for FFY 2012 FFY 2012 Measurable and Rigorous Targets Indicator 5A ≥ 80% Indicator 5B < 40% Indicator 5C Separate 56% 13.5% 1.3% Indicator 5A, 5B, and 5C — Actual Target Data for FFY 2012 number of children percentage of children Indicator 5A ≥ 80% Indicator 5B < 40% Indicator 5C Separate 69,600 17,339 2,204 62% 15% 2% number of students aged 6–21 with IEPs 112,895 Arizona exceeded the target for Indicator 5A. Arizona did not meet the target for Indicator 5B. Arizona did not meet the target for Indicator 5C. Part B Arizona Annual Performance Report for FFY 2012 57 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Data Data Source The data were collected through the October 1, 2012, Child Count report and are the same as the State’s data reported under section 618, Table 3, Part B, Individuals with Disabilities Education Act Implementation of FAPE Requirements. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because ESS collected, maintained, and reported the October 1, 2012, child count data and the placement data using internal edit checks. The State requires public education agencies (PEAs) to assure their data are accurate and reliable by having them submit signed verification letters. Explanation of Progress and Slippage That Occurred for FFY 2012 Arizona exceeded the target for Indicator 5A, but did not meet the targets for Indicators 5B and 5C. Slight progress was made from FFY 2011 on Indicators 5A, from 60.4% to 62%, and on 5C, from 2.8% to 2.0%. There was slight slippage on Indicator 5B from 14.68% to 15%. To address Indicators 5B and 5C, the ESS/Comprehensive System of Personnel Development (CSPD) unit sponsors the Autism Spectrum Disorders project: this project is designed to expand the number of Arizona educators who are skilled in meeting the needs of students with autism so that less restrictive environments are possible for these students. The two-year training program builds skills in evaluation, instructional planning and strategies, and environmental design that will address the academic, communication, behavioral, and social needs of students. The program, when implemented at school sites, improves the decision-making abilities of IEP team members regarding the least restrictive environment (LRE) for students with autism. On an annual basis, the ADE/ESS program specialists review least restrictive environment data with school administrators at each PEA in the State. If a PEA’s data do not meet State targets for LRE, then the concern is discussed with the administrator. If a PEA is in Year 4 of the monitoring cycle, then a selfassessment in this area may be one of the monitoring activities. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Conduct interviews with special education directors and site administrators about available service delivery models and LRE data as components of all on-site monitorings (documentation Sub-Activities (Objectives or Action Steps) a) Gather data from interview responses and supporting documentation of placement decisions and service delivery models for students with IEPs b) Revise interview questions and Timeline Complete Activity completed 6/1/11. Projected 10/1/10– 6/30/11 ADE/ESS Directors and Specialists 7/1/11– 6/30/12 ADE/ESS Directors and Interviews were conducted with special education and site administrators during all on-site monitorings to discuss service delivery models. Results of the interviews and the supporting documentation will be analyzed. Activity completed from 7/1/11 to 6/30/12. Resources (Planned) Part B Arizona Annual Performance Report for FFY 2012 58 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 requires an individualized decision-making process for placement) documentation requirements based on results related to LRE targets c) Conduct revised interviews and gather supporting documentation 2) Provide school teams with autism training designed to improve teams’ ability to make LRE decisions that will maintain or increase the amount of time students with autism are served in general education settings Specialists Interview question results and decision-making process documents were analyzed. Outcomes demonstrated a high level of both compliance and understanding related to LRE. No revisions were made. No revisions were necessary. a) Collect baseline data (FFY 2010 service codes) for students with autism enrolled in schools that participate in autism training Activity completed 12/1/12. b) Provide training that, if components are implemented, can increase teams’ decision-making abilities to maintain or increase the amount of time students with autism are served in general education settings by giving teams the:  knowledge of traits and characteristics of students with autism  ability to apply instructional supports and practices consistent with the needs of students with autism  skills to decrease ineffective behaviors of Activity completed from 7/1/12 to 6/30/13. 7/1/12– 6/30/13 ADE/ESS Directors and Specialists 7/1/11– 6/30/13 PEAs’ FFY 2010 service code data 7/1/11– 6/30/13 CSPD Staff Eleven teams participated in the ASD Year 1 2012–2014 cohort, representing 168 students receiving services under the category of autism. Within this group, approximately 18% received services in the A service code; 10%, in the B service code, and 71%, in the C service code. The training was completed for the ASD Year 1 cohort (11teams). An overview of autism was provided to the teams during the first training session. Team training included a series of lectures, activities, and training materials about Effective Practices in Autism over the course of 6 sessions. Additional supports during the training included supplying teams with the Ziggurat Model, a research-based model capitalizing on students’ strengths to address underlying deficits, to help teams recognize and identify characteristic behaviors in their students. (Eleven of 11 teams Team Training Materials School Administrator General Education Teacher(s) Special Education Teacher(s) Part B Arizona Annual Performance Report for FFY 2012 59 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 students with autism c) Provide capacity building coaching support to each team to increase team effectiveness during training completed this task.) Teams received supports for completing the Ziggurat worksheet and understanding the model through at least one face-to-face visit per team from the project coordinator and also received written feedback on completed products. Each team was observed to practice using methods to develop behavioral hypotheses in 2 sessions. Of the teams, 11 of 11 submitted a completed Reinforcer Survey, and of those, all used that data along with their Ziggurat information to develop an IEP goal, chose an instructional approach, and determine a reinforcement and reinforcement schedule. All teams submitted data indicating that the instructional approach was achieving the desired results. Activity completed from 7/1/12 to 6/30/13. Coaching was provided to the ASD Year 1 2012–2014 cohort and the ASD Year 2 2011–2013 cohort. The CSPD coaching coordinator provided training, forms, and feedback to the coaches. Twelve of the 13 coaches attended Coaching Refresher Training. Feedback provided by the teams for all the team trainings indicated that 81% of teams agreed or strongly agreed with the statement, “The team improves their ability to work collaboratively, to think deeper, to plan, and to reflect based on interactions with the coach.” 7/1/11– 6/30/13 CSPD Coaching Coordinator Coaching Refresher Training Coaching Presenter Coaching Materials Part B Arizona Annual Performance Report for FFY 2012 60 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 d) Provide training that, if implemented, would give teams the necessary processes to plan, execute, and evaluate the effectiveness of their activities Activity completed from 7/1/12 to 6/30/13 Of the 11 ASD Year 1 cohort teams, 84% completed all required implementation activities. 7/1/11– 6/30/13 CSPD Staff Implementati on Portfolios Training was provided to the ASD Year 2 cohort (7 teams) as follows: Social Skills/Student-level activities: To increase social success in regular classrooms for students with autism, teams received training on strategies, as well as an evidence-based tool to measure social skills. Six of the teams used the evaluation process to develop IEP goals supported by the data collected. Of those teams that developed goals, 5 teams provided data on the effectiveness of the strategy selected to teach the goals to the target student(s). Supports for the teams for this effort included site visits by the project coordinator (4 per year), phone consultations, and a review of implementation activities, as well as feedback to the teams on their progress. Sensory Integration/Schoolwide activities: To further increase student success in regular classrooms, teams received training on strategies and guidance on assessment of sensory needs for students with autism. Six of the teams developed a Sensory Strategy Action Plan that targeted their team’s local needs for addressing sensory issues within their school-based system. Four of those teams chose to develop visually based supports for Part B Arizona Annual Performance Report for FFY 2012 61 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 their school. Professional development (PD) plans were developed for dissemination of the planned supports. Of those PD plans,3 teams provided data in regard to the effectiveness of the professional development (using the PD tools provided). Supports for this effort included site visits by the project coordinator (4 per year), phone consultations, and review of implementation activities, as well as feedback to the teams on their progress. e) Collect FFY 2010 and FFY 2011 census data for students with autism enrolled in schools that participate in autism training Activity completed 10/1/11. f) Aggregate and disaggregate service code data for students with autism enrolled in schools that participate in autism training Seven teams met in January 2013 to analyze their LRE data. 7/1/11– 6/30/13 CSPD staff PEAs’ FFY 2010 and FFY 2011 census data All teams submitted the initial 10/1/11 census data, including the LRE code for students with autism. The teams submitted 10/1/12 census and LRE data for students with autism; Activity completed 01/2013 7/1/11– 6/30/13 CSPD Staff PEA Staff Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 62 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 6: Preschool LRE Percent of children aged 3 through 5 with IEPs attending a: A. Regular early childhood program and receiving the majority of special education and related services in the regular early childhood program; and B. Separate special education class, separate school, or residential facility. (20 U.S.C. 1416 (a)(3)(A)) Measurement A. Percent = [(# of children aged 3 through 5 with IEPs attending a regular early childhood program and receiving the majority of special education and related services in the regular early childhood program) divided by the (total # of children aged 3 through 5 with IEPs)] times 100. B. Percent = [(# of children aged 3 through 5 with IEPs attending a separate special education class, separate school, or residential facility) divided by the (total # of children aged 3 through 5 with IEPs)] times 100. Data Data Source The data were collected through the October 1, 2012, Child Count report and are the same as the State’s data reported under section 618, Table 3, Part B, Individuals with Disabilities Education Act Implementation of FAPE Requirements. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because ESS collected, maintained, and reported the October 1, 2012, child count data and the placement data using internal edit checks. The State requires PEAs to assure their data are accurate and reliable by having them submit signed verification letters. Target Data for FFY 2012 Targets were set based on the FFY 2011 baseline data and input from the stakeholder groups. Measurable and Rigorous Targets FFY 2012 Indicator 6A Indicator 6B 48.5% 45.5% Actual Target Data for FFY 2012 Part B Arizona Annual Performance Report for FFY 2012 63 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Actual Targets FFY Indicator 6A Indicator 6B 49.8% 44.81% 2012 Arizona met the targets. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity(GOAL) 1) Provide statewide baseline data to school districts 2) Continue to promote Head Start programs as least restrictive environment options Sub-Activities (Objectives or Action Steps) a) Provide school districts access to individual baseline data for Early Childhood Quality Improvement Process (ECQUIP) teams Timeline Complete Activity completed 6/30/13. Projected Resources (Planned) 7/1/12– 6/30/13 ADE/Early Childhood Special Education (ECSE) Baseline data from 2011 school year were sent to preschool coordinators at the local education agencies (LEAs). Data were also discussed during visits with Early Childhood staff and LEA staff throughout the school year. b) Provide school districts baseline data with 5-yearold kindergarten student data extrapolated to view preschool LRE data Activity completed 6/30/13. 7/1/12– 6/30/13 ADE/ECSE Provide resources through newsletters to continue the collaboration with Head Start and Arizona Early Intervention Program (AzEIP) to promote LRE and highlight Head Start’s mandate for 10% enrollment of children with disabilities Activity completed 7/1/12–6/30/13. 7/01/12– 6/30/13 ADE/ECSE Quarterly meetings held throughout the year. Head Start State Collaboration Office Arizona Head Start Association Part B Arizona Annual Performance Report for FFY 2012 64 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 3) Promote First Things First (FTF)– funded programs as a least restrictive environment option a) Provide information to school district programs that receive FTF scholarship grants through FTF/Early Childhood Education Quality mentors Activity completed 7/1/12–6/30/13. 7/01/12– 6/30/13 ADE/ECSE 4) Provide professional development opportunities that promote the least restrictive environment for preschool students with IEPs a) Continue collaboration with Arizona Council for Exceptional Children/Division of Early Childhood (CEC/DEC) for spring conference and presentations at the Exceptional Student Services (ESS) Directors Institute Activity completed 8/29/12. 5) Provide information to PEAs about preschool least restrictive environment data collection and reporting a) Post OSEP letter regarding preschool LRE on Web site (previously sent to all early childhood special education programs) Activity completed 8/1/12. 7/1/12– 6/30/13 ADE/ECSE b) Conduct webinars and professional development regarding preschool LRE data Activity completed 6/4/13. 7/1/12– 6/30/13 ADE/ECSE c) Include LRE information in new Help for Early Learning Professionals (HELP) manual and on Web site Activity completed 4/30/13. FTF/Early Childhood Education Quality Mentors Discussions occurred between ADE/ECSE staff and preschool coordinators throughout the year. 7/1/12– 6/30/13 ADE/ECSE CEC/DEC Presentations were given at the ESS Directors Institute on LRE and quality environments for all children. ESS Data Management Specialist A series of in-person workshops and webinars were conducted 5/7/13– 6/4/13. 7/1/12– 6/30/13 ADE/ECSE Revised HELP manual posted on Web site, which included an enhanced LRE section. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 65 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 7: Preschool Outcomes Percent of preschool children aged 3 through 5 with IEPs who demonstrate improved: A. Positive social-emotional skills (including social relationships); B. Acquisition and use of knowledge and skills (including early language/communication and early literacy); and C. Use of appropriate behaviors to meet their needs. (20 U.S.C. 1416 (a)(3)(A)) Measurement Outcomes A. Positive social-emotional skills (including social relationships); B. Acquisition and use of knowledge and skills (including early language/communication and early literacy); and C. Use of appropriate behaviors to meet their needs. Progress Categories for A, B, and C a. Percent of preschool children who did not improve functioning = [(# of preschool children who did not improve functioning) divided by (# of preschool children with IEPs assessed)] times 100. b. Percent of preschool children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers = [(# of preschool children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers) divided by (# of preschool children with IEPs assessed)] times 100. c. Percent of preschool children who improved functioning to a level nearer to same-aged peers but did not reach it = [(# of preschool children who improved functioning to a level nearer to sameaged peers but did not reach it) divided by (# of preschool children with IEPs assessed)] times 100. d. Percent of preschool children who improved functioning to reach a level comparable to sameaged peers = [(# of preschool children who improved functioning to reach a level comparable to same-aged peers) divided by (# of preschool children with IEPs assessed)] times 100. e. Percent of preschool children who maintained functioning at a level comparable to same-aged peers = [(# of preschool children who maintained functioning at a level comparable to same-aged peers) divided by (# of preschool children with IEPs assessed)] times 100. Summary Statements for Each of the Three Outcomes Summary Statement 1: Of those preschool children who entered or exited the preschool program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 6 years of age or exited the program. Measurement for Summary Statement 1: Percent = # of preschool children reported in progress category (c) plus # of preschool children reported in category (d) divided by [# of preschool children reported in progress category (a) plus # of preschool children reported in progress category (b) plus # of preschool children reported in progress category (c) plus # of preschool children reported in progress category (d)] times 100. Summary Statement 2: The percent of preschool children who were functioning within age expectations Part B Arizona Annual Performance Report for FFY 2012 66 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 in each Outcome by the time they turned 6 years of age or exited the program. Measurement for Summary Statement 2: Percent = # of preschool children reported in progress category (d) plus [# of preschool children reported in progress category (e) divided by the total # of preschool children reported in progress categories (a) + (b) + (c) + (d) + (e)] times 100. Data Data Source PEAs reported FFY 2012 preschool assessment data using Teaching Strategies GOLD, a Web-based data collection system. Sampling was not used for this Indicator as all preschool children with disabilities have their entry status and exit status assessed. Additionally, this assessment system is statewide and inclusive of all children in state-funded preschool programs, as well as all Head Start programs and one Tribal Head Start program. Instruments All early childhood programs must administer the one assessment tool approved by the Arizona State Board of Education for ongoing progress monitoring. Teaching Strategies is the publisher of the GOLD assessment. Trainings were provided to Arizona educators and administrators by Teaching Strategies staff in 2012 and 2013. These trainings continue to take place throughout the State for new users. Online training modules, including interrater reliability certification courses, are available through the GOLD subscriptions. Ongoing technical assistance has been provided by the Arizona Department of Education/Early Childhood Education (ADE/ECE) staff. ADE/ECE staff also train PEAs on a continual basis regarding the use of data to drive instruction and program improvement. Further guidance is provided through Arizona’s Early Childhood Assessment System for Ongoing Progress Monitoring (http://www.azed.gov/earlychildhood/files/2012/02/arizonas-early-childhood-assessment-system-for-on-going-progress-monitoring2012-13.pdf) and monthly GOLD Nugget publications (http://www.azed.gov/earlychildhood/2011/11/14/teaching-strategies-gold/). Data Analysis The preschool outcome data was analyzed by Teaching Strategies. Valid and Reliable Data The Arizona Department of Education/Early Childhood Special Education (ADE/ECSE) assures the validity and reliability of the early childhood outcomes (ECO) data by offering professional development on recording quality observations of children. PEAs have online access to an interrater reliability certification course provided by Teaching Strategies. Training is provided to administrators on accessing the Documentation Status Report that provides information on the number of observations used to assess the child and assign ratings. The Early Childhood Quality Improvement Process (ECQUIP) also incorporates the early childhood outcomes assessment process in the districts’ self-assessment rubric. The ECQUIP Manual is available at http://www.azed.gov/early-childhood/files/2011/11/ecquip-biner-final-11.28.12.pdf. Target Data for FFY 2012 Part B Arizona Annual Performance Report for FFY 2012 67 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Measurable and Rigorous Targets Positive SocialEmotional Skills Acquiring and Using Knowledge and Skills Taking Appropriate Action to Meet Needs 1. Of those children who entered or exited the program below age expectations, the percent who substantially increased their rate of growth by the time they exited. 72.2 75.0 71.9 2. Percent of children who were functioning at a level comparable to same-aged peers by the time they exited. 58.8 57.9 63.2 Positive SocialEmotional Skills Acquiring and Using Knowledge and Skills Taking Appropriate Action to Meet Needs 1. Of those children who entered or exited the program below age expectations, the percent who substantially increased their rate of growth by the time they exited. 79.9% 79.0% 76.2% 2. Percent of children who were functioning at a level comparable to same-aged peers by the time they exited. 63.3% 62.0% 67.0% FFY 2012 Actual Data for Preschool Children Exiting in FFY 2012 (2012-13) Actual Targets FFY 2012 Arizona met the targets. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Provide professional development activities around quality assessment practices 2) Increase the percentage of PEAs that collect and report timely preschool Sub-Activities (Objectives or Action Steps) a) Provide professional development “How to Improve the Quality of your Ongoing Progress Monitoring Data” within areas of need as identified through the Early Childhood Quality Improvement Practices (ECQUIP) process and upon request of districts a) Crosscheck child count data with PEA preschool assessment data Timeline Complete Activity completed from 7/1/12 to 6/30/13. Projected 7/1/10– 6/30/13 ADE/ECSE Director and Specialist 7/1/10– 6/30/13 ADE/ECSE Director and Specialist Fifteen “How to Improve the Quality of your Ongoing Progress Monitoring Data” trainings were held throughout Arizona and were attended by 319 educators. Activity completed from 7/1/12 to 6/30/13. Child count data were crosschecked with PEA Resources (Planned) Part B Arizona Annual Performance Report for FFY 2012 68 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 assessment data b) Notify PEAs if preschool assessment data are not submitted on time 3) Establish an Early Childhood Data Collaborative (ECDC) a) Obtain stakeholder input at quarterly meetings during transition to Teaching Strategies GOLD b) Use stakeholder input to guide development of Statelevel policies and procedures related to implementation of Teaching Strategies GOLD 4) Develop and disseminate publications statewide to serve as ongoing guidance and communication a) Develop Arizona’s Early Childhood Assessment System manual (guidance document) and disseminate statewide b) Develop monthly bulletins to alert PEAs about updates and new guidance for preschool assessment instrument preschool assessment data. The number of PEAs not reporting on time decreased from 34 in FFY 2010 to 11 in FFY 2011 and 5 in FFY2012. Activity completed from 7/1/12 to 6/30/13. 7/1/10– 6/30/13 Five PEAs received notification about submitting data within timelines. Activity completed from 7/1/12 to 6/30/13. ADE/ECSE Director and Specialist 7/1/11– 6/30/13 ADE/ECSE Four meetings were held with ECDC members representing PEAs and Head Start programs. Activity completed from 7/1/12 to 6/30/13. EC Data Collaborati ve 7/1/11– 6/30/13 ADE/ECSE EC Data Collaborati ve Input from ECDC members created guidance in the use of GOLD. This guidance was communicated to PEAs through the Arizona’s Early Childhood Assessment System manual and a monthly electronic publication, GOLD Nuggets. Both documents were disseminated via email and Web site posting. Activity completed from 7/1/12 to 6/30/13. 7/1/11– 6/30/13 ADE/ECSE The Arizona’s Early Childhood Assessment System was updated, posted on the ADE Web site, and disseminated to PEAs via email. Activity completed from 7/1/12 to 6/30/13. 7/1/11– 6/30/13 ADE/ECSE GOLD Nuggets are posted to the Website Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable Part B Arizona Annual Performance Report for FFY 2012 69 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 70 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: FAPE in the LRE Indicator 8: Parent Involvement Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities. (20 U.S.C. 1416 (a)(3)(A)) Measurement Percent = [(# of respondent parents who report schools facilitated parent involvement as a means of improving services and results for children with disabilities) divided by the (total # of respondent parents of children with disabilities)] times 100. Target Data for FFY 2012 FFY Measurable and Rigorous Target 2012 65% Actual Target Data for FFY 2012 Number of respondent parents who report schools facilitated parent involvement as a means of improving services and results for children with disabilities Total number of respondent parents of children with disabilities 2,938 5,341 Actual Target Data for FFY 2012 55% 2,938  5,341 = 0.55  100 = 55% Arizona did not meet the target. Data Data Source The data are taken from the Arizona Parent Involvement Survey. Arizona uses a 25-question parent survey developed by the National Center for Special Education Accountability Monitoring (NCSEAM). The survey is the same survey as the one that has been used for past years and has not been revised. Data Description Part B Arizona Annual Performance Report for FFY 2012 71 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 The Arizona Parent Involvement Survey uses a Web-based data collection system to collect confidential demographic information and parental responses to the 25-question NCSEAM rating scale. A paper version of the survey is available in English and Spanish and in a large font in both languages. Parents complete the demographic data and 25 survey items. The data from the surveys are analyzed using WINSTEPS statistical software. Following NCSEAM guidelines, a threshold score of 600 has been established for a positive response to the item “The school explains what options parents have if they disagree with a decision of the school.” The instrument measure implies that agreement with this threshold item indicates high likelihood of agreement with items located “under” it on the scale. A score of 600 is required for any parent’s survey response to be considered positive. Sampling Procedures Each school year a new cohort of PEAs is selected to administer the survey. The cohort is composed of PEAs: a) in the assigned year of the ESS monitoring cycle, or b) with a total student population of 50,000 or greater. Every parent who has a child with an individualized education program (IEP) within these PEAs has an opportunity to complete the survey using either the Web-based data collection system or a paper response that is mailed in. Thus, within the cohort, a census of parents has the opportunity to complete the survey. The ADE/ESS ensures all newly opened PEAs (typically, charter schools) are included in a cohort and administer the parent survey during that cohort year. The use of these procedures allows the State to meet the requirement to report on each PEA at least once during the SPP cycle. The response data are representative of Arizona’s demographics by race/ethnicity, as shown in Table 8.1, and by child age group, as shown in Table 8.2. (Both tables are shown below.) The data set includes charter schools, unified districts, union high school districts, and elementary districts. In addition, urban and rural schools that are distributed throughout the geographical regions of the State are represented. Valid and Reliable Data Arizona ensures that the data are valid and reliable by offering extensive ongoing technical assistance to PEAs. Initial survey instructions detail the steps that PEAs must follow to distribute survey instructions and confidential user codes/passwords to all parents who have a child with a disability. PEAs are given surplus user codes/passwords to have ready for transfer students. PEAs also receive guidance on how to maximize their parental response and involvement rates as demonstrated in the improvement activities. Table 8.1 Comparison of Parent Responses by Race / Ethnicity to State Special Education Population Percentage of Special Education Population (Child Count) Race/Ethnicity of Child of Parent Respondent Number of Responses Percentage of Responses Number of Special Education Population (Child Count) Hispanic/Latino of Any Race 1,748 32.72% 53,411 41.64% American Indian or Alaska Native 692 12.95% 8,248 6.43% Asian 54 1.01% 1,821 1.42% Part B Arizona Annual Performance Report for FFY 2012 72 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Black or AfricanAmerican 304 5.69% 8,073 6.29% Native Hawaiian or Other Pacific Islander 45 0.84% 252 0.20% White 2,070 38.75% 53,021 41.33% Two or More Races 342 6.4% 3,455 2.69% Total 5,341 128,281 Note: 86 respondents did not indicate the race/ethnicity of their child. Table 8.1 shows that the response rate by race/ethnicity is in alignment with the race/ethnicity of children in special education in Arizona for Asian, Black or African-American, Native Hawaiian or Other Pacific Islander, and White racial/ethnic populations. The response rate for American Indian or Alaska Native parents is significantly higher than the State special education population data for that race/ethnicity group. The response rate for American Indian or Alaska Native parents (12.95%) compared to State data (6.43%) can be attributed to the number of reservation-area PEAs in this year’s cohort and the outreach efforts by the Parent Involvement Survey coordinator. The response rate for Hispanic parents (32.72%) is lower than the State special education population data (41.64%) for that race/ethnicity group. It is possible that the responses in the multi-racial category (which were self-reported as to race/ethnicity) and the responses that did not report ethnicity (which combined would account for 8.01% of the responses) may have been reported differently when other data-collection methods were used. Some of these variances in race/ethnicity responses may be affecting the percentage of Hispanic/Latino participation. It should be noted that the percentage of respondents who selected the two or more races is significantly higher than the State race/ethnicity statistics for that group. Table 8.2 Comparison of Parent Responses by Child Age Group to State Special Education Population Percentage of Special Education Population (Child Count) Child Age Group Number of Responses Percentage of Responses Number of Special Education Population (Child Count) Ages 3–5 698 13.07% 15,386 11.99% Ages 6–13 3,059 57.26% 4,382 57.98% Ages 14–22 1,356 25.38% 38,513 30.02% Total 5,341 128,281 Note: 228 respondents did not indicate the age of their child. Part B Arizona Annual Performance Report for FFY 2012 73 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Table 8.2 shows the response rate is in alignment with the age group statistics for parents of children ages 3–5 and 6–13. The response rate is slightly lower than the age group statistics for parents of children aged 14–22. The significant gain achieved in FFY 2010 remains constant with modest growth in both FFY 2011 and FFY 2012. Explanation of Slippage That Occurred for FFY 2012 Arizona exceeded annual Indicator 8 targets from FFY 2005 through FFY 2011. In FFY 2012 there was a decrease from 60.4% (FFY 2011) to 55% (FFY 2012) in the percentage of parents reporting that schools facilitated parent involvement as a means of improving services. The current measurement of parent involvement is below the FFY 2012 65% target. This slippage may be attributed to the framework of PEAs participating in the FFY 2012 survey which was a different cohort compared to the group that completed the survey in FFY 2011. Because of the cohort variance, the drop is not indicative of a decline in the quality of statewide parent/school partnerships that impact student outcomes. Only two PEAs with a student population greater than 50,000 were participants in both survey years. One district maintained a 51% measurement; however, the second district’s measurement rose significantly from 63% to 68%. Comparing the FFY 2011 and FFY 2012 target measurements of parental involvement to determine improvement, or lack thereof, could lead to a faulty supposition when combined results of the only PEAs surveyed in both years shows progress. Although there was slippage, the Parent Involvement Survey coordinator was in regular contact with participating PEAs. The coordinator provided monthly response rate updates and tips aligned with principles of effective parent involvement reflected in the NCSEAM survey questions: when these tips are put into practice by PEAs, a higher measurement of satisfactory parental involvement results. The ADE/ESS Parent Information Network (PIN) specialists and Raising Special Kids (Arizona’s parent training and information center) family resource specialists received survey training and advice on how to encourage schools and families to participate in the survey process and promote family engagement. Special education resources and toll-free assistance telephone lines to families and schools were marketed throughout the State. Consultation with school staff, parent group leaders, and ADE program specialists will guide development of strategies to enhance parent involvement in FFY 2013. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Evaluate PEA’s feedback of the parent involvement survey process as a means of improving distribution to families and use of results to enhance parent involvement Sub-Activities (Objectives or Action Steps) a) Develop and administer a survey to PEAs that conducted the parent involvement survey during the 2005–2011 SPP Timeline Completed Activity completed from 7/1/11 to 6/30/12. Survey methodologies were researched and questions were drafted. The survey was administered during the first half of FFY 2012 to solicit special education directors’ feedback about the survey Projected 7/1/10– 6/30/13 Resources (Planned) Arizona Find Coordinator ADE/ESS PIN Specialists ADE/ESS Program Specialists Part B Arizona Annual Performance Report for FFY 2012 74 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Examine PEA survey results to improve the parent involvement survey process and to advise PEAs on strategies for using the parent survey results to improve family involvement 2) Increase opportunities for PEAs and parents to gain knowledge about the parent involvement survey and related family involvement projects a) Plan and develop a new ESS parent involvement survey Web site combining current links with access to researchbased family involvement literature and effective projects process and parent involvement. Analysis and action plans were completed during the second half of FFY 2012. 7/1/11– 6/30/13 Arizona Find Coordinator ADE/ESS PIN Specialists ADE/ESS Program Specialists Activity completed from 7/1/11 to 6/30/13. 7/1/10– 6/30/13 Research-based documents were collected and effective projects were identified. Discussions with other ADE units that have parent involvement components were initiated; shared knowledge of applicable family engagement strategies will be incorporated in the Web site. Arizona Find Coordinator ADE/ESS PIN Specialists Parent involvement resources were posted on the ADE Parent Web page and the ESS, Parent Information Network and Promising Practices Web sites. b) Test, revise, and launch the new ESS parent involvement survey Web site Activity completed from 7/1/12 to 6/30/13. ADE Web site analytics and unit protocols were used to guide the choice and placement of parent involvement resources. 7/1/11– 6/30/13 Arizona Find Coordinator ADE/ESS PIN Specialists Part B Arizona Annual Performance Report for FFY 2012 75 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 76 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Disproportionality Indicator 9: Racial / Ethnic Disproportionality Percent of districts with disproportionate representation of racial and ethnic groups in special education and related services that is the result of inappropriate identification. (20 U.S.C. 1416 (a)(3)(C)) Measurement Percent = [(# of districts with disproportionate representation of racial and ethnic groups in special education and related services that is the result of inappropriate identification) divided by the (# of districts in the State)] times 100. Data Data Source The ADE/ESS collected the data from the PEAs through the October 1, 2012, child count report. The data are the same as the data collected and reported on Table 1, Report of Children with Disabilities Receiving Special Education under Part B of the Individuals with Disabilities Education Act, As Amended, for all children with disabilities aged 6–21 served under IDEA. Note that the source of this data is from FFY 2012. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because it collected, maintained, and reported the October 1, 2012, child count data using internal edit checks. In addition, the State requires the PEAs to assure data accuracy and reliability by submitting a signed verification letter. Definition of Disproportionate Representation Disproportionate Representation Over representation Weighted Risk Ratio Minimum n Size Target Racial/Ethnic Group Minimum n Size Racial / Ethnic Groups in Special Education and Related Services ≥ 3.00 30 30 Methodology The data were analyzed using Statistical Analysis Software (SAS) to produce a weighted risk ratio (WRR) that identified all racial/ethnic groups for all PEAs in the State. Data for over representation were examined. PEAs with a cell size of 30 or more students in the target racial/ethnic group and in the other racial/ethnic groups and that met the weighted risk ratio criteria for over representation were flagged for a Part B Arizona Annual Performance Report for FFY 2012 77 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 review of policies, procedures, and practices by the State. PEAs with a lower cell size in the target groups were not flagged because false positives were identified as a function of the small number rather than as a result of noncompliant policies, procedures, and practices. Arizona included the total number of PEAs in the State (602) in the denominator. Of the 602 PEAs, 13 were eliminated from the analyses because a weighted risk ratio could not be calculated for any racial/ethnic group. Arizona’s Procedures to Determine if Disproportionate Representation Is the Result of Inappropriate Identification Arizona ensures that PEAs’ policies, procedures, and practices are reviewed as required by 34 CFR §§ 300.173, 300.600(d)(3), and 300.602(a). The data are analyzed annually and PEAs may be flagged each year for over representation, according to the State’s definition. When a PEA is flagged, then the policies, procedures, and practices of the PEA are reviewed annually to determine if the disproportionate representation is the result of inappropriate identification. Arizona’s Review of PEAs’ Policies and Procedures On an annual basis, Arizona requires all PEAs to have special education policies and procedures in compliance with the requirements of 34 CFR § 300.111, § 300.201, and § 300.301 through § 300.311 prior to having Part B-IDEA Basic Entitlement Grant funds approved by the ADE/ESS. Each year, if the PEA makes any changes to the policies and procedures, the PEA must resubmit them to the State for review and acceptance. Each year, if the PEA does not make any changes to the policies and procedures, the PEA must submit a Statement of Assurance that says: “The PEA has not altered or modified the policies and procedures implementing the State and Federal requirements for services to children with disabilities previously submitted to and accepted by the Arizona Department of Education, Exceptional Student Services. If the PEA proposes to alter or modify the policies and procedures previously submitted to the Exceptional Student Services, the PEA must resubmit the policies and procedures to the Exceptional Student Services for review and acceptance.” In addition, the PEAs that are flagged for disproportionate representation must submit their policies and procedures related to child find, evaluation, and eligibility to an ADE/ESS specialist for review. Arizona’s Review of PEAs’ Practices On an annual basis, Arizona calculates the WRR for each PEA and uses the data as a trigger to flag PEAs with disproportionate representation. If a PEA is flagged, then an investigation of the practices is required to determine whether the disproportionate representation is a result of inappropriate identification. Review of practices when a PEA is flagged for over representation the first year:  The ESS specialist reviews current monitoring data, if applicable.  The PEA conducts a self assessment of the agency’s child find, evaluation, and eligibility practices to determine whether the disproportionate representation is a result of inappropriate identification. The self assessment consists of a series of questions requiring narrative responses and a review of student files using the State’s monitoring forms. The ADE/ESS specialists conduct on-site visits and/or desk audits during the self assessments to validate the decisions made by the PEAs during the file reviews.  Upon completion of the self assessments, the PEAs have the option to begin immediately revising their policies, procedures, and practices related to child find, evaluation, and eligibility and to correct any noncompliance. No more than 60 days after completion of the self assessment, the ESS specialists then interview the special education administrators and review Part B Arizona Annual Performance Report for FFY 2012 78 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 student files via on-site visits and/or desk audits to verify correction of instances of any noncompliance, including child specific, and to ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. Review of practices when a PEA is flagged for over representation for two or more consecutive years:  If the PEA did not have disproportionate representation as a result of inappropriate identification the first year, then the ESS program specialist: o Reviews current monitoring data, if applicable, and; o Validates the prior year’s self assessment by reviewing a sample of student files.  If the PEA had disproportionate representation as a result of inappropriate identification the first year, then the PEA is required to: o Review current monitoring data, if applicable; o Review the prior year’s self assessment and describe the issues identified; o Describe the steps taken to resolve those issues; o Describe any current concerns regarding possible inappropriate identification; o Describe the resources and technical assistance used to help address the issues related to disproportionate representation within the agency; and o Review individual student files using the State’s monitoring forms:  The ADE/ESS specialists conduct on-site visits and/or desk audits during the file reviews to validate the decisions made by the PEAs.  The ESS specialists verify correction of instances of any noncompliance, including child specific, through on-site visits and/or desk audits.  The ESS specialists ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. When Arizona makes findings of noncompliance as a result of the review of policies, practices and procedures, the PEA has one year from the date of written notification from the State to correct the noncompliance. Target Data for FFY 2012 FFY Measurable and Rigorous Target FFY 2012 0% Actual Target Data for FFY 2012 0% Arizona met the target. PEAs with Disproportionate Representation of Racial and Ethnic Groups That Was the Result of Inappropriate Identification Year Total Number of PEAs* Number of PEAs with Disproportionate Number of PEAs with Disproportionate Representation of Racial and Ethnic Groups That Percent of PEAs Part B Arizona Annual Performance Report for FFY 2012 79 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Representation FFY 2012 (2012– 2013) 602 Was the Result of Inappropriate Identification 0 0 0.00% *Arizona included the total number of PEAs in the denominator. PEAs with Disproportionate Representation of Racial and Ethnic Groups Arizona identified zero PEAs with disproportionate representation of racial and ethnic groups in special education and related services (0.0%). PEAs with Disproportionate Representation of Racial and Ethnic Groups That Was the Result of Inappropriate Identification Arizona identified zero PEAs with disproportionate representation of racial and ethnic groups in special education and related services that was the result of inappropriate identification (0.0%). Table 9.1 PEAs with Over Representation by Racial / Ethnic Group Indicator 9 – Over Representation # of PEAs flagged for over representation # of PEAs found to have disproportionate representation (over representation) as a result of inappropriate identification Hispanic/Latino of Any Race American Indian or Alaska Native Asian Black or African American Native Hawaiian or Other Pacific Islander White Two or More Races As shown in Table 9.1, no PEAs were flagged for over representation. Arizona made no findings of noncompliance as a result of the review of child find, evaluation, and eligibility policies, procedures, and practices. Part B Arizona Annual Performance Report for FFY 2012 80 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Provide support for PEAs that are flagged as at risk for disproportionate representation with a WRR ≤ 0.40 for under representation and ≥ 2.5 for over representation Sub-Activities (Objectives or Action Steps) a) Analyze data on an annual basis to flag PEAs that are at risk for disproportionate representation b) On an annual basis, notify PEAs that are flagged as at risk for disproportionate representation c) On an annual basis, provide assessment tools and resources to PEAs that are flagged as at risk to conduct a root cause analysis 2) Provide support for PEAs that are flagged for disproportionate representation with a WRR ≤ 0.30 for under representation and a WRR ≥ 3.0 for over representation a) On an annual basis, notify PEAs that are flagged for disproportionate representation b) Provide technical assistance to PEA staff during their review of policies, procedures, and practices 3) Investigate strategies to assist PEAs that are flagged with disproportionate representation a) Investigate resources from the regional Equity Center, NCCRESt, and ADE/OELAS (Office of English Language Acquisition Timeline Complete Activity completed 2/15/13. Projected 7/1/10– 6/30/13 Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk WRRs for disproportionality. Resources (Planned) ADE/ESS Directors and Program Specialists ADE Research and Evaluation ADE/ESS Directors and Program Specialists Activity completed from 3/1/13 to 6/30/13. 7/1/10– 6/30/13 ESS specialists reviewed disproportionality data during on-site meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 3/1/13 to 6/30/13. 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists ESS specialists gave self assessment tools and resources to PEAs that were identified as at risk. Activity completed 4/1/13. ESS notified PEAs that were flagged for disproportionality. Activity completed from 4/1/13 to 6/30/13. 7/1/10– 6/30/13 ESS specialists made onsite visits to work with PEA staff during the review of policies, procedures, and practices related to child find, eligibility, and evaluation. Activity completed 7/1/11 to 9/1/11. 7/1/11– 12/31/11 ADE/ESS Directors ESS directors researched resources offered by Equity Centers and ADE/OELAS. Part B Arizona Annual Performance Report for FFY 2012 81 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Services) b) Obtain input from stakeholders via regional groups and Special Education Advisory Panel c) Develop new strategies to assist PEAs that are flagged with disproportionate representation d) Implement new strategies to assist PEAs that are flagged with disproportionate representation Activity completed 9/30/11. 7/1/11– 12/31/11 Presented resources to Special Education Advisory Panel and received comments from panel members. Activity completed 1/1/12 to 4/1/12. ADE/ESS Deputy Associate Superinten dent and Directors 1/1/12– 6/30/12 ESS reviewed available resources for PEAs and added new strategies. Activity completed 4/1/13 to 6/30/13 ADE/ESS Directors and Program Specialists 7/1/12– 6/30/13 ADE/ESS Directors and Program Specialists Correction of FFY 2011 Findings of Noncompliance (if State did not report 0%) Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 0% Arizona made no findings of noncompliance in FFY 2011. Actions Taken if Noncompliance Not Corrected Not applicable. Arizona made no findings of noncompliance in FFY 2011. Verification of Correction (either timely or subsequent) Not applicable. Arizona made no findings of noncompliance in FFY 2011. Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2011 Not applicable. Arizona made no findings of noncompliance in FFY 2011. Correction of Remaining FFY 2010 Findings of Noncompliance Not applicable. Verification of Correction of Remaining FFY 2010 Findings Not applicable. Part B Arizona Annual Performance Report for FFY 2012 82 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2009 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 83 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Disproportionality Indicator 10: Racial / Ethnic Disproportionality by Disability Percent of districts with disproportionate representation of racial and ethnic groups in specific disability categories that is the result of inappropriate identification. (20 U.S.C. 1416 (a)(3)(C)) Measurement Percent = [(# of districts with disproportionate representation of racial and ethnic groups in specific disability categories that is the result of inappropriate identification) divided by the (# of districts in the State)] times 100. Data Data Source The ADE/ESS collected the data from the PEAs through the October 1, 2012, child count report. The data are the same as the data collected and reported on Table 1, Report of Children with Disabilities Receiving Special Education under Part B of the Individuals with Disabilities Education Act, As Amended, for all children with disabilities aged 6–21 served under IDEA. Note that the source of this data is from FFY 2012. The total number of PEAs in Arizona varies from year to year because of the number of charter schools that may open and close from year to year. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data because it collected, maintained, and reported the October 1, 2012, child count data using internal edit checks. In addition, the State requires the PEAs to assure data accuracy and reliability by submitting a signed verification letter. Definition of Disproportionate Representation Disproportionate Representation Over representation Weighted Risk Ratio Minimum n Size Target Racial / Ethnic Group Minimum n Size Racial / Ethnic Groups in Special Education and Related Services ≥ 3.00 30 30 Methodology The data were analyzed using Statistical Analysis Software (SAS) to produce a weighted risk ratio (WRR) that identified all racial/ethnic groups and six disability categories for all PEAs in the State. Data for over representation were examined. PEAs with a cell size of 30 or more students in the target racial/ethnic Part B Arizona Annual Performance Report for FFY 2012 84 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 group and in the other racial/ethnic groups and meeting the weighted risk ratio criteria for over representation were flagged for a review of policies, procedures, and practices by the State. PEAs with a lower cell size in the target groups were not flagged because false positives were identified as a function of the small number rather than as a result of noncompliant policies, procedures, and practices. Arizona included the total number of PEAs in the State (602) in the denominator. Of the 602 PEAs, 13 were eliminated from the analyses because a weighted risk ratio could not be calculated for any racial/ethnic group. Arizona’s Procedures to Determine if Disproportionate Representation Is the Result of Inappropriate Identification Arizona ensures that PEAs’ policies, procedures, and practices are reviewed, as required by 34 CFR §§ 300.173, 300.600(d)(3), and 300.602(a). The data are analyzed annually and PEAs may be flagged each year for over representation, according to the State’s definition. When a PEA is flagged, then the policies, procedures, and practices of the PEA are reviewed annually to determine if the disproportionate representation is the result of inappropriate identification. Arizona’s Review of PEAs’ Policies and Procedures On an annual basis, Arizona requires all PEAs to have special education policies and procedures in compliance with the requirements of 34 CFR § 300.111, § 300.201, and § 300.301 through § 300.311 prior to having Part B-IDEA Basic Entitlement Grant funds approved by the ADE/ESS. Each year, if the PEA makes any changes to the policies and procedures, the PEA must resubmit them to the State for review and acceptance. Each year, if the PEA does not make any changes to the policies and procedures, the PEA must submit a Statement of Assurance that says: “The PEA has not altered or modified the policies and procedures implementing the State and Federal requirements for services to children with disabilities previously submitted to and accepted by the Arizona Department of Education, Exceptional Student Services. If the PEA proposes to alter or modify the policies and procedures previously submitted to the Exceptional Student Services, the PEA must resubmit the policies and procedures to the Exceptional Student Services for review and acceptance.” In addition, the PEAs that are flagged for disproportionate representation must submit their policies and procedures related to child find, evaluation, and eligibility to an ADE/ESS specialist for review. Arizona’s Review of PEAs’ Practices On an annual basis, Arizona calculates the WRR for each PEA and uses the data as a trigger to flag PEAs with disproportionate representation. If a PEA is flagged, then an investigation of the practices is required to determine whether the disproportionate representation is a result of inappropriate identification. Review of practices when a PEA is flagged for over representation the first year:  The ESS specialist reviews current monitoring data, if applicable.  The PEA conducts a self assessment of the agency’s child find, evaluation, and eligibility practices to determine whether the disproportionate representation is a result of inappropriate identification. The self assessment consists of a series of questions requiring narrative responses and a review of student files using the State’s monitoring forms. The ADE/ESS specialists conduct on-site visits and/or desk audits during the self assessments to validate the decisions made by the PEAs during the file reviews.  Upon completion of the self assessments, the PEAs have the option to begin immediately revising their policies, procedures, and practices related to child find, evaluation, and eligibility Part B Arizona Annual Performance Report for FFY 2012 85 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 and to correct any noncompliance. No more than 60 days after completion of the self assessment, the ESS specialists then interview the special education administrators and review student files via on-site visits and/or desk audits to verify correction of instances of any noncompliance, including child specific, and to ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. Review of practices when a PEA is flagged for over representation for two or more consecutive years:  If the PEA did not have disproportionate representation as a result of inappropriate identification the first year, then the ESS program specialist: o Reviews current monitoring data, if applicable, and; o Validates the prior year’s self assessment by reviewing a sample of student files.  If the PEA had disproportionate representation as a result of inappropriate identification the first year, then the PEA is required to: o Review current monitoring data, if applicable; o Review the prior year’s self assessment and describe the issues identified; o Describe the steps taken to resolve those issues; o Describe any current concerns regarding possible inappropriate identification; o Describe the resources and technical assistance used to help address the issues related to disproportionate representation within the agency; and o Review individual student files using the State’s monitoring forms:  The ADE/ESS specialists conduct on-site visits and/or desk audits during the file reviews to validate the decisions made by the PEAs.  The ESS specialists verify correction of instances of any noncompliance, including child specific, through on-site visits and/or desk audits.  The ESS specialists ensure that regulatory requirements are being implemented based on subsequent file reviews of updated data. When Arizona makes findings of noncompliance as a result of the review of policies, practices and procedures, the PEA has one year from the date of written notification from the State to correct the noncompliance. Target Data for FFY 2012 FFY Measurable and Rigorous Target FFY 2012 0% Actual Target Data for FFY 2012 0% Arizona met the target. PEAs with Disproportionate Representation of Racial and Ethnic Groups in Specific Disability Categories That Was the Result of Inappropriate Identification Part B Arizona Annual Performance Report for FFY 2012 86 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Year Total Number of PEAs* FFY 2012 (2012– 2013) Number of PEAs with Disproportionate Representation of Racial and Ethnic Groups in Specific Disability Categories That Was the Result of Inappropriate Identification Number of PEAs with Disproportionate Representation 602 2 0 Percent of PEAs 0.00% *Arizona included the total number of PEAs in the denominator. PEAs with Disproportionate Representation of Racial and Ethnic Groups Arizona identified two PEAs with disproportionate representation of racial and ethnic groups in specific disability categories. PEAs with Disproportionate Representation of Racial and Ethnic Groups That Was the Result of Inappropriate Identification Arizona identified zero PEAs with disproportionate representation of racial and ethnic groups in specific disability categories that was the result of inappropriate identification (0.0%). Table 10.1 PEAs, and Cases, with Over Representation by Racial / Ethnic Group and Disability Note: The cases give a duplicated count of PEAs. Cases of Over Representation Autism Emotional Disturbance 1 1 Intellectual Disabilities Other Health Impairments Specific Learning Disability Speech or Language Impairment Hispanic/Latino of Any Race American Indian or Alaska Native Asian Black or African American Native Hawaiian or Other Pacific Islander White 1 Two or More Races Part B Arizona Annual Performance Report for FFY 2012 87 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 # of PEAs flagged for over representation 2 # of PEAs found to have disproportionate representation (over representation) as a result of inappropriate identification 0 The following is a breakdown of Table 10.1:    One PEA was flagged for over representation due to a WRR of 3.0 or above for a total of two cases, one for autism and one for emotional disturbance. One PEA was flagged for over representation due to a WRR of 3.0 or above for one case for Other Health Impaired. No PEAs were found to have disproportionate representation as a result of inappropriate identification. The following describes the investigation of the policies, procedures, and practices of the two PEAs:  The two PEAs flagged for over representation submitted special education policies and procedures that were in compliance with the requirements of 34 CFR § 300.111, § 300.201, and § 300.301 through § 300.311 prior to having Part B-IDEA Basic Entitlement Grant funds approved. The ADE/ESS specialists reviewed the child find, evaluation, and eligibility policies and procedures during the PEA’s self assessment and found them to be in compliance.  The two PEAs were flagged for more than one consecutive year, and it was determined that the agencies did not have disproportionate representation as a result of inappropriate identification the prior year. In order to confirm this conclusion, the ADE/ESS specialists reviewed individual student files from FFY 2012 and the first few months of FFY 2013 and current monitoring data, where available. The practices of the PEAs were found to be consistent with 34 CFR § 300.173 and § 300.600(d)(3). It was determined that the PEAs did not have disproportionate representation as a result of inappropriate identification. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Provide support for PEAs that are flagged as at risk for disproportionate representation with a WRR ≤ 0.40 for under representation and ≥ 2.5 for over representation Sub-Activities (Objectives or Action Steps) a) Analyze data on an annual basis to flag PEAs that are at risk for disproportionate representation Timeline Complete Activity completed 2/15/13. Data analyzed by the ADE Research and Evaluation specialist and by ESS directors. ESS flagged PEAs that had at-risk WRRs for disproportionality. Projected 7/1/10– 6/30/13 Resources (Planned) ADE/ESS Directors and Program Specialists ADE Research and Evaluation Part B Arizona Annual Performance Report for FFY 2012 88 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) On an annual basis, notify PEAs that are flagged as at risk for disproportionate representation c) On an annual basis, provide assessment tools and resources to PEAs that are flagged as at risk to conduct a root cause analysis 2) Provide support for PEAs that are flagged for disproportionate representation with a WRR ≤ 0.30 for under representation and a WRR ≥ 3.0 for over representation a) On an annual basis, notify PEAs that are flagged for disproportionate representation b) Provide technical assistance to PEA staff during their review of policies, procedures, and practices 3) Investigate strategies to assist PEAs that are flagged with disproportionate representation a) Investigate resources from the regional Equity Center, NCCRESt, and ADE/OELAS (Office of English Language Acquisition Services) b) Obtain input from stakeholders via regional groups and Special Education Advisory Panel c) Develop new strategies to assist PEAs that are flagged with disproportionate representation Activity completed from 3/1/13 to 6/30/13. 7/1/10– 6/30/13 ESS specialists reviewed disproportionality data during on-site meetings with special education administrators and alerted PEAs to their at-risk status. Activity completed from 3/1/13 to 6/30/13. ADE/ESS Directors and Program Specialists 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists 7/1/10– 6/30/13 ADE/ESS Directors and Program Specialists ADE/ESS Directors and Program Specialists ESS specialists gave self assessment tools and resources to PEAs that were identified as at risk. Activity completed 4/1/13. ESS notified PEAs that were flagged for disproportionality. Activity completed from 4/1/13 to 6/30/13. 7/1/10– 6/30/13 ESS specialists made onsite visits to work with PEA staff during the review of policies, procedures, and practices related to child find, eligibility, and evaluation. Activity completed 7/1/11 to 9/1/11. 7/1/11– 12/31/11 ADE/ESS Directors 7/1/11– 12/31/11 ADE/ESS Deputy Associate Superinten dent and Directors 1/1/12– 6/30/12 ADE/ESS Directors and Program Specialists ESS directors researched resources offered by Equity Centers and ADE/OELAS. Activity completed 9/30/11. Presented resources to Special Education Advisory Panel and received comments from panel members. Activity completed 1/1/12 to 4/1/12. ESS reviewed available resources for PEAs and added new strategies. Part B Arizona Annual Performance Report for FFY 2012 89 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 d) Implement new strategies to assist PEAs that are flagged with disproportionate representation Activity completed 4/1/13 to 6/30/13 7/1/12– 6/30/13 ADE/ESS Directors and Program Specialists Correction of FFY 2011 Findings of Noncompliance (if State reported more than 0% compliance) Level of compliance (actual target data) State reported for FFY 2011 for this Indicator: 0% Arizona made no findings of noncompliance in FFY 2011. Actions Taken if Noncompliance Not Corrected Not applicable. Arizona made no findings of noncompliance in FFY 2011. Verification of Correction (either timely or subsequent) Not applicable. Arizona made no findings of noncompliance in FFY 2011. Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2011 Not applicable. Arizona made no findings of noncompliance in FFY 2011. Correction of Remaining FFY 2010 Findings of Noncompliance Not applicable. Verification of Correction of Remaining FFY 2010 Findings Not applicable. Specific Actions the State Took to Verify the Correction of Findings of Noncompliance Identified in FFY 2010 Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2009 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Part B Arizona Annual Performance Report for FFY 2012 90 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 91 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Effective General Supervision Part B / Child Find Indicator 11: Evaluation Timelines Percent of children who were evaluated within 60 days of receiving parental consent for initial evaluation or, if the State establishes a timeframe within which the evaluation must be conducted, within that timeframe. (20 U.S.C. 1416 (a)(3)(B)) Measurement a. # of children for whom parental consent to evaluate was received. b. # of children whose evaluations were completed within 60 days (or State-established timeline). Account for children included in a but not included in b. Indicate the range of days beyond the timeline when the evaluation was completed and any reasons for the delays. Percent = [(b) divided by (a)] times 100. Target Data for FFY 2012 FFY Measurable and Rigorous Target FFY 2012 100% Actual Target Data for FFY 2012 97% Arizona did not meet the target. Data Data Source The data for Indicator 11 are from the Arizona monitoring system. A public education agency (PEA) is selected for monitoring each fiscal year based on the results of a review of the agency’s data, including data from the SPP/APR, dispute resolution results, audit findings, and annual determinations. While Arizona has maintained a six-year monitoring cycle with assigned activities always occurring in Year 4, a PEA’s monitoring schedule can be adjusted and Year 4 monitoring activities can occur when the data reviews indicate systemic issues. Data Collection Part B Arizona Annual Performance Report for FFY 2012 92 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Data are collected from the PEAs during one of three types of monitorings:  Data Review — PEAs review student files focusing on Indicator 11. The ADE/ESS specialist validates the compliance calls. The student file forms are submitted to ESS for data entry.  Self-Assessment — PEAs review student files and collect data for Indicator 11. The PEAs also focus on identified SPP/APR Indicators in which agency results have not met the State target. The ADE/ESS specialist validates the compliance calls. The student file forms are submitted to ESS for data entry.  On-Site — PEAs and the ADE/ESS team review student files, collect data through surveys and interviews, and collect data for Indicator 11. The ADE/ESS staff input data. The data that Arizona collects and reports for this Indicator include all children whose permissions to evaluate were received during FFY 2012 and for whom initial evaluations including eligibility determinations were completed during either FFY 2012 or FFY 2013. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data as it is collected, maintained, and reported through the State monitoring system. Training is provided to all ESS program specialists who monitor to ensure interrater reliability on compliance calls that are based on regulatory requirements. The ADE/ESS staff conduct trainings for PEA staff who will participate in monitorings. The ESS specialists validate and verify the data through on-site visits or desk audits. Evaluation Timeline Arizona has established a 60-day timeline for initial evaluations. The Arizona Administrative Code (A.A.C.) R7-2-401 (E)(3) states that the initial evaluation shall not exceed 60 calendar days from receipt of informed written consent. However, the 60-day evaluation period may be extended for an additional 30 days if it is in the best interests of the child and the parents and the public education agency agree in writing to do so (A.A.C. R7-2-401 (E)(4)). Definition of Finding for Monitoring for FFY 2012 During FFY 2012, a finding for Indicator 11 was issued when the line item for the evaluation timeline was found to be noncompliant. The finding was a written notification to the PEA by the State that the line item was noncompliant, and the finding included a description of a Federal or State statute or regulation. The source of information on which to base a finding of noncompliance was an individual student file. Children Evaluated Within 60 Days (or State-established timeline) a. Number of children for whom parental consent to evaluate was received b. Number of children whose evaluations were completed within 60 days (or Stateestablished timeline) 936 911 Part B Arizona Annual Performance Report for FFY 2012 93 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Percent of children with parental consent to evaluate, who were evaluated within 60 days (or State-established timeline) (Percent = [(b) divided by (a)] times 100) 97% Children Included in a (above) and Not Included in b (above) FFY 2012 Noncompliance number of findings by incidence of noncompliance number of findings by incidence corrected prior to one-year timeline as of 1/15/14 25 25 Arizona made 25 findings of noncompliance in FFY 2012. Although the PEAs have one year to correct the noncompliance, all 25 findings have been corrected as of January 15, 2014. Range of Days Beyond the Timeline and Reasons for the Delays Table 11.1 Range of Days Beyond Timeline Range of days 1–150 Mean 26 Median 24 Mode 9,10 The 150 days beyond the 60-day timeline occurred at a school district that was unaware that evaluations need to be completed within the timeline regardless of summer break. The evaluation was completed, although it was past the timeline. The ADE/ESS specialist verified the correction of the child specific noncompliance and verified that a new strategy was developed to ensure compliance with the 60-day evaluation timelines. During follow-up visits to ensure sustainability, the ESS specialist verified that new evaluations are being done within the timeline. Table 11.2 Reasons Given for Delays Unavailability of required personnel (parent, general education teacher, etc.) 2 Miscalculation of 60-day timeline 4 Lack of an adequate timeline tracking system 4 Shortage of evaluators 9 Lack of understanding of evaluation process 2 Interruption in school calendar 4 Part B Arizona Annual Performance Report for FFY 2012 94 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Develop and conduct Webinars pertaining to the requirements for compliant evaluations and IEPs Sub-Activities (Objectives or Action Steps) a) Develop Webinar trainings for evaluation and IEP requirements b) Conduct statewide Webinars for evaluation and IEP requirements c) Collect and analyze training feedback from participants 2) Review the Arizona Technical Assistance System (AZTAS) Evaluation and Eligibility document and revise, as necessary d) Collect corrective action close-out (timeline) data for evaluation and IEP monitoring line items a) Review the AZTAS Evaluation and Eligibility document to determine if it is current with statutes and regulations b) Revise the AZTAS Evaluation and Eligibility document, if appropriate c) Disseminate revised AZTAS Evaluation and Eligibility document via ESS Web site and Timeline Complete Activity completed 12/1/10 to 6/30/11. Projected Resources (Planned) 12/1/10– 6/30/11 ADE/ESS Directors and Specialists 7/1/12 – 12/31/12 ADE/ESS Directors and Specialists 1/1/12– 4/30/12 ADE/ESS Directors and Specialists 5/1/13 – 6/30/13 ADE/ESS Directors and Specialists Activity completed 7/1/11 to 6/30/12. 7/1/11– 12/31/11 ESS reviewed AZTAS document for current requirements. Activity completed by 6/30/12. 1/1/12– 6/30/12 ADE/ESS Deputy Associate Superintende nt and Directors ADE/ESS Deputy Associate Superintende nt, Directors, Specialists ADE/ESS Directors and Specialists Monitoring director and monitoring team created Webinars to train PEAs on the evaluation process and required timelines. Webinars were reviewed and revised and presented statewide on th th November 15 , 19 and th December 6 2012. Feedback surveys were emailed to participants directly concluding the webinar, via Go-To Meeting. Analyzing the outcomes was completed December 2012. Activity completed 6/30/13 ESS made revisions to the AZTAS document. Activity Completed by 6/30/13 7/1/12– 6/30/13 ESS disseminated the Part B Arizona Annual Performance Report for FFY 2012 95 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 ESS specialists revised AZTAS document by posting on the ADE Web site and distributing it via ESS specialists. Correction of FFY 2011 Findings of Noncompliance (if State reported less than 100% compliance) Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 97% 1. Number of findings of noncompliance the State made during FFY 2011 (the period from July 1, 2011, through June 30, 2012) 32 2. Number of FFY 2011 findings the State verified as timely corrected (corrected within one year from the date of notification to the LEA of the finding) 32 3. Number of FFY 2011 findings not verified as corrected within one year [(1) minus (2)] 0 Correction of FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 0 5. Number of FFY 2011 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 0 6. Number of FFY 2011 findings not verified as corrected [(4) minus (5)] 0 The ADE/ESS specialists reviewed the child specific files from the monitorings to determine that the PEAs completed the evaluation for any child whose initial evaluation was not timely, unless the child was no longer within the PEA. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine that the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to the evaluation process in conformity with 34 CFR § 300.301 (c) (1). Actions Taken if Noncompliance Not Corrected All FFY 2011 noncompliance has been corrected, and Arizona has verified correction for all FFY 2011 findings. When findings of noncompliance are made, Arizona works collaboratively with the PEAs as the agencies identify the root causes of their continuing noncompliance through drill downs in the specific focus area that affects this Indicator. However, when noncompliance is not corrected within one year, the ADE/ESS uses a variety of methods to ensure that all public agencies meet the requirements of State and Federal Part B Arizona Annual Performance Report for FFY 2012 96 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 statutes related to special education. The enforcement actions taken by ESS for the PEAs that are unable to demonstrate compliance within one year from the date of written notification are as follows:  Enforcement of corrective action plan (CAP) activities as outlined in the current agency CAP, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Review and revision of the current CAP to develop targeted activities that address the continued noncompliance, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Assignment of a special monitor.  Interruption of IDEA payments until adequate compliance is achieved. For charter schools not receiving IDEA funds, a request to begin withholding 10% of State funds.  For charter schools, a request to the appropriate board for a notice of intent to revoke the charter.  With Arizona State Board of Education approval, interruption of Group B weighted State aid or redirection of funds pursuant to 34 CFR § 300.227 (a).  Referral to the Arizona Attorney General for legal action. Verification of Correction of FFY 2011 Noncompliance (either timely or subsequent) FFY 2011 Verification of Correction from Monitoring As specified in OSEP’s July 2013 Arizona Part B FFY 2011 SPP/APR Response Table, Arizona verified that each PEA with noncompliance reflected in the data: 1) is correctly implementing 34 CFR § 300.301 (c) (1) (i.e., achieved 100% compliance) based on a review of updated data such as data subsequently collected through on-site monitoring; and 2) has completed the evaluation, although late, for any child whose initial evaluation was not timely, unless the child is no longer within the jurisdiction of the LEA, consistent with OSEP Memorandum 09-02, dated October 17, 2008. Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2011 The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data:  ADE/ESS specialists conducted follow-up on-site visits and/or desk audits after the monitoring to verify correction of all instances of noncompliance, including those that were child specific. The specialists reviewed the child specific files to determine that the evaluation was completed within 60 calendar days from the date of written notification of noncompliance.  ADE/ESS specialists reviewed updated data from subsequent files and/or conducted interviews with the special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including those that were child specific, were corrected and to ensure ongoing sustainability of the implementation of the regulatory requirements regarding initial evaluations. Part B Arizona Annual Performance Report for FFY 2012 97 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Correction of Remaining FFY 2010 Findings of Noncompliance Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2009 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response Because the State reported less than 100% compliance for FFY 2011, the State must report on the status of correction of noncompliance reflected in the data the State reported for this Indicator. In the FFY 2012 APR for Indicator 11 and Indicator 15, Arizona reported on the status of the correction of noncompliance (FFY 2011 data). In reporting on correction of findings of noncompliance in the FFY 2012 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2011 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2012 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2012 APR for Indicator 11 and Indicator 15, Arizona reported that it verified each PEA with noncompliance (FFY 2011 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 98 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator 12: Early Childhood Transition Percent of children referred by Part C prior to age 3, who are found eligible for Part B, and who have an IEP developed and implemented by their third birthdays. (20 U.S.C. 1416(a)(3)(B)) Measurement a. # of children who have been served in Part C and referred to Part for Part B eligibility determination. b. # of those referred determined to be NOT eligible and whose eligibilities were determined prior to their third birthdays. c. # of those found eligible who have an IEP developed and implemented by their third birthdays. d. # of children for whom parent refusal to provide consent caused delays in evaluation or initial services or to whom exceptions under 34 CFR § 300.301(d) applied. e. # of children who were referred to Part C less than 90 days before their third birthdays. Account for children included in a but not included in b, c, d, or e. Indicate the range of days beyond the third birthday when eligibility was determined and the IEP developed and the reasons for the delays. Percent = [(c) divided by (a – b – d – e)] times 100. Target Data for FFY 2012 FFY Measurable and Rigorous Target FFY 2012 100% Actual Target Data for FFY 2012 99% Arizona did not meet the target. Data Data Source The data for Indicator 12 are reported annually by all public education agencies (PEAs) in Arizona that have children who transition from Part C to Part B. Data are included for the entire reporting year, from July 1, 2012, through June 30, 2013. Data Collection The data are collected through the Annual Special Education Data Collection, an Arizona Department of Education (ADE) Web-based data collection system. Part B Arizona Annual Performance Report for FFY 2012 99 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Valid and Reliable Data The Arizona Department of Education (ADE)/Early Childhood Special Education (ECSE) unit assures the validity and reliability of the data as it is collected, maintained, and reported through internal edit checks. Training is provided to school personnel by the ESS Data Management Unit regarding the operation of the data system and interpretation of the questions that are components of the measurement. The State requires an assurance from the PEAs through the submission of a signed form attesting to the validity of the data. Random verification checks require that a selected district submit a copy of the front page of the IEP that shows the date of the IEP and the child’s birthday for children that transitioned from early intervention service or a Prior Written Notice (PWN) of children found ineligible by the child’s third birthday. Definition of Finding A finding of noncompliance for Indicator 12 is defined as the number of PEAs with noncompliance. The finding of noncompliance is a written notification to the PEA by the State that the PEA is noncompliant. Actual State Data (Numbers) for FFY 2012 a. # of children who have been served in Part C and referred to Part B for Part B eligibility determination 2,512 b. # of those referred determined to be NOT eligible and whose eligibility was determined prior to third birthday 354 c. # of those found eligible who have an IEP developed and implemented by their third birthdays 2,052 d. # for whom parent refusals to provide consent caused delays in evaluation or initial services or to whom exceptions under 34 CFR § 300.301(d) applied 63 e. # of children who were referred to Part C less than 90 days before their third birthdays 31 # in a but not in b, c, d, or e 12 Percent of children referred by Part C prior to age 3 who are found eligible for Part B and who have an IEP developed and implemented by their third birthdays Percent = [(c) / (a-b-d-e)] * 100 99% FFY 2012 Noncompliance # findings of noncompliance # of findings corrected prior to one-year timeline as of 1/15/14 9 9 Part B Arizona Annual Performance Report for FFY 2012 100 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Arizona made nine finding of noncompliance in FFY 2012. Although the PEAs have one year to correct the noncompliance, all nine findings have been corrected as of January 15, 2014. Account for Children Included in a, but not in b, c, d, or e — Reasons for Delays Late referrals from Part C 9 Failed hearing or vision screening 1 Shortage of personnel 1 Interruption of school schedule 1 Total 12 Each year since FFY 2009, the number of children not transitioned on time due to late referrals from Part C has decreased. Currently, in FFY 2012, 9 children were not transitioned on time due to late referrals from Arizona Early Intervention Program (AzEIP) as compared with 12 in FFY 2011, 21 in FFY 2010, and 39 children in FFY 2009. School districts are asked to submit an alert to the ADE/ECSE any time they receive a late referral from AzEIP that was not in category d (parent refusals to provide consent caused delays in evaluation or initial services) or category e (children who were referred to Part C less than 90 days before their third birthdays). Each late referral from AzEIP to a district is reported to the State AzEIP office. The State AzEIP office provides technical assistance and follow-up to the local service-providing agency. Similarly, if a local service-providing agency is reporting difficulty with a school district, the local agency issues an alert to the State AzEIP office. The ADE/ECSE provides technical assistance and follow-up to the school district. The ADE/ECSE and AzEIP maintain a shared database to track resolution of the alerts. Challenges with the completion of hearing and vision screenings and the resulting follow-ups are an inherent part of evaluating young children, which at times causes delays in transition. Arizona has worked diligently to provide resources and facilitate collaborative efforts between Head Start organizations, school districts, and Part C agencies. This has helped Part C service coordinators encourage families to have regular hearing screenings. Range of Days beyond Third Birthday Range of days 1–270 The 270 days beyond the child’s third birthday was due to a late referral from Part C. Explanation of Progress That Occurred for FFY 2012 Arizona maintained the same results (99%) from FFY 2011 to FFY 2012. The ADE/ECSE unit has been working with the Part C lead agency, Arizona Early Intervention Program (AzEIP), to implement changes based on the new Part C regulations and has been using the alert system to improve the transition Part B Arizona Annual Performance Report for FFY 2012 101 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 process for families and children. Each instance of late referral from an AzEIP provider was followed up by the ADE/ECSE unit together with the AzEIP agency, and systemic issues were identified and corrected. The continued collaboration between AzEIP and ECSE also contributed to continued positive outcomes. Additionally, the ADE/ESS specialists reviewed files during their annual site visits, provided technical assistance, and alerted ECSE of any problems during the year. Discussion of Improvement Activities Completed for FFY 2012 1) Promote and support “I’m Turning 3: What’s Next for Me?” parent trainings a) Post “I’m Turning 3: What’s Next for Me?” on ECSE Web site Activities completed from 7/1/12 to 6/30/13. 2) Maintain alert system between Part C and Part B to examine and resolve systemic and situational issues a) Respond to individual alerts at local level to resolve issues Activities completed from 7/1/12 to 6/30/13. b) Maintain database to track number of alerts reported to ADE/ECSE and AzEIP Activities completed from 7/1/12 to 6/30/13. a) Provide phone and email consultation to PEAs found to be noncompliant Activities completed from 7/1/12 to 6/30/13. 3) Conduct targeted technical assistance to PEAs found to be noncompliant 7/1/10– 6/30/13 ADE/ECSE AzEIP Staff PEA Staff Parent Information Network Specialists Raising Special Kids 7/1/10– 6/30/13 ADE/ECSE Director ADE/ECSE Specialist AzEIP Staff 7/1/10– 6/30/13 ADE/ECSE Director ADE/ECSE Specialist AzEIP Staff 7/1/10– 6/30/13 ADE/ECSE Director ADE/ECSE Specialist AzEIP Staff The parent transition booklet, “I’m Turning 3: What’s Next for Me?” was posted on the ECSE Web site and was distributed during parent trainings. A bilingual version is under development. Arizona’s PTI, Raising Special Kids/Pilot Parents of Southern Arizona, and ADE/ESS Parent Information Network Specialists (PINS) provided 36 trainings statewide (382 participants). ECSE and AzEIP staff responded to all alerts from the field. School districts and AzEIP service coordinators were assisted with resolving issues. Database was maintained to track number and sources of alerts in order to drive professional development and technical assistance decisions. ECSE staff, in collaboration with AzEIP staff, provided ongoing consultation with PEAs found noncompliant. Part B Arizona Annual Performance Report for FFY 2012 102 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Review noncompliant PEAs’ policies, procedures, and practices via desk audits and monthly reviews of data 4) Revise the Intergovernmental Agreement (IGA) with Arizona Early Intervention Program a) Review and revise 2010 IGA as appropriate based on adoption of Part C regulations Activities completed from 7/1/12 to 6/30/13. 7/1/10– 6/30/13 ADE/ECSE Director ADE/ECSE Specialist AzEIP Staff 7/1/12– 6/30/13 ADE/ECSE Director AzEIP Staff Noncompliant PEAs’ written policies and procedures were reviewed, along with their operating practices. Technical assistance was provided. File reviews were completed to ensure compliance with corrective action plans. This improvement activity will be implemented this year based on the implementation of Part C regulations on 7/1/12. Correction of FFY 2011 Findings of Noncompliance (if State reported less than 100% compliance in its FFY 2011 APR) Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 99% 1. Number of findings of noncompliance the State made during FFY 2011 (the period from July 1, 2011, through June 30, 2012) 7 2. Number of FFY 2011 findings the State verified as timely corrected (corrected within one year from the date of notification to the LEA of the finding) 7 3. Number of FFY 2011 findings not verified as corrected within one year [(1) minus (2)] 0 Correction of FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 0 5. Number of FFY 2011 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 0 6. Number of FFY 2011 findings not verified as corrected [(4) minus (5)] 0 Arizona made seven findings of noncompliance in FFY 2011. Although the PEAs have one year to correct the noncompliance, all seven findings were corrected as of January 15, 2013, as reported in the FFY 2011 APR on February 15, 2013. Part B Arizona Annual Performance Report for FFY 2012 103 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Actions Taken if Noncompliance Not Corrected All FFY 2011 noncompliance has been corrected, and Arizona has verified correction for all FFY 2011 findings. When findings of noncompliance are made, Arizona works in a collaborative process with school districts and AzEIP to identify the root causes of continuing noncompliance through individual verification and collaborative development of written process and procedures between districts, AzEIP service coordinators, Head Start Programs, and other programs that may be within the district of residence. When noncompliance is not corrected within one year, the ADE/ECSE interrupts 619 funds until full compliance is demonstrated. Verification of Correction (either timely or subsequent) As specified in OSEP’s July 2013 FFY 2011 SPP/APR Response Table, Arizona verified that each PEA with noncompliance reflected in the data: 1) is correctly implementing 34 CFR § 300.124 (b) (i.e., achieved 100% compliance) based on a review of updated data, such as data subsequently collected through on-site monitoring; and 2) has corrected each individual case of noncompliance, unless the child is no longer within the jurisdiction of the local education agency (LEA), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2011 The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance, including child-specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data include the following actions:  The ADE/ECSE specialists reviewed the written process and procedures for the PEAs’ early intervention transitions, including those that were collaboratively developed and agreed upon with AzEIP service coordinators.  The ADE/ECSE specialists reviewed student data during subsequent visits and/or desk audits of updated data to determine if the PEAs corrected all instances of noncompliance, including child specific instances, and to ensure ongoing sustainability with the implementation of the regulatory requirements. Correction of Remaining FFY 2010 Findings of Noncompliance Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2009 or Earlier Not applicable. Part B Arizona Annual Performance Report for FFY 2012 104 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response Because the State reported less than 100% compliance for FFY 2011, the State must report on the status of correction of noncompliance reflected in the data the State reported for this Indicator. In the FFY 2012 APR for Indicator 12 and Indicator 15, Arizona reported on the status of the correction of noncompliance (FFY 2011 data). In reporting on correction of findings of noncompliance in the FFY 2012 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2011 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2012 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2012 APR for Indicator 12 and Indicator 15, Arizona reported that it verified each PEA with noncompliance (FFY 2011 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 105 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Effective General Supervision Part B / Effective Transition Indicator 13: Secondary Transition Percent of youth with IEPs aged 16 and above with an IEP that includes appropriate measurable postsecondary goals that are annually updated and based upon an age appropriate transition assessment, transition services, including courses of study, that will reasonably enable the student to meet those postsecondary goals, and annual IEP goals related to the student’s transition services needs. There also must be evidence that the student was invited to the IEP Team meeting where transition services are to be discussed and evidence that, if appropriate, a representative of any participating agency was invited to the IEP Team meeting with the prior consent of the parent or student who has reached the age of majority. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent = [(# of youth with IEPs aged 16 and above with an IEP that includes appropriate measurable postsecondary goals that are annually updated and based upon an age appropriate transition assessment, transition services, including courses of study, that will reasonably enable the student to meet those postsecondary goals, and annual IEP goals related to the student’s transition services needs. There also must be evidence that the student was invited to the IEP Team meeting where transition services are to be discussed and evidence that, if appropriate, a representative of any participating agency was invited to the IEP Team meeting with the prior consent of the parent or student who has reached the age of majority) divided by the (# of youth with an IEP age 16 and above)] times 100. Target Data for FFY 2012 FFY Measurable and Rigorous Target FFY 2012 100% Actual Target Data for FFY 2012 80% Arizona did not meet the target. Year Total number of youth aged 16 and above with an IEP Total number of youth aged 16 and above with an IEP that meets the requirements Percent of youth aged 16 and above with an IEP that meets the requirements FFY 2012 509 409 80% Part B Arizona Annual Performance Report for FFY 2012 106 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 (2012– 2013) FFY 2012 Findings of Noncompliance number of findings by incidence of noncompliance number of findings by incidence corrected prior to one-year timeline as of 1/15/14 100 44 Arizona made 100 findings of noncompliance in FFY 2012. Although the PEAs have one year to correct the noncompliance, 44 findings have been corrected as of January 15, 2014. Correction of the remaining noncompliance (56 findings) will be reported in the FFY 2013 APR. Data Data Source The data for Indicator 13 are from the Arizona monitoring system. A public education agency (PEA) is selected for monitoring each fiscal year based on the results of a review of the agency’s data, including data from the SPP/APR, dispute resolution results, audit findings, and annual determinations. While Arizona has maintained a six-year monitoring cycle with assigned activities always occurring in Year 4, a PEA’s monitoring schedule can be adjusted and Year 4 monitoring activities can occur when the data reviews indicate systemic issues. The National Secondary Transition Technical Assistance Center (NSTTAC) Indicator 13 Checklist was used as a guide for the eight components that comprise the monitoring line item from which the data are pulled. The eight components are:         Measurable post-secondary goals Postsecondary goals updated annually Postsecondary goals based upon age-appropriate transition assessments Transition services Courses of study Annual IEP goals related to transition service needs Student invited to IEP meeting Representative of participating agency invited to IEP meeting with prior consent of parent or student who has reached the age of majority Data Collection Data are collected from the PEAs during one of three types of monitorings:  Data Review — PEAs review student files focusing on Indicator 13. The ADE/ESS specialists validate the compliance calls. The student file forms are submitted to ESS for data entry.  Self-Assessment — PEAs review student files and collect data for Indicator 13. The PEAs also focus on reviewing files for other identified SPP/APR Indicators on which their agencies have not Part B Arizona Annual Performance Report for FFY 2012 107 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 met the State targets. The ADE/ESS specialists validate the compliance calls. The student file forms are submitted to ESS for data entry.  On-Site — PEAs and the ADE/ESS teams review student files, collect data through surveys and interviews, and collect data for Indicator 13. The ADE/ESS staff inputs data. Valid and Reliable Data The ADE/ESS assures the validity and reliability of the data as it is collected, maintained, and reported through the State monitoring system. Training is provided to all ESS program specialists who monitor to ensure interrater reliability for compliance calls according to regulatory requirements. The ADE/ESS staff conducts trainings for PEA staff who will participate in monitorings. The ESS specialists validate and verify the data through on-site visits or desk audits. Definition of Finding for Monitoring for FFY 2012 During FFY 2012, a finding for Indicator 13 was issued when the line item for secondary transition was found to be noncompliant. The finding was a written notification to the PEA by the State that the line item was noncompliant, and the finding included a description of a Federal or State statute or regulation. The source of information on which to base a finding of noncompliance is an individual student file. Explanation of Progress That Occurred for FFY 2012 Although Arizona did not meet the target, it made progress from FFY 2011 (78%) to FFY 2012 (80%.) The ADE/ESS specialists who monitor provide ongoing technical assistance throughout the year during on-site visits to review files and to teach the school personnel how to review and analyze their own student files. These site visits and follow-ups by the ADE/ESS specialists continue to focus on Indicator 13. On an annual basis, the ESS secondary transition specialists update and implement a comprehensive strategic plan that includes analysis of Indicator 13 data; staff development for PEAs; provision of capacity building grants; collaboration with national technical assistance centers and with other State agencies; and organization of a statewide transition conference (see the discussion of improvement activities below). Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Revise, implement, and evaluate a comprehensive plan for training PEAs to increase compliance with postsecondary Sub-Activities (Objectives or Action Steps) a) On an annual basis, identify PEAs in Years 2 and 3 of the monitoring cycle through collaboration with ESS specialists Timeline Complete Activity completed 8/1/12. The FFY 2012 PEA list was completed August 2012. The Annual Site Visit Log (ASVL) from school year 2011-2012 was used to identify PEAs most in need of training and technical assistance (TA) for Indicator 13. Projected 7/1/12– 6/30/13 Resources (Planned) ADE/ESS Transition Specialists ADE/ESS Program Specialists Part B Arizona Annual Performance Report for FFY 2012 108 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 requirements related to Indicator 13 b) On an annual basis, review, revise (if necessary), and implement the comprehensive training plan, emphasizing the eight required components of Indicator 13 Activities completed from 7/1/12 to 6/30/13. 7/1/12– 6/30/13 ADE/ESS Transition Specialists The FFY 2012 review and revision of the Strategic Plan for Statewide Transition Planning was completed July 2012. Implementation of the Transition Strategic Plan was completed from July 2012 to June 2013. The FFY 2012 Transition Strategic Plan includes seven main components: 1. Provide training to targeted PEAs and in response to requests from non-targeted PEAs on Indicator 13; 2. Organize Arizona’s Twelfth Annual Transition Conference focusing on improving post school outcomes for students with disabilities by providing sessions on transition planning and dropout prevention; 3. Provide training to special education directors from across the State at the annual ADE/ESS Directors Institute; 4. Provide capacity building grants to PEAs to facilitate intra/interagency collaboration and build local capacity to improve post school outcomes through local interagency work, as well as provide intensive training and support for PEAs to achieve 100% compliance on Indicator 13; 5. Collaborate with national technical assistance centers and organizations including NSTTAC, NPSO, NDPC-SD, and the NASDSE IDEA Partnership Community of Practice on Transition and Transition Coalition; 6. Participate with other Arizona State agencies including Part B Arizona Annual Performance Report for FFY 2012 109 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Rehabilitation Services of Arizona/Vocational Rehabilitation (RSA/VR), Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN); 7. Collaborate with other ADE sections: High School Renewal and Improvement (AZHSRI), Career and Technical Education (CTE), Dropout Prevention, and School Counselors and ADE/ESS areas: Program Support, Assistive Technology, and Parent Information Network. All components of Arizona’s FFY 2012 Strategic Plan for Statewide Transition Planning were immediately implemented upon revision, from 8/1/12 to 6/30/13. Activities completed:  1,581 participants from 519 targeted and non-targeted PEAs received secondary transition training, including Indicator 13 trainings from ADE/ESS transition specialists at 76 sites statewide. Trainings were delivered in regional or direct school locations.  Arizona’s Twelfth Annual Transition Conference was held in October 2012 and offered sessions focused on improving compliance with the eight components of Indicator 13. 903 participants attended the conference, including education and agency professionals, youth, young adults, family members of youth with disabilities, and vendors/exhibitors.  Two sessions on secondary transition were provided by ADE/ESS transition specialists at the annual ADE/ESS Directors Part B Arizona Annual Performance Report for FFY 2012 110 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Institute (DI). Of the 716 in attendance at the DI, 59 participants attended sessions related to secondary transition.  28 PEAs participated in Year 1 (14 PEAs in the 2012–2014 cohort) or Year 2 (14 PEAs in the 2011–2013 cohort) team trainings of the Secondary Transition Mentoring Project (STMP) capacity building grant. Through a contract with the University of Kansas Transition Coalition and in collaboration with ADE/ESS, training was provided to achieve and sustain 100% compliance on Indicator 13 using State and local data results from Indicators 1, 2, and 14 as a tool to inform transition planning and practices in a coordinated effort to improve transition education, services, and outcomes for students with disabilities. Team training included: three two-day face-toface training sessions, webinars, a web site for project participants containing all training materials and extensive resources, an online short course for Year 1 teams, and attendance at Arizona’s Twelfth Annual Transition Conference.  Collaboration with national technical assistance centers and organizations occurred throughout the year and included: participation in NPSO and NSTTAC Community of Practice calls; use of resources from the NSTTAC Web site; participation in the NASDSE IDEA Partnership, Community of Practice National Meeting; and attendance at the National Secondary Transition Planning Institute where OSEP, NPSO, NSTTAC, and NDPC-SD provided guidance. Additionally, ADE/ESS maintained ongoing collaboration with the University of Kansas Transition Coalition and the Mountain Plains Regional Part B Arizona Annual Performance Report for FFY 2012 111 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Resource Center.  At the State level, ADE/ESS collaborated with RSA/VR, Division of Developmental Disabilities (DDD), Division of Behavioral Health Services (DBHS), and the Office for Children with Special Health Care Needs (OCSHCN), and met every other month with these agency stakeholders through the Arizona Community of Practice on Transition (AZCoPT). In FFY 2012, AZCoPT revised a presentation used statewide through RSA/VR and DBHS teleconferencing media to introduce participants to the supports/services available to school-aged and adult individuals with disabilities.  ADE/ESS collaboration meetings with the ADE areas of High School Renewal and Improvement, Career and Technical Education, Dropout Prevention, and Career and College Readiness were scheduled and held throughout the year and resulted in crosstraining for conferences sponsored by different ADE sections that included the topic of secondary transition.  Intra-ADE/ESS collaborative efforts included: monthly meetings with PINS (Parent Information Network specialists) as fellow AZCoPT members, as well as involvement with PINS during quarterly Transition Conference Planning Committee meetings; at least quarterly meetings with ESS Program Support to discuss the use of the Annual Site Visit Log (ASVL); review/revision of the secondary transition section of the monitoring manual and needed secondary transition trainings for ESS program specialists and PEAs; and the development of Part B Arizona Annual Performance Report for FFY 2012 112 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 c) On an annual basis, create and disseminate information through a variety of sources: annual statewide conference, monitoring alerts, Web site, and listserv announcements collaborative presentations with the ADE/ESS assistive technology unit. Activities completed from 8/1/12 through 6/30/13. 7/1/12– 6/30/13 ADE/ESS Transition Specialists 7/1/12– 6/30/13 ADE/ESS Transition Specialists  Two secondary transition presentations were offered at the ADE/ESS Directors Institute in August 2012 for approximately 59 participants.  102 sessions either directly or indirectly related to Indicator 13 were offered at Arizona’s Twelfth Annual Transition Conference held in October 2012.  Between July 2012 and June 2013, the ADE/ESS Secondary Transition Web site was updated to include additional and/or revised Indicator 13 materials from NSTTAC, IDEA Partnership, and other secondary transition technical assistance centers. The Web address is http://www.azed.gov/specialeducation/specialprojects/secondary-transition/. Activities completed from 7/1/12 to 6/30/13. d) On an annual basis, analyze preand post-training data collected  Analysis of ASVL pre-training through the Annual data was completed in January Site Visit Log (ASVL) 2013 after all ESS program for each PEA to specialists were able to complete determine level of at least one PEA annual site visit compliance on all during fall 2012. The analysis eight required showed a 72.8% average for components of compliance with the eight items of Indicator 13 Indicator 13. ADE/ESS Program Specialists  Post-training data analysis of all PEAs that received a pre-training annual site visit and were trained in secondary transition during FFY 2012 showed an 85.8% average for compliance with the eight items of Indicator 13. Through pre- and post-training analysis, an increase of 13.3% in compliance for Indicator 13 was Part B Arizona Annual Performance Report for FFY 2012 113 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 2) Provide a twoyear capacity building grant to PEAs to participate in the Secondary Transition Mentoring Project (STMP) Team Training a) On an annual basis, identify PEAs who meet eligibility requirements and extend invitations to them to participate in STMP trainings b) On an annual basis, provide indepth and ongoing professional development on transition requirements and best practices demonstrated after training and technical assistance was provided to PEAs by transition and education program specialists. Activities completed from 1/1/12 to 7/30/13. 7/1/12– 6/30/13  Using Annual Site Visit (ASV) data for PEAs currently in Years 1, 2, or 3 of the monitoring cycle, 23 PEAs achieving less than 100% compliance on Indicator 13 were invited to apply for a noncompetitive Secondary Transition Mentoring Project (STMP) capacity building grant.  Fourteen PEAs were accepted for participation in Year 1 of the STMP Team training in July 2012. Activities completed from 7/1/12 through 6/30/13. ADE/ESS Transition Specialists ADE/ESS Program Specialists 7/1/12– 6/30/13 ADE/ESS Transition Specialists  STMP participants attended Arizona’s Twelfth Annual Transition Conference, which included a STMP team orientation and designated sessions.  Training materials and activities were designed, created, and disseminated by ADE/ESS in collaboration with the University of Kansas Transition Coalition. Trainings focused on: identifying PEA barriers to meeting transition requirements; developing an action plan to eliminate barriers; creating IEPs that meet transition requirements; implementing training to build intra-PEA capacity to attain 100% compliance on secondary transition requirements; and determining improvements made and targeting areas still in need of improvement.  Team training for each cohort consisted of three two-day faceto-face trainings throughout the year. Additionally, Year 1 teams completed an online short course specially designed for STMP Part B Arizona Annual Performance Report for FFY 2012 114 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 teams and focused on best practices in transition planning. Activity completed from 7/1/12 through 6/30/13. c) On an annual basis, analyze preand post-training data collected during  Measures to determine STMP trainings for effectiveness of STMP training each PEA that included the use of: participated to a) team process checklist; determine level of b) interrater reliability measures compliance on all for IEP file review; eight required c) training session evaluations; components of d) STMP team self-reported Indicator 13 progress toward completion of action plans; and e) feedback provided by ADE/ESS monitoring program specialists on compliance. 7/1/12– 6/30/13 ADE/ESS Transition Specialists Data from the measures are as follows: k) April 2013 trainings for both STMP cohorts included the completion of a 37-item “team process” checklist. Results from the checklist use indicated both Year 1 and Year 2 teams reported an average of 92.15% for the item “all team members are highly knowledgeable of transition compliance requirements” and an average of 80.8% for the item “STMP team collaboratively develops and implements an action plan that addresses the prioritized needs.” l) Training for both cohorts included instruction and practice activities using interrater reliability measures. m) All STMP training sessions during FFY 2012 included instruction, activities, and Indicator 13 file reviews. Based on a 5-point Likert scale (1 lowest and 5 highest), evaluation results indicated an average score of 4.27 across all activities for both cohorts. Additionally, all team members completed a pre-/postevaluation of their competency Part B Arizona Annual Performance Report for FFY 2012 115 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 in meeting Indicator 13 requirements that asked the question, “How prepared do you feel?” Based on a 5-point Likert Scale (1 equaled unprepared and 5 equaled prepared) and using a Paired Samples T-Test, all scores indicated a significant increase in knowledge from the beginning to the end of the training for FFY 2012 for both cohorts (Years 1 and 2). The areas reporting the greatest increase in knowledge were transition assessments and documenting transition services and course of study that will support the student in reaching postsecondary goals. n) Review of each STMP team’s action plan indicated all teams set goals to build and sustain systems to ensure 100% compliance on Indicator 13 school- or district-wide. All teams reported progress in meeting these goals. o) During the April 2012 training, Year 2 teams met with ADE/ESS program specialists to receive informal feedback on IEPs written during the second year of the STMP training program. Feedback indicated significant improvement in PEA knowledge and compliant practices. Correction of FFY 2011 Findings of Noncompliance (if State reported less than 100% compliance) Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 78% 1. Number of findings of noncompliance the State made during FFY 2011 (the period from July 1, 2011, through June 30, 2012) 99 2. Number of FFY 2011 findings the State verified as timely corrected (corrected within one year from the date of notification to the LEA of the finding) 77 3. Number of FFY 2011 findings not verified as corrected within one year [(1) minus (2)] 22 Part B Arizona Annual Performance Report for FFY 2012 116 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Correction of FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 22 5. Number of FFY 2011 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 22 6. Number of FFY 2011 findings not verified as corrected [(4) minus (5)] 0 The ADE/ESS specialists reviewed the child specific files from the monitoring to determine that the PEA implemented the eight components of the secondary transition requirements for the children, unless they were no longer within the jurisdiction of the PEA. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine that the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to secondary transition in conformity with 34 CFR §§ 300.320 (b) and 300.321 (b). Actions Taken if Noncompliance Not Corrected All FFY 2011 noncompliance has been corrected, and Arizona has verified correction for all FFY 2011 findings. When findings of noncompliance are made, Arizona works collaboratively with the PEAs as the agencies identify the root causes of their continuing noncompliance through drill downs in the specific focus area that affects this Indicator. However, when noncompliance is not corrected within one year, the ADE/ESS uses a variety of methods to ensure that all public agencies meet the requirements of State and Federal statutes related to special education. The enforcement actions taken by ESS for the PEAs that are unable to demonstrate compliance within one year from the date of written notification are as follows:  Enforcement of corrective action plan (CAP) activities as outlined in the current agency CAP, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Review and revision of the current CAP to develop targeted activities that address the continued noncompliance, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Assignment of a special monitor.  Interruption of IDEA payments until adequate compliance is achieved. For charter schools not receiving IDEA funds, a request to begin withholding 10% of State funds.  For charter schools, a request to the appropriate board for a notice of intent to revoke the charter.  With Arizona State Board of Education approval, interruption of Group B weighted State aid or redirection of funds pursuant to 34 CFR § 300.227 (a).  Referral to the Arizona Attorney General for legal action. Verification of Correction of FFY 2011 Noncompliance (either timely or subsequent) FFY 2011 Verification of Correction from Monitoring Part B Arizona Annual Performance Report for FFY 2012 117 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 As specified in OSEP’s June 2013 Arizona Part B FFY 2011 SPP/APR Response Table, Arizona verified that each PEA with noncompliance reflected in the data: 1) is correctly implementing 34 CFR §§ 300.320 (b) and 300.321 (b) (i.e., achieved 100% compliance) based on a review of updated data such as data subsequently collected through onsite monitoring; and 2) has completed the evaluation, although late, for any child whose initial evaluation was not timely, unless the child is no longer within the jurisdiction of the LEA, consistent with OSEP Memorandum 09-02, dated October 17, 2008. Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2011 The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data:  ADE/ESS specialists conducted follow-up on-site visits and/or desk audits after the monitoring to verify correction of all instances of noncompliance, including those that were child specific. The specialists reviewed the child specific files to determine that the evaluation was completed within 60 calendar days from the date of written notification of noncompliance.  ADE/ESS specialists reviewed updated data from subsequent files and/or conducted interviews with the special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including those that were child specific, were corrected and to ensure ongoing sustainability of the implementation of the regulatory requirements regarding initial evaluations. Correction of Remaining Findings of Noncompliance from FFY 2010 or Earlier Not applicable. Correction of Any Remaining Findings of Noncompliance from FFY 2009 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response Because the State reported less than 100% compliance for FFY 2011, the State must report on the status of correction of noncompliance reflected in the data the State reported for this Indicator. In the FFY 2012 APR for Indicator 13 and Indicator 15, Arizona reported on the status of the correction of noncompliance (FFY 2011 data). In reporting on correction of findings of noncompliance in the FFY 2012 APR, the State must report that it verified that each PEA with In the FFY 2012 APR for Indicator 13 and Indicator 15, Arizona reported that it verified each PEA with noncompliance (FFY 2011 data), consistent with Part B Arizona Annual Performance Report for FFY 2012 118 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 noncompliance identified in FFY 2011 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2012 APR, the State must describe the specific actions that were taken to verify the correction. OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 119 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Effective General Supervision Part B / Effective Transition Indicator 14: Post School Outcomes Percent of youth who are no longer in secondary school, had IEPs in effect at the time they left school, and were: A. Enrolled in higher education within one year of leaving high school. B. Enrolled in higher education or competitively employed within one year of leaving high school. C. Enrolled in higher education or in some other postsecondary education or training program; or competitively employed or in some other employment within one year of leaving high school. (20 U.S.C. 1416 (a)(3)(B)) Measurement A. Percent enrolled in higher education = [(# of youth who are no longer in secondary school, had IEPs in effect at the time they left school and were enrolled in higher education within one year of leaving high school) divided by the (# of respondent youth who are no longer in secondary school and had IEPs in effect at the time they left school)] times 100. B. Percent enrolled in higher education or competitively employed within one year of leaving high school = [(# of youth who are no longer in secondary school, had IEPs in effect at the time they left school and were enrolled in higher education or competitively employed within one year of leaving high school) divided by the (# of respondent youth who are no longer in secondary school and had IEPs in effect at the time they left school)] times 100. C. Percent enrolled in higher education, or in some other postsecondary education or training program; or competitively employed or in some other employment = [(# of youth who are no longer in secondary school, had IEPs in effect at the time they left school and were enrolled in higher education, or in some other postsecondary education or training program; or competitively employed or in some other employment) divided by the (# of respondent youth who are no longer in secondary school and had IEPs in effect at the time they left school)] times 100. Data Data Source and Collection Methods The ADE/ESS used a sampling procedure to collect Post School Outcome (PSO) data. Over the course of the State Performance Plan (SPP), each PEA serving students 16 years old and older is asked to collect and report post school outcomes data during the second year of the six-year monitoring cycle. The monitoring cycle is a representative sample of Arizona’s districts and charter schools and the representative sample is based on the categories of disability, race, and gender. The ADE/ESS sampling plan was approved by OSEP. Federal Fiscal Year (FFY) 2012 marked the second year of cycle two of Arizona’s PSO Data Collection requirement, in which all eligible PEAs are assigned to a collection year for inclusion in the SPP. Of Arizona’s eligible PEAs, 50 were identified to participate in the PSO Survey requirement for FFY 2012. Of this number, nine PEAs did not have any leavers who met the criteria (youth with a current IEP who aged out, graduated, or dropped out) to be eligible to participate in the PSO Survey during the 2011–2012 Part B Arizona Annual Performance Report for FFY 2012 120 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 school year. This resulted in 41 PEAs that had eligible leavers and these PEAs were required to conduct the PSO Survey during FFY 2012. It should be noted that of these 41 PEAs, four PEAs failed to meet the requirement to participate in the FFY 2012 PSO data collection. Three of the four PEAs that did not participate only had one exiter each. In order to participate in the PSO Survey, PEAs gather contact information on student leavers and either input the data into the online PSO data collection system or maintain contact information locally. The PSO data collection system uses a secure application as part of the ADE Common Logon. The application includes an auto-population of student demographic information and exit reason imported directly from the Student Accountability Information System (SAIS), a Web-based system for reporting all student-level details to the ADE. PEAs designate district or charter school personnel to contact student leavers or designated family members (i.e., parent, grandparent, or guardian), conduct phone interviews, and input survey data into the online PSO data collection system. Youth or family members were contacted between July 1 and September 30, 2013, after being out of school for at least one year. Arizona’s two PEAs with an average daily membership exceeding 50,000 are included in the data collection each year. Definitions The following definitions are used by the ADE/ESS in the data collection and reporting for Indicator 14: Higher Education includes youth who have been enrolled on a full- or part-time basis in a community college (two-year program) or a college/university (four- or more year program) for at least one complete term, at any time in the year since leaving high school. Competitive Employment includes youth who have worked for pay at or above the minimum wage in a setting with others who are nondisabled for a period of 20 hours a week for at least 90 days at any time in the year since leaving high school. This includes military employment. Other Postsecondary Education or Training includes youth enrolled on a full- or part-time basis for at least one complete term at any time in the year since leaving high school in an education or training program (e.g., Job Corps, adult education, workforce development program, or vocational technical school that is less than a two-year program). Some Other Employment includes youth who have worked for pay or been self-employed for a period of at least 90 days at any time in the year since leaving high school. This includes working in a family business (e.g., farm, store, ranching, catering services, etc.). Respondents are youth, young adults, or designated family members who answer the PSO Survey. Leavers are youth or young adults, who left school by graduating, aging out, or leaving school early (i.e., dropped out) or who were expected to return to school and did not. Response Rate and Representativeness The FFY 2012 response rate was 73%. Table 14.1 shows that Arizona’s FFY 2012 sample included 1467 youth who were eligible to take the survey. Interviews were conducted with 1068 youth, young adults, or their family members. The FFY 2012 73% response rate represents a 3% increase in response rate over FFY 2011. Of Arizona’s41 PEAs required to participate in the FFY 2012 PSO Survey, 68% of the youth eligible to complete the PSO survey exited from one of three PEAs. ADE/ESS considers this an anomaly of the FFY 2012 sample. Table 14.1 Response Rate Calculation Number of leavers in the sample 1602 Part B Arizona Annual Performance Report for FFY 2012 121 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Subtract the number of youth ineligible (those who had returned to school, or were deceased, or whose data were uploaded by the PEA to the SAIS system in error) 122 + 5+ 8 = 135 Number of youth eligible to contact 1467 Number of completed surveys 1068 Response rate (1068 / 1467) * 100 73% The ADE/ESS used the NPSO Response Calculator to calculate representativeness of the respondent group on the characteristics of (a) disability type, (b) ethnicity, (c) gender, and (d) exit status (e.g., dropout) to determine whether the youth who responded to the interviews were similar to, or different from, the total population of youth with an IEP who exited school during school year (SY) 2011–2012. According to the NPSO Response Calculator, differences between the Respondent Group and the Target Leaver Group +/− 3% are important. Negative differences indicate an under representativeness of the group and positive differences indicate over representativeness. In Table 14.2, bolded text is used to indicate a difference exceeding a +/− 3% interval. As shown in Table 14.2, respondents were representative of all 2011–2012 target leavers based on category of disability, gender, ethnicity and youth who graduated. However, as in previous years, dropouts were under-represented compared to the target leaver group. FFY 2012 saw an end to the positive trend of yearly improvement in response rates for dropouts compared to the target group of dropouts. In FFY 2012, a – 5.7% difference between respondents and target leavers group existed. This represents almost a 1% increase in the difference between respondents and target leavers over FFY 2011. ADE/ESS will continue its efforts to increase response rates, especially among youth who drop out. Technical assistance and highlighting tips provided in the NPSO guidance document for contacting hard to reach youth will be provided to PEAs that participate in the PSO Survey. Table 14.2 Representativeness of Responders to Leavers FFY 2012 Representativeness Overall Target Leaver Totals Response Totals Target Leaver Representation Respondent Representation Difference LD ED MR AO Female Minority ELL Dropout 1467 971 141 120 235 487 369 54 372 1068 708 85 100 175 351 254 40 210 66.19 9.61 8.18% 16.02 33.20% 25.15% 3.68 25.36% 66.29 7.96% 9.3600% 16.39% 32.87% 23.78% 3.75 19.66% .10 −1.65% 1.18% .37% -.33 −1.37% 0.06 −5.69% Missing Data Arizona’s PSO response rate for FFY 2012 was 73% (1467 youth eligible to contact and 1068 respondents). The FFY 2012 PSO Survey is missing data on 27% or 399 former students. An analysis of missing data indicated that the largest segments of missing data were the result of either schools not being able to contact leavers after three attempts (165 or 41% of the missing data) or not having correct contact information (177 or 44% of the missing data). Selection Bias Part B Arizona Annual Performance Report for FFY 2012 122 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 The State is encouraged by the 3% increase in response rate for FFY 2012 as well as the continued representativeness of target populations, with the exception of dropouts. The State will continue to work with NPSO to identify strategies to encourage survey responses from youth in the dropout category. Table 14.3 FFY 2012 Engagement Data FFY 2012 (2011–2012) Number of respondent leavers Percentage of respondent leavers Total respondents Indicator 14 Engagement Data 14A 14B 14C 209 532 715 19.6% 49.8% 66.9% 1068 Discussion of FFY 2012 Data There were 1068 total respondents to the PSO Survey. Each leaver was counted once in the highest category: 1) 209 respondent leavers were enrolled in “higher education” 2) 323 respondent leavers were engaged in “competitive employment” (and not counted in number 1 above) 3) 103 respondent leavers were enrolled in “some other postsecondary education or training” (and not counted in numbers 1 or 2 above) 4) 80 respondent leavers were engaged in “some other employment” (and not counted in numbers 1, 2, or 3 above) Thus: 14A = 209 (#1) divided by 1,068 (total respondents) = 19.6% 14B = 209 (#1) + 323 (#2) divided by 1,068 (total respondents) = 49.8% 14C = 209 (#1) + 323 (#2) + 103 (#3) +80 (#4) divided by 1,068 (total respondents) = 66.9% Figure 14.1, Arizona PSO Survey FFY 2012 Cohort, 2011–2012 School Year Exiters, shows the outcome categories, including the not engaged category, the number of leavers in each category, and the percentage of leavers in each outcome category. Table 14.3, FFY 2012 Engagement Data, shows the percentages for each measure A, B, and C. As shown in Figure 14.1, the largest percentage of leavers was not engaged 33.1% (n = 353 of leavers counted in this category). The second largest percentage of leavers was competitively employed with 30.2% (n = 323). The remaining categories, from largest percentage to smallest, were enrolled in higher education, 19.6% (n = 209); enrolled in other postsecondary education or training, 9.6% (n = 103); and some other employment, 7.5% (n =80). Part B Arizona Annual Performance Report for FFY 2012 123 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Figure 14.1 Arizona IDEA Part B SPP/APR Indicator #14: Post-School Outcomes for 2011-12 School Year Exiters 1: Enrolled in higher education , 209, 20% Not Engaged, 353, 33% 2: Competitive employment , 323, 30% 4: Some other employment, 80, 7% 3: Enrolled in other postsecondary education or training, 103, 10% SPP #14 Meaurement A: 20% Equals Segment 1 SPP #14 Meaurement B: 50% Equals Segments 1+2 SPP #14 Meaurement C:_ 67% Equals Segments 1+2+3+4 1: Enrolled in higher education 2: Competitive employment 3: Enrolled in other postsecondary education or training 4: Some other employment Not Engaged Part B Arizona Annual Performance Report for FFY 2012 124 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Targets for FFY 2012 Targets were set based on the FFY 2011 baseline data and input from stakeholders. FFY Measurable and Rigorous Targets 14A 14B 14C 26.6% 60.2% 74.1% 2012 FFY Actual Measurable and Rigorous Targets Achieved 14A 14B 14C 19.6% 49.8% 66.9% 2012 Arizona did not meet the targets. Disaggregated Outcomes by Subgroups The ADE/ESS used the NPSO Data Display Templates to allow for a thorough understanding of the post school outcomes of Arizona’s youth and young adults. The outcomes were examined by each subgroup: gender, disability type, ethnicity, and exit type. Post School Outcomes by Gender As displayed in Figure 14.2, Indicator 14: Arizona’s Post School Outcomes by Gender, female leavers in Arizona had similar outcomes to males in the areas of some other employment and enrolled in other postsecondary education or training. Analysis of engagement in higher education and competitive employment reveals that females are slightly more likely (2%) to have enrolled in higher education (21% vs. 19%) and slightly less likely (11%) to be competitively employed than their male counterparts 23% vs. 34%). The ADE/ESS will share this information with PEAs and assist districts and charters in analyzing root causes for these gender discrepancies. Part B Arizona Annual Performance Report for FFY 2012 125 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Figure 14.2, Indicator 14: Arizona’s Post School Outcomes by Gender Post School Outcomes by Disability Category Figure 14.3, Indicator 14: Arizona’s Post School Outcomes Respondents by Type of Disability, shows that individuals with a disability category of mental retardation (MR) were more likely to be enrolled in other postsecondary education or training (22% vs. 10% state wide). If employed, individuals with a disability category of mental retardation are less likely to be competitively employed than individuals identified in any other disability category (15% vs. 35% (SLD), 25% (ED), and 23% (all other disabilities). It should be noted that the “mental retardation” category was used for this analysis since the terminology update to “intellectually disabled” (ID) in this disability category had not yet been reflected in the SAIS system when this data was collected. However, for discussion purposes in this APR, the term intellectual disability (ID) is used instead of mental retardation. Further analysis of the outcomes of respondents of individuals with an emotional disability or intellectual disability reveal that both groups experience the same nonengagement rate of 46%. This rate is 13% higher than the state rate of 33%. When considering competitive employment rates, individuals with a specific learning disability were employed at a higher rate (35%) than any other disability category. Furthermore, engagement rates in higher education were highest for individuals in the category of all other disabilities (30%), followed by individuals with a specific learning disability (20%), representing a rate much higher than peers with an intellectual disability (6%) Based on this information, in collaboration with our local and State community of practice/community Part B Arizona Annual Performance Report for FFY 2012 126 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 transition teams, ADE/ESS will continue to work on developing strategies and resources targeted to assist local PEAs improve overall engagement for individuals with an intellectual or emotional disability. Since FFY 2011 was a reset of the baseline, ADE analyzed FFY 2012 results compared to the FFY 2011 reset baseline. This comparison revealed a 7% decrease in overall engagement for FFY 2012. Most notably, the non-engagement rate rose from 26% to 33%. Despite the overall decrease in engagement, slight increases for students with intellectual disabilities were noted. Enrollment in higher education doubled from 3% to 6% and competitive employment rose by 1% from 14% to 15%, while nonengagement dropped from 53% to 46%. Individuals from every other disability category saw decreases in engagement in higher education and competitive employment and increases in non-engagement. Based on this analysis, ADE will continue its efforts in assisting PEAs to use knowledge of predictors of post school success in program development and implement strategies targeted at improving engagement for students with intellectual disabilities. Figure 14.3, Indicator 14: Arizona’s Post School Outcomes Respondents by Type of Disability Post School Outcomes by Ethnicity As displayed in Figure 14.4, Indicator 14: Arizona’s Post School Outcomes by Ethnicity, Arizona youth enrolled in higher education at a rate ranging from 67% for students identified as Asian to 7% for those identified as American Indian/Alaska Native. Students identified as Black or African American (26%) and White (23%) had enrollment rates higher than the average statewide rate of 20%, while Hispanic/Latinos had a rate of 18%, which is slightly lower than the statewide rate. The ethnic group with the largest percentage of youth who were competitively employed was White (35%), as compared with peers who identified as Hispanic/Latino (33%), Black/African American (24%), and American Indian/Alaska Native Part B Arizona Annual Performance Report for FFY 2012 127 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 (15%).The percentage of youth enrolled in other postsecondary education or some other employment was consistent across all groups, with Arizona reporting an engagement rate of less than 17% for all ethnic groups. Engaging youth who are identified as American Indian/Alaska Native continues to be a concern for ADE/ESS since those individuals experience the least successful outcomes compared to their peers in terms of engagement in higher education and competitive employment, while experiencing the highest rate of non-engagement. Figure 14.4, Indicator 14: Arizona’s Post School Outcomes by Ethnicity Post School Outcomes by Type of Exit Figure 14.5, Indicator 14: Arizona’s Post School Outcomes by Type of Exit, discloses that individuals who earned a high school diploma had better outcomes, especially in the areas of higher education and competitive employment. Only 2% of dropouts enrolled in higher education compared to 24% of graduates; similarly, 22% of dropouts were engaged in competitive employment as compared to 32% of graduates. Dropouts had a lower rate of engagement (43%) compared to graduates (73%). Part B Arizona Annual Performance Report for FFY 2012 128 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Figure 14.5, Indicator 14: Arizona’s Post School Outcomes by Type of Exit Characteristics of Nonresponders A review of the 609 leavers who did not respond to the survey by demographic subgroup indicates that 66% were male, 43% identified themselves as Hispanic/Latino, 74% exited with a high school diploma, and 65% were individuals identified with the disability category of specific learning disability. The ADE/ESS will continue to share this information with PEAs and encourage schools to target these subgroups for greater participation in the PSO survey. Trend Data With FFY 2011 being a new baseline, ADE/ESS will not discuss trend data until three years of data are available in FFY 2013. Explanation of Slippage That Occurred for FFY 2012 Examination of the data indicates a 6.6% decrease in measurement A (26.6% target vs. 19.6% actual), a 10.2% decrease in measurement B (60.2% target vs. 49.8% actual) and 7.1% decrease in measurement Part B Arizona Annual Performance Report for FFY 2012 129 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 C (74.1% vs. 66.9%). The ADE/ESS believes the reasons for slippage are difficult to determine due to anomalies in the FFY 2012 sampling procedure. Although the sample was comprised of 50 PEAs, the actual number of PEAs that had eligible exiters during FFY 2012 was 41. Additionally, 67% of youth eligible to be contacted for the survey in the sample (988 of 1467) came from just three PEAs. The FFY 2012 measurements A, B, and C are primarily reflective of outcomes from three PEAs. It is difficult to generalize statewide results or understand measured slippage based on results from three PEAs. As a result of the FFY 2012 data collection sampling anomalies, the ADE/ESS will reexamine the State’s sampling plan during FFY 2013 to determine if changes are needed. Although Arizona did not meet the targets, the following increases were noted: 3% increase in response rate; 3% increased engagement of students with intellectual disabilities in higher education and overall engagement rate for students with intellectual disabilities rose from 47% (FFY 2011) to 54% (FFY 2012). ADE/ESS’ concerns over the observed 7% increase in non-engagement rate of 26% in FFY 2011 to 33% in 2012 led to a deeper analysis of the FFY 2012 non-engagement rate. Of the 353 individuals that are counted in the non-engaged category, 219 did not attempt any postsecondary education/training or employment; however, 134 respondents (38%) indicated that they attempted postsecondary education/training or employment but did not meet the criteria to be counted in an engagement category. If those 134 youth who attempted engagement had been successful, Arizona’s “Measurement C” (engagement rate) would have increased from 67% to 79%. The ADE/ESS will share this information with schools and stakeholders. Technical assistance from NPSO will be requested by ESS to assist in drilling down into the data to determine specific reasons for non-engagement by individuals who attempted engagement but did not meet the criteria to be counted as engaged. Dissemination of FFY 2012 Data To ensure broad dissemination of Indicator 14 results, ESS will post results on the ADE/ESS Post School Outcomes Web site at http://www.azed.gov/special-education/special-projects/secondary-transition /postschool-outcomes-survey by May 2014. Additionally, ESS will present PSO results to Arizona’s Special Education Advisory Panel (SEAP) and Arizona’s Community of Practice on Transition as well as in presentations at Arizona’s Annual Directors’ Institute and Transition Conference. PEAs that participated in the 2013 PSO Survey will be invited to attend a 2013 PSO Results webinar, which will include State results and information on how to access and use local FFY 2012 PSO results. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Develop, implement, and evaluate procedures and trainings needed to assure participation in Post School Outcomes (PSO) Survey by identified PEAs Sub-Activities (Objectives or Action Steps) a) Revise PSO application and survey questions to align with new Indicator 14 Table, requirements, and definitions Timeline Complete Activity completed 7/1/12 to 6/30/13 and discontinued. Enhancements to the online PSO Survey application were completed. Enhancements included upgrades to reporting features allowing PEAs to export reports to excel, a reporting tool to analyze non-engaged response categories and creation of reports to analyze participation and Projected 7/1/10– 6/30/13 Resources (Planned) ADE/ESS Transition Specialists ADE/IT Specialists Part B Arizona Annual Performance Report for FFY 2012 130 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 results by county and size. b) Provide training to PEAs on Indicator 14 changes and the ADE/ESS PSO Survey Application Activity completed 7/1/12 to 6/30/13. 7/1/10– 6/30/13 ADE/ESS Transition Specialists 7/1/10– 6/30/13 ADE/ESS Transition Specialists PEAs in the FFY 2012 PSO Survey participation cohort were identified and targeted for training. Two trainings on Indicator 14 and the enhanced online PSO Survey application were provided at the annual ADE/ESS Directors Institute in August 2012. An additional two trainings on Indicator 14 and the enhanced online PSO Survey application were provided at Arizona’s Twelfth Annual Transition Conference in October 2012. Over 75 people attended at least one of these sessions. c) Analyze PSO training evaluations and survey results to determine effectiveness of trainings Throughout FFY 2012 the following webinar topics were offered to all PEAs including the FFY 2011 and FFY2012 reporting cohort: Getting Ready for the PSO Survey, Essentials of the PSO Survey and Results of the PSO Survey. Activity completed from 7/1/12 through 6/30/13. Review and analysis of PSO training participant evaluations was completed after each training and also at the conclusion of FFY 2012. Participant evaluation forms showed significant increases on self-rating measures of knowledge from pre- to post- training. Using a scale of 1 to 5 (1 = Part B Arizona Annual Performance Report for FFY 2012 131 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 d) Create PSO data reports for participating PEAs to use as a measure for analyzing and improving transition practices 2) Develop, implement, and sustain local community transition teams during Year 2 of the STMP capacity building team training grant a) Provide training to STMP teams on evidence-based practices in developing local community transition teams low and 5 = high), participants reported an average growth of 1.5 points. Activity completed 7/1/12 to 6/30/13. Two webinar trainings that highlighted and discussed FFY 2011 State PSO results and included a demonstration of how to access PEA reports was conducted. PSO data reports for participating PEAs were created and made available to PEAs at both the district and school levels in the online PSO Survey application to enable PEAs to obtain response rates and results by subcategories. Activity completed 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Transition Specialists ADE/IT Specialists 7/1/10– 6/30/13 ADE/ESS Transition Specialists This activity was completed and discontinued because it is integrated into the STMP training program. ADE/ESS, in collaboration with STMP grant coaches from the University of Kansas/Transition Coalition, provided training and created instructional materials designed to facilitate the development of local Community Transition Teams (CTTs) for Year 2 STMP grant participants. The goals included: developing interagency CTTs; working across stakeholders to identify and prioritize community transition needs and challenges to attaining successful post school outcomes; developing protocols for working across stakeholders to increase Part B Arizona Annual Performance Report for FFY 2012 132 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Participate in PSO survey and share results with local community transition teams 3) Provide technical assistance to PEAs on strategies to reach exiters to increase response rate, especially targeting dropouts and individuals from minority groups a) Develop and disseminate flyers and printed materials for use by PEAs to inform students and families and encourage participation in the PSO survey employability and postsecondary participation of students as they leave high school; and improving post school outcome data. An overview of CTTs was provided to Year 1 STMP teams. Activity completed 7/1/11 7/1/10– to 6/30/12. 6/30/13 This activity was completed and discontinued because it is integrated into the STMP training program. STMP teams were provided training, reports, and materials for use with local CTT teams on State and local PSO response rates, representativeness, and outcome data. Activity completed 7/1/11 to 6/30/12. ADE/ESS Transition Specialists STMP Grant Year 2 PEAs 7/1/10– 6/30/13 ADE/ESS Transition Specialists This activity was completed and discontinued. The ESS Web site is updated at least annually. A one-page summary of Arizona’s PSO Survey, an Arizona PSO glossary, and Arizona PSO parent and student (English and Spanish versions) announcement flyers were adapted from NPSO, disseminated via e-mail, and posted on the ADE/ESS Secondary Transition Web site: http://www.azed.gov /special-education /special-projects /secondary-transition /post-school-outcomessurvey/. These support documents were also included on the homepage of the online PSO Survey Part B Arizona Annual Performance Report for FFY 2012 133 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 application. b) Encourage use of the Parent Advocacy Coalition for Educational Rights (PACER)/NPSO– created technical assistance video “Be a Superstar—Take the Survey” YouTube video and provide a link to the video on the ADE/ESS Web site c) Provide session(s) at Arizona’s Annual Transition Conference devoted to increasing participation in the PSO Survey d) Survey PEAs to determine use of strategies Activity completed 7/1/11 to 6/30/12. 7/1/10– 6/30/13 ADE/ESS Transition Specialists 7/1/10– 6/30/13 ADE/ESS Transition Specialists 7/1/10– 6/30/13 ADE/ESS Transition Specialists This activity was completed and discontinued. The ESS Web site is updated at least annually. ADE/ESS transition specialists and STMP trainers highlighted and encouraged the use of the PACER/NPSO–created technical assistance video “Be a Superstar—Take the Survey” YouTube video. A link to the video and the flyers was posted on the ADE/ESS Secondary Transition Web site: http://www.azed.gov /special-education /special-projects /secondary-transition/postschool-outcomes-survey/. Activity completed 7/1/11 to 6/30/12. This activity was completed and discontinued. PSO sessions are included in the conference schedule each year at Arizona’s Annual Transition Conference. Two sessions were offered during Arizona’s Eleventh Annual Transition Conference, and two sessions were planned for Arizona’s Twelfth Annual Transition Conference. This activity was completed and discontinued during FFY 2012. Part B Arizona Annual Performance Report for FFY 2012 134 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 4) Work with the National PostSchool Outcomes (NPSO) Technical Assistance Center as an “intensive state” a) Implement technical assistance received from NPSO 5) Revise Arizona’s online PSO data collection system to include missing data and enable future trend analysis a) Revise PSO online data collection system to include reason for PEA failure to collect survey information b) Revise PSO online data collection system to allow for the exploration of additional data related to nonengaged youth 6) Increase the number of youth who respond to the post school outcomes survey from 61% (FFY 2010) to 67.5% a) Design and implement a marketing plan to target increased participation by male and ethnic minority youth and young adults b) Enhance the online PSO Survey application to filter and group data by size of PEA and county Activity completed 7/1/12 to 6/30/13. ADE/ESS has taken steps to implement technical assistance received from NPSO as evidenced by enhancements made to the online PSO Survey application, updated trainings provided at conferences and via webinars to local PEAs, and incorporation of NPSO data collection and analysis tools. Activity completed 7/1/11 to 6/30/12. A review of the enhancements made to the PSO online data collection system showed that no additional changes were required related to PEA failure to collect survey information at this time. This activity is discontinued. Activities completed 7/1/12 to 6/30/13. This activity is completed and discontinued, a tool was developed and implemented for nonengagement analysis. Activities completed 1/1/12 to 6/30/12. This activity was completed and discontinued, the goal was exceeded, as the response rate for FFY 2011 was 70%, an increase of 2.5 percentage points above the target of 67.5%. This activity was completed and discontinued 6/30/2013 ADE/ESS is now able to filter and group data by size of PEA and County. 5/1/11– 6/30/13 ADE/ESS Transition Specialists NPSO Technical Assistance Center 7/1/10– 6/30/13 ADE/ESS Transition Specialists ADE/IT Specialists 7/1/10– 6/30/13 ADE/ESS Transition Specialists ADE/IT Specialists 1/1/12– 6/30/12 ADE/ESS Transition Specialists Parent Information Network Specialists 11/1/11– 6/30/12 ADE/ESS Transition Specialists ADE/IT Specialists Part B Arizona Annual Performance Report for FFY 2012 135 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 c) Use the NPSO response calculator to track responses during the data collection to monitor response rates 7) Increase the rate of engagement for students with a disability category of mental retardation from 47% (FFY 2011 baseline) to 48% a) Offer 3 regional trainings a year specifically designed to increase awareness of available options for postsecondary education, training, and employment for students with intellectual disabilities Activity completed 7/1/11 to 6/30/12. 7/1/12– 9/30/12 ADE/ESS Transition Specialists The Arizona PSO Online application was enhanced to enable it to track response rates. Activities completed 1/1/12 to 6/30/12. 1/1/12– 6/30/13 ADE/ESS Transition Specialists This improvement goal is revised to reflect the new FFY 2011 baseline data. Three regional trainings cofacilitated by ADE/ESS and the Division of Developmental Disabilities (DDD) were held highlighting local options for postsecondary engagement for students with intellectual disabilities. Review and analysis of evaluations from these trainings indicated an increase in knowledge pre/post-training regarding post school options for students with intellectual disabilities served through DDD. AZ Community of Practice on Transition (AZCoPT) Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 136 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 15: Effective General Supervision General supervision system (including monitoring, complaints, hearings, etc.) identifies and corrects noncompliance as soon as possible but in no case later than one year from identification. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent of noncompliance corrected within one year of identification: a. # of findings of noncompliance. b. # of corrections completed as soon as possible but in no case later than one year from identification. Percent = [(b) divided by (a)] times 100. States are required to use the “Indicator 15 Worksheet” to report data for this indicator. Target Data for FFY 2012 FFY Measurable and Rigorous Target FFY 2012 100% Actual Target Data for FFY 2012 89% Arizona did not meet the target. Process to Select PEAs for Monitoring The ADE/ESS conducts compliance monitoring for IDEA procedural requirements. Arizona uses a sixyear cycle for monitoring with assigned activities always occurring in Year 4 of the cycle. The ADE/ESS directors and program specialists review each PEA’s data annually. The ADE/ESS can adjust a PEA’s monitoring schedule, and Year 4 monitoring activities can occur any time systemic issues arise related to the review of data. The data for FFY 2012 included the PEAs in Year 5 of the cycle; that is, data included those PEAs that were on a Corrective Action Plan to correct findings of noncompliance identified as a result of their monitoring activities during FFY 2011. Part B Arizona Annual Performance Report for FFY 2012 137 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 The PEAs monitored each year represent a regional balance across the State. The monitoring cycle year has a mix of elementary, unified, and union high school districts, charter schools, and other public agencies such as secure care and accommodation districts. Definition of Finding for Monitoring for FFY 2011 During FFY 2011, a finding was issued when any line item was less than 100% compliant based upon the review of the components within the line item. The finding was a written notification to the PEA by the State that the line item(s) was noncompliant, and each finding included a description of a Federal or State statute or regulation. Explanation of Progress That Occurred for FFY 2012 Although Arizona did not meet the target, there was progress from FFY 2011 (88%) to FFY 2012 (89%). Four PEAs extended past the one-year corrective action timeline in FFY 2012 compared to nine PEAs in FFY 2011. The ADE/ESS continued to focus on the correction of noncompliance within one year. ADE/ESS program specialists provided support and technical assistance to each PEA in corrective action and added increased support to those PEAs that were not meeting timelines. The ADE/ESS provided staff development for ESS program specialists to increase understanding of the validity of compliance calls and to ensure interrater reliability. ESS mandates attendance by the specialists at a three-day summer monitoring training (six days for newly employed specialists). Follow-up meetings and workshops are scheduled throughout the school year. This staff development was planned and presented by the Monitoring Team, a group composed of the monitoring director and veteran specialists. The ADE/ESS mentoring program is another type of staff development for new ESS program specialists. Mentors were members of the Monitoring Team. New specialists shadowed the mentors on monitorings, CAP follow-up visits, technical assistance visits, and periodic meetings with PEAs. The specialists and mentors also communicated about issues that arose from regular interactions with the assigned PEA staff. The mentoring was maintained for up to one year. The director reviewed the CAPs on a weekly basis to check each PEA’s progress and scheduled followup visits and desk audits by the specialist. The director communicated with the assigned specialist for detailed updates if the CAP closeout was not progressing at a reasonable pace. Additionally, the monitoring director sent a monthly CAP Progression Report to the specialists and ESS directors that identified timelines toward the one-year closeout for each open monitoring. This alerted specialists to the remaining days for one-year closeouts. The director asked specialists to respond if difficulties existed that were impeding timely closeout. Strategies were then identified to assist the PEAs to close out the monitoring within the one-year timeline. The Monitoring Team played a crucial role in making progress with this Indicator. This established group, guided by the monitoring director, is a team of experienced ESS specialists that met monthly, at a minimum, to evaluate the monitoring process and system. In addition, the Monitoring Team members carried out their regular duties throughout the year. That is, they reviewed and revised the monitoring manual, which included forms and guide steps (an annual project); planned continuing support for program specialists; and designed resources and support materials for PEA administrators, evaluators, and teachers. Part B Arizona Annual Performance Report for FFY 2012 138 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Develop and conduct webinars pertaining to the requirements for compliant evaluations and IEPs Sub-Activities (Objectives or Action Steps) a) Develop webinar trainings for evaluation and IEP requirements b) Conduct statewide webinars for evaluation and IEP requirements 2) Improve the general supervision system of PEAs by enhancing internal staff development Timeline Complete Activity completed 6/30/11. Monitoring director and monitoring team created webinars to train PEAs about evaluation process and timelines. Webinars were reviewed and revised and presented statewide on November 15, th th 19 and December 6 . Projected Resources (Planned) 12/1/10– 6/30/11 ADE/ESS Directors and Specialists 7/1/12 – 12/31/12 ADE/ESS Directors and Specialists c) Collect and analyze training feedback from participants Feedback surveys were emailed to participants directly concluding the webinar, via Go-To Meeting. Analyzing the outcomes was completed December 2012 1/1/12– 4/30/12 ADE/ESS Directors and Specialists d) Collect corrective action close-out (timeline) data for evaluation and IEP monitoring line items a) Review and revise, if necessary, the ADE/ESS mentoring system for ESS monitoring specialists This activity was completed by June 2013 for FFY 2012 APR updates. 5/1/13 – 6/30/13 ADE/ESS Directors and Specialists Activity completed from 7/1/12 to 6/30/13 The manual for the ESS mentoring system was updated. 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team b) Implement the ESS mentoring system for the monitoring specialists, based on demand and need c) Develop three-day summer monitoring training each year for ESS monitoring specialists This activity is completed. Activity completed from 7/1/12 to 6/30/13 The revised ESS mentoring system was implemented for all new ESS specialists who monitor. This activity is completed. Activity completed by. 6/30/13 The director of program support and the monitoring team (M Team) developed the three-day summer Part B Arizona Annual Performance Report for FFY 2012 139 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 monitoring training. d) Implement three-day summer monitoring training each year for ESS monitoring specialists e) Provide follow-up staff development for ESS monitoring specialists periodically throughout the year 3) Conduct Dispute Resolution presentations for PEAs and parent groups a) Review and revise presentation, support materials, and resources 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team 7/1/10– 6/30/13 ADE/ESS Director of Program Support and Monitoring Team 7/1/11– 6/30/13 ADE/ESS Dispute Resolution Director and Dispute Resolution Coordinator ADE/ESS Dispute Resolution Director and Complaint Investigator Monitoring training was provided on July 18, 19 and 26 2012 to all ESS specialists who monitor. This activity is completed. Activities completed from. 7/1/12 to 6/30/13 Follow-up trainings and electronic communications were provided to all ESS specialists who monitor. This activity is completed. Activity completed as of July 2012 a) Review dispute resolution database Materials on dispute resolution were reviewed, revised, and widely disseminated statewide. Activity completed 7/1/12 to 6/30/13 Dispute resolution director and staff trained statewide at various conferences and workshops, including Directors Institute and Principals Institute. Activity completed 7/1/11 through 6/30/12. b) Reconstruct dispute resolution database Corrective Action Compliance Monitor assessed the efficacy of the previously utilized database and targeted its deficiencies. Activity completed as of 9/30/12. b) Conduct statewide presentations at various regional and statewide venues 4) Develop dispute resolution database to improve collection, maintenance, and reporting of data This activity is completed. Activity completed by. 6/30/13 Corrective Action Compliance Monitor created a new database utilizing Microsoft Access to track and aggregate all corrective action data. 7/1/11– 6/30/13 7/1/11– 6/30/12 ADE/ESS Dispute Resolution Director and Corrective Action Compliance Monitor 7/1/12– 12/31/12 ADE/ESS Dispute Resolution Director and Corrective Action Compliance Monitor Part B Arizona Annual Performance Report for FFY 2012 140 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 c) Test phase  alpha-test to debug  beta-test to ensure ease of use d) Fully implement reconstructed dispute resolution database Activity completed 3/31/13 Database was fully tested initially with one year’s corrective action data to ensure it would provide the needed platform for storage and monitoring of corrective action data. Reporting features were tested with a variety of data aggregations. Activity Completed 4/1/13 1/1/13– 6/30/13 Dispute Resolution Team 7/1/13 Dispute Resolution Team The database has been fully implemented with corrective action data and has additionally been used to store and aggregate data regarding administrative complaint issues, as well as issues according to LEA and student disability categories. DB has made corrective action monitoring more efficient and has allowed for readily available data reporting on administrative complaints. Timely Correction of FFY 2011 Findings of Noncompliance (corrected within one year from identification of the noncompliance) 1. Number of findings of noncompliance the State made during FFY 2011 (the period from July 1, 2011, through June 30, 2012) (Sum of Column a on the Indicator B15 Worksheet) 520 2. Number of findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) (Sum of Column b on the Indicator B15 Worksheet) 463 3. Number of findings not verified as corrected within one year [(1) minus (2)] 57 FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance and/or Not Corrected) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 57 Part B Arizona Annual Performance Report for FFY 2012 141 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 5. Number of FFY 2011 findings the State has verified as corrected beyond the one-year timeline (“subsequent correction”) 57 6. Number of FFY 2011 findings not yet verified as corrected [(4) minus (5)] 0 Verification of Correction for All Findings of Noncompliance Identified in FFY 2011 (either timely or subsequent) As specified in OSEP’s July 2013 Arizona Part B FFY 2011 SPP/APR Response Table, Arizona verified that each PEA with findings of noncompliance: 1) is correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) based on a review of updated data such as data subsequently collected through on-site monitoring; and 2) has corrected each individual case of noncompliance, unless the child is no longer within the jurisdiction of the PEA, consistent with OSEP Memorandum 09-02, dated October 17, 2008. Specific Actions Taken to Verify the Correction of Findings of Noncompliance Identified in FFY 2011 (including any revisions to general supervision procedures, technical assistance provided, and/or any enforcement actions taken) Specific Actions for Correction of FFY 2011 Findings of Noncompliance for Indicator 4B Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 0.17% 1. Number of findings of noncompliance the State made during FFY 2011 (the period from July 1, 2011, through June 30, 2012) 1 2. Number of FFY 2011 findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) 1 3. Number of FFY 2011 findings not verified as corrected within one year [(1) minus (2)] 0 Correction of FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 0 5. Number of FFY 2011 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 0 6. Number of FFY 2011 findings not verified as corrected [(4) minus (5)] 0 Part B Arizona Annual Performance Report for FFY 2012 142 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 All findings of noncompliance from FFY 2011 related to Indicator 4B have been corrected and verified. Upon receipt of the finding of noncompliance, the PEA revised its discipline policies and procedures and corrected all noncompliance. The ADE/ESS program specialist verified that the PEA corrected all instances of noncompliance, including child specific. Based on subsequent file reviews of updated data, the specialist determined that the PEA implemented sustainable practices to meet the regulatory requirements. There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Specific Actions for Correction of FFY 2011 Findings of Noncompliance for Indicator 11 Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 97% 1. Number of findings of noncompliance the State made during FFY 2011 (the period from July 1, 2011, through June 30, 2012) 32 2. Number of FFY 2011 findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) 32 3. Number of FFY 2011 findings not verified as corrected within one year [(1) minus (2)] 0 Correction of FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 0 5. Number of FFY 2011 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 0 6. Number of FFY 2011 findings not verified as corrected [(4) minus (5)] 0 All findings of noncompliance from FFY 2011 related to Indicator 11 have been corrected and verified. The ADE/ESS specialists reviewed the child specific files from the monitorings to determine whether the PEAs completed the evaluation for any child whose initial evaluation was not timely, unless the child was no longer within the PEA’s jurisdiction. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine that the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to the evaluation process in conformity with 34 CFR § 300.301 (c) (1). There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Specific Actions for Correction of FFY 2011 Findings of Noncompliance for Indicator 12 Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 99% 1. Number of findings of noncompliance the State made during FFY 2011 (the 7 Part B Arizona Annual Performance Report for FFY 2012 143 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 period from July 1, 2011, through June 30, 2012) 2. Number of FFY 2011 findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) 7 3. Number of FFY 2011 findings not verified as corrected within one year [(1) minus (2)] 0 Correction of FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 0 5. Number of FFY 2011 findings the State has verified as corrected beyond the oneyear timeline (“subsequent correction”) 0 6. Number of FFY 2011 findings not verified as corrected [(4) minus (5)] 0 The PEAs submitted to ECSE the policies and procedures for early intervention transitions that were mutually agreed upon with the AzEIP service coordinators. The ADE/ECSE specialists reviewed the child specific files from the PEAs to determine whether the IEPs were developed and implemented, although late, unless the child was no longer within the PEA’s jurisdiction. The ADE/ESS specialists conducted follow-up on-site visits and/or desk audits and reviewed updated data based on subsequent student files to verify that each PEA was correctly implementing the regulatory requirements (i.e., achieved 100% compliance) in conformity with 34 CFR § 300.124 (b). There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Specific Actions for Correction of FFY 2011 Findings of Noncompliance for Indicator 13 Level of compliance (actual target data) State reported for FFY 2011 for this indicator: 78% 1. Number of findings of noncompliance the State made during FFY 2011 (the period from July 1, 2011, through June 30, 2012) 99 2. Number of FFY 2011 findings the State verified as timely corrected (corrected within one year from the date of notification to the PEA of the finding) 77 3. Number of FFY 2011 findings not verified as corrected within one year [(1) minus (2)] 22 Correction of FFY 2011 Findings of Noncompliance Not Timely Corrected (corrected more than one year from identification of the noncompliance) 4. Number of FFY 2011 findings not timely corrected (same as the number from (3) above) 22 5. Number of FFY 2011 findings the State has verified as corrected beyond the one- 22 Part B Arizona Annual Performance Report for FFY 2012 144 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 year timeline (“subsequent correction”) 6. Number of FFY 2011 findings not verified as corrected [(4) minus (5)] 0 All findings of noncompliance from FFY 2011 related to Indicator 13 have been corrected and verified. The ADE/ESS specialists reviewed the child specific files from the monitorings to determine whether the PEAs developed and implemented IEPs that included the secondary transition regulatory requirements for any child aged 16 and above, unless the child was no longer within the PEA’s jurisdiction. The ESS specialists reviewed updated data from subsequent files during follow-up visits to determine whether the PEAs were correctly implementing the specific regulatory requirements (i.e., achieved 100% compliance) related to secondary transition in conformity with IDEA requirements. There were no revisions to general supervision procedures, technical assistance, or enforcement actions. Specific Actions for Additional Related Requirements Identified Through the Monitoring System (Indicators 4, 9, 10, 11, and 13) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2011, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data were as follows. There were no revisions to general supervision procedures, technical assistance, or enforcement actions.  ADE/ESS specialists reviewed student files during follow-up on-site visits and/or desk audits to verify correction of all instances of noncompliance, including child specific, and to ensure that regulatory requirements were being implemented.  ADE/ESS specialists reviewed updated data from subsequent files and conducted interviews with special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including child specific, were corrected and to ensure ongoing sustainability with the implementation of the regulatory requirements.  ADE/ESS specialists reviewed the drill down analyses completed by the PEAs in targeted areas of both compliance and results SPP/APR Indicators to determine if PEAs had conducted genuine and thorough examinations of root causes. The analyses resulted in action plans to address systemic issues and to ensure sustainability of compliance. Specific Actions for Dispute Resolution (Indicators 18 and 19) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2011, including child specific noncompliance, and were correctly implementing the regulatory requirements were as follows. The review of data did not identify systemic noncompliance; therefore, the correction was at the student level. There were no revisions to general supervision procedures, technical assistance, or enforcement actions.  The Corrective Action Compliance Monitor (CACM) maintained a database of all corrective actions and tracked timelines to ensure timely correction of noncompliance.  As direct follow-up to a child specific finding of noncompliance, the CACM reviewed the student file, generally via desk audit but occasionally via site visit, to verify correction of any instance of noncompliance, including child specific, and to ensure that regulatory requirements were being implemented. Part B Arizona Annual Performance Report for FFY 2012 145 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012  The CACM reviewed the corrective action plan documentation via desk audit to ensure that all instances of noncompliance were corrected. The corrective action plan documentation may include such actions as a written action plan, professional development, and/or a letter of assurance. Each PEA submitted documentation evidencing correction of all the noncompliance.  The CACM approved all corrective action plan documentation via desk audit to verify that the PEAs corrected all instances of noncompliance, including child specific, and were adhering to the regulatory requirements. Specific Actions for Additional Related Requirements for Early Childhood Transitions (Indicator 12) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2011, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data, were as follows. There were no revisions to general supervision procedures, technical assistance, or enforcement actions.  The ADE/ECSE specialists reviewed the written process and procedures for the PEAs’ early intervention transitions.  The ADE/ESS specialists and/or ADE/ECSE specialists reviewed student files during subsequent on-site visits and/or desk audits of updated data to determine if the PEAs corrected all instances of noncompliance, including child specific, and to ensure ongoing sustainability with the implementation of the regulatory requirements. Specific Actions for Other Related Requirements Pertaining to Graduation, Dropout, Assessment, School Age and Preschool LRE, Preschool Outcomes, Parent Involvement, and Post School Outcomes (Indicators 1, 2, 3, 5, 6, 7, 8, and 14) The specific methods Arizona used to verify that PEAs corrected all instances of noncompliance identified in FFY 2011, including child specific noncompliance, and were correctly implementing the regulatory requirements, based on subsequent file reviews of updated data were as follows. There were no revisions to general supervision procedures, technical assistance, or enforcement actions.  ADE/ESS specialists reviewed student files during follow-up on-site visits and/or desk audits to verify correction of all instances of noncompliance, including child specific, and to ensure that regulatory requirements were being implemented.  ADE/ESS specialists reviewed updated data from subsequent files and conducted interviews with the special education administrators during follow-up visits and/or desk audits to determine if all instances of noncompliance, including child specific, were corrected and to ensure ongoing sustainability with the implementation of the regulatory requirements.  ADE/ESS specialists reviewed the drill down analyses completed by the PEAs in targeted areas of both compliance and results SPP/APR Indicators to determine if the PEAs had conducted genuine and thorough examinations of root causes. The analyses resulted in action plans to address systemic issues and to ensure sustainability of compliance. Actions Taken if Noncompliance Not Corrected Part B Arizona Annual Performance Report for FFY 2012 146 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 When findings of noncompliance are made, Arizona works collaboratively with the PEAs as the agencies identify the root causes of their continuing noncompliance through drill downs in the specific focus area that affects this Indicator. However, when noncompliance is not corrected within one year, the ADE/ESS uses a variety of methods to ensure that all public agencies meet the requirements of State and Federal statutes related to special education. The enforcement actions taken by ESS for the PEAs that are unable to demonstrate compliance within one year from the date of written notification are as follows:  Enforcement of corrective action plan (CAP) activities as outlined in the current agency CAP, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Review and revision of the current CAP to develop targeted activities that address the continued noncompliance, which may include the diversion of IDEA funds to cover any costs associated with those activities.  Assignment of a special monitor.  Interruption of IDEA payments until adequate compliance is achieved. For charter schools not receiving IDEA funds, a request to begin withholding 10% of State funds.  For charter schools, a request to the appropriate board for a notice of intent to revoke the charter.  With Arizona State Board of Education approval, interruption of Group B weighted State aid or redirection of funds pursuant to 34 CFR § 300.227 (a).  Referral to the Arizona Attorney General for legal action. Although these enforcement actions are in place, the ESS monitoring system is designed to work with the PEAs to correct the findings of noncompliance as soon as possible but no later than one year. ESS directors and specialists meet periodically throughout the year to discuss and better understand the PEAs’ data pertaining to the APR indicators, student population, and other data elements. This information is used to make decisions about the type of monitoring and need for technical assistance for each PEA. The ESS specialists use the same data during their on-site visits to the education agencies to address concerns and offer resources. Another form of technical assistance offered by ESS is the annual site visit by the ESS specialist. These site visits are designed to assist PEAs with understanding IDEA regulatory requirements through technical assistance provided as a result of student file reviews. Information from these visits is logged in a database to track the need for additional technical assistance. The ESS secondary transition specialists use the file review results related to the components of Indicator 13 to target their trainings and grants. The monitoring director, facilitator, and Monitoring Team develop the monitoring manual and train the specialists throughout the year to ensure interrater reliability for compliance calls according to regulatory requirements. The ADE/ESS staff conducts on-site and regional trainings for the PEAs to enhance understanding of compliance and the monitoring process. A major component of this is the Corrective Action Plan follow-up, which includes a strict schedule after a monitoring to ensure timely correction of noncompliance. The Early Childhood Special Education specialists also offer targeted technical assistance to districts that are not in compliance through individual trainings, monthly audits, and consultations. In addition, ESS specialists review files of preschool students during the annual site visits to provide assistance and resources when needed. Correction of Remaining FFY 2010 Findings of Noncompliance Not applicable. Part B Arizona Annual Performance Report for FFY 2012 147 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Although Arizona reported < 100% for this Indicator in the FFY 2011 APR, all remaining FFY 2010 findings were subsequently corrected. 1. Number of remaining FFY 2010 findings noted in OSEP’s FFY 2011 APR response table for this indicator 0 2. Number of remaining FFY 2010 findings the State has verified as corrected 0 3. Number of remaining FFY 2010 findings the State has NOT verified as corrected [(1) minus (2)] 0 Correction of Any Remaining Findings of Noncompliance Identified FFY 2009 or Earlier Not applicable. Additional Information Required by the OSEP APR Response Table for This Indicator Statement from the Response Table State’s Response In reporting on correction of findings of noncompliance in the FFY 2012 APR, the State must report that it verified that each PEA with noncompliance identified in FFY 2011 is (1) correctly implementing the specific regulatory requirements; and (2) has corrected each individual case of noncompliance, consistent with OSEP Memo 09-02. In the FFY 2012 APR, the State must describe the specific actions that were taken to verify the correction. In the FFY 2012 APR for Indicators 4B, 11, 12, 13, and 15, Arizona reported that it verified each PEA with noncompliance (FFY 2011 data), consistent with OSEP Memorandum 09-02, dated October 17, 2008. Arizona described the specific actions that were taken to verify the correction of noncompliance. In addition, in reporting on Indicator 15 in the FFY 2012 APR, the State must use the Indicator 15 Worksheet. In the FFY 2012 APR, Arizona uses the Indicator 15 Worksheet to report on Indicator 15. Further, in responding to Indicators 4B, 11, 12, and 13, in the FFY 2012 APR, the State must report on correction of the noncompliance described in this table under those indicators. In the FFY 2012 APR, Arizona reports on the correction of noncompliance for Indicator 4B within the Indicator 4B section and within Indicator 15, reports on the correction of noncompliance for Indicator 11 within the Indicator 11 section and within Indicator 15, reports on the correction of noncompliance for Indicator 12 within the Indicator 12 section and within Indicator 15, and reports on the correction of noncompliance for Indicator 13 within the Indicator 13 section and within Indicator 15. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 148 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 INDICATOR 15 WORKSHEET Indicator/Indicator Clusters 1. Percent of youth with IEPs graduating from high school with a regular diploma. 2. Percent of youth with IEPs dropping out of high school. 14. Percent of youth who had IEPs, are no longer in secondary school and who have been competitively employed, enrolled in some type of postsecondary school, or both, within one year of leaving high school. 3. Participation and performance of children with disabilities on statewide assessments. 7. Percent of preschool children with IEPs who demonstrated improved # of PEAs Issued findings in FFY 2011 (7/1/11 to 6/30/12) (a) # of findings (individual student files) of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings Monitoring System: 20 Monitoring System: 25 Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other General Supervision System Components (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Monitoring System: 21 # of findings of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) for which correction was verified later than one year from identification 1 corrected and verified within 15 months from identification 1 corrected and verified within 17 months from identification 2 corrected and verified within 18 months from identification Complaints: 0 Complaints: 0 Complaints: 0 Due process: 0 Due process: 0 Due process: 0 Monitoring System: 32 Monitoring System: 32 Monitoring System: 28 1 corrected and verified within 14 months from identification 1 corrected and verified within 15 months from identification 1 corrected and verified within 17 months from identification 1 corrected and verified within 18 months from identification Part B Arizona Annual Performance Report for FFY 2012 149 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator/Indicator Clusters outcomes. 4A. Percent of districts identified as having a significant discrepancy in the rates of suspensions and expulsions of children with disabilities for greater than 10 days in a school year. 4B. Percent of districts that have: (a) a significant discrepancy, by race or ethnicity, in the rate of suspensions and expulsions of greater than 10 days in a school year for children with IEPs; and (b) policies, procedures or practices that contribute to the significant discrepancy and do not comply with requirements relating to the development and implementation of IEPs, the use of positive behavioral interventions and supports, and procedural safeguards. General Supervision System Components Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings # of PEAs Issued findings in FFY 2011 (7/1/11 to 6/30/12) (a) # of findings (individual student files) of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) Complaints: 0 Complaints: 0 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 6 Monitoring System: 6 Monitoring System: 5 APR: 1 APR: 1 APR: 1 Complaints: 2 Complaints: 2 Complaints: 2 Due Process: 0 Due Process: 0 Due Process: 0 # of findings of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) for which correction was verified later than one year from identification 1 corrected and verified within 17 months from identification (corrected from monitoring outside of APR) Part B Arizona Annual Performance Report for FFY 2012 150 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator/Indicator Clusters 5. Percent of children with IEPs aged 6 through 21 educational placements. 6. Percent of preschool children aged 3 through 5 – early childhood placement. 8. Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities. General Supervision System Components Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings # of PEAs Issued findings in FFY 2011 (7/1/11 to 6/30/12) (a) # of findings (individual student files) of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) Monitoring System: 33 Monitoring System: 54 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Monitoring System: 48 # of findings of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) for which correction was verified later than one year from identification 2 corrected and verified within 15 months from identification 2 corrected and verified within 17 months from identification 2 corrected and verified within 18 months from identification Complaints: 23 Complaints: 55 Complaints: 55 Due Process: 2 Due Process: 4 Due Process: 4 Monitoring System: 32 Monitoring System: 71 Monitoring System: 63 2 corrected and verified within 15 months from identification 3 corrected and verified within 17 months from identification 3 corrected and verified within 18 months from identification Complaints: 13 Complaints: 19 Complaints: 19 Due Process: 0 Due Process: 0 Due Process: 0 Part B Arizona Annual Performance Report for FFY 2012 151 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator/Indicator Clusters 9. Percent of districts with disproportionate representation of racial and ethnic groups in special education that is the result of inappropriate identification. 10. Percent of districts with disproportionate representation of racial and ethnic groups in specific disability categories that is the result of inappropriate identification. General Supervision System Components Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other # of PEAs Issued findings in FFY 2011 (7/1/11 to 6/30/12) (a) # of findings (individual student files) of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) Monitoring System: 30 Monitoring System: 30 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Monitoring System: 26 APR: 0 APR: 0 APR: 0 # of findings of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) for which correction was verified later than one year from identification 1 corrected and verified within 14 months from identification 1 corrected and verified within 15 months from identification 1 corrected and verified within 17 months from identification 1 corrected and verified within 18 months from identification Part B Arizona Annual Performance Report for FFY 2012 152 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator/Indicator Clusters 11. Percent of children who were evaluated within 60 days of receiving parental consent for initial evaluation or, if the State establishes a timeframe within which the evaluation must be conducted, within that timeframe. 12. Percent of children referred by Part C prior to age 3, who are found eligible for Part B, and who have an IEP developed and implemented by their third birthdays. General Supervision System Components Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings Monitoring Activities: Data Collected by Census, SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other # of PEAs Issued findings in FFY 2011 (7/1/11 to 6/30/12) (a) # of findings (individual student files) of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) Complaints: 0 Complaints: 0 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 29 Monitoring System: 93 Monitoring System: 85 # of findings of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) for which correction was verified later than one year from identification 2 corrected and verified within 14 months from identification 2 corrected and verified within 15 months from identification 2 corrected and verified within 17 months from identification 2 corrected and verified within 18 months from identification Complaints: 15 Complaints: 22 Complaints: 22 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 0 Monitoring System: 0 Monitoring System: 0 Indicator 12 Census (early childhood transition): 7 Indicator 12 Census (early childhood transition): 7 Indicator 12 Census (early childhood transition): 7 Part B Arizona Annual Performance Report for FFY 2012 153 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator/Indicator Clusters 13. Percent of youth aged 16 and above with IEP that includes coordinated, measurable, annual IEP goals and transition services that will reasonably enable student to meet the post-secondary goals. Other areas of noncompliance: General Supervision System Components Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings Monitoring Activities: SelfAssessment/ Local APR, Data Review, Desk Audit, On-Site Visits, or Other Dispute Resolution: Complaints, Hearings # of PEAs Issued findings in FFY 2011 (7/1/11 to 6/30/12) (a) # of findings (individual student files) of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) Complaints: 0 Complaints: 0 (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 18 Monitoring System: 99 Monitoring System: 77 # of findings of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) for which correction was verified later than one year from identification 17 corrected and verified within 17 months from identification 5 corrected and verified within 18 months from identification Complaints: 0 Complaints: 0 Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Monitoring System: 0 Monitoring System: 0 Monitoring System: 0 Complaints: 0 Complaints: 0 Complaints: 0 Due Process: 0 Due Process: 0 Due Process: 0 Part B Arizona Annual Performance Report for FFY 2012 154 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Indicator/Indicator Clusters General Supervision System Components # of PEAs Issued findings in FFY 2011 (7/1/11 to 6/30/12) Sum the numbers down Column a and Column b Percent of noncompliance corrected within one year of identification = (column (b) sum divided by column (a) sum) times 100. (b) / (a) X 100 = % (a) # of findings (individual student files) of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) (b) # of findings (individual student files) of noncompliance from (a) for which correction was verified no later than one year from identification # of findings of noncompliance identified in FFY 2011 (7/1/11 to 6/30/12) for which correction was verified later than one year from identification 520 463 57 463 / 520 = 0.8903 X 100 = 89% Part B Arizona Annual Performance Report for FFY 2012 155 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 18: Resolution Session Effectiveness Percent of hearing requests that went to resolution sessions that were resolved through resolution session settlement agreements. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent = (3.1(a) divided by 3.1) times 100. Target Data for FFY 2012 FFY Measurable and Rigorous Target 2012 76% Actual Target Data for FFY 2012 number of hearing requests that went to resolution sessions that were resolved through resolution session settlement agreements number of hearing requests that went to resolution sessions 15 31 Actual Target Data for FFY 2012 48% (3.1 (a)  3.1)  100 = X 15  31 = 0.4838  100 = 48% Arizona did not meet the target. Data Data Source Data are the same as the data submitted under section 618, Table 7, Report of Dispute Resolution, under Part B of the Individuals with Disabilities Education Act. Valid and Reliable Data The ADE/ESS collects and maintains the dispute resolution data in its internal database and assures the accuracy, reliability, and validity of the data. The dispute resolution data are the same as the data reported under section 618, Dispute Resolution, Table 7. Arizona uses a Resolution Session Tracking Form, which is sent to each PEA, along with the Notice of Hearing, when a due process complaint is filed. Part B Arizona Annual Performance Report for FFY 2012 156 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 This comprehensive form, which must be submitted to the ADE/ESS and the assigned administrative law judge, provides, among other things, information about the date and outcome of each resolution session held. Explanation of Progress That Occurred for FFY 2012 During FFY 2012, 31 resolution sessions were held pursuant to due process hearing requests, with 15 matters (48%) resulting in resolution agreements. This result reflects progress from FFY 2011 (45%). Of the 31 resolution sessions held: 15 resulted in resolution agreements; one matter resulted in a fully adjudicated hearing; two matters were pending a due process hearing as of June 30, 2013; two matters were withdrawn by the complaining party; and nine matters resulted in settlement agreements outside of the resolution session process. It is noteworthy that of the 81 due process complaints filed:  1 resulted in a fully adjudicated hearing;  9 were pending as of June 30, 2013;  15 were resolved through a formal resolution agreement;  13 were resolved via mediation agreement;  26 were resolved via private settlement; and  17 were dismissed or withdrawn. Thus, although Arizona did not meet its target of 76% on this Indicator, of the 71 matters that were resolved without a hearing by or before June 30, 2013, 54 or 76% were resolved by resolution agreement, mediation agreement, or a private settlement. This indicates that, overall, the State’s due process system is successful in resolving due process complaints without the need for a fully adjudicated due process hearing. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Train PEAs on resolution session effectiveness 2) Conduct dispute resolution presentations for PEAs and parent groups Sub-Activities (Objectives or Action Steps) a) Identify qualified trainer Activity completed as of April 2011. 10/1/10– 5/1/11 b) Provide training at a statewide conference A qualified trainer was identified to present at the next Directors Institute. Activity completed as of 6/30/12. ADE/ESS Dispute Resolution Director and Coordinator 5/1/11– 6/30/12 Two sessions on leading an effective resolution session were offered at the 2011 Directors Institute. Activity completed as of July 2013. ADE/ESS Dispute Resolution Director and Coordinator 7/1/11– 6/30/13 ADE/ESS Dispute Resolution Director and Coordinator a) Review and revise presentation, support materials, and resources Timeline Complete Materials on dispute resolution were reviewed, revised, and widely Projected Resources (Planned) Part B Arizona Annual Performance Report for FFY 2012 157 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 disseminated statewide. b) Conduct statewide presentations at various regional and statewide venues 3) Review and revise dispute resolution brochure a) Revise and update brochure b) Disseminate brochure statewide and post on ADE/ESS Web site Activity completed 7/1/12 to 6/30/13. 7/1/11– 6/30/13 Dispute resolution director and staff trained statewide at various conferences and workshops, including Directors Institute and Principals Institute. Activity completed as of July 2011. ADE/ESS Dispute Resolution Director and Complaint Investigator 7/1/11– 6/30/13 Dispute resolution brochure was revised and updated and is currently in circulation. Activity completed 7/1/12 to 6/30/13. ADE/ESS Dispute Resolution Director and Coordinator 7/1/11– 6/30/13 ADE/ESS Dispute Resolution Director and Coordinator Revised and updated brochure was disseminated at local and statewide conferences and via the Parent Information Network specialists. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 158 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 19: Mediation Agreements Percent of mediations held that resulted in mediation agreements. (20 U.S.C. 1416 (a)(3)(B)) Measurement Percent = [(2.1(a)(i) + 2.1(b)(i)) divided by 2.1] times 100. Target Data for FFY 2012 FFY Measurable and Rigorous Target 2012 85.5% Actual Target Data for FFY 2012 number of mediations held that resulted in mediation agreements number of mediations 32 37 Actual Target Data for FFY 2012 86% (2.1 (a) (i) + 2.1 (b) (i))  (2.1)  100 = X 13 + 19  37 = 0.8648  100 = 86% Arizona met the target. Data Data Source Data are the same as the data submitted under section 618, Table 7, Report of Dispute Resolution under Part B of the Individuals with Disabilities Education Act. Valid and Reliable Data The ADE/ESS collects and maintains the dispute resolution data in its internal database and assures the accuracy, reliability, and validity of the data. The dispute resolution data are the same as the data reported under section 618, Dispute Resolution, Table 7. Explanation of Progress That Occurred for FFY 2012 Part B Arizona Annual Performance Report for FFY 2012 159 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Arizona experienced progress as compared to FFY 2011 (83%), and exceeded its target of 85.5% for FFY 2012. The progress of 3 percentage points is attributed to the skill level of the mediators and various statewide trainings that encourage the use of mediation as a means to resolve special education-related disputes. Arizona maintains a list of independent contractors to serve as mediators. The State has maintained the same mediators for many years and they are becoming more experienced in the area of special education mediation. The mediators are required to complete a 40-hour course in mediation, have 20 hours of hands-on mediation experience, and have a background in education. As part of their ongoing training, mediators continue to have the opportunity to participate in quarterly mediator conference calls through the Mountain Plains Regional Resource Center and attend the ADE’s annual Directors Institute, which includes a full-day private training specifically tailored for state administrative complaint investigators, administrative law judges, and mediators. Information about Arizona’s mediation system is disseminated to PEAs through trainings and conferences and upon request. Additionally, the Deputy Director of Legal Services works with Arizona’s PTI to ensure that information on mediation is widely disseminated to parents. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Train mediators on current developments in special education law 2) Conduct dispute resolution presentations for PEAs and parent groups Sub-Activities (Objectives or Action Steps) a) Invite mediators to attend the ADE/ESS Directors Institute Timeline Complete Activity completed as of July 2011. Projected 7/1/10– 6/30/13 Mediators were invited to attend the ADE’s annual Directors Institute. b) Invite mediators to participate in the Dispute Resolution in Special Education (DRSE) Consortium quarterly conference calls for mediators a) Review and revise presentation, support materials, and resources Activity completed from 7/1/12 to 6/30/13. 7/1/10– 6/30/13 The mediators participated in the quarterly conference calls. Activity completed as of July 2011. 7/1/11– 6/30/13 Materials on dispute resolution were reviewed, revised, and widely disseminated statewide. Resources (Planned) ADE/ESS Dispute Resolution Director and Coordinator Technical Assistance for Excellence in Special Education (TAESE) ADE/ESS Dispute Resolution Director and Coordinator TAESE ADE/ESS Dispute Resolution Director and Coordinator Part B Arizona Annual Performance Report for FFY 2012 160 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Conduct statewide presentations at various regional and statewide venues 3) Review and revise dispute resolution brochure a) Revise and update brochure b) Disseminate brochure statewide and post on ADE/ESS Web site Activity completed 7/1/12 to 6/30/13. 7/1/11– 6/30/13 Dispute Resolution director and staff trained statewide at various conferences and workshops, including Directors Institute and Principals Institute. Activity completed as of July 2011. ADE/ESS Dispute Resolution Director and Complaint Investigator 7/1/11– 6/30/13 Dispute resolution brochure was revised and updated and is currently in circulation. Activity completed 7/1/12 to 6/30/13. ADE/ESS Dispute Resolution Director and Coordinator 7/1/11– 6/30/13 ADE/ESS Dispute Resolution Director and Coordinator Revised and updated brochure was disseminated at local and statewide conferences and via Raising Special Kids. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 161 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Monitoring Priority: Effective General Supervision Part B / General Supervision Indicator 20: State Reported Data State reported data (618 and State Performance Plan and Annual Performance Report) are timely and accurate. (20 U.S.C. 1416 (a)(3)(B)) Measurement State reported data, including 618 data, State Performance Plan, and Annual Performance Reports, are: a. Submitted on or before due dates (first Wednesday in February for child count, including race and ethnicity; and educational environments; first Wednesday in November for exiting, discipline, personnel and dispute resolution; December 15 for assessment; May 1 for Maintenance of Effort & Coordinated Early Intervening Services; and February 1 for Annual Performance Reports). b. Accurate, including covering the correct year and following the correct measurement. Target Data for FFY 2012 FFY Measurable and Rigorous Target 2012 100% Actual Target Data for FFY 2012 100% Arizona met the target. Data Data Source Arizona collects the 618 data and the SPP/APR data through the following sources:  Student Accountability Information System (SAIS), a Web-based system for the collection of all student data from the PEAs;  Arizona’s Instrument to Measure Standards (AIMS) and Arizona’s Instrument to Measure Standards Alternate (AIMS A), the statewide student assessment system used by the Arizona Department of Education for AMO and AZ LEARNS determinations;  Arizona Safety Accountability for Education (AZ SAFE), a Web-based system for PEAs to submit data on the discipline elements; Part B Arizona Annual Performance Report for FFY 2012 162 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012  Annual Special Education Data Collection, a Web-based system for PEAs to submit data on the preschool transition, personnel, and exit elements;  Teaching Strategies Gold, a web-based data collection system for PEAs to submit preschool outcome data;  Arizona Parent Survey, a Web-based system for parents to submit survey responses;  Arizona Monitoring System, a Web-based system to collect monitoring data; and  Dispute Resolution spreadsheet to collect, maintain, and report all dispute resolution information. Data Description Based on the Part B Indicator 20 Self-Scoring Rubric, Arizona’s results for submission of timely and accurate data were 100% for FFY 2012.  Child Count and Educational Environment, due February 6, 2013, were submitted on time and were accurate. This data applied to Indicators 4, 5, 6, 9, and 10.  Personnel, due November 6, 2013, was submitted on time and was accurate.  Exit, due November 6, 2013, was submitted on time and was accurate. This data applied to Indicators 1 and 2.  Discipline, due November 6, 2013, was submitted on time and was accurate. This data applied to Indicator 4.  Dispute Resolution, due November 6 2013, was submitted on time and was accurate. This data applied to Indicators 18, and 19.  Assessment, due December 19, 2013, was submitted on time and was accurate. This data applied to Indicator 3.  Maintenance of Effort/Coordinated Early Intervening Services, due May 1, 2013, was submitted on time and was accurate.  Annual Performance Report, due February 3, 2014, was submitted on time and was accurate. Discussion of Improvement Activities Completed for FFY 2012 Primary Activity (GOAL) 1) Provide SPP/APR Indicator data to each PEA in secure format Sub-Activities (Objectives or Action Steps) a) Develop Data Profiles each federal fiscal year Timeline Complete Activity completed from 7/1/12 to 6/30/13. ADE contracted with third party to develop PEA data profiles to include Report Card, Report Card Snapshot, and Trend Report for FFY 2011. Projected 7/1/10– 6/30/13 Resources (Planned) ADE/ESS Directors and Data Management Specialist Part B Arizona Annual Performance Report for FFY 2012 163 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Disseminate Data Profiles each federal fiscal year 2) Review and revise the ADE Student Accountability Information System (SAIS) to improve timely and accurate special education data a) ADE/ESS will meet with Information Technology (IT) staff periodically to revise procedures as necessary and address problems b) ADE/ESS will write business rules for the SAIS revisions c) ADE/ESS will analyze SAIS operations for timely and accurate collection and reporting of special education data d) Investigate the creation of two FTE positions: 1) a PEA data support, and 2) an IT SAIS developer 3) Provide information to PEAs about data accuracy and timeliness a) Develop webinars and workshops for PEAs Activity completed from 7/1/12 to 6/30/13. 7/1/10– 6/30/13 ADE contracted with third party to generate user names and passwords for special education administrators allowing each PEA to access PEA Data Profiles via a secure Web-based application. Activity completed from 7/1/12 to 6/30/13. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 Daily stand-up and weekly scrum meetings held to review current work items and backlog items. Activity completed from 7/1/12 to 6/30/13. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 Collaborated with School Finance business analyst to develop new and revised business rules. Activity completed from 7/1/12 to 6/30/13. 7/1/11– 6/30/13 ADE IT Staff ADE/ESS Directors and Data Management Specialist ADE IT Staff ADE/ESS Directors and Data Management Specialist Participated in IT meetings to establish SAIS processing schedule. Monitored schedule to ensure valid and timely collection and reporting of special education data. Activity completed from 7/1/12 to 6/30/13. 7/1/11– 6/30/13 Two contracted positions maintained: business analyst and quality assurance staff. Activity completed from 7/1/12 to 6/30/13. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 ADE/ESS Directors and Data Management Specialist ESS Data Management developed presentations for use at the annual Directors Institute, regional workshops, and webinars. ADE IT Staff Part B Arizona Annual Performance Report for FFY 2012 164 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 b) Conduct data workshops at annual Directors Institute c) Conduct webinars and workshops for PEAs Activity completed from 7/1/12 to 6/30/13. 7/1/11– 6/30/13 ESS Data Management presented three sessions on three different topics related to the collection and reporting of special education data at the Directors Institute. Activity completed from 7/1/12 to 6/30/13. ADE/ESS Directors and Data Management Specialist 7/1/11– 6/30/13 ADE/ESS Directors and Data Management Specialist ESS Data Management conducted a total of thirtyone regional workshops and webinars on the collection and reporting of special education data. Part B – Indicator 20 Self-Scoring Rubric Part B Indicator 20 – SPP/APR Data APR Indicator Valid and Reliable Correct Calculation Total 1 1 1 2 3A 3B 3C 4A 4B 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 5 6 7 8 9 10 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 11 12 13 14 15 18 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 19 1 1 Subtotal 2 38 Part B Arizona Annual Performance Report for FFY 2012 165 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 APR Score Calculation Timely Submission Points – If the FFY 2012 APR was submitted on time, place the number 5 in the cell on the right. 5 Grand Total – (Sum of the subtotal and Timely Submission Points) = 43.00 Part B Indicator 20 – 618 Data Table Table 1 – Child Count Due Date: 2/1/12 Table 2 – Personnel Due Date: 11/7/12 Table 3 – Ed. Environments Due Date: 2/1/12 Table 4 – Exiting Due Date: 11/7/12 Table 5 – Discipline Due Date: 11/7/12 Table 6 – State Assessment Due Date: 12/19/12 Table 7 – Dispute Resolution Due Date: 11/7/12 Table 8 – MOE/CEIS Due Date: 5/1/12 Timely Complete Data Passed Edit Check Responded to Date Note Requests Total 1 1 1 1 4 1 1 1 N/A 3 1 1 1 1 4 1 1 1 N/A 3 1 1 1 N/A 3 1 N/A N/A N/A 1 1 1 1 N/A 3 1 1 N/A N/A 2 Subtotal 23 Grand Total (Subtotal X 1.8695) = 618 Score Calculation 43.00 Indicator 20 Calculation A. APR Grand Total B. 618 Grand Total C. APR Grand Total (A) + 618 Grand Total (B) = Total N/A in APR Total N/A in 618 Base D. Subtotal (C divided by Base*) = E. Indicator Score (Subtotal D x 100) = 43.00 43.00 86.00 0 0 86.00 1.00 100.00 Part B Arizona Annual Performance Report for FFY 2012 166 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 *Note any cell marked as N/A will decrease the denominator by 1 for APR and 1.8695 for 618. Additional Information Required by the OSEP APR Response Table for This Indicator Not applicable. Revisions, with Justification, to Proposed Targets / Improvement Activities / Timelines / Resources for FFY 2013 There are no revisions for FFY 2013. Part B Arizona Annual Performance Report for FFY 2012 167 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Attachments The following are attachments to the FFY 2012 APR: Attachment 1  List of Acronyms and Terms Part B Arizona Annual Performance Report for FFY 2012 168 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 Attachment 1 List of Acronyms and Terms AAC Arizona Administrative Code ADE Arizona Department of Education AIMS Arizona’s Instrument to Measure Standards AIMS A Arizona’s Instrument to Measure Standards Alternate ALJ Administrative Law Judge AMO Annual Measurable Objective APR Annual Performance Report ARS Arizona Revised Statutes ASVL Annual Site Visit Log AYP Adequate Yearly Progress AZCoPT Arizona Community of Practice on Transition AzEIP Arizona Early Intervention Program for Infants and Toddlers AZHSRI Arizona High School Renewal and Improvement Initiative Az SAFE Arizona Safety Accountability for Education AZ TAS Arizona Technical Assistance System CACM Corrective Action Compliance Monitor CAP Corrective Action Plan CEIS Coordinated Early Intervening Services CFR Code of Federal Regulations CoP Community of Practice CSPD Comprehensive System of Personnel Development CTE Career and Technical Education CTT Community Transition Team DAC Data Accountability Center Part B Arizona Annual Performance Report for FFY 2012 169 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 DBHS Division of Behavioral Health Services DDD Division of Developmental Disabilities DI Directors Institute ECE Early Childhood Education ECQUIP Early Childhood Quality Improvement Practices Process ECSE Early Childhood Special Education ESEA Elementary and Secondary Education Act ESS Exceptional Student Services FAPE Free Appropriate Public Education FFY Federal Fiscal Year GOLD Teaching Strategies GOLD (early childhood assessment) Group B Arizona Funding Category for Significant Disabilities IDEA The Individuals with Disabilities Education Act IEP Individualized Education Program IGA Intergovernmental Agreement IT Information Technology LRE Least Restrictive Environment M Team Monitoring Team MOE Maintenance of Effort MPRRC Mountain Plains Regional Resource Center NASDSE National Association of State Directors of Special Education NCCRESt National Center for Culturally Responsive Educational Systems NDPC-SD National Dropout Prevention Center for Students with Disabilities NPSO National Post School Outcomes Center NSTTAC National Secondary Transition Technical Assistance Center OCSHCN Office for Children with Special Health Care Needs Part B Arizona Annual Performance Report for FFY 2012 170 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 OELAS Office of English Language Acquisition Services OSEP Office of Special Education Programs/U.S. Department of Education PBISAz Positive Behavioral Interventions and Supports of Arizona PEA Public Education Agency PINS Parent Information Network Specialist PSO Post School Outcome PTI Parent Training and Information Center R and E Research and Evaluation RSA/VR Rehabilitation Services of Arizona/Vocational Rehabilitation RTI Response to Intervention SAIS Student Accountability Information System SEAP Special Education Advisory Panel SEAS-Math Special Education Achieving Success in Mathematics SPP State Performance Plan STMP Secondary Transition Mentoring Project SUMS Special Education Using Mathematics for School Improvement TA Technical Assistance TAESE Technical Assistance for Excellence in Special Education WRR Weighted Risk Ratio Part B Arizona Annual Performance Report for FFY 2012 171 Arizona Part B Arizona Annual Performance Report (APR) for FFY 2012 The contents of this publication were developed with funds allocated by the U.S. Department of Education under The Individuals with Disabilities Education Act. These contents do not necessarily represent the guideline of the agency, nor should endorsement by the federal government be assumed. The Arizona Department of Education of the State of Arizona does not discriminate on the basis of race, religion, color, national origin, sex, disability or age in its programs, activities or in its hiring and employment practices. For questions or concerns regarding this statement, please contact Administrative Services at 602-542-3186. Printed in Phoenix, Arizona, by the Arizona Department of Education Part B Arizona Annual Performance Report for FFY 2012 172