Arizona’s School
Accountability System
2005
Technical Manual

Tom Horne
Superintendent of Public Instruction
Arizona Department of Education
Volume II: Adequate Yearly Progress
Research & Policy
Arizona Department of Education
Robert Franciosi, Deputy Associate Superintendent
Achieve@azed.gov
Published by the Arizona Department of Education, April 2005.
The Arizona Department of Education of the State of Arizona does not discriminate on the basis of race,
religion, color, national origin, sex, disability or age in its programs, activities or in its hiring and
employment practices. If you have questions or grievances related to this policy, please contact the
Administrative Services DAS at (602)542-3186.

Table of Contents
Table of Contents............................................................................................................................. i
List of Tables and Figures............................................................................................................... ii
1. Introduction................................................................................................................................. 3
2. Overview of the NCLB Evaluation System................................................................................ 5
Schools to Be Evaluated ............................................................................................................. 5
Proficiency Standards ................................................................................................................. 5
Percentage of Students Assessed ................................................................................................ 9
Applicable Subgroups................................................................................................................. 9
Additional Indicators of School Performance............................................................................. 9
Putting It All Together .............................................................................................................. 10
3. Data Verification....................................................................................................................... 11
Timeline .................................................................................................................................... 11
4. Meeting the Annual Measurable Objectives for Proficiency.................................................... 12
Calculation of Annual Measurable Objectives (AMOs)........................................................... 12
Data used................................................................................................................................... 15
Special rule................................................................................................................................ 15
5. Meeting the Standard for Number of Students Tested.............................................................. 16
Calculation ................................................................................................................................ 16
Data used................................................................................................................................... 16
Applicable subgroups................................................................................................................ 16
Special rules .............................................................................................................................. 17
6. Other Indicators of School Performance................................................................................... 18
Attendance Rate ........................................................................................................................ 18
Graduation Rate ........................................................................................................................ 18
7. Calculation of Adequate Yearly Progress for K-2 Schools ...................................................... 20
Meeting the Annual Measurable Objectives for Proficiency for K-2 Schools ......................... 20
Attendance Criteria for K-2 Schools......................................................................................... 20
8. Calculation of Adequate Yearly Progress for Small Schools ................................................... 22
Meeting the Annual Measurable Objectives Small Schools..................................................... 22
9. Determining Adequate Yearly Progress for School Districts and Charter Holders.................. 23
Differences between District and School AYP Evaluation Methods ....................................... 24

Arizona’s Accountability System: Volume II

Arizona Department of Education

i

List of Tables and Figures
Table 1.1. Comparison of Arizona's Accountability Systems……………………………….…...4
Table 2.1. Calculation of Performance Starting Points…………………………………………..6
Table 2.2. Starting Points for State Performance Standards……………………………………..6
Table 2.3. Annual Measurable Objectives (AMOs) and Intermediate Goals…………………....7
Figure 2.1. Intermediate Goals: Grade 3 Reading……………………………………………....8
Table 2.4. Categories Evaluated Under NCLB for a K-5 Elementary School………………….10
Table 4.1. Rules for Determining UB99 for Small n and p………………………………….….13
Table 7.1. AYP Determination for K-2 Schools………………………………………………....21

Arizona’s Accountability System: Volume II

Arizona Department of Education

ii

1. Introduction
The federal No Child Left Behind Act of 2001 (NCLB) requires states to establish an
accountability system to evaluate the performance of local public schools and school districts,
including charter schools. Specifically, states are required to:
•

Institute performance standards for reading/language arts, mathematics, and science.

•

Develop and administer tests to measure whether students meet these standards. By the
2005-06 academic year, states must give tests in reading/language arts and math for
grades three through eight. By the 2007-08 academic year, states must also administer a
test to evaluate student performance in science in elementary (grades 3-5), middle (grades
6-8), and high school (grades 10-12).

•

Establish a timeline to ensure that all students are proficient according to state standards
by 2013-2014.

•

Create a statewide accountability system to evaluate school progress in meeting the goals
of the timeline, and issue report cards informing parents of school performance.

In 2001 Arizona voters also approved Proposition 301 that among other things called for
a state accountability system for public schools. In 2001, Arizona also had in place state
standards and a test to measure whether students met them: Arizona’s Instrument to Measure
Standards (AIMS).
Since the passage of NCLB and Proposition 301 the staff of the Arizona Department of
Education (ADE) has worked with scholars, school officials ranging from superintendents to
teachers, and members of the public to develop an accountability system that fulfills the
requirements of both laws. The result is a system that consists of two linked components.
Arizona LEARNS was created to comply with Proposition 301. Its primary focus is on
longitudinal change through time of student performance as measured by AIMS and the
TerraNova tests. The system created to comply with NCLB, commonly referred to as Adequate
Yearly Progress (AYP), provides a single-year snapshot of school performance as measured by
AIMS. Table 1.1 provides a brief comparison of the two accountability systems.
The State of Arizona’s complete plan to meet the requirements of NCLB is contained in
the workbook submitted to the U.S. Department of Education. The workbook is available at
http://www.ade.az.gov/azlearns/workbook.asp.

Arizona’s Accountability System: Volume II

Arizona Department of Education

3

Table 1.1 Comparison of Arizona’s Accountability Systems

NCLB
Required by federal law

Arizona LEARNS
Required by state law

One-year snapshot of student performance

Longitudinal examination of student performance

Components of evaluation
• AIMS scores
• Percent students assessed
• Attendance/Graduation rates

Components of evaluation
• AIMS scores
• Measure of Academic Progress
• Graduation/dropout rates
• AYP
Labels schools on a graded scale:

Labels schools on a yes/no system

•
•
•
•
•

Arizona’s Accountability System: Volume II

Failing to meet academic standards
Underperforming
Performing
Highly performing
Excelling

Arizona Department of Education

4

2. Overview of the NCLB Evaluation System
This section provides an overview of the determination of adequate yearly progress
(AYP). More detailed discussions of the methodology used to determine AYP, including
descriptions of equations, algorithms, and data used are given in subsequent chapters.
The No Child Left Behind Act requires that every public school and district in a state—as
well as the state itself—be evaluated on three measures:
1. Progress toward meeting the goal of 100 percent proficiency in state standards;
2. Percentage of students assessed; and
3. An additional measure of school performance. NCLB mandates that for high
schools this indicator be the graduation rate. States may select an alternative
indicator for elementary schools. Arizona, along with many other states, has
chosen attendance rate for the other indicator for elementary schools.
If an entity—school, district, or state—passes on all three measures, then it is deemed to
have made adequate yearly progress (AYP).

Schools to Be Evaluated
All schools—including extremely small schools, new schools, and schools that only offer
grades K-2—must receive an AYP determination. Similarly, although the state’s system for
school accountability, Arizona LEARNS, allows alternative schools to be evaluated under
different criteria, NCLB requires all public schools in the state to be given an AYP designation
based on the same criteria.

Proficiency Standards
NCLB requires that every student in Arizona meet state standards in reading/language
arts and mathematics—that is, pass AIMS—by the year 2013-2014. To further this goal, the
state must set annual measurable objectives (AMOs) for each grade and subject evaluated. The
annual measurable objectives describe the yearly growth in the fraction of students passing
AIMS that is necessary for Arizona to reach the 100 percent requirement by 2013-2014. The
AMOs are then used to set intermediate goals. To make AYP an entity must reach the
intermediate goals for every subject in each grade it offers. If an entity fails to reach an
intermediate goal, it still may be deemed to have made adequate yearly progress if it satisfies the
safe harbor provisions that will be described later.
The Arizona Department of Education established the starting points, annual measurable
objectives, and intermediate goals in the manner specified by the No Child Left Behind Act. To
determine the baselines for each subject/grade combination, all schools in Arizona were ranked
in descending order according to the percentage of students passing AIMS for that subject and
grade. Then, cumulative enrollment was calculated adding upward from the bottom of the list of
schools. The baseline was then set to be equal to the fraction of students passing AIMS for that

Arizona’s Accountability System: Volume II

Arizona Department of Education

5

grade and subject in the school where the cumulative enrollment was equal to 20 percent of state
enrollment for that grade. The data used for this calculation were AIMS results for the spring of
2002. As required by NCLB, students with invalid scores such as English language learners and
special education students who received nonstandard accommodations were included in the
setting of the baselines.
Table 2.1 provides a hypothetical example of how the baselines were established. In this
case, we assume there are only eight schools in the state that offer third grade.
Table 2.1. Calculation of Performance Starting Points

Grade
3

Subject
Math

School
1
2
3
4
5
6
7
8

Percent
pass
100
75
70
61
55
48
32
15

Enrollment
10
40
30
30
20
30
20
20

Cumulative
percent of total
state enrollment
100
95
75
60
45
35
20
10

These eight schools are ranked in descending order by the percentage of their students
who passed the AIMS for third grade math (fourth column). The third grade enrollment for each
school is given in the fifth column. Starting from the bottom of the list, enrollment is summed
until the total equals 20 percent of the state’s total enrollment for that grade. In table 2.1 this
point is reached at School Seven, where the cumulative sum equals forty students (40/200 =
0.20). The percent of students passing for School 7 (32 percent) is then taken as the starting
point for the state for third grade math.
Table 2.2 provides the starting points for each of the subjects and grades evaluated in
2003. These served as the AMOs for the 2004 AYP determinations.
Table 2.2 Starting Points for State Performance Standards
Subject/Grade
Grade 3
Grade 5
Grade 8
High School

Reading
44
32
31
23

Mathematics
32
20
7
10

The annual measurable objectives were calculated as six equal percentage-point increments from
the 2002 starting point to the 2014 goal of 100 percent. The AMO for third grade reading, for
example, is 9.3 percentage points ([100-44]/6). The AMOs cover three-year increments through

Arizona’s Accountability System: Volume II

Arizona Department of Education

6

2010 and one-year increments thereafter. This leads to a stepwise increase in the intermediate
goals until 2010, followed by a linear increase until 2014 (see table 2.3). Figure 2.1 shows an
example using third grade reading for the increase in the intermediate goals.
Table 2.3 Annual Measurable Objectives (AMOs) and Intermediate Goals
Grade 3

2004-05
2007-08
2010-11
2011-12
2012-13
2013-14
Grade 5

2004-05
2007-08
2010-11
2011-12
2012-13
2013-14
Grade 8

2004-05
2007-08
2010-11
2011-12
2012-13
2013-14
High
School
2004-05
2007-08
2010-11
2011-12
2012-13
2013-14

Intermediate
Goals
1
2
3
4
5
6
Intermediate
Goals
1
2
3
4
5
6
Intermediate
Goals
1
2
3
4
5
6
Intermediate
Goals
1
2
3
4
5
6

Reading
AMO
9.3
9.3
9.3
9.3
9.3
9.3
Reading
AMO
11.3
11.3
11.3
11.3
11.3
11.3
Reading
AMO
11.5
11.5
11.5
11.5
11.5
11.5
Reading
AMO

Arizona’s Accountability System: Volume II

12.8
12.8
12.8
12.8
12.8
12.8

Reading
Proficiency
(percent)
53.3
62.6
71.9
81.2
90.5
100
Reading
Proficiency
(percent)
43.3
54.6
65.9
77.2
88.5
100
Reading
Proficiency
(percent)
42.5
54.0
65.5
77.0
88.5
100
Reading
Proficiency
(percent)
35.8
48.6
61.4
74.2
87.0
100

Math
AMO
11.3
11.3
11.3
11.3
11.3
11.3
Math
AMO
13.3
13.3
13.3
13.3
13.3
13.3
Math
AMO
15.5
15.5
15.5
15.5
15.5
15.5
Math
AMO
15
15
15
15
15
15

Math
Proficiency
(percent)
43.3
54.6
65.9
77.2
88.5
100
Math
Proficiency
(percent)
33.3
46.6
59.9
73.2
86.5
100
Math
Proficiency
(percent)
22.5
38.0
53.5
69.0
84.5
100
Math
Proficiency
(percent)
25
40
55
70
85
100

Arizona Department of Education

7

Figure 2.1 Intermediate Goals: Grade 3 Reading

100!

Percent Students Proficient

100
80
71.9
60
62.6
53.3
40

44

20
0
01-02

02-03

03-04

04-05

05-06

06-07

07-08

08-09

09-10

10-11

11-12

12-13

13-14

The reasons for setting all annual measurable objectives (and corresponding intermediate
goals) in this stepwise manner were:
1.) The ADE completed a grade-level articulation of Arizona’s Academic Content Standards
in 2003. The progressive setting of annual measurable objectives and corresponding
intermediate goal allows schools the necessary time to align these grade-level standards
with school curricula/resources and implement these standards via instruction.
2.) The ADE has developed new assessments for grades four, six, and seven for reading and
mathematics and is developing a science assessment to be administered on an annual
basis in grades three, five, eight, and high school as mandated by NCLB. The
progressive setting of annual measurable objectives and intermediate goals allows
schools the opportunity to effectively prepare students for these assessments.
3.) Currently, the academic performance of several disaggregated student subgroups is below
(in some cases, far below) the state’s starting points in reading and mathematics. Many
schools and districts have initiated scientifically based research programs and other
instructional practices to assist students in these groups. In addition, the ADE has
implemented a comprehensive K-3 reading program designed to have all students
proficient in the state’s reading standards by the third grade. By setting the state’s annual
measurable objectives and corresponding intermediate goals in a progressive manner,
schools, districts, and the state are given the necessary time to effectively implement
these programs and initiatives, giving students in this circumstance an opportunity to
catch up with the aggregated student population as represented by the state’s starting
points.
There are two additional steps taken when determining if a school has met the AMO for a
specific subject and grade. First, rather than comparing the actual percentage of students who
are proficient to the AMO, a 99 percent confidence interval is calculated to estimate the percent

Arizona’s Accountability System: Volume II

Arizona Department of Education

8

proficient. If the upper bound of this confidence interval is above the AMO, the school is
deemed to have met the objective.
Second, if a school fails to meet the objective after the confidence interval is applied, it
may still be deemed to have met the AMO if it meets the safe harbor provision. Safe harbor is a
two-part test that requires schools to demonstrate sufficient progress over the previous year in the
percentage of students failing to meet the standard and meet a threshold set by the Arizona
Department of Education for an additional indicator. Both of these refinements will be discussed
in more detail later.

Percentage of Students Assessed
In order for a school, district, or the state to make adequate yearly progress it must assess
95 percent of its students for each subject in every grade offered, including each applicable
subgroup. Students count as assessed if they had a valid score for AIMS or the alternative
assessment for the severely disabled, AIMS-A.
All the students enrolled for the day of testing (high school) or the first day of the week
the test was given (elementary) represent the population to be assessed.

Applicable Subgroups
In addition to assessing 95 percent of its students and meeting the intermediate goals for
all subject/grade combinations it encompasses, an entity must also meet the same objectives for
every applicable subgroup within each subject/grade combination. NCLB specifies the
following subgroups be evaluated: the five major ethnic groups—Hispanic, White, AfricanAmerican, Asian-Pacific Islander, and Native American—English Language Learners (ELL),
students with disabilities, and students from low-income families.

Additional Indicators of School Performance
NCLB requires that an additional indicator be used for AYP determinations. The law
mandates that a four-year graduation rate be used for high schools, but allows states to select the
standard schools must meet. The performance goal for the high school graduation rate was set at
71 percent, the state average graduation rate for 2001. To make adequate yearly progress, a high
school must have a four-year graduation rate of 71 percent, or show a 1 percentage-point
improvement in the graduation rate over the previous year.
NCLB allows states to select the additional indicator used for elementary schools.
Arizona has chosen to use the school-wide attendance rate. The performance goal for the
attendance rate was set at 90 percent, the implicit expectation for school attendance rates set by
the state’s school finance laws in A.R.S. § 15-902 A. To make AYP, elementary schools must
have a school-wide attendance rate of 90 percent, or show a 1 percentage-point improvement in
the attendance rate over the previous year.

Arizona’s Accountability System: Volume II

Arizona Department of Education

9

Putting It All Together
Table 2.4 provides an example of how the three performance measures—proficiency in
state standards, percentage of students assessed, and an additional indicator—are combined to
determine whether a school has made AYP. The example given is for a K-5 school. The school
is evaluated based on student performance on AIMS reading and mathematics tests for grades 3
and 5, the percentage of students evaluated for each test and attendance rates. Since our example
is an elementary school, all the combinations for which a typical elementary school would be
evaluated under NCLB are provided; there are 73 separate combinations examined.
NCLB requires that schools be evaluated using a conjunctive model. That is, to make
AYP, a school must meet the performance objective in every category in which it is evaluated.
For example, if the school in table 2.4 fails to meet the objective in any one of the cells in the
table, it fails to make AYP.
Table 2.4. Categories Evaluated Under NCLB for a K-5 Elementary School
Grade
Subject
Subgroup

All students
African American
Asian-Pacific
Islander
Hispanic
Native American
White
Special Education
English Language
Learner
Low Income
Met Other School
wide Indicator:
Attendance Rate?

Third
Math
Met
AMO?

Fifth

Met
95%
tested?
Yes/No
Y/N
Y/N

Y/N
Y/N
Y/N

Reading
Met
Met
95%
AMO?
tested?
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N

Y/N

Y/N

Y/N

Arizona’s Accountability System: Volume II

Y/N
Y/N
Y/N

Reading
Met
Met
95%
AMO?
tested?
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N
Y/N
Y/N
Y/N
Y/N

Y/N

Y/N

Y/N

Y/N

Met
95%
tested?
Y/N
Y/N
Y/N

Math
Met
AMO?

Y/N

Arizona Department of Education

10

3. Data Verification
Districts and charter holders are solely responsible for submitting the data necessary for
calculating achievement profiles for their schools and for ensuring its accuracy. Because of the
stakes involved and the volume and scope of the data used, the ADE considered it prudent to
allow districts and charter holders to review their data before preliminary AYP evaluations were
carried out.
From June 22, 2005, through August 12, 2005 schools and districts were given an
opportunity to review and correct data through the common logon on the ADE web site. The
primary purpose of the application was to allow districts and charter holders to correct the
information for individual students. A link was provided through the common logon that
allowed schools to download student-level testing data in order to make any necessary
corrections. Schools also had the opportunity to correct data up through the close of the appeals
window on August 12, 2005.
IMPORTANT NOTE: The criteria used to select AIMS scores for evaluation of AYP differ
from the criteria used to select scores for Arizona LEARNS. Indeed, the criteria differ among
the separate components of the AYP evaluation. The criteria also differ from the scores provided
to schools by the testing contractor, the scores publicly reported by ADE, and the scores
available through the ADE’s AIMS wizard located at www.ade.az.gov/profile/publicview.

Timeline
The timeline for AYP Determination was:
•

June 22, 2005. Opening of data verification process.

•

July 13, 2005. Public release of AIMS Scores.

•

August 1, 2005. Preliminary release of AYP evaluations for all schools and districts;
opening of appeals process.

•

August 12, 2005. Closing of appeals process.

•

September 1, 2005. Public release of AYP evaluations for all schools and districts.

Arizona’s Accountability System: Volume II

Arizona Department of Education

11

4. Meeting the Annual Measurable Objectives
for Proficiency
Calculation of Annual Measurable Objectives (AMOs)
This section describes the calculation used to determine if schools met the annual
measurable objectives (AMOs) for student proficiency in math and reading/language arts.
NCLB requires that schools meet the AMOs set by the state in order to make AYP. A
description of how the AMOS were set is given in section two. Schools must meet the AMOs
for each subject/grade combination and all the applicable subgroups.
The formula used to calculate the percentage of students passing is:

Percent Pass =

Number of students meeting/exceeding the standard on AIMS
Number of students tested

This fraction is rounded to two digits, e.g.: .941=.94; .946=.95.
To ensure that the decision regarding whether a school met the AMOs is statistically
reliable and not overly influenced by random factors, the determination for meeting the AMOs is
made employing a 99 percent (one-tailed) confidence interval. The confidence interval
methodology is designed to ensure that 99 out of 100 times the confidence interval will contain a
school’s true performance level. If the AMO in question is below the upper bound of the
confidence interval calculated for the school, the school is deemed to have met the standard.
Example. Twenty-nine percent of a school’s third graders passed the AIMS mathematics test.
The upper bound of the 99 percent confidence interval for this subject/grade combination for this
school is calculated to be 35 percent. Since this is greater than the intermediate goal of 32
percent, the school is considered to have met the standard.
Let p=the percent of students in a group passing the AIMS and n=the number of students
in the group. Then the equation for the upper bound of the 99 percent confidence interval
(UB99) is:

UB99 = p + 2.33 p(1 − p) n .
As can be seen from the equation, the confidence interval depends upon the percent of
students who passed the test, and the number of students tested. Thus, the confidence interval
will differ among grades, subjects, and schools.

Arizona’s Accountability System: Volume II

Arizona Department of Education

12

The equation is an approximation of the confidence interval for a binomially distributed
variable. It uses the standard normal distribution and is sufficiently accurate if the group size
and percentage of students passing are large enough. For small values of n and small p, a more
accurate estimate of the confidence interval is made using statistical tables that provide
confidence intervals for a binomially distributed variable. 1 The tables were applied using the
rules given in table 4.1.
Table 4.1. Rules for Determining UB99 for Small n and p.

If n>=30 and n < 35, and
p>=0 and p <.05, UB99=.16
p>=.05 and p < .10,
UB99=.25
p>=.10 and p<.15, UB99=.33
p>=.15 and p<.20, UB99=.38
p>=.20 and p<.25, UB99=.45
p>=.25 and p<.30, UB99=.51

If n>=40 and n<45, and
p>=0 and p <.05,
UB99=.13
p>=.05 and p < .10,
UB99=.22
p>=.10 and p<.15,
UB99=.28
p>=.15 and p<.20,
UB99=.35

If n>=35 and n < 40, and
If n>=45 and n<50, and
p>=0 and p <.05, UB99=.15
p>=.05 and p < .10,
UB99=.24
p>=.10 and p<.15, UB99=.30
p>=.15 and p<.20, UB99=.36
p>=.20 and p<.25, UB99=.43

p>=0 and p <.05,
UB99=.12
p>=.05 and p < .10,
UB99=.21
p>=.10 and p<.15,
UB99=.27
n>=50 and n < 55, and
p>=0 and p <.05,
UB99=.11
p>=.05 and p < .10,
UB99=.20
If n>=55 and n<60, and
p=0, UB99=.10
If n>=60 and n<100 and
p=0, UB99=.09
If n>=100 and n<200 and
p=0, UB99=.06
If n>=200 and p=0,
UB99=0

1

Mansfield, Edwin. 1991. Statistics for Business and Economics, 4th Edition. New York: W.W. Norton and
Company. 280-284.

Arizona’s Accountability System: Volume II

Arizona Department of Education

13

Even if after calculating the confidence interval the percent of students proficient in a
subgroup still falls short of the AMO, the group may still make AYP if its achievement
indicators meet certain safe harbor provisions. To make safe harbor a subgroup has to meet the
following two-part test:
a) Make a 10 percent decrease in the fraction of students failing to meet the standard
(i.e. failing AIMS) from the previous year, and
b) Have a 90 percent attendance rate for that group, or make a one-percentage point
improvement in the group’s attendance rate over the previous year. Since graduation
rate data was not available for the applicable subgroups, attendance rate was used as
the additional safe-harbor indicator for high schools as well.
Examples
1.

In 2004, 20 percent of fifth graders in Gila Monster Elementary passed the AIMS
reading test. The upper bound of the confidence interval was 25 percent, still
below the annual measurable objective of 32 percent. However, in 2003, 10
percent of fifth graders passed the AIMS reading test, thus Gila Monster
Elementary saw a decrease of 11 percent in the percentage failing [(80-90)/90 = 11 percent]. Furthermore, the attendance rate for Gila Monster’s fifth grade was
96 percent, greater than the standard of 90 percent. So, Gila Monster’s fifth
graders make AYP in reading.

2.

In 2004, 20 percent of eighth graders in Javelina Middle School passed the AIMS
reading test. The upper bound of the confidence interval was 27 percent, still
below the annual measurable objective of 31 percent. In 2002, 15 percent of fifth
graders passed the AIMS reading test, thus Javelina Middle School saw a
decrease of only 6 percent in the percentage failing [(80-85)/85 = -6 percent].
Even though the attendance rate for Javelina’s eighth grade was 96 percent,
greater than the standard of 90 percent, it fails to make the safe harbor provisions,
and so does not make AYP in eighth grade reading.

3.

In 2004, 30 percent of third graders in Gila Monster Elementary passed the AIMS
reading test. The upper bound of the confidence interval was 40 percent, still
below the annual measurable objective of 44 percent. However, in 2003, 20
percent of third graders passed the AIMS reading test, thus Gila Monster El. saw
an improvement of 13 percent in performance [(70-80)/80 = -13 percent].
However, the attendance rate for Gila Monster’s third grade was 85 percent, less
than the standard of 90 percent and identical to last year’s attendance rate, so Gila
Monster’s third graders fail to make AYP in reading.

4.

In 2004, 20 percent of third graders in Saguaro Elementary passed the AIMS
reading test. The upper bound of the confidence interval was 30 percent, still
below the annual measurable objective of 32 percent. However, in 2003, 10
percent of fifth graders passed the AIMS reading test, thus Saguaro Elementary
saw an improvement of over 11 percent in performance [(80-90)/90 = -11
percent]. The attendance rate for Saguaro’s third grade was 80 percent, less than
the standard of 90 percent. However, in 2002, the attendance rate for Saguaro’s
third grade was 81 percent. Since Saguaro saw an 11 percent improvement in the
fraction of third graders meeting the standard in math and a 1 percent

Arizona’s Accountability System: Volume II

Arizona Department of Education

14

improvement in the attendance rate for third graders, it meets the safe harbor
provision for third grade math, and thus makes AYP.
If a group had less than 40 students tested in 2005, it was automatically considered to
have met the safe harbor provision. If a group had an ADM of less than 40 in 2005, then it was
automatically considered to have met the attendance rate criterion for safe harbor.

Data used
Students are included in the calculation if they meet the following criteria:
•

Have taken either the AIMS or AIMS-A and received a score of FFB or above,

•

Began the year in the same school. (Answered yes or left blank question number 3 on
the AIMS demographic questionnaire; the field STARTYR in the AIMS data set = Y
or blank.)

Students in each of the following subgroups in every subject/grade combination are
required to meet the annual measurable objective.
•

Ethnicity. The fraction of students meeting the standard is calculated for each of the five
ethnic groups—White, Black, Asian/Pacific Islander, American Indian, and Hispanic.

•

English Language Learners (ELL). ELL status is determined using the answer to
question number eight on the AIMS test sheet (ELLPROF in the ADE AIMS database.)
Students with ELLPROF = 1 are considered English language learners. Students with
ELLRPOF = 2 or blank are considered English proficient.

•

Special Education Students. A student is identified as special education if she takes the
AIMS-A, or is specified as a member of a special education program in SAIS.

•

Low income. A student is identified as being from a low-income family if the AIMS
demographic information indicates she is eligible for a free or reduced lunch.

Special rule
Minimum group size. A group or subgroup is not evaluated if it had less than 40 test scores that
meet the selection criteria. A sample size of 40 was considered large enough to provide
statistically meaningful results.

Arizona’s Accountability System: Volume II

Arizona Department of Education

15

5. Meeting the Standard for Number of Students
Tested
Calculation
This section describes the calculation used to determine if a school has assessed 95
percent of its students. To make AYP, schools must test 95 percent of their students in reading
and mathematics in all grades in which AIMS is administered, and must test 95 percent of their
students in each applicable subgroup.
The formula used to calculate the percentage of students tested is:
Percent Tested =

Number of students tested
Number of students enrolled

The fraction of percent tested is rounded to two digits, e.g.: .941=.94; .946=.95.

Data used
Number of students tested. All students who take either the AIMS or AIMS-A and received a
score of FFB or above. Students who receive a score of Did Not Attempt (DNA) are excluded
from the calculation.
Number of students enrolled. The denominator for the percent tested calculation is an
unduplicated enrollment count at the school level. For grades three, five, and eight, enrollment
used for all subjects is the first day of the week of testing: April 11, 2005—as reported to the
Student Accountability Information System (SAIS). For grade 10, enrollment used is for the day
the test was administered: April 12, 2005 for mathematics, and February 23, 2005 for reading.
The rules used to select students for enrollment are:
•

Students were counted in both schools if they were concurrently enrolled in more than
one school on the relevant day.

•

Students are not selected if they had more than one attribute on the relevant day. For
example, a student could not have been assigned to more than one grade or more than one
ethnicity on that particular day.

Applicable subgroups
•

Ethnicity. Schools are required to have tested 95 percent of their students in the five
ethnic groups—White, Black, Asian/Pacific Islander, American Indian, and Hispanic.
The ethnicity of students tested was taken from the answer to the ethnic information
question on the AIMS test document. The number of students enrolled for each ethnicity
was taken from SAIS enrollment data for the selected days described in chapter two.

Arizona’s Accountability System: Volume II

Arizona Department of Education

16

•

English Language Learners (ELL). Schools are required to test 95 percent of their
English language learners in order to make adequate yearly progress (AYP). The number
of ELL students tested is the sum of all students identified as ELL students in the AIMS
test file. ELL status is determined using the answer to question 8 on the AIMS test sheet
(ELLPROF in the ADE AIMS database.) Students with ELLPROF = 1 are considered
English language learners. Students with ELLRPOF = 2 or blank are considered English
proficient. The number of ELL students enrolled is taken from SAIS. Students must
have been enrolled as English Language Learners on the relevant day in order to be
included in the enrollment counts for this subgroup.

•

Special Education Students. Schools are required to test 95 percent of their special
education students in order to make adequate yearly progress (AYP). The number of
special education students tested is the sum of all students identified as special education
in the AIMS test file. A student is identified as special education if he/she took the
AIMS-A, or was specified as a member of a special education program. The number of
special education students enrolled is taken from the SAIS system. Students must have
been enrolled in a special education program on the relevant day in order to be included
in the enrollment counts for this subgroup.

Special rules
100 percent tested. If a school tested 100 percent of its students overall in a subject/grade
category, it is assumed that it tested 100 percent for each subgroup in that subject/grade category,
regardless of how students were labeled or mislabeled on testing documents.
Consistency Check for Number Tested. Subgroups for a subject/grade are deemed to have met
the 95 percent goal for percentage of students tested if the data for the entire subject/grade
implies that missing the goal was mathematically impossible.
Example. Data for Gila Monster Elementary show 98 students have taken AIMS for
fifth grade reading and 100 fifth grade students enrolled. However, data submitted to
ADE show 50 students enrolled in special education programs but only 45 students
with special education status indicated on their AIMS demographic information.
Since only two students did not take the test, it is mathematically impossible for Gila
Monster to have tested less than 48 of its special education students—there must be
special education students in the data who are not labeled as such. Since 48/50 = .96
> .95, Gila Monster meets the 95 percent assessed threshold for its special education
students.
Minimum group size. A group or subgroup is not evaluated if it had less than 40 students
enrolled on the relevant day. A sample size of 40 was considered large enough to provide
statistically meaningful results.

Arizona’s Accountability System: Volume II

Arizona Department of Education

17

6. Other Indicators of School Performance
Attendance Rate
This section describes the calculation used to determine if a school met the other
performance indicators for AYP. NCLB requires that schools be evaluated on a third
performance indicator as well as percentage of students assessed and percentage of students
proficient in the standard. The law requires that graduation rate be used for the third indicator
for high schools, and gives states the discretion to choose the third indicator for elementary
schools. Arizona has chosen the school-wide attendance rate as the third indicator for
elementary schools. To make AYP a high school must have a graduation rate of 71 percent; an
elementary school must have an attendance rate of 90 percent.
Calculation. The formula used to calculate the attendance rate is:
Schoolwide Attendance Rate =

Average Daily Attendance
Average Daily Membership

The attendance rate is rounded to two digits, e.g.: e.g.: .891=.89; .896=.90.
Data used. The average daily attendance (ADA) and average daily membership (ADM) for the
100-day counts for all grades offered by a school, except for pre-school and kindergarten, are
used in the calculation.
Safe Harbor. If a school demonstrates a one-percentage point improvement in its attendance
rate from the previous year, it is deemed to have met the performance standard. The growth rate
is rounded to the nearest hundredth of a point, e.g. .009 = .01, .004=.00.
Example. Gila Monster Elementary had an attendance rate in 2005 of 88 percent, less
than the standard of 90 percent. However, its 2004 attendance rate was 86 percent. Gila
Monster Elementary demonstrated an improvement of two percentage points over the
previous year, and so is deemed to have met the requirements for attendance rate.
Special rules. A school’s attendance rate is not evaluated if it had an ADM of less than 40.

Graduation Rate
The graduation rate is an important complement to the AYP determination for high
schools. Graduation rates indicate the success of students in meeting course requirements and
achieving passing grades in subject areas not covered by the AIMS test. Graduation rates are
used solely in the calculation of high school AYP. High school status was granted to any school
that reported data in grade ten for each of the relevant school years (2001-2002, 2002-2003 and
2003-2004).
The Graduation Rate is a four-year, longitudinal measure of how many students graduate
from high school. By examining a cohort of students who began high school at the same time,

Arizona’s Accountability System: Volume II

Arizona Department of Education

18

the graduation rate assesses how many students actually complete high school within four years
of beginning high school.
Calculation. The formula used to calculate the graduation rate is:
N u m b e r o f C o h o rt m e m b e rs w h o g ra d u a te d w it h in fo u r y e a rs

G ra d u a t io n
R a te

=

X 100
O r ig in a l C o h o rt
M e m b e rs h ip

+

T ra n s fe rs
In

-

T ra n s fe rs

-

D eceased

O ut

The graduation rate is rounded to three digits, e.g.: .7045=.705; .7044=.704.
Data used. Federal requirements mandate that Arizona use the four-year graduation rate rather
than the five-year rate used for Arizona LEARNS. The threshold graduation rate was for the
cohort class of 2004, which represents the most recent graduation rate statistics. The graduation
rate for the cohort class of 2003 was used for the determination of safe harbor.
Safe Harbor. If a school demonstrates a one-half-percentage point improvement in its
graduation rate from the previous year, it is deemed to have met the performance standard. The
growth rate is rounded to the nearest thousandth of a point, e.g. .0045 = .005, .0044=.004.
Example. Gila Monster High School had a graduation rate in 2004 of 69.0 percent, less
than the standard of 70.5 percent. However, its 2003 graduation rate was 67.0 percent.
Gila Monster High demonstrated an improvement of two percentage points over the
previous year, and so is deemed to have met the requirements for graduation rate.
Special rules. A school’s graduation rate is not evaluated if it had an ADM of less than 40.

Arizona’s Accountability System: Volume II

Arizona Department of Education

19

7. Calculation of Adequate Yearly Progress for
K-2 Schools
The No Child Left Behind Act requires that a state evaluate all schools. Consequently,
an alternative methodology for determining adequate yearly progress (AYP) had to be developed
for schools that did not offer any of the grades in which AIMS is administered. In Arizona, this
group consisted of schools that offered grades two and below.

Meeting the Annual Measurable Objectives for Proficiency for K-2 Schools
K-2 schools are evaluated based on two criteria: whether they meet the annual
measurable objectives and attendance rate. Because AIMS is not administered in these schools,
the AMO evaluation used the performance of their graduates on the third grade AIMS. As for
other schools, the conjunctive model is used. A K-2 school has to meet both the AMO and the
performance standard for attendance rate to make AYP. The percentage of students assessed is
not used in determining AYP for K-2 Schools
The Arizona Department of Education has the ability to track test scores across years.
ADE will create rosters of students who attend K-2 schools in the 2003 – 2004 school year. It
will then match these rosters to the students’ 2005 third grade AIMS test scores. The AIMS
scores of matched students are used to calculate the AMO for the K-2 school. This is then
compared to third grade AMO for the subject.
For a K-2 school to make the AMO in 2005, 53.3 percent of the matched students had to
pass reading and 43.3 percent of the matched students had to pass math.

Attendance Criteria for K-2 Schools
Minimum group size. A subject group is not evaluated if it had less than 40 test scores that met
the selection criteria.
Attendance rate was calculated the same way as regular school. To make AYP an
elementary school must have an attendance rate of 90 percent.
Calculation. The formula used to calculate the attendance rate is:
Schoolwide Attendance Rate =

Average Daily Attendance
Average Daily Membership

The attendance rate is rounded to two digits, e.g.: .891=.89; .896=.90.
Data used. The average daily attendance (ADA) and average daily membership (ADM) for the
100-day counts for all grades offered by a school, except for pre-school and kindergarten, are
used in the calculation.
Summary of AYP Determination for K-2 Schools

Arizona’s Accountability System: Volume II

Arizona Department of Education

20

To make AYP a school has to meet the AMO and the school-wide attendance goal of 90
percent.
Table 7.1 below summarizes the AYP determination process for K-2 schools. The
conjunctive model requires that all cells in the table have a “yes” for a school to make AYP.
Table 7.1. AYP Determination for K-2 Schools
Met AMO for Reading?
Yes/No

Met AMO for Math?
Yes/No

Arizona’s Accountability System: Volume II

Met Goal for Attendance Rate?
Yes/No

Arizona Department of Education

21

8. Calculation of Adequate Yearly Progress for
Small Schools
The No Child Left Behind Act requires that a state evaluate all schools. Consequently,
an alternative methodology for determining adequate yearly progress (AYP) had to be developed
for schools that did not have any grade with 40 students enrolled. All the calculations are done
the same way for small schools as the regular schools. There are two differences: (a) Three years
of data is used in the calculations (b) Small schools do not get safe harbor part of the calculation.
This is explained in detail below.
Meeting the 95% tested Requirement
For this calculation, the current year percent tested is calculated as well as the three year average.
In the current year, if 95 % of the students were tested, the school has met the 95% requirement.
The formula used to calculate the percent tested in the current year is:
Percent tested =

# tested in 2005
# enrolled in 2005

Data is aggregated across three years to evaluate whether 95% of the students were tested in the
past three years. The formula used to calculate Percent tested is:
Percent tested =

# tested in 2003 + # tested in 2004 + # tested in 2005
# enrolled in 2003 + # enrolled in 2004 + # enrolled in 2005

Meeting the Annual Measurable Objectives Small Schools
Annual measurable Objectives is calculated by aggregating data for the past three years.
Students excluded from this calculation are students who did not start the year at the school and
students who do not have a score on the test. For small schools, there is no safe harbor because
improvement cannot be determined.
The formula used to calculate the percent passing is:

Percent passing =

# passed in 2003 + # passed in 2004 + # passed in 2005
# tested in 2003 + # tested in 2004 + # enrolled in 2005

The upperbound of 99% confidence interval is also calculated for small schools. Please
refer to regular school calculations which are discussed in an earlier chapter.
Arizona’s Accountability System: Volume II

Arizona Department of Education

22

Meeting the Additional Indicator
Attendance rate was calculated the same way as regular school but a three year average is
determined using data from three years. To make AYP an elementary school must have an
attendance rate of 90 percent.
Calculation. The formula used to calculate the attendance rate is:
Schoolwide Attendance Rate =

Average Daily Attendance
Average Daily Membership

The attendance rate is rounded to two digits, e.g.: .891=.89; .896=.90.
Data used. The average daily attendance (ADA) and average daily membership (ADM) for the
100-day counts for all grades offered by a school, except for pre-school and kindergarten, are
used in the calculation.
Graduation Rate
To make AYP a high school must have a 4-year graduation rate of 71%. A three average
graduation rate of 71% is required to meet the additional indicator.

9. Determining Adequate Yearly Progress for
School Districts and Charter Holders

Arizona’s Accountability System: Volume II

Arizona Department of Education

23

The No Child Left Behind Act requires that local education agencies (LEAs), districts
and charter holders, be evaluated for Adequate Yearly Progress. The method for determining
AYP (AYP) for districts is analogous to that used for schools with data being aggregated to the
district level as if a district were one large school. 2 The details of the AYP calculation for
districts are nearly identical to that for schools.
•

Districts are evaluated for percentage of students passing AIMS, percentage of students
assessed, and a third indicator.

•

Annual Measurable Objectives (AMOs) and the performance goals for percentage of
students assessed, attendance rate, and graduation rate are the same for districts as they
are for schools.

•

The applicable subgroups for AYP evaluation are the same for districts as they are for
schools.

•

Confidence intervals, safe harbor provisions, and minimum group size requirements are
applied to district AYP using the same methodology and parameters as for school AYP.

•

District AYP uses a conjunctive model. To make AYP, a district must meet all the
performance standards for all subjects, grades, and subgroups that are applicable.

Differences between District and School AYP Evaluation Methods
There are four differences between the AYP evaluation method used for districts and that
used for schools.
1. Measure of student mobility. NCLB requires that students mobile with respect to an
entity are not included in the AMO part of the AYP evaluation. For a school, this meant
excluding students who did not start the year at that school. District level mobility is
determined by whether the student started the school year at the district. If the student
did not start the school year at the district, she is excluded from the AMO calculation.
2. Limit on the number of students with alternative assessment who count toward meeting
the proficiency standard. NCLB mandates that the number of students who take an
alternative assessment who count as being proficient may not be greater than 1 percent of
the total number enrolled in the grades tested. For the 2005 AYP determination, students
who took the AIMS-A are considered to have taken an alternate assessment. Federal
guidance requires that students be treated consistently at all levels of accountability.
Therefore a student who is deemed not proficient because her district exceeded the 1
percent cap will be deemed not proficient when determining if her school met AYP as
well.
Example. Gila Monster Elementary District has 1000 students enrolled in grades
three, five, eight and ten. Only one percent can be counted as proficient for AMO for
AIMS-A. One percent of 1000 is 10, therefore, if 20 students too the AIMS-A and 15
of them were proficient, only 10 of them will be counted as proficient when

2

All statements in this section apply to both districts and charter holders. For the sake of brevity, we use “district”
to refer to both types of entities/LEAs.

Arizona’s Accountability System: Volume II

Arizona Department of Education

24

determining if Gila Monster Elementary District met the AMO. The other five
students will be counted as not proficient.
3. Graduation/Attendance Rates. Graduation rate is used as the third indicator required by
NCLB for unified and high school districts. Attendance rate is used for elementary
districts.

Arizona’s Accountability System: Volume II

Arizona Department of Education

25

10. Adequate Yearly Progress (AYP) Appeals
Process
Procedure and Timeline
PROCEDURE
The Adequate Yearly Progress (AYP) Appeals Process developed by the Arizona
Department of Education (ADE) provided districts and schools the opportunity to appeal 2004 –
2005 AYP determinations. In accordance with Title I, Section 1116 of the No Child Left Behind
Act of 2001 (NCLB), the ADE allowed districts and schools to appeal their respective AYP
determinations for statistical and/or substantive reasons.
Step 1: Data Correction
The first step in completing the AYP Appeals Process required all districts and schools to
review all data in order to confirm its accuracy. The data correction took place utilizing the AZ
LEARNS/Adequate Yearly Progress (NCLB) Application through the Common Logon located at
the ADE’s Website. Data correction took place June 27– August 12, 2005. In the application,
schools/districts were asked to verify and correct:
•

•

SAIS/Student Details Demographic Data, which included Student’s SAIS number,
date, month and year of birth, number of years student has been classified at level of
English proficiency (ELLYear), did the student receive a 504 accommodation on the test
(AP504), did the student take the test in Braille, did the student take the test using a test
booklet printed in large print (Lprint).
All other data could be corrected in the SAIS system. The data in SAIS will overwrite
the data in AIMS download.

It is important to note that districts and charter holders were solely responsible for
verifying information for their districts and schools. If a district or charter holder did not verify
the information for its district and schools through the correction process, the ADE assumed the
schools on file and the data available were correct as listed.
Districts and/or schools failing to complete the data correction process in its entirety
by the deadline forfeited their right to file an NCLB or AZ LEARNS statistical appeal.

Arizona’s Accountability System: Volume II

Arizona Department of Education

26

Step 2: Appeal Application
Administrators choosing to appeal a district or school AYP determination must have
completed the AYP Appeal Application(s), which was accessible via the Common Logon during
the specified appeal timeframe in order to indicate the exact issue(s) of the appeal(s). Appeals
were only accepted through the website application. Appeals sent to ADE via email, fax, or
mail/delivery were not be accepted.
Districts and schools were able to appeal AYP determinations in two categories: data
(statistical) and non-data (substantive) reasons – districts and schools were not limited to one
category and were able to appeal in both if necessary.
Statistical Appeals
Appeals based on statistical arguments could argue one or more of the following:
1. Calculation of 95% tested. This included appeals that addressed the accuracy of
enrollment data and/or number of test documents in the analysis.
2. Calculation of AMO due to the inclusion of invalid scores. This type of appeal included
appealing the inclusion of English Language Learners not yet considered English
proficient and/or the inclusion of Special Education students with non-standard
accommodations.
3. Calculation of additional indicators: attendance (elementary schools) and graduation rate
(high schools).
Substantive Appeals
Districts and schools that appealed based on substantive arguments could argue that
mitigating circumstances, outside of the district’s/school’s control, negatively impacted the
quantity or quality of test data. This included circumstances that affected test conditions, test
scores, percent tested, and/or additional indicators (attendance – elementary schools, graduation
rate – high schools).
Important Notes for the Appeal Process
Administrators that chose to appeal a district or school AYP determination must have
clearly articulated the issue(s) they believe merited an appeal through the AYP appeal
application. Administrators must have submitted evidence that the issue(s) they believe merited
an appeal directly resulted in a significant decrease in student academic achievement as
demonstrated on AIMS and/or a decrease in student participation during the administration of
AIMS. The evidence must have been submitted to ADE at the time the appeal was submitted.
Failure to provide this evidence resulted in the appeal not being granted. Evidence submitted
after the appeal deadline closed was not considered. Once appeals were submitted through the
Common Logon, the school/district/charter holder received an email verifying that the appeal
was received.

Arizona’s Accountability System: Volume II

Arizona Department of Education

27

The ADE, if necessary, requested that a district or school administrator provide additional
information/evidence to assist in the appeals process. Only those requests for additional
information that were provided during the specified timeframe allotted were included in the
appeals process. Requests submitted after the specified timeframe were excluded from the
appeals process. Unsolicited additional information submitted after the appeal deadline was not
accepted.
Both district and school AYP determinations were separate and distinct. Districts
and schools had to submit separate appeals for both if necessary. Appealing the school
determination did not have an impact on the district determination. Appealing the school
determination did not have an impact on the district determination or vice versa.
Step 3: Appeal Resolution
After all appeals were submitted and the appeals window closed, the ADE began to
process the appeals. Appeals were addressed categorically, not necessarily in the order received,
so the fact that a district or school submitted its appeal during the first day of the appeal window
did not mean it necessarily received a decision first during the resolution process. The appeal
resolution process was implemented in three stages.
Stage 1 – Statistical Appeals Process
All appeals of a statistical nature based on data discrepancies were reviewed.
Appeals that challenged the calculation of 95% tested and/or attendance/graduation were
processed by verifying that the information taken from the Student Accountability Information
System (SAIS) and the numbers used in the calculations were true and accurate.
Note: It is the responsibility of the school/district/charter holder to ensure that the
information reported to SAIS was accurate and the district’s/school’s numbers match those
reported to ADE.
Appeals that challenged the calculation of AMO due to the inclusion of invalid scores
were evaluated using three modified data sets. The first was a data set that excluded the English
Language Learners (ELL) from the calculations to determine if the school would in fact have
met the AMO Objective had it not been for the special education group of students. The second
modified data set excluded the Special Education students from the calculations to determine if
AYP would be met without the special population of students. The third modified data set
excluded both the ELL and Special Education students from the calculations.
All statistical appeals needed to be supported with compelling evidence. For example, if
the percent of students tested objective was not met in the ELL subgroup because of the
miscoding of ELL students on the test, evidence of miscoding needed to be provided. Simply
stating “ELL numbers at ADE don’t match the district’s or school’s” was not compelling; ADE
needed to know why the numbers were different; meaning that particular student needed to be
identified as miscoded.

Arizona’s Accountability System: Volume II

Arizona Department of Education

28

Note: In the past, some schools, when providing information in the appeals, mentioned
specific details about students such as name, id #, ethnicity, and specific attendance/student
record information which violates guidelines set forth by FERPA. Schools were strongly
encouraged to follow FERPA guidelines in the future. When referring to students in
appeals, identifying student information such as name, id#, etc. was not to be submitted
with the appeal. Instead, students were to be referred to as student #1, student #2, etc.
Stage 2 – Substantive Appeals Process
Substantive appeals were resolved in a committee process. All committee members
represented a diverse background to ensure that appeals were considered from multiple
perspectives.
Once the committee was assembled, the appeals were evaluated utilizing an appeals
rubric that evaluated the significance of the argument presented and how the circumstances
presented in the argument affected the district’s or the school’s performance. The committee
based their decisions on the following criteria:
1. Was the circumstance that affected the school outside the school’s control? If the
district or school was negligent in its test administration and/or data collection, the appeal
was not deemed relevant and the appeal was not considered. For example, if the district
or school forgot to test a certain class in a certain grade and remembered after the test
window closed, that circumstance was not outside of the district’s/school’s control and
therefore not a valid argument for appeal. Conversely, if the district or school did test
everyone and some of the tests were lost by the testing contractor, then that would be
outside of their control.
2. Did the special circumstance actually have an impact on the performance? Not all
circumstances at a district or school impacted test data. For example, if the district or
school had a long-lasting construction project on campus, did the actual test environment
suffer during the test week? How? Or, if a teacher left mid-year, did the learning
environment suffer? How? If the answers to these questions did not show adequate
impact on testing environment, then the event most likely did not affect the actual
performance at the district or school. Conversely, if it could be demonstrated that the
event did influence the scores then that was a valid argument.
3. Was this problem one that was recurring and likely to happen in the future? Appeals
made based on policy(s) at the district or school that impacted test collection/data results,
which contradicted ADE/NCLB policy(s), were not accepted. For example, if the
district’s or school’s enrollment numbers were inflated and they failed to make the 95%
tested objective because the school did not withdraw students after the 10th consecutive
missed day, opting instead to withdraw students after the 20th day, they could not appeal
the enrollment figures because they were not following ADE policy and the problem was
likely to happen again in the future if district/school policy did not change.
4. Was the problem eligible for appeal? Arguments that targeted NCLB regulations and
ADE policy were not valid. For example, districts or schools could not argue that the
95% tested threshold be lowered for their school or that certain subgroups be excluded
from the requirements.
Note: As mentioned in the statistical section, certain populations could be excluded

Arizona’s Accountability System: Volume II

Arizona Department of Education

29

from the analysis of AMO such as ELL and Special Education students after the
initial calculations were computed, but they could not be exempted from the 95%
tested and/or additional indicator requirements. All students and subgroups have
to be tested, attend school, and are expected to graduate.
5. Did the district or school provide compelling evidence of the circumstance?
Compelling evidence of impact needed to be provided to support all substantive appeals.
For example, if percent of students tested objective was not met, specific details to
support the claim needed to be provided with the appeal at the time it was submitted.
Simply stating “Students were absent and unable to make up the test” was not
compelling; the committee needed to know why the students were unable to make up the
test such as being extremely ill, suspended, incarcerated, or dealing with a family
emergency for the entire test window.
Note: In the past, some schools, when providing information in the appeals, mentioned
specific details about students such as name, id#, ethnicity, and specific attendance/student
record information which violated guidelines set forth by FERPA. Schools were strongly
encouraged to follow the FERPA guidelines in the future. When referring to students in
appeals, identifying student information such as name, id#, etc. was not to be submitted
with the appeal. Instead, students were to be referred to as student #1, student #2, etc.
Appeal Resolution Notes
If the district/school/charter submitted both a statistical and a substantive appeal, the
statistical appeal was evaluated first. Only after the statistical arguments were exhausted was the
appeal sent to the substantive committee for evaluation.
The appeals submitted should have addressed the appropriate category. During the first
year of appeals, some appeals addressed issues that were not relevant. For example, some
districts and schools presented arguments that ELL and Special Education students should be
taken out of the analysis (an appeal for the AMO category) yet their school failed to meet the
95% tested objective. Therefore, the appeal was not relevant and was denied. Since everyone
had to be tested at the 95% level, the school could not argue that ELL and SPED students be
excluded from this requirement. Districts and schools in these situations should have addressed
their appeal rationales as to why the school did not test 95% of the students.
Districts and schools needed to be certain that if they failed in two AYP categories the
appeal addressed both deficiencies. Some appeals submitted in the first year addressed only one
deficient area. While those arguments were compelling in that criterion, the overall AYP
designation for the school did not change because part of the AYP designation was not addressed
in the appeal.
Districts and schools needed to be certain to provide all information/support when
submitting the appeal; late information to support the appeal was not accepted (unless ADE
specifically asked for additional information as noted above).

Arizona’s Accountability System: Volume II

Arizona Department of Education

30

Again, both district and school AYP determinations were separate and distinct.
Districts and schools had to submit separate appeals for both if necessary. Appealing the
school determination will not have an impact on the district determination or vice versa.
Stage 3 – Notification of Result Sent to Districts and Schools
Once all appeals were resolved, notifications were sent to the districts and/or schools that
filed appeals. The contact person of record for the district/school would receive an email from
Achieve with directions as to how to access appeal information via the Common Logon when the
appeal had been processed. Districts and schools were notified before the final public release of
the AYP determinations as to the outcome of the appeal process. All appeals were final.

Arizona’s Accountability System: Volume II

Arizona Department of Education

31