Arizona’s School
Accountability System
2006
Technical Manual

Tom Horne
Superintendent of Public Instruction
Arizona Department of Education
Volume II: Adequate Yearly Progress
Research & Policy
Arizona Department of Education
Robert Franciosi, Deputy Associate Superintendent
Achieve@azed.gov
Published by the Arizona Department of Education, January 2007.
The Arizona Department of Education of the State of Arizona does not discriminate on the basis of race,
religion, color, national origin, sex, disability or age in its programs, activities or in its hiring and
employment practices. If you have questions or grievances related to this policy, please contact the
Administrative Services DAS at (602)542-3186.

Table of Contents
Table of Contents............................................................................................................................. i
List of Tables and Figures............................................................................................................... ii
1. Introduction................................................................................................................................. 3
2. Overview of the NCLB Evaluation System................................................................................ 5
Schools to Be Evaluated ............................................................................................................. 5
Proficiency Standards ................................................................................................................. 5
Percentage of Students Assessed ................................................................................................ 8
Applicable Subgroups................................................................................................................. 8
Additional Indicators of School Performance............................................................................. 8
Putting It All Together ................................................................................................................ 9
3. Timeline .................................................................................................................................... 10
4. Meeting the Annual Measurable Objectives for Proficiency.................................................... 12
Calculation of Annual Measurable Objectives (AMOs)........................................................... 12
Data Used.................................................................................................................................. 15
Special Rules............................................................................................................................. 15
5. Meeting the Standard for Number of Students Tested.............................................................. 16
Calculation ................................................................................................................................ 16
Data Used.................................................................................................................................. 16
Special Rules............................................................................................................................. 16
6. Other Indicators of School Performance................................................................................... 18
Attendance Rate ........................................................................................................................ 18
Graduation Rate ........................................................................................................................ 18
7. Calculation of Adequate Yearly Progress for K-2 Schools ...................................................... 20
Meeting the Annual Measurable Objectives for Proficiency for K-2 Schools ......................... 20
Attendance Criteria for K-2 Schools......................................................................................... 20
8. Calculation of Adequate Yearly Progress for Small Schools ................................................... 22
Meeting the 95 Percent Tested Requirement ............................................................................ 22
Meeting the Annual Measurable Objectives in Small Schools................................................. 22
Meeting the Additional Indicator.............................................................................................. 22
9. Determining Adequate Yearly Progress for School Districts and Charter Holders.................. 23
Differences between District and School AYP Evaluation Methods ....................................... 23
10. Adequate Yearly Progress (AYP) Appeals Process................................................................ 25
Procedure and Timeline ............................................................................................................ 25

Arizona’s Accountability System: Volume II

Arizona Department of Education

i

List of Tables and Figures
Table 1.1.
Table 2.1.
Table 2.2.
Table 2.3.
Table 4.1.

Comparison of Arizona's Accountability Systems……………………………….…...4
Calculation of Performance Starting Points…………………………………………...6
Annual Measurable Objectives (AMOs)…............…………………………………...7
Categories Evaluated Under NCLB for a K-5 Elementary School..…………….…....9
Rules for Determining UB99 for Small n and p……………………………….…….13

Arizona’s Accountability System: Volume II

Arizona Department of Education

ii

1. Introduction
The federal No Child Left Behind Act of 2001 (NCLB) requires states to establish an
accountability system to evaluate the performance of local public schools and school districts,
including charter schools. Specifically, states are required to:
•

Institute performance standards for reading/language arts, mathematics, and science.

•

Develop and administer tests to measure whether students meet these standards. By the
2005-06 academic year, states must give tests in reading/language arts and math for
grades three through eight. By the 2007-08 academic year, states must also administer a
test to evaluate student performance in science in elementary (grades 3-5), middle (grades
6-8), and high school (grades 10-12).

•

Establish a timeline to ensure that all students are proficient according to state standards
by 2013-2014.

•

Create a statewide accountability system to evaluate school progress in meeting the goals
of the timeline, and issue report cards informing parents of school performance.

In 2001 Arizona voters also approved Proposition 301 that among other things called for
a state accountability system for public schools. In 2001, Arizona also had in place state
standards and a test to measure whether students met them: Arizona’s Instrument to Measure
Standards (AIMS).
Since the passage of NCLB and Proposition 301 the staff of the Arizona Department of
Education (ADE) has worked with scholars, school officials ranging from superintendents to
teachers, and members of the public to develop an accountability system that fulfills the
requirements of both laws. The result is a system that consists of two linked components.
Arizona LEARNS was created to comply with Proposition 301. Its primary focus is on
longitudinal change through time of student performance. The system created to comply with
NCLB, commonly referred to as adequate yearly progress (AYP), provides a single-year
snapshot of school performance. Table 1.1 provides a brief comparison of the two accountability
systems.
The State of Arizona’s complete plan to meet the requirements of NCLB is contained in
the workbook submitted to the U.S. Department of Education. The workbook is available at
http://www.ade.az.gov/azlearns/workbook.asp.

Arizona’s Accountability System: Volume II

Arizona Department of Education

3

Table 1.1 Comparison of Arizona’s Accountability Systems
NCLB
Required by federal law

Arizona LEARNS
Required by state law

One-year snapshot of student performance

Longitudinal examination of student performance

Components of evaluation
• AIMS scores
• Percent students assessed
• Attendance/Graduation rates

Components of evaluation
• AIMS scores
• Measure of Academic Progress
• Graduation/dropout rates
• AYP
Labels schools on a graded scale:

Labels schools on a yes/no system

•
•
•
•
•

Arizona’s Accountability System: Volume II

Failing to meet academic standards
Underperforming
Performing
Highly performing
Excelling

Arizona Department of Education

4

2. Overview of the NCLB Evaluation System
This section provides an overview of the determination of adequate yearly progress
(AYP). More detailed discussions of the methodology used to determine AYP, including
descriptions of equations, algorithms, and data used are given in subsequent chapters.
The No Child Left Behind Act requires that every public school and district in a state—
as well as the state itself—be evaluated on three measures:
1. Progress toward meeting the goal of 100 percent proficiency in state standards;
2. Percentage of students assessed; and
3. An additional measure of school performance. NCLB mandates that for high
schools this indicator be the graduation rate. States may select an alternative
indicator for elementary schools. Arizona, along with many other states, has
chosen attendance rate for the other indicator for elementary schools.
If an entity—school, district, or state—passes on all three measures, then it is deemed to
have made adequate yearly progress (AYP).

Schools to Be Evaluated
All schools—including extremely small schools, new schools, and schools that only offer
grades K-2—must receive an AYP determination. Similarly, although the state’s system for
school accountability, Arizona LEARNS, allows alternative schools to be evaluated under
different criteria, NCLB requires all public schools in the state to be given an AYP designation
based on the same criteria.

Proficiency Standards
NCLB requires that every student in Arizona meet state standards in reading/language
arts and mathematics—that is, pass AIMS—by the year 2013-2014. To further this goal, the
state must set annual measurable objectives (AMOs) for each grade and subject evaluated. The
annual measurable objectives describe the yearly growth in the fraction of students passing
AIMS that is necessary for Arizona to reach the 100 percent requirement by 2013-2014. To
make AYP an entity must reach the AMOs for every subject in each grade it offers. If an entity
fails to reach an AMO, it still may be deemed to have made adequate yearly progress if it
satisfies the safe harbor provisions that will be described later.
For the 2003 through 2005 school years, AYP evaluations were based on the AIMS tests
administered in grades 3, 5, 8, and 10. The AIMS was first administered operationally in grades
4, 6, and 7 in the 2005 school year. However, the results from those grades were not
incorporated into AYP evaluations until the 2006 school year—the year required by law for
states to test in all grades.

Arizona’s Accountability System: Volume II

Arizona Department of Education

5

The Arizona Department of Education established the starting points and annual
measurable objectives in the manner specified by the No Child Left Behind Act. To determine
the baselines for each subject/grade combination, all schools in Arizona were ranked in
descending order according to the percentage of students passing AIMS for that subject and
grade. Then, cumulative enrollment was calculated adding upward from the bottom of the list of
schools. The baseline was then set to be equal to the fraction of students passing AIMS for that
grade and subject in the school where the cumulative enrollment was equal to 20 percent of state
enrollment for that grade.
The starting points and AMOs for grades 3, 5, 8, and 10 were set in 2003 with the 2002
AIMS test results used to set the baseline. The starting points and AMOs for grades 4, 6 and 7
were set in 2006 with the 2005 AIMS test results used to set the baseline.
Table 2.1 provides a hypothetical example of how the baselines were established. In this
case, we assume there are only eight schools in the state that offer third grade.

Grade
3

Table 2.1. Calculation of Performance Starting Points
Cumulative
percent of total
Percent
pass
state enrollment
Subject
School
Enrollment
Math
1
100
10
100
2
75
40
95
3
70
30
75
4
61
30
60
5
55
20
45
6
48
30
35
7
32
20
20
8
15
20
10

These eight schools are ranked in descending order by the percentage of their students
who passed the AIMS for third grade math (fourth column). The third grade enrollment for each
school is given in the fifth column. Starting from the bottom of the list, enrollment is summed
until the total equals 20 percent of the state’s total enrollment for that grade. In table 2.1 this
point is reached at School Seven, where the cumulative sum equals forty students (40/200 =
0.20). The percent of students passing for School 7 (32 percent) is then taken as the starting
point for the state for third grade math.
The annual measurable objectives were calculated as equal percentage-point increments
from the starting point to the 2014 goal of 100 percent. The AMOs cover three-year increments
through 2010 and one-year increments thereafter. This leads to a stepwise increase until 2010,
followed by a linear increase until 2014 (see table 2.2).

Arizona’s Accountability System: Volume II

Arizona Department of Education

6

Table 2.2 Annual Measurable Objectives (AMOs)
Reading AMO (percent
Math AMO (percent
passing)
passing)
Grade 3
2005-07
53.3
43.3
2008-10
62.6
54.6
2011
71.9
65.9
2012
81.2
77.2
2013
90.5
88.5
2014
100
100
Grade 4
2005-07
45
54.0
2008-10
56
63.2
2011
67
72.4
2012
78
81.6
2013
89
90.8
2014
100
100
Grade 5
2005-07
43.3
33.3
2008-10
54.6
46.6
2011
65.9
59.9
2012
77.2
73.2
2013
88.5
86.5
2014
100
100
Grade 6
2005-07
45
43
2008-10
56
54.4
2011
67
65.8
2012
78
77.2
2013
89
88.6
2014
100
100
Grade 7
2005-07
49
48
2008-10
59.2
58.4
2011
69.4
68.8
2012
79.6
79.2
2013
89.8
89.6
2014
100
100
Grade 8
2005-07
42.5
22.5
2008-10
54.0
38.0
2011
65.5
53.5
2012
77.0
69.0
2013
88.5
84.5
2014
100
100
High School
2005-07
35.8
25
2007-08
48.6
40
2010-11
61.4
55
2011-12
74.2
70
2012-13
87.0
85
2013-14
100
100

Arizona’s Accountability System: Volume II

Arizona Department of Education

7

There are two additional steps taken when determining if a school has met the AMO for a
specific subject and grade. First, rather than comparing the actual percentage of students who
are proficient to the AMO, a 99 percent confidence interval is calculated to estimate the percent
proficient. If the upper bound of this confidence interval is above the AMO, the school is
deemed to have met the objective.
Second, if a school fails to meet the objective after the confidence interval is applied, it
may still be deemed to have met the AMO if it meets the safe harbor provision. Safe harbor is a
two-part test that requires schools to demonstrate sufficient progress over the previous year in the
percentage of students failing to meet the standard and meet a threshold set by the Arizona
Department of Education for an additional indicator. Both of these refinements will be discussed
in more detail later.

Percentage of Students Assessed
In order for a school, district, or the state to make adequate yearly progress it must assess
95 percent of its students for each subject in every grade offered, including each applicable
subgroup. Students count as assessed if they had a valid score for AIMS or the alternative
assessment for the severely disabled, AIMS-A. Starting in 2006, in compliance with federal
guidance, students who tested with alternate accommodations were not counted as tested.
All the students enrolled for the day of testing (high school) or the first day of the week
the test was given (elementary) represent the population to be assessed.

Applicable Subgroups
In addition to assessing 95 percent of its students and meeting the annual measurable
objectives for all subject/grade combinations it encompasses, an entity must also meet the same
objectives for every applicable subgroup within each subject/grade combination. NCLB
specifies the following subgroups be evaluated: the five major ethnic groups—Hispanic, White,
African-American, Asian-Pacific Islander, and Native American—English Language Learners
(ELL), students with disabilities (SPED), and students from low-income families. A student is
identified as being from a low-income family if SAIS demographic information indicates she is
eligible for a free or reduced lunch. Students are considered program members (ELL, SPED, or
free or reduced lunch) if they were a member of that program at any time during the school year
at the school in which they were tested.

Additional Indicators of School Performance
NCLB requires that an additional indicator be used for AYP determinations. The law
mandates that a four-year graduation rate be used for high schools, but allows states to select the
standard schools must meet. The performance goal for the high school graduation rate was set at
71 percent, the state average graduation rate for 2001. To make adequate yearly progress, a high
school must have a four-year graduation rate of 71 percent, or show a 1 percentage-point
improvement in the graduation rate over the previous year.

Arizona’s Accountability System: Volume II

Arizona Department of Education

8

NCLB allows states to select the additional indicator used for elementary schools.
Arizona has chosen to use the school-wide attendance rate. The performance goal for the
attendance rate was set at 90 percent. To make AYP, elementary schools must have a schoolwide attendance rate of 90 percent, or show a 1 percentage-point improvement in the attendance
rate over the previous year.

Putting It All Together
Table 2.3 provides an example of how the three performance measures—proficiency in
state standards, percentage of students assessed, and an additional indicator—are combined to
determine whether a school has made AYP. The example given is for a middle school serving
grades 7 and 8. The school is evaluated based on student performance on AIMS reading and
mathematics tests for these two grades, the percentage of students evaluated for each test and
attendance rates. All the combinations for which a typical middle school would be evaluated
under NCLB are provided; there are 73 separate combinations examined.
NCLB requires that schools be evaluated using a conjunctive model. That is, to make
AYP, a school must meet the performance objective in every category in which it is evaluated.
For example, if the school in table 2.3 fails to meet the objective in any one of the cells in the
table, it fails to make AYP.

Table 2.3. Categories Evaluated Under NCLB for a K-5 Elementary School
Seventh
Eighth
Math
Reading
Math
Reading
Met
Met
Met
Met
Met
Met
Met
Met
95%
AMO?
95%
AMO?
95%
AMO?
95%
AMO?
tested?
tested?
tested?
tested?
All students
Yes/No Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
African American Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Asian-Pacific
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Islander
Hispanic
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Native American
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
White
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Special Education Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
English Language Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Learner
Low Income
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Y/N
Met Other School
Y/N
wide Indicator:
Attendance Rate?
Grade
Subject
Subgroup

Arizona’s Accountability System: Volume II

Arizona Department of Education

9

3. Timeline
Districts and charter holders are solely responsible for submitting the data necessary for
calculating achievement profiles for their schools and for ensuring its accuracy. Because of the
stakes involved and the volume and scope of the data used, the ADE considers it prudent to
allow districts and charter holders to review their data before preliminary AYP evaluations are
carried out.
From May 8, 2006 through June 29, 2006 schools and districts were given the
opportunity to review and correct the data used for calculating the four-year graduation rate used
in the AYP evaluation. From June 14, 2006 through August 15, 2006 schools and districts were
given an opportunity to review and correct their testing data through the common logon on the
ADE web site. The primary purpose of the application was to allow districts and charter holders
to correct the information for individual students. A link was provided through the common
logon that allowed schools to download student-level testing data in order to make any necessary
corrections.
The 2006 evaluations were the first in which all program membership and demographic
information relevant to AYP evaluations was taken by matching test records to the state’s SAIS
database of student records. Consequently the only information that schools needed to correct in
the ADE AIMS testing file were students’ SAIS ID’s (need for matching) and if the student
received alternate accommodations (only collected on the testing document). If program
membership or other information was incorrect, schools and districts were required to correct it
in the SAIS database.
IMPORTANT NOTE: The criteria used to select AIMS scores for evaluation of AYP differ
from the criteria used to select scores for AZ LEARNS. Indeed, the criteria differ among the
separate components of the AYP evaluation. The criteria also differ from the scores provided to
schools by the testing contractor, the scores publicly reported by ADE, and the scores available
through the ADE AIMS wizard located at www.ade.az.gov/profile/publicview.

Timeline
The timeline for the 2006 AYP determinations was:
•

May 8, 2006.

Start of statistical review and appeals of graduation rate data.

•

June 14, 2006.

Start of testing data verification process.

•

June 29, 2006.

Closing of statistical review and appeals of graduation rate data.

•

July 12, 2006.

Public release of AIMS Scores.

•

August 1, 2006.

Preliminary release of AYP evaluations for all schools and
districts; opening of appeals process.

Arizona’s Accountability System: Volume II

Arizona Department of Education 10

•

August 11, 2006.

Closing of appeals process.

•

August 15, 2006.

Deadline for all corrections to AIMS testing file and SAIS.

•

September 1, 2006.

Public release of AYP evaluations for all schools and districts.

Arizona’s Accountability System: Volume II

Arizona Department of Education 11

4. Meeting the Annual Measurable Objectives
for Proficiency
Calculation of Annual Measurable Objectives (AMOs)
This section describes the calculation used to determine if schools met the annual
measurable objectives (AMOs) for student proficiency in math and reading/language arts.
NCLB requires that schools meet the AMOs set by the state in order to make AYP. A
description of how the AMOS were set is given in section two. Schools must meet the AMOs
for each subject/grade combination and all the applicable subgroups.
The formula used to calculate the percentage of students passing is:
Percent Pass =

Number of students meeting/exceeding the standard on AIMS
Number of students tested

This fraction is rounded to two digits, e.g.: .941=.94; .946=.95.
To ensure that the decision regarding whether a school met the AMOs is statistically
reliable and not overly influenced by random factors, the determination for meeting the AMOs is
made employing a 99 percent (one-tailed) confidence interval. The confidence interval
methodology is designed to ensure that 99 out of 100 times the confidence interval will contain a
school’s true performance level. If the AMO in question is below the upper bound of the
confidence interval calculated for the school, the school is deemed to have met the standard.
Example. Twenty-nine percent of a school’s third graders passed the AIMS mathematics test.
The upper bound of the 99 percent confidence interval for this subject/grade combination for this
school is calculated to be 35 percent. Since this is greater than the intermediate goal of 32
percent, the school is considered to have met the standard.
Let p=the percent of students in a group passing the AIMS and n=the number of students
in the group. Then the equation for the upper bound of the 99 percent confidence interval
(UB99) is:

UB99 = p + 2.33 p(1 − p) n .
As can be seen from the equation, the confidence interval depends upon the percent of
students who passed the test, and the number of students tested. Thus, the confidence interval
will differ among grades, subjects, and schools.
The equation is an approximation of the confidence interval for a binomially distributed
variable. It uses the standard normal distribution and is sufficiently accurate if the group size
and percentage of students passing are large enough. For small values of n and small p, a more
accurate estimate of the confidence interval is made using statistical tables that provide

Arizona’s Accountability System: Volume II

Arizona Department of Education 12

confidence intervals for a binomially distributed variable. 1 The tables were applied using the
rules given in table 4.1.
Table 4.1. Rules for Determining UB99 for Small n and p.
If n>=30 and n < 35, and
p>=0 and p <.05, UB99=.16
p>=.05 and p < .10,
UB99=.25
p>=.10 and p<.15, UB99=.33
p>=.15 and p<.20, UB99=.38
p>=.20 and p<.25, UB99=.45
p>=.25 and p<.30, UB99=.51

If n>=40 and n<45, and
p>=0 and p <.05,
UB99=.13
p>=.05 and p < .10,
UB99=.22
p>=.10 and p<.15,
UB99=.28
p>=.15 and p<.20,
UB99=.35

If n>=35 and n < 40, and
If n>=45 and n<50, and
p>=0 and p <.05, UB99=.15
p>=.05 and p < .10,
UB99=.24
p>=.10 and p<.15, UB99=.30
p>=.15 and p<.20, UB99=.36
p>=.20 and p<.25, UB99=.43

p>=0 and p <.05,
UB99=.12
p>=.05 and p < .10,
UB99=.21
p>=.10 and p<.15,
UB99=.27
n>=50 and n < 55, and
p>=0 and p <.05,
UB99=.11
p>=.05 and p < .10,
UB99=.20
If n>=55 and n<60, and
p=0, UB99=.10
If n>=60 and n<100 and
p=0, UB99=.09
If n>=100 and n<200 and
p=0, UB99=.06
If n>=200 and p=0,
UB99=0

Even if after calculating the confidence interval the percent of students proficient in a
subgroup still falls short of the AMO, the group may still make AYP if its achievement
indicators meet certain safe harbor provisions. To make safe harbor a subgroup has to meet the
following two-part test:

1

Mansfield, Edwin. 1991. Statistics for Business and Economics, 4th Edition. New York: W.W. Norton and
Company. 280-284.

Arizona’s Accountability System: Volume II

Arizona Department of Education 13

a) Make a 10 percent decrease in the fraction of students failing to meet the standard
(i.e. failing AIMS) from the previous year, and
b) Meet the performance goal or make a one-percentage point improvement for the
additional indicator: graduation rate for high school and attendance rate for other
grades.
Examples
1.

In 2004, 20 percent of fifth graders in Gila Monster Elementary passed the AIMS
reading test. The upper bound of the confidence interval was 25 percent, still
below the annual measurable objective of 32 percent. However, in 2003, 10
percent of fifth graders passed the AIMS reading test, thus Gila Monster
Elementary saw a decrease of 11 percent in the percentage failing [(80-90)/90 = 11 percent]. Furthermore, the attendance rate for Gila Monster’s fifth grade was
96 percent, greater than the standard of 90 percent. So, Gila Monster’s fifth
graders make AYP in reading.

2.

In 2004, 20 percent of eighth graders in Javelina Middle School passed the AIMS
reading test. The upper bound of the confidence interval was 27 percent, still
below the annual measurable objective of 31 percent. In 2003, 15 percent of fifth
graders passed the AIMS reading test, thus Javelina Middle School saw a
decrease of only 6 percent in the percentage failing [(80-85)/85 = -6 percent].
Even though the attendance rate for Javelina’s eighth grade was 96 percent,
greater than the standard of 90 percent, it fails to make the safe harbor provisions,
and so does not make AYP in eighth grade reading.

3.

In 2004, 30 percent of third graders in Gila Monster Elementary passed the AIMS
reading test. The upper bound of the confidence interval was 40 percent, still
below the annual measurable objective of 44 percent. However, in 2003, 20
percent of third graders passed the AIMS reading test, thus Gila Monster El. saw
an improvement of 13 percent in performance [(70-80)/80 = -13 percent].
However, the attendance rate for Gila Monster’s third grade was 85 percent, less
than the standard of 90 percent and identical to last year’s attendance rate, so Gila
Monster’s third graders fail to make AYP in reading.

4.

In 2004, 20 percent of third graders in Saguaro Elementary passed the AIMS
reading test. The upper bound of the confidence interval was 30 percent, still
below the annual measurable objective of 32 percent. However, in 2003, 10
percent of fifth graders passed the AIMS reading test, thus Saguaro Elementary
saw an improvement of over 11 percent in performance [(80-90)/90 = -11
percent]. The attendance rate for Saguaro’s third grade was 80 percent, less than
the standard of 90 percent. However, in 2002, the attendance rate for Saguaro’s
third grade was 81 percent. Since Saguaro saw an 11 percent improvement in the
fraction of third graders meeting the standard in math and a 1 percent
improvement in the attendance rate for third graders, it meets the safe harbor
provision for third grade math, and thus makes AYP.

Arizona’s Accountability System: Volume II

Arizona Department of Education 14

Data Used
Students are included in the calculation if they meet the following criteria:
•

Have taken either the AIMS or AIMS-A and received a score of FFB or above;

•

Did not take the test with alternate accommodations;

•

Were enrolled in the school for the full academic year. A student was considered
enrolled for the full academic year if she enrolled in a school during the first 10
school days of the school year and remained enrolled up through the testing date.
This information was obtained from SAIS. Students not matched to SAIS were
assumed to be continuously enrolled.

Special Rules
Minimum group size. A group or subgroup is not evaluated if it had less than 40 test scores that
meet the selection criteria.
English language learners. For AMO determinations, former English language learners who
have become proficient are included in the English language learner subgroup for two additional
years.

Arizona’s Accountability System: Volume II

Arizona Department of Education 15

5. Meeting the Standard for Number of Students
Tested
Calculation
This section describes the calculation used to determine if a school has assessed 95
percent of its students. To make AYP, schools must test 95 percent of their students in reading
and mathematics in all grades in which AIMS is administered, and must test 95 percent of their
students in each applicable subgroup.
The formula used to calculate the percentage of students tested is:
Percent Tested =

Number of students tested
Number of students enrolled

The fraction of percent tested is rounded to two digits, e.g.: .941=.94; .946=.95.

Data Used
Number of students tested. All students who take either the AIMS or AIMS-A and received a
score of FFB or above. Students who receive a score of Did Not Attempt (DNA) or took the test
with alternate accommodations are excluded from the calculation.
Number of students enrolled. The denominator for the percent tested calculation is enrollment
count at the school level. For grades 3 through 8 enrollment used for all subjects is the first day
of the week of testing as reported to the SAIS. For grade 10, enrollment used is for the day the
test was administered. Students are counted in both schools if they were concurrently enrolled in
more than one school on the relevant day. Students are counted in the enrollment of a program
subgroup (ELL, special education, free or reduced lunch) if they participated in that program at
the school in which they were tested at anytime during that school year.

Special Rules
Minimum group size. A group or subgroup is not evaluated if it had less than 40 students
enrolled on the relevant day. A sample size of 40 was considered large enough to provide
statistically meaningful results.
Best of current year or three-year average. If a school does not test 95 percent of its students in
a subgroup for the current year, a three-year average of percent of students tested is calculated
using the following formula:
Percent tested =

# tested in 2004 + # tested in 2005 + # tested in 2006
# enrolled in 2004 + # enrolled in 2005 + # enrolled in 2006

Arizona’s Accountability System: Volume II

Arizona Department of Education 16

If the three-year average is greater than or equal to 95 percent then the subgroup is deemed to
have met the goal of testing 95 percent of its students.

Arizona’s Accountability System: Volume II

Arizona Department of Education 17

6. Other Indicators of School Performance
Attendance Rate
This section describes the calculation used to determine if a school met the other
performance indicators for AYP. NCLB requires that schools be evaluated on a third
performance indicator as well as percentage of students assessed and percentage of students
proficient in the standard. The law requires that graduation rate be used for the third indicator
for high schools, and gives states the discretion to choose the third indicator for elementary
schools. Arizona has chosen the school-wide attendance rate as the third indicator for
elementary schools. To make AYP a high school must have a graduation rate of 71 percent; an
elementary school must have an attendance rate of 90 percent.
Calculation. The formula used to calculate the attendance rate is:
Schoolwide Attendance Rate =

Average Daily Attendance
Average Daily Membership

The attendance rate is rounded to two digits, e.g.: e.g.: .891=.89; .896=.90.
Data used. The average daily attendance (ADA) and average daily membership (ADM) for the
100-day counts for all grades offered by a school, except for pre-school and kindergarten, are
used in the calculation.
Safe Harbor. If a school demonstrates a one-percentage point improvement in its attendance
rate from the previous year, it is deemed to have met the performance standard. The growth rate
is rounded to the nearest hundredth of a point, e.g. .009 = .01, .004=.00.
Example. Gila Monster Elementary had an attendance rate in 2005 of 88 percent, less
than the standard of 90 percent. However, its 2004 attendance rate was 86 percent. Gila
Monster Elementary demonstrated an improvement of two percentage points over the
previous year, and so is deemed to have met the requirements for attendance rate.
Special rules. A school’s attendance rate is not evaluated if it had an ADM of less than 40.

Graduation Rate
The Graduation Rate is a four-year, longitudinal measure of how many students graduate
from high school. The formula used to calculate the graduation rate is:

Arizona’s Accountability System: Volume II

Arizona Department of Education 18

Graduation Rate =

Number in cohort that graduated in within four years
Original cohort + transfers in - transfers out

The graduation rate is rounded to two digits, e.g.: .705=.71; .704=.70.
Data used. Federal requirements mandate that Arizona use the four-year graduation rate rather
than the five-year rate used for Arizona LEARNS. The threshold graduation rate was for the
cohort class of 2005, which represents the most recent graduation rate statistics. The graduation
rate for the cohort class of 2004 was used for the determination of safe harbor.
Safe Harbor. If a school demonstrates a one percentage point improvement in its graduation rate
from the previous year, it is deemed to have met the performance standard. The growth rate is
rounded to the nearest hundredth of a point, e.g. .005 = .01, .004=.00.
Example. Gila Monster High School had a graduation rate in 2004 of 69 percent, less
than the standard of 71 percent. However, its 2003 graduation rate was 67 percent. Gila
Monster High demonstrated an improvement of two percentage points over the previous
year, and so is deemed to have met the requirements for graduation rate.
Special rules. A school’s graduation rate is not evaluated if it had a cohort of less than 40.

Arizona’s Accountability System: Volume II

Arizona Department of Education 19

7. Calculation of Adequate Yearly Progress for
K-2 Schools
The No Child Left Behind Act requires that a state evaluate all schools. Consequently,
an alternative methodology for determining adequate yearly progress (AYP) had to be developed
for schools that did not offer any of the grades in which AIMS is administered. In Arizona, this
group consisted of schools that offered grades two and below.

Meeting the Annual Measurable Objectives for Proficiency for K-2 Schools
K-2 schools are evaluated based on two criteria: whether they meet the annual
measurable objectives and attendance rate. Because AIMS is not administered in these schools,
the AMO evaluation used the performance of their graduates on the third grade AIMS. As for
other schools, the conjunctive model is used. A K-2 school has to meet both the AMO and the
performance standard for attendance rate to make AYP. The percentage of students assessed is
not used in determining AYP for K-2 Schools
The Arizona Department of Education has the ability to track test scores across years.
ADE created rosters of students who attended K-2 schools in the 2004-05 school year. It then
matched these rosters to the students’ 2006 third grade AIMS test scores. The AIMS scores of
matched students are used to calculate the AMO for the K-2 school. This was then compared to
third grade AMO for the subject. If the percentage of students proficient (using a confidence
interval) was equal to or greater than the AMO, then the group was deemed to have met the
AMO in that subject. The evaluation was carried out for the entire group of students. Subgroups
were not evaluated.
Minimum group size. A subject group is not evaluated if it had less than 40 test scores that met
the selection criteria.

Attendance Criteria for K-2 Schools
Attendance rate was calculated the same way as regular school. To make AYP an
elementary school must have an attendance rate of 90 percent.
Calculation. The formula used to calculate the attendance rate is:
Schoolwide Attendance Rate =

Average Daily Attendance
Average Daily Membership

The attendance rate is rounded to two digits, e.g.: .891=.89; .896=.90.

Arizona’s Accountability System: Volume II

Arizona Department of Education 20

Data used. The average daily attendance (ADA) and average daily membership (ADM) for the
100-day counts for all grades offered by a school, except for pre-school and kindergarten, are
used in the calculation.
Minimum group size. A subject group is not evaluated if it had less than 40 test scores that met
the selection criteria.

Arizona’s Accountability System: Volume II

Arizona Department of Education 21

8. Calculation of Adequate Yearly Progress for
Small Schools
The No Child Left Behind Act requires that a state evaluate all schools. Consequently,
an alternative methodology for determining adequate yearly progress (AYP) had to be developed
for schools that did not have any grade with 40 students enrolled. All the calculations are done
the same way for small schools as the regular schools. There are two differences: (a) Three years
of data is used in the calculations (b) Small schools do not get safe harbor part of the calculation.
This is explained in detail below.

Meeting the 95 Percent Tested Requirement
For this calculation, the current year percent tested is calculated as well as the three year
average. In the current year, if 95 percent of the students were tested, the school has met the 95
percent requirement. The formula used to calculate the percent tested in the current year is:
Percent tested =

# tested in 2006
# enrolled in 2006

Data is aggregated across three years to evaluate whether 95 percent of the students were tested
in the past three years. The formula used to calculate percent tested is:

Percent tested =

# tested in 2004 + # tested in 2005 + # tested in 2006
# enrolled in 2004 + # enrolled in 2005 + # enrolled in 2006

Meeting the Annual Measurable Objectives in Small Schools
Annual measurable objectives are calculated by aggregating data for the past three years.
The same rules are used for excluding students as with other schools. For small schools, there is
no safe harbor because improvement cannot be determined.
The formula used to calculate the percent passing is:
Percent passing =

# passed in 2004 + # passed in 2005 + # passed in 2006
# tested in 2004 + # tested in 2005 + # tested in 2006

The upper bound of a 99 percent confidence interval is also calculated for small schools.
Please refer to regular school calculations which are discussed in an earlier chapter.

Meeting the Additional Indicator
Additional indicators for small schools are calculated in the same manner as for other
schools.

Arizona’s Accountability System: Volume II

Arizona Department of Education 22

9. Determining Adequate Yearly Progress for
School Districts and Charter Holders
The No Child Left Behind Act requires that local education agencies (LEAs), districts
and charter holders, be evaluated for adequate yearly progress. The method for determining
AYP (AYP) for districts is analogous to that used for schools with data being aggregated to the
district level as if a district were one large school. 2 The details of the AYP calculation for
districts are nearly identical to that for schools.
•

Districts are evaluated for percentage of students passing AIMS, percentage of students
assessed, and a third indicator.

•

Annual measurable objectives (AMOs) and the performance goals for percentage of
students assessed, attendance rate, and graduation rate are the same for districts as they
are for schools.

•

The applicable subgroups for AYP evaluation are the same for districts as they are for
schools.

•

Confidence intervals, safe harbor provisions, and minimum group size requirements are
applied to district AYP using the same methodology and parameters as for school AYP.

•

District AYP uses a conjunctive model. To make AYP, a district must meet all the
performance standards for all subjects, grades, and subgroups that are applicable.

Differences between District and School AYP Evaluation Methods
There are four differences between the AYP evaluation method used for districts and that
used for schools.
1. Measure of student mobility. NCLB requires that students mobile with respect to an
entity are not included in the AMO part of the AYP evaluation. For a school, this meant
excluding students who did not start the year at that school. District level mobility is
determined by whether the student started the school year at the district. If the student
did not start the school year at the district, she is excluded from the AMO calculation.
2. Limit on the number of students with alternative assessment who count toward meeting
the proficiency standard. NCLB mandates that the number of students who take an
alternative assessment who count as being proficient may not be greater than 1 percent of
the total number enrolled in the grades tested. For the 2005 AYP determination, students
who took the AIMS-A are considered to have taken an alternate assessment. Federal
guidance requires that students be treated consistently at all levels of accountability.
Therefore a student who is deemed not proficient because her district exceeded the 1
2

All statements in this section apply to both districts and charter holders. For the sake of brevity, we use “district”
to refer to both types of entities/LEAs.

Arizona’s Accountability System: Volume II

Arizona Department of Education 23

percent cap will be deemed not proficient when determining if her school met AYP as
well.
Example. Gila Monster Elementary District has 1000 students enrolled in grades
three, five, eight and ten. Only one percent can be counted as proficient for AMO for
AIMS-A. One percent of 1000 is 10, therefore, if 20 students took the AIMS-A and
15 of them were proficient, only 10 of them will be counted as proficient when
determining if Gila Monster Elementary District met the AMO. The other five
students will be counted as not proficient.
3. Graduation/Attendance Rates. Graduation rate is used as the third indicator required by
NCLB for unified and high school districts. Attendance rate is used for elementary
districts.

Arizona’s Accountability System: Volume II

Arizona Department of Education 24

10. Adequate Yearly Progress (AYP) Appeals
Process
The Adequate yearly progress (AYP) Appeals Process developed by the Arizona
Department of Education (ADE) provides districts and schools the opportunity to appeal their
AYP determinations. In accordance with Title I, Section 1116 of the No Child Left Behind Act
of 2001 (NCLB), the ADE allows districts and schools to appeal their respective AYP
determinations for statistical and/or substantive reasons.

Procedure and Timeline
Step 1: Data Correction. The first step in completing the AYP Appeals Process required all
districts and schools to review all data in order to confirm its accuracy. Data correction took
place May 8 through August 15, 2006. It is important to note that districts and charter holders
were solely responsible for verifying information for their districts and schools. If a district or
charter holder did not verify the information for its district and schools through the correction
process, the ADE assumed the schools on file and the data available were correct as listed.
Step 2: Appeal Application. Administrators choosing to appeal a district or school AYP
determination completed the AYP Appeal Application, which was accessible via the common
logon during the specified appeal window. Appeals were only accepted through the website
application. Appeals sent to ADE via email, fax, or mail/delivery were not accepted.
Districts and schools were able to appeal AYP determinations in two categories: data
(statistical) and non-data (substantive) reasons – districts and schools were not limited to one
category and were able to appeal in both if necessary. Statistical appeals are appeals of the
accuracy of the data used in the AYP determination. Given the extensive time allowed to view
and correct the data, it is expected that any errors should be corrected by the time preliminary
profiles are released. Statistical appeals were not granted unless the underlying data was
corrected. Substantive appeals are arguments by districts and schools that circumstances outside
of the district’s/school’s control negatively affected school performance on any of the AYP
indicators.
Administrators that chose to appeal a district or school AYP determination must have
clearly articulated the issues they believe merited an appeal. Administrators must have
submitted evidence that the issues they believe merited an appeal directly resulted in a significant
decrease in student academic achievement as demonstrated on the AIMS and/or a decrease in
student participation during the administration of AIMS. The evidence must have been
submitted to ADE at the time the appeal was submitted. Failure to provide this evidence resulted
in the appeal not being granted. Evidence submitted after the appeal deadline closed was not
considered.
Once appeals were submitted through the Common Logon, the
school/district/charter holder received an email verifying that the appeal was received.

Arizona’s Accountability System: Volume II

Arizona Department of Education 25

NOTE: In order to protect student privacy and the integrity of the appeals process,
schools were asked to refer to a specific student only by that student’s SAIS ID. The SAIS ID
was required so that ADE staff could verify the contentions in the appeal.
The ADE, if necessary, requested that a district or school administrator provide additional
information/evidence to assist in the appeals process. Only those requests for additional
information that were provided during the specified timeframe allotted were included in the
appeals process. Requests submitted after the specified timeframe were excluded from the
appeals process. Unsolicited additional information submitted after the appeal deadline was not
accepted.
District and school AYP determinations were separate and distinct. Districts and schools
had to submit separate appeals for both if necessary. Appealing the school determination did not
have an impact on the district determination or vice versa.
Step 3: Appeal Resolution. After all appeals were submitted and the appeals window
closed, the ADE began to process the appeals. Appeals were addressed categorically, not
necessarily in the order received, so the fact that a district or school submitted its appeal during
the first day of the appeal window did not mean it necessarily received a decision first during the
resolution process.
Statistical appeals were resolved only through recalculation of the AYP evaluation by
ADE staff using any corrected data submitted by the school. The purpose of a statistical appeal
is principally to advise ADE staff that data was in error and has been corrected. Calculations
submitted by schools via an appeal were not taken at face value nor used to alter an AYP
evaluation if the underlying data was not corrected.
Substantive appeals were resolved in a committee process. Committee members
represented a diverse background of ADE staff and school administrators to ensure that appeals
were considered from multiple perspectives. Appeals were evaluated using an appeals rubric
that evaluated the significance of the argument presented and how the circumstances presented in
the argument affected the district’s or the school’s performance. The committee based their
decisions on the following criteria:
1. Was the circumstance that affected the school outside of its control? Appeals involving
the adverse affect of school or district policies; errors made by school or district
personnel regarding test administration or data entry; or events whose impact could have
been foreseen and mitigated by school or district action were not considered valid
appeals.
2. Did the special circumstance actually have an impact on performance? Schools or
districts must have shown that the adverse circumstance had a real impact on test scores
or other performance measures.
3. Was this problem one that was recurring and likely to happen in the future? Appeals
regarding recurring events or circumstances, such as student demographics, were not
considered valid.

Arizona’s Accountability System: Volume II

Arizona Department of Education 26

4. Was the problem eligible for appeal? Arguments that targeted NCLB regulations and
ADE policy were not valid. For example, districts or schools could not argue that the 95
percent tested threshold be lowered for their school or that certain subgroups be excluded
from the requirements.
5. Did the district or school provide compelling evidence of the circumstance?
Compelling evidence of impact needed to be provided to support all substantive appeals.
For example, if the percent of students tested objective was not met, specific details to
support the claim needed to be provided with the appeal at the time it was submitted.
Simply stating “Students were absent and unable to make up the test” was not
compelling; the committee needed to know why the students were unable to make up the
test such as being extremely ill, suspended, incarcerated, or dealing with a family
emergency for the entire test window.
Once all appeals were resolved, notifications were sent to the districts and/or schools that
filed appeals. The contact person of record for the district/school received an email from
Achieve with directions as to how to access appeal information via the Common Logon when the
appeal had been processed. Districts and schools were notified before the final public release of
the AYP determinations as to the outcome of the appeal process. All appeals were final.

Arizona’s Accountability System: Volume II

Arizona Department of Education 27