Arizona’s School
Accountability System
2009
Technical Manual

Tom Horne
Superintendent of Public Instruction
Arizona Department of Education

Volume I: AZ LEARNS Achievement Profiles
Research & Evaluation
Arizona Department of Education
Robert Franciosi, Deputy Associate Superintendent
Achieve@azed.gov
Published by the Arizona Department of Education, March 2010.
The Arizona Department of Education of the State of Arizona does not discriminate on the basis of race, religion,
color, national origin, sex, disability or age in its programs, activities or in its hiring and employment practices. If
you have questions or grievances related to this guideline, please contact the Administrative Services DAS at
(602)542-3186.

Table of Contents
1. INTRODUCTION ...................................................................................................................................................4

WHAT’S NEW FOR THE 2009 AZ LEARNS ACHIEVEMENT PROFILES ......................................... 4
2. OVERVIEW OF THE AZ LEARNS EVALUATION SYSTEM ........................................................................6

GENERAL PROCESS TO CALCULATE AN ACHIEVEMENT PROFILE ................................................. 7
3. TIMELINE ............................................................................................................................................................... 8

TIMELINE ..................................................................................................................................... 8
4. CALCULATION OF AIMS STATUS AND GROWTH POINTS ......................................................................9

AWARDING STATUS POINTS ......................................................................................................... 9
AWARDING IMPROVEMENT POINTS ............................................................................................ 10
COMPUTATION OF THE AVERAGE ANNUAL CHANGE IN DIFFERENCE OF ACHIEVEMENT............ 11
CALCULATION OF SUBJECT/GRADE SCALE POINTS FROM AIMS ............................................... 12
SPECIAL CASES .......................................................................................................................... 13
5. THE MEASURE OF ACADEMIC PROGRESS ................................................................................................ 15

THE GROWTH INDEX .................................................................................................................. 15
CALCULATING EXPECTED GROWTH ........................................................................................... 15
AWARDING AZ LEARNS POINTS ............................................................................................. 16
SCHOOLS WITHOUT MAP ........................................................................................................... 17
6. GRADUATION AND DROPOUT RATES ......................................................................................................... 18

GRADUATION RATE ................................................................................................................... 18
DROPOUT RATE ......................................................................................................................... 19
7. EVALUATING PERFORMANCE ON THE ENGLISH LANGUAGE ASSESSMENT ............................... 21
8. EVALUATING THE TOTAL SCALE SCORE VALUE TO DETERMINE A SCHOOL
CLASSIFICATION ................................................................................................................................................... 22
9. APPLICATION OF THRESHOLD CRITERIA FOR EXCELLING AND HIGHLY PERFORMING
SCHOOLS .................................................................................................................................................................. 24
10. AZ LEARNS ACHIEVEMENT PROFILE APPEALS PROCESS ................................................................ 28

PROCEDURE AND TIMELINE ....................................................................................................... 28
11. AZ LEARNS ACHIEVEMENT PROFILES FOR ALTERNATIVE SCHOOLS ........................................ 32

DEFINITION OF AN ALTERNATIVE SCHOOL ................................................................................ 32
GENERAL PROCESS TO PRODUCE ACHIEVEMENT PROFILES FOR ALTERNATIVE SCHOOLS ......... 33
CALCULATION OF AN ACHIEVEMENT PROFILE FOR AN ALTERNATIVE SCHOOL ......................... 33
CLASSIFICATION SCALES FOR ALTERNATIVE SCHOOLS ............................................................. 34
12. AZ LEARNS ACHIEVEMENT PROFILES FOR K-2 SCHOOLS ............................................................... 35

Arizona's Accountability System: Volume I

Arizona Department of Education

i

List of Tables and Figures
TABLE 4.1 STATUS POINTS AWARDED ........................................................................................... 10
TABLE 4.2. PERCENTAGE OF STUDENTS ......................................................................................... 11
TABLE 4.3. YEAR-TO-YEAR CHANGE IN PERCENTAGES ................................................................. 11
TABLE 4.4. IMPROVEMENT POINTS ................................................................................................ 12
TABLE 4.5. RULES FOR DETERMINING UB95 FOR SMALL N AND P ................................................. 14
TABLE 5.1. PARAMETERS FOR CALCULATING EXPECTED GROWTH FOR MATH ............................. 16
TABLE 5.2. PARAMETERS FOR CALCULATING EXPECTED GROWTH FOR READING ......................... 16
TABLE 8.1. ELEMENTARY SCHOOL CLASSIFICATION CUT POINTS .................................................. 22
TABLE 8.2. HIGH SCHOOL CLASSIFICATION CUT POINTS ............................................................... 23
TABLE 8.3. COMBINATION SCHOOL CLASSIFICATION CUT POINTS ................................................. 23
TABLE 9.1. PARAMETERS FOR CALCULATING Z-SCORES ............................................................... 26
TABLE 9.2. NUMBER OF STUDENTS EXCEEDING THE STANDARD—THIRD GRADE ........................ 26
TABLE 9.3. AVERAGE Z-SCORES .................................................................................................... 27
TABLE 11.1. ALTERNATIVE SCHOOL CLASSIFICATION CUT POINTS ............................................... 34
TABLE 12.1. AZ LEARNS SCALE FOR K-2 SCHOOLS .................................................................... 35

Arizona's Accountability System: Volume I

Arizona Department of Education

ii

Appendices
APPENDIX I: SUBSTANTIVE APPEAL COMMITTEE EVALUATION RUBRIC FOR
AZ LEARNS ............................................................................................................................. 36
APPENDIX II: SUBSTANTIVE APPEAL RUBRIC FOR AZ LEARNS ............................................... 37
APPENDIX III: SUBSTANTIVE APPEAL COMMITTEE EVALUATION RUBRIC FOR
AZ LEARNS (FAILING SCHOOLS)............................................................................................. 38
APPENDIX IV: SUBSTANTIVE APPEAL RUBRIC FOR AZ LEARNS (FAILING SCHOOLS) ............ 39

Arizona's Accountability System: Volume I

Arizona Department of Education

iii

1. Introduction
In November of 2001, Arizona voters approved Proposition 301 which provided funds to
the Arizona Department of Education (ADE) to develop ―a system to measure school
performance based on student achievement, including student performance on the AIMS test.‖
The legislative requirements for the accountability system are stated in section 15-241 (ARS §
15-241) of the Arizona Revised Statutes. The accountability system created to satisfy the statute
is referred to as the Arizona LEARNS. The school evaluation given by ADE to each school is
referred to as the school’s achievement profile. This manual describes the method and processes
used to generate the 2009 AZ LEARNS achievement profiles. It provides formulas, parameters,
and business rules that make up the profile calculation. Its intent is to document and explain the
methods used and justify the policies adopted.

What’s New for the 2009 AZ LEARNS Achievement Profiles
Five changes to the AZ LEARNS school evaluation formula were approved by the State
Board of Education on March 23, 2009. They are:
1. The baseline year for aggregate growth was moved from 2005 to 2006.
2. The results of the federal adequate yearly progress (AYP) evaluations were removed
from the AZ LEARNS formula.
3. The rules for graduation and dropout rates were updated.
4. The rules for the calculation of profiles for alternative schools were updated.
5. The rules for K-12 schools that do not have points from graduation/ dropout rates or
from MAP were clarified.
The baseline was moved from 2005 to 2006 for two reasons. First 2006 was the first year
that the use of alternative accommodations was discontinued by the majority of schools in the
state. Second, it was the first year the test used for ELL classification was uniform across the
state. The intent is to have 2006 remain the baseline year in the future. This change is described
in detail in Chapter 4.
The AYP point was removed to simplify the AZ LEARNS evaluation. Originally school
AYP determinations were included because it was first understood that federal law required that
Arizona’s system had to be linked to AYP in some way. The Arizona Department of Education
obtained clarification from the United States Department of Education (US ED) on the issue. The
US ED’s interpretation of the ―single system‖ requirement is that it applies only to AYP.
Although the US ED would like the states to incorporated components of AYP in any state
system, the law does not currently require that state systems account for or incorporate AYP.
To account for the potential loss of a point, greater weight is now given to the MAP points via a
modified MAP formula. Details for these changes are included in Chapter 5.

Arizona's Accountability System: Volume I

Arizona Department of Education

4

The rules for graduation and dropout rates needed updating and clarification as they had
not been addressed since 2003. The seven changes in graduation/dropout rates were instituted for
AZ Learns in 2009. These included moving the baseline year for both graduation and dropout
rates, changing the amount of growth needed to be contingent on the current year, re-defining the
formula for average annual growth, clarifying when a school will be evaluated on grad-rate, the
minimum number of students in a cohort or group and specifying how points will be assigned for
schools without a grad-rate or a dropout rate. These changes are detailed in Chapter 6.
The calculation of profiles for alternative schools was updated to bring this ―parallel‖
profile more in line with that for other schools. This update includes both how the AIMS points
will be calculated and the number of points required for a profile of ―performing‖, as well as
requiring dropout rate to be evaluated for all schools and a single profile for schools serving both
elementary middle grades and high school grades. The details of these changes are explained in
Chapter 11.
Certain K-12 schools either 1) do not have sufficient students to be evaluated on MAP or
2) do not have sufficient students to be evaluated on graduation and dropout rates. The total
points available to the first type of schools is the same as for high schools so they were evaluated
using the high school scale. The total points available for the second type of schools is the same
as for elementary schools so they were evaluated using the elementary/middle school scale.
Details of these scales are presented in Chapter 8.

Arizona's Accountability System: Volume I

Arizona Department of Education

5

2. Overview of the AZ LEARNS Evaluation
System
This section provides an overview of how AZ LEARNS achievement profiles are
determined. More detailed discussions of the methodology used to determine the profiles,
including descriptions of equations, algorithms, and data used are given in subsequent chapters.
Arizona law (ARS § 15-241) mandates that the Arizona Department of Education shall
compile an annual achievement profile for each public school. It specifies that the profiles of
schools serving grades K-8 shall be based on:
Percent of students who pass AIMS.
Arizona Measure of Academic Progress (MAP).
Performance on the English language proficiency test.
The law specifies that the profiles of high schools shall be based on:
Percent of students who pass AIMS.
Dropout rate
Graduation rate
Performance on the English language proficiency test.
Starting in 2008, a school that serves both grades K-8 and high school receives a unified
achievement profile based on all of the above performance measures.
The law also calls for the ADE to use a research based methodology that shall:
Include performance of pupils at all achievement levels
Account for pupil mobility
Account for the distribution of pupil achievement
Include longitudinal indicators of academic performance.
A research based methodology is defined as ―the systematic and objective application of
statistical and quantitative research principles to determine a standard measurement of acceptable
academic progress for each school‖.

Arizona's Accountability System: Volume I

Arizona Department of Education

6

The law also calls for a system of parallel achievement profiles for accommodation
schools/alternative schools as defined by the Board of Education.

General Process to Calculate an Achievement Profile
The achievement profile for a school serving grades 3-8 consists of the following
performance measures:
1. A status measure based on the performance of students on all three sections of the AIMS
(reading, writing, and mathematics) in the current year.
2. A measure of improvement in aggregate student performance on the AIMS compared to
the baseline year.
3. A measure of growth in individual student performance. This is the Measure of Academic
Progress (MAP).
4. A measure of student performance on the state’s English language proficiency
assessment: AZELLA.
5. In order to create the incentive for schools to improve the achievement of average and
above-average students, a school cannot earn the highly performing or excelling labels
unless the percentage of its students exceeding the standard on AIMS met specific
thresholds.
The achievement profile for a high school is made up of all of the above components
except for measure 3 (MAP). In addition, the following performance measures are used for high
schools:
6. Dropout rate.
7. Graduation rate.
Schools are awarded scale score points based on their performance on measures one
through four, six, and seven. Scale score points are then summed up for each school and
compared to a scale that relates scale score points to the five profile labels: excelling, highly
performing, performing plus, performing, and underperforming. Performance measure five is
then examined to determine if the school has earned the highest labels of highly performing or
excelling.
Note: Prior to 2009, one additional point was available to schools for ―making‖ AYP. To
compensate for the possible loss of this point the student-level growth formula was modified to
give greater weight to the available MAP points.

Arizona's Accountability System: Volume I

Arizona Department of Education

7

3. Timeline
Districts and charter holders are solely responsible for submitting the data necessary for
calculating achievement profiles for their schools and for ensuring its accuracy. From June 13,
2009 through June 19, 2009 schools and districts were given an opportunity to review and
correct their testing data through on-line applications. The primary purpose of the process was to
allow districts and charter holders to correct the information for individual students. From March
3, 2009 through May 30, 2009 schools and districts were given the opportunity to review and
correct the data used for calculating the five-year graduation rate and dropout rates used in the
AZ LEARNS profile.
All program membership and demographic information relevant to AZ LEARNS profiles
were taken by matching test records to the state’s SAIS database of student records.
Consequently the only information that schools needed to correct in the ADE AIMS testing file
were students’ SAIS IDs (needed for matching). If program membership or other information
was incorrect, schools and districts were required to correct it in the SAIS database. Schools
were not allowed to correct the indicators for alternate testing modifications.
IMPORTANT NOTE: The criteria used to select AIMS scores for AZ LEARNS profiles differ
from the criteria used to select scores for adequate yearly progress under NCLB. Indeed, the
criteria differ among the separate components of the AZ LEARNS evaluation. The criteria also
differ from the scores provided to schools by the testing contractor, and the scores publicly
reported by ADE which are available here: http://azed.gov/researchpolicy/AIMSResults/.

Timeline
The timeline for AZ LEARNS Achievement Profile process was:
March 3, 2009.

Start of statistical review and appeals of graduation rate and
dropout rate data.

June 13, 2009.

Opening of test data verification process.

April 30, 2009.

Closing of application process for alternative schools.

May 30, 2009.

Closing of statistical review and appeals of graduation rate data.

June 15, 2009.

Preliminary release of AZ LEARNS achievement profiles for all
schools.

June 15, 2009.

Opening of window for appeals submissions.

June 26, 2009.

Closing of appeals window for all schools.

July 29, 2009.

Public release of AZ LEARNS achievement profiles for all
schools.

Arizona's Accountability System: Volume I

Arizona Department of Education

8

4. Calculation of AIMS Status and Growth
Points
In the AZ LEARNS profile, schools are awarded scale score points based on student
performance on the AIMS. Points are determined by performance in the current year, and
improvement in student performance from the baseline year.

Awarding Status Points
The following method is used to calculate status points for every subject and grade
offered by a school in which the AIMS test is administered. First, calculate the percent passing in
the current year. The percentage is compared to the scale given in table 4.1 which in turn gives
the status points achieved for the subject and grade.
Percent Passing in the current year

# Students Passing AIMS in the current year
# Students tested in the current year

The points are rounded to the nearest hundredth e.g. .675 = .68; .672 = .67.
Data Used
A student’s score is excluded from the calculation if any of the following criteria are met:
1. The student was not matched to SAIS with a valid ID.
2. Student received no score on the test.
3. The student received an alternate accommodation on the test.
4. Student was not English proficient. A student was considered not proficient if it
was indicated that she was a participant in the English language learner program
for three years or less.
5. The student was not enrolled in the school for the full academic year. A student
was considered enrolled for the full academic year if she enrolled in a school
during the first 10 school days of the school year and remained enrolled up
through the testing date.
The calculation for high schools includes all students in grades 10 through 12 who have
taken the AIMS either in the fall or spring. If a high school student took the test twice in a school
year in the same school, the higher of the student’s two scores was used.

Arizona's Accountability System: Volume I

Arizona Department of Education

9

Table 4.1 Status Points Awarded
Grade

Subject

3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
HS
HS
HS

Math
Read
Writ
Math
Read
Writ
Math
Read
Writ
Math
Read
Writ
Math
Read
Writ
Math
Read
Writ
Math
Read
Writ

Status
Points 1
< 51%
< 46
< 56
< 46
< 40
< 42
< 42
< 42
< 47
< 34
< 38
< 49
< 35
< 41
< 64
< 27
< 35
< 58
<5
< 16
< 18

Status
Points 2

Status
Points 3

Status
Points 4

Status
Points 5

51 to 64%
46 to 59
56 to 67
46 to 60
40 to 53
42 to 53
42 to 56
42 to 56
47 to 57
34 to 49
38 to 52
49 to 60
35 to 49
41 to 54
64 to 73
27 to 41
35 to 48
58 to 70
5 to 13
16 to 28
18 to 30

65 to 78%
60 to 73
68 to 78
61 to 75
54 to 69
54 to 65
57 to 72
57 to 72
58 to 69
50 to 66
53 to 69
61 to 73
50 to 66
55 to 69
74 to 82
42 to 59
49 to 65
71 to 82
14 to 29
29 to 46
31 to 48

79 to 88%
74 to 84
79 to 87
76 to 86
70 to 81
66 to 77
73 to 85
73 to 84
70 to 80
67 to 81
70 to 82
74 to 83
67 to 81
70 to 82
83 to 89
60 to 76
66 to 79
83 to 90
30 to 51
47 to 65
49 to 67

89 to 94%
85 to 92
88 to 93
87 to 93
82 to 90
78 to 85
86 to 92
85 to 92
81 to 87
82 to 90
83 to 91
84 to 90
82 to 90
83 to 90
90 to 94
77 to 87
80 to 88
91 to 95
52 to 70
66 to 79
68 to 80

Status
Points 6
≥ 95%
≥ 93
≥ 94
≥ 94
≥ 91
≥ 86
≥ 93
≥ 93
≥ 88
≥ 91
≥ 92
≥ 91
≥ 91
≥ 91
≥ 95
≥ 88
≥ 89
≥ 96
≥ 71
≥ 80
≥ 81

A school is awarded status points for each subject/grade it offers in which the AIMS test
is administered.
Example. In the current year, 66 percent of the students in Gila Monster Elementary passed the
math portion of the third grade AIMS. This value places the subject/grade in Status grouping
three. Gila Monster Elementary has earned three status points for this particular subject/grade.

Awarding Improvement Points
Determining a school’s improvement points for each subject/grade combination is based
on the average of the annual changes in the difference between the percentage of students
passing and the percentage of students scoring ―falls far below‖ on the AIMS. The improvement
points earned are determined by the average of the annual changes from 2006 or when a school
opened, whichever is latest. Improvement points are awarded for all grades three through eight
and high school. No minimum group size is applied.

Arizona's Accountability System: Volume I

Arizona Department of Education

10

Computation of the Average Annual Change in Difference of Achievement
Improvement points for a subject/grade are calculated in the following five steps:
1. For every year for which the school was open, or until 2006, whichever is latest, for
each subject/grade combination the percentage of student passing and the percentage
of students scoring ―falls far below‖ is computed.
2. The difference in these percentages is then taken.
3. The annual change between the differences is found.
4. The average of the annual changes is determined.
5. The average of the annual changes is compared to the new Improvement Points Table
(table 4.4) to determine the AZ LEARN points for the subject/grade combination.
All values are rounded to the nearest hundredth e.g. .675 = .68; .672 = .67.
Data Used
The rules for using student scores for the growth measure are the same as for the status
measure. For high school students who took the test multiple times, each student’s highest score
in each year is used.
Example. The following example demonstrates how growth points are calculated for a single
grade and subject. Table 4.2 shows the percentage of students passing and scoring ―falls far
below‖, and the difference in the two percentages from 2006 to 2009 of AIMS scores for a single
subject and grade for a hypothetical school.

Year
2006
2007
2008
2009

Table 4.2. Percentage of Students
Percent
Percent Falls
Passing
Far Below
52
18
54
17
57
15
60
10

Difference
34
37
42
50

Table 4.3 shows the year-to-year changes in the differences from table 4.2.
Table 4.3. Year-to-Year Change in Percentages
Years
Difference
Year-to year change
2006 to 2007
37 - 34
3
2007 to 2008
42 - 37
5
2008 to 2009
50 - 42
8

Arizona's Accountability System: Volume I

Arizona Department of Education

11

The growth measure is the average (3 + 5 + 8)/3 = 5.3.
The improvement point thresholds are given in table 4.4. This table is used to convert the
average annual improvement measure to AZ LEARNS improvement points.

Grade Subject
3-8
Math
3-8
Read
3-8
Writ
HS
Math
HS
Read
HS
Writ

AZ
LEARNS
Points 1
< -16%
< -14
< -16
< -16
< -12
< -10

Table 4.4. Improvement Points
AZ
AZ
AZ
LEARNS LEARNS LEARNS
Points 2
Points 3
Points 4
-16 to -9% -8 to 0%
1 to 9%
-14 to -8
-7 to 0
1 to 7
-16 to -8
-7 to 0
1 to 11
-16 to -7
-6 to 4
5 to 15
-12 to -5
-4 to 4
5 to 13
-10 to -2
-1 to 7
8 to 16

AZ
LEARNS
Points 5
10 to 17%
8 to 14
12 to 20
16 to 25
14 to 21
17 to 25

AZ
LEARNS
Points 6
≥ 18%
≥ 15
≥ 21
≥ 26
≥ 22
≥ 26

Calculation of Subject/Grade Scale Points from AIMS
The total scale score points derived from AIMS performance for each subject/grade
combination are calculated by adding the status points awarded to the AZ LEARNS
improvement points awarded. A 70 percent weight is given to the school’s strongest point value
(status or improvement) and a 30 percent weight to the other point value.
Example. In third grade mathematics, Gila Monster Elementary has earned three status points
and four improvement points. Because it has earned more scale points for its improvement, the
improvement points are given a 70 percent weight and the status points are given a 30 percent
weight. Thus, the total scale points earned for third grade math are (3 X 0.3) + (4 X 0.7) = 3.7.
Example. In mathematics, Desert Mountain Vista High School has earned five status group scale
points and two improvement points. Because it has earned more scale points for its status, the
status scale points are given a 70 percent weight and the improvement points are given a 30
percent weight. Thus, the total scale points earned for math are (5 X 0.7) + (2 X 0.3) = 4.1.
If a subject/grade group earned six status points, 100 percent of the weight is given to the
status points. As long as data is available for the past two years, for a subject/grade combination,
improvement points are both computed and combined with the status points.
Example. Gila Monster Elementary is missing data for fifth grade writing for both 2006 and
2007 but has scores for 2008 and 2009. Its status points for that subject/grade combination put it
in status group five and it received three improvement points. Because it has earned more scale
points for its status, the status points are given a 70 percent weight and the improvement points
are given a 30 percent weight. Thus, the total scale points earned for fifth grade writing are (5 X
0.7) + (3 X 0.3) = 4.4.

Arizona's Accountability System: Volume I

Arizona Department of Education

12

Example. The newly opened High Mountain Elementary has no test scores for any year prior to
2008. Its status points for fourth grade reading put it in status group three but received no
improvement points because there was no comparison year. 100 percent weight is given to the
status group in this case. Thus it receives three scale points for fourth grade reading.
The points derived from AIMS for all subject/grade combinations for a school are
averaged across grades by subject and added to the scale score values for other performance
measures.

Special Cases
Missing Data. If a school is missing current year AIMS test data for a subject/grade combination
it offers, it receives zero growth/status scale points for that subject/grade for the achievement
profile calculation for the year.
Small Grades. Subject/grade combinations with less than ten students meeting the selection
criteria in the current year were not evaluated and so receive zero growth/status scale points.
Second Look for Small Schools. Because of the high-stakes consequences of being labeled an
underperforming school, and because of the uncertainty of measurement involved with small
sample sizes, a school with a small number of students is given a ―second look‖ if it faces the
possibility of receiving an underperforming profile. If the preliminary label of a school is
underperforming, then the AIMS scale score points for that school are recalculated for each
subject/grade group that has less than 16 students. For each subject/grade combination with less
than 16 students, the upper bound of the 95-percent confidence interval is used to calculate to
which baseline group the school belongs. If the recalculated points move the school into a
classification higher than underperforming, the school receives a profile performing, regardless
of the points earned.
Let p = the percent of students in a group passing the AIMS and n=the number of
students in the group. Then the equation for the upper bound of the 95 percent confidence
interval (UB95) is:

UB95

p 1.96 p(1 p) n.

As can be seen from the equation, the confidence interval depends upon the percent of
students who passed the test and the number of students tested. Thus, the confidence interval will
differ among grades, subjects, and schools.
The equation is an approximation of the confidence interval for a binomially distributed
variable. It uses the standard normal distribution and is sufficiently accurate if the group size and
percentage of students passing are large enough. For small values of n and small p, a more
accurate estimate of the confidence interval is made using statistical tables that provide

Arizona's Accountability System: Volume I

Arizona Department of Education

13

confidence intervals for a binomially distributed variable.1 The tables are applied using the rules
given in table 4.5.
Table 4.5. Rules for Determining UB95 for Small n and p
If n ≥ 0 and n < 8, and
p ≥ 0.00 and p < 0.04, then UB95=0.42;
p ≥ 0.04 and p < 0.10, then UB95=0.50;
p ≥ 0.10 and p < 0.20, then UB95=0.60.

If n ≥ 16 and n < 20, and
p ≥ 0.00 and p < 0.04, then UB95=0.24;
p ≥ 0.04 and p < 0.10, then UB95=0.32;
p ≥ 0.10 and p < 0.20, then UB95=0.44.

If n ≥ 8 and n < 10, and
p ≥ 0.00 and p < 0.04, then UB95=0.37;
p ≥ 0.04 and p < 0.10, then UB95=0.45;
p ≥ 0.10 and p < 0.20, then UB95=0.55.

If n ≥ 20 and n < 24, and
p ≥ 0.00 and p < 0.04, then UB95=0.21;
p ≥ 0.04 and p < 0.10, then UB95=0.29;
p ≥ 0.10 and p < 0.20, then UB95=0.42.

If n ≥ 10 and n < 12, and
p ≥ 0.00 and p < 0.04, then UB95=0.33;
p ≥ 0.04 and p < 0.10, then UB95=0.41;
p ≥ 0.10 and p < 0.20, then UB95=0.52.

If n ≥ 24 and n < 30, and
p ≥ 0.00 and p < 0.04, then UB95=0.18;
p ≥ 0.04 and p < 0.10, then UB95=0.27;
p ≥ 0.10 and p < 0.20, then UB95=0.38.

If n ≥ 12 and n < 16, and
p ≥ 0.00 and p < 0.04, then UB95=0.27;
p ≥ 0.04 and p < 0.10, then UB95=0.35;
p ≥ 0.10 and p < 0.20, then UB95=0.47.

If n ≥ 30 and n < 40, and
p ≥ 0.00 and p < 0.04, then UB95=0.15;
p ≥ 0.04 and p < 0.10, then UB95=0.23;
p ≥ 0.10 and p < 0.20, then UB95=0.36.

1

Mansfield, Edwin. 1991. Statistics for Business and Economics, 4th Edition. New York: W.W. Norton and
Company. 280-284.

Arizona's Accountability System: Volume I

Arizona Department of Education

14

5. The Measure of Academic Progress
The measure of academic progress (MAP) is a measure of individual student academic
growth measured by performance on the AIMS reading and mathematics tests. (Writing is not
included in the MAP analysis.) It uses a conventional value-added approach to measuring student
progress.
Calculating the number of points earned from MAP has three steps:
1. Individual expectations of one year’s growth (OYG) are calculated for each student.
2. The expectation of OYG is subtracted from the actual growth achieved by the student
to determine a growth index.
3. The average growth index for school is calculated by averaging growth indices for
individual students across all grades and subjects. Schools are awarded AZ LEARNS
points based on this average.
Actual growth, expected growth, and the growth index are expressed in AIMS scale score points.

The Growth Index
In order to control for a ceiling effect and for student mobility, the MAP analysis is done
using a standard value-added model. The value-added model is used to calculate an estimate of
the expected for each student for each subject. The expected growth is then subtracted from
actual growth to determine a student’s growth index.
Example. Student A scored 478 on the 6th grade math test in 2008. Her expected growth for 7th
grade in 2009 is 38. In 2009 she scores a 528 on the 7th grade math test. Her actual growth is 528
– 478 = 50. Her growth index is 50 – 38 = 12.
Example. Student B scored 490 on the 4th grade reading test in 2008. His expected growth for 5th
grade in 2009 is 12. In 2009 he scores a 500 on the 5th grade reading test. His actual growth is
500 – 490 = 10. His growth index is 10 – 12 = -2.
The results of the MAP analysis for students and schools are reported in terms of the
growth index. Growth indices for individual students can be averaged across schools, grades, and
subjects to measure performance.

Calculating Expected Growth
The expected growth for an individual student is calculated using the following formula:
Expected Growth = A + B X (2008 scale score) + C X (FAY);

Arizona's Accountability System: Volume I

Arizona Department of Education

15

where FAY = 1 if the student has been enrolled for the full 2009 academic year and FAY = 0 if
not. A student is considered to have been enrolled the full 2009 academic year if she has enrolled
within the first two weeks of the school year and remained enrolled up to the first day of the
testing window.
The parameters A, B, and C are given in the tables below:
Table 5.1. Parameters for Calculating Expected Growth for Math
Student’s Grade
Current Year
A
B
C
4
98.9308
-0.1514
6.027
5
107.7715
-0.174
5.7754
6
75.6373
-0.1268
6.999
7
121.1295
-0.1896
7.1203
8
54.1785
-0.0823
7.0308

Table 5.2. Parameters for Calculating Expected Growth for Reading
Student’s Grade
Current Year
A
B
C
4
120.9638
-0.2269
3.7108
5
153.1619
-0.2879
4.106
6
105.8317
-0.2016
4.343
7
88.3119
-0.1492
5.1193
8
89.8856
-0.1688
7.0786
Example. Student A scored 478 on the 6th grade math test in 2008. She has been enrolled in her
current school all year. Her expected growth for 7th grade in 2009 is 121.1295 + (-0.1896) * 478
+ 7.1203 = 37.62 = 38.
Example. Student B scored 490 on the 4th grade reading test in 2008. He has not been enrolled in
his current school for the full year. His expected growth for 5th grade in 2009 is 153.1619 +
(-0.2879) * 490 = 12.09 = 12.
Please see the 2006 MAP report available on the ADE web site for a description of how
the parameters were developed.

Awarding AZ LEARNS Points
The Growth Index for a school is calculated by averaging the student-level growth
indices across all grades for math and reading. The number of AZ LEARNS points earned by a
school is calculated with the following formula:
AZ LEARNS points = 5.9 + .22 * (School’s growth index.)

Arizona's Accountability System: Volume I

Arizona Department of Education

16

The scale is bounded at the bottom by 2. So if the result of the above formula is less than
2, a school would earn 2 points. No maximum is placed on the number of points a school may
earn. This is to award schools that make outstanding growth with students. The point formula
was re-developed for 2009 to compensate for the potential loss of an AYP point that was
removed from the AZ LEARNS formula.
Example. School A’s growth index is -1.7. The number of AZ LEARNS points awarded to the
school is 5.9 + .22 * (-1.7) = 5.5.
Example. School B’s growth index is 3.2. The number of AZ LEARNS points awarded to the
school is 5.9 + .22 * (3.2) = 6.6.
Example. School C’s growth index is -20.0. Since 5.9 + .22 * (-20.0) = 1.5 is less than 2. The
number of AZ LEARNS points awarded to the school is 2.

Schools without MAP
MAP is only evaluated for grades four through eight using student scores from grades
three through eight.) Schools do not receive points from MAP if they do not have a grade
evaluated for MAP or if they have less than 16 scores in the MAP analysis. The AZ LEARNS
profile scale thresholds were adjusted in 2009 for schools not being evaluated on MAP to
compensate these schools for the potential loss of an AYP point that was removed from the AZ
LEARNS formula.

Arizona's Accountability System: Volume I

Arizona Department of Education

17

6. Graduation and Dropout Rates
The rules for schools earning points for their graduation and dropout rates were updated
in 2009 for the first time since 2003. High school continue to be able to earn a maximum of two
points total for their graduation rate and dropout rate in their AZ LEARNS profiles.

Graduation Rate
The Graduation Rate is a five-year, longitudinal measure of how many students graduate
from high school. The formula to calculate the one-year graduation rate (used to compute
average annual growth and for determining the growth needed to earn points) is:
One - year Graduation Rate

Number in cohort who graduated in within five years
Original cohort transfers in - transfers out

The three-year average is calculated by taking the total number of combined five-year
graduates for the most recent three graduating classes and dividing by the total number of
students in the combined classes.
Three - year Graduation Rate

2007 2008 2009 five - year graduates
Combined number of students in three adjusted cohorts

The graduation rate is rounded to two digits, e.g.: .705=.71; .704=.70.
A school can earn one AZ LEARNS scale point for its graduation rate in one of three
ways:
1. If its three-year average graduation rate is 90 percent or greater.
2. If its three-year average graduation rate is less than 90 percent and its current year
rate is greater than or equal to 74 percent, then a school will earn one point if the
average annual growth of its graduation rate is 1 percentage point.
3. If its three-year average graduation rate is less than 90 percent and its current year
rate is less than 74 percent, then a school will earn one point if the average annual
growth of its graduation rate is 2 percentage points.
In 2009 the baseline year for graduation rate was moved to 2006 or the school’s first year
serving grade 12, whichever is latest. A school’s annual average growth is calculated by
subtracting the baseline year’s rate from the current year’s rate and dividing by the number of
years spanned by the calculation.

Arizona's Accountability System: Volume I

Arizona Department of Education

18

Average Annual Growth

Current One - year Rate - Baseline One - year Rate
Number of Years in Span

A school will not be evaluated on graduation rate until it serves grade 12 for two years.
Additionally, there is a minimum cohort size of 15. Schools that are not evaluated on graduation
rate will receive double the number of points earned via the dropout rate.

Dropout Rate
The dropout rate is an annual measure of how many students drop out of a school during
a twelve-month reporting period. The formula to calculate the one-year dropout rate (used to
compute average annual decrease and for determining the decrease needed to earn points) is:

One - year Dropout Rate

Number of students who dropped out
Number of students who were enrolled during the school year

The three-year average is calculated by taking the total number of dropouts for the most
recent three years and dividing by the total number of students enrolled.

Three - year dropout rate

Total number of dropouts for 2007, 2008 and 2009
Total number of students enrolled in 2007, 2008 and 2009

Like the graduation rate, a school can earn one AZ LEARNS scale point for its dropout
rate in one of three ways:
1. If the three-year average dropout rate is 6 percent or less, the target is automatically
met.
2. If its three-year average dropout rate is greater than 6 percent and its current year rate
is greater than or equal to 9 percent, then a school will earn one point if the average
annual decrease of its dropout rate is 2 percentage points.
3. If its three-year average graduation rate is less than 6 percent and its current year rate
is less than or equal to 9 percent, then a school will earn one point if the average
annual decrease of its dropout rate is 1 percentage point.

In 2009 the baseline year from dropout rate was moved to 2006 or the school’s first year
of operation, whichever is latest. A school’s annual average growth is calculated by subtracting
the baseline year’s rate from the current year’s rate and dividing by the number of years spanned
by the calculation.

Arizona's Accountability System: Volume I

Arizona Department of Education

19

Average Annual Decrease

Baseline One - year Rate - Current One - year Rate
Number of Years in Span

A school will not be evaluated on dropout rate if it has less than 15 students in the group.
Schools that are not evaluated on dropout rate will receive double the number of points earned
via the graduation rate.

Arizona's Accountability System: Volume I

Arizona Department of Education

20

7. Evaluating Performance on the English
Language Assessment
State law governing AZ LEARNS now requires that the ―results of English language
learners tests‖ be included as part of the AZ LEARNS school evaluation. {ARS 15-241 (D) (3)
and ARS 15-241 (E) (4)} In order to comply with this requirement, the following change was
made to the AZ LEARNS profile.
A school receives one scale score point if the percentage of English language learner
students reclassified during the year across all grades served is greater or equal to 30 percent.
Only groups of 16 or more student were evaluated for this point. Students included in this
analysis were those identified as continuing English language learners (ELL), were continuously
enrolled in the ELL program within the school for at least 150 calendar days, were not
withdrawn from the program either due to Special Education criteria or by parent request, and
passed ELL integrity within SAIS. ELL integrity is a multiple step process which includes but is
not limited to requiring that:
1)
2)
3)
4)
5)

the student be enrolled in the school
the student have a recent assessment indicating need for an ELL membership
the student is in a grade past preschool
the student be attending a public or charter school within Arizona
the student be attending a public or charter school within a public or charter LEA

Arizona's Accountability System: Volume I

Arizona Department of Education

21

8. Evaluating the Total Scale Score Value to
Determine a School Classification
The tables below show the total number of scale score points that schools must earn in
order to receive a given classification. A school may receive up to 18 scale points from status
and growth; unlimited scale points from MAP if it is an elementary or middle school; a single
point if it was able to reclassify 30 percent of its English language learners, and up to two
graduation/drop out scale points if it is a high school. In 2009, the scale point a school could
possibly receive if it made adequate yearly progress (AYP) was removed. To compensate
schools who receive MAP points for the possible loss the AYP point, the formula for MAP
points was adjusted. To compensate schools who do not receive MAP points for the possible loss
of the AYP point, the profile performance thresholds were adjusted.
For each school, the applicable scale score cut points for classification labels are
calculated by averaging the AIMS scale points received for each subject; adding the ELL point
received; adding points from MAP if it is an elementary school; and adding points from the
graduation/dropout rate if it is a high school. The classification cut points for elementary schools
are given in the first two columns of table 8.1 and those for high schools are given in the first
column of table 8.2. In order for schools to be classified as highly performing or excelling, there
is an additional Z-score criterion that has to be met, which is explained in the next chapter.

Table 8.1. Elementary School Classification Cut points
Performance Thresholds
Profile

Underperforming
Performing
Performing Plus
Highly Performing
Excelling

Scale with
MAP

Non-MAP
Scale

<13
13 to 15.9
16 or more
16 or more
19 or more

<7.7
7.7 to 12.0
12.1 or more
12.1 or more
14.1 or more

Arizona's Accountability System: Volume I

z-score

NA
NA
0.45 to 0.99
1.00 or more

Arizona Department of Education

22

Table 8.2. High School Classification Cut points
Performance Thresholds
Profile
Scale Points
Underperforming
Performing
Performing Plus
Highly Performing
Excelling

z-score

<9.4
9.4 to 14.6
14.7 or more
14.7 or more
16.5 or more

NA
NA
0.45 to 0.99
1.00 or more

Table 8.3. Combination School Classification Cut points
Performance Thresholds
Profile

Underperforming
Performing
Performing Plus
Highly Performing
Excelling

Scale with
MAP

Non-MAP
Scale

<14
14 to 16.9
17 or more
17 or more
20 or more

<9
9 to 13.9
14 or more
14 or more
16 or more

z-score

NA
NA
0.45 to 0.99
1.00 or more

Special Rules: Schools with no subject/grade combination with at least ten students meeting the
selection criteria for the status measure in the current year were automatically assigned a
―Performing‖ profile.
A combination (K-12) school that has insufficient students to be evaluated on MAP will be
evaluated using the scale for high schools.
A combination (K-12) school that has insufficient students to be evaluated on graduation/dropout
rates will be evaluated using the scale for elementary/middle schools.

Arizona's Accountability System: Volume I

Arizona Department of Education

23

9. Application of Threshold Criteria for
Excelling and Highly Performing Schools
To ensure continued focus on improving the academic achievement of all students,
including those students currently demonstrating proficiency, threshold criteria are applied to
determine excelling and highly performing schools. To be deemed a highly performing or
excelling school, a school must meet certain levels in the percentage of its students exceeding the
standard on the AIMS. Schools must not only receive a total scale value that places them into
either excelling or highly performing, but must also meet the requisite percentage of students in
the exceeds-the-standard category on AIMS to be designated as either excelling or highly
performing schools. The application of threshold criteria for excelling and highly performing
schools results in the following scenarios.
1. A school that receives a total scale value that places it in the excelling classification and
meets the requisite percentage of students in the Exceeds level on AIMS necessary for an
excelling classification will be designated an excelling school.
2. A school that receives a total scale value that places it in the excelling classification and
did not meet the requisite percentage of students in the Exceeds level on AIMS necessary
for a excelling classification, but did met the requisite percentage of students in the
exceeds category on AIMS necessary for a highly performing classification will be
designated as a highly performing school.
3. A school that receives a total scale value that places it in the excelling classification and
did not meet either the requisite percentage of students in the Exceeds category on AIMS
necessary for the excelling classification or the highly performing classification will be
designated as a performing school. Such schools are recognized with the non-statutory
designation of performing plus.
4. A school that receives a total scale value that places it in the highly performing
classification and meets the requisite percentage of students in the Exceeds category on
AIMS necessary for an excelling classification will be designated as a highly performing
school.
5. A school that receives a total scale value that places it in the highly performing
classification and meets the requisite percentage of students in the Exceeds category on
AIMS necessary for a highly performing classification will be designated as a highly
performing school.
6. A school that receives a total scale value that places it in the highly performing
classification and did not meet either the requisite percentage of students in the Exceeds
category on AIMS necessary for an excelling classification or highly performing

Arizona's Accountability System: Volume I

Arizona Department of Education

24

classification will be designated as a performing school. Such schools are recognized
with the non-statutory designation of performing plus.
7. A school that receives a total scale value that places it in the performing classification
will be designated as a performing school, regardless if the school meets the requisite
percentage of students in the Exceeds category on AIMS necessary for an excelling
classification or a highly performing classification.
8. A school that receives a total scale value that places it in the underperforming
classification will be designated as an underperforming school, regardless if the school
meets the requisite percentage of students in the Exceeds category on AIMS necessary
for an excelling classification or a highly performing classification.
The determination of whether a school met the goals for percent exceeding is based on a
z-score calculated in the following manner:
1. For each school, the percent exceeding is calculated by grade across all three subjects for
2009 and for 2007 through 2009. The one-year estimation of percent exceeding gives
credit to schools that have shown improvement in the percent of students exceeding the
standard.
2. The percent-exceeding is converted into a z-score for each grade by subtracting the
statewide average for that grade for percent exceeding and dividing by the statewide
standard deviation for that grade. (These parameters are given in table 8.1)
Note: The same statewide parameters are used to calculate both the one-year and threeyear z-score. This is because the purpose of the z-score is to create standard scores
comparable across grades, not to create scores comparing a school’s performance to the
state norm. The z-scores are fixed parameters so if a school shows improvement over
time its z-score will increase regardless of the movement of the state average over the
same period.
3. The one- and three-year z-scores are averaged across all grades for a school. Each school
will have two z-scores: one based on its three-year average for percent exceeding, the
other based on the percent exceeding for the current year.
4. The higher of the one- and three-year averages are taken and compared to the
performance thresholds. To be a highly performing school, the school must first meet its
appropriate highly performing classification scale cut point, as explained in chapter 8,
plus the average z-score for a school must be greater than or equal to 0.45. To be an
excelling school, the school must first meet its appropriate excelling classification scale
cut point plus the average z-score for a school must be greater than or equal to 1.00.

Arizona's Accountability System: Volume I

Arizona Department of Education

25

Table 9.1. Parameters for Calculating ZScores
Grade
Average
Standard
Deviation
3
0.124576
0.093475
4
0.111652
0.087267
5
0.092426
0.080033
6
0.095892
0.085301
7
0.06592
0.0666
8
0.052172
0.056997
10
0.058823
0.082827

Example. The following table shows the third grade AIMS scores for Gila Monster Elementary
over the past three years.

Subject
Year
2007
2008
2009
Total

Table 9.2. Number of Students Exceeding the Standard—Third Grade
Reading
Mathematics
Writing
# Exceeding #Tested # Exceeding #Tested # Exceeding #Tested
25
100
24
100
23
100
24
105
23
105
22
105
26
99
25
99
24
99
75
304
72
304
69
304

The three-year average for percent of students exceeding the standard is:
Percent

75 72 69
304 304 304

Exceeding

23.6%.

The one-year average for percent of students exceeding the standard is:
Percent Exceeding

25 25 24
99 99 99

25.3%.

The three-year z-score for third grade is:
z - score

.236 - .125
1.19.
.093

The one-year z-score for third grade is:
z - score

Arizona's Accountability System: Volume I

.253 - .125
1.38.
.093

Arizona Department of Education

26

Gila Monster Elementary serves grades K-6. The one- and three-year average z-scores for the
entire school are:

Grade
K-2
3
4
5
6
Average

Table 9.3. Average Z-scores
One-year
Three-year
NA
NA
1.38
1.19
1.02
.86
.89
.72
.95
.80
1.06
.89

The highest z-score for Gila Monster Elementary is 1.06 for the one-year average. This z-score is
greater than 1.00, so Gila Monster Elementary would earn an excelling — if it has earned
sufficient scale points.
Special Rules: The minimum group size for computing z-scores is 16. If a school has less than
16 students with usable test scores for either the one-year or three-year calculation, then that zscore is not evaluated and the other is assigned. If a school has less than 16 students with usable
test scores for both the one-year and three-year calculations then the z-score is not evaluated for
the school.
For high schools, students who take an a subject test more than once during fall or spring testing
within the time period (one- or three- years) have only their highest score used within the
calculation.

Arizona's Accountability System: Volume I

Arizona Department of Education

27

10. AZ LEARNS Achievement Profile Appeals
Process
Procedure and Timeline
In accordance with A.R.S. §15-241, school administrators are allowed the opportunity to
appeal an achievement profile classification on behalf of the schools for which they are
responsible.
Step 1: Data Correction. The first step in completing the AZ LEARNS Appeals Process required
districts and schools to review their data in order to confirm its accuracy. Data correction took
place March 3, 2009 through June 19, 2009. It is important to note that districts and charter
holders were solely responsible for verifying information for their districts and schools. If a
district or charter holder did not verify the information for its district and schools through the
correction process, the ADE assumed the schools on file and the data available were correct as
listed.
Step 2: Appeal Application. Administrators choosing to appeal an AZ LEARNS profile
completed the AZ LEARNS Appeal Application, which was accessible via the common logon
during the specified appeal window. Appeals were only accepted through the website
application. Appeals sent to ADE via email, fax, or mail/delivery were not accepted.
Schools were able to appeal AZ LEARNS profiles in two categories: data (statistical) and
non-data (substantive) reasons – districts and schools were not limited to one category and were
able to appeal in both if necessary. Statistical appeals are appeals of the accuracy of the data used
in the AZ LEARNS profile. Given the extensive time allowed to view and correct the data, it is
expected that any errors should be corrected by the time preliminary profiles are released.
Statistical appeals were not granted unless the underlying data was corrected. Substantive
appeals are arguments by districts and schools that that circumstances outside of the
district’s/school’s control negatively affected school performance.
Administrators that chose to appeal an AZ LEARNS profile must have clearly articulated
the issues they believe merited an appeal. Administrators must have submitted evidence that the
issues they believe merited an appeal directly resulted in a significant decrease in student
academic achievement as demonstrated on the AIMS. The evidence must have been submitted to
ADE at the time the appeal was submitted. Failure to provide this evidence resulted in the appeal
not being granted. Evidence submitted after the appeal deadline closed was not considered. Once
appeals were submitted through the Common Logon, the school/district/charter holder received
an email verifying that the appeal was received.
NOTE: In order to protect student privacy and the integrity of the appeals process, schools were
asked to refer to a specific student only by that student’s SAIS ID. The SAIS ID was required so
that ADE staff could verify the contentions in the appeal.

Arizona's Accountability System: Volume I

Arizona Department of Education

28

The ADE, if necessary, requested that a district or school administrator provide additional
information/evidence to assist in the appeals process. Only those requests for additional
information that were provided during the specified timeframe allotted were included in the
appeals process. Requests submitted after the specified timeframe were excluded from the
appeals process. Unsolicited additional information submitted after the appeal deadline was not
accepted.
Step 3: Appeal Resolution. After all appeals were submitted and the appeals window closed, the
ADE began to process the appeals. Appeals were addressed categorically, not necessarily in the
order received, so the fact that a district or school submitted its appeal during the first day of the
appeal window did not mean it necessarily received a decision first during the resolution process.
Statistical appeals were resolved only through recalculation of the AZ LEARNS profile
by ADE staff using any corrected data submitted by the school. The purpose of a statistical
appeal is principally to advise ADE staff that data was in error and has been corrected.
Calculations submitted by schools via an appeal were not taken at face value nor used to alter a
profile if the underlying data was not corrected.
Schools that were labeled underperforming for a third consecutive year were entitled to a
site visit to determine if the label was warranted. These site visits were important as the third
year underperforming label merited an alternative designation of failing to meet the academic
standards in accordance with statute. After the statistical review of appeals took place, schools
labeled underperforming for the third consecutive time were scheduled to receive that visit.
Teams from the Research and Evaluation and State Intervention sections of the ADE visited each
of the third year underperforming schools to gather additional supporting data for the appeal as
well as gather information related to the school’s Arizona school improvement plan (ASIP) as
required by statute. All information gathered from the site visits was taken to the substantive
appeal committee for use in the third stage of the appeal process. Only third year
underperforming schools received site visits as part of the appeals process.
Substantive appeals were resolved in a committee process. Committee members
represented a diverse background of ADE staff and school administrators to ensure that appeals
were considered from multiple perspectives. Appeals were evaluated using an appeals rubric
approved by the State Board of Education that evaluated the argument presented and whether or
not the evidence provided to support the argument was compelling. The appeals rubric consisted
of a three-tiered system for appeal evaluation: 1) Initial review of the appeal to determine its
merit. 2) Review of the evidence provided. 3) Committee recommendation.
Initial Review. The substantive appeal rubric provided for three categories that apply during the
initial review. Each appeal was classified into the categories based on the information provided
in the appeals.
1. Mitigating Factors Outside of the School’s Control. Appeals of this nature referenced
when the school indicated significant issues that affected test scores outside of the
school’s control. If a school provided information detailing a significant event that
impacted test scores, which was clearly outside the school’s control, the appeal was
deemed as passing the initial review. Appeals involving the adverse affect of school or

Arizona's Accountability System: Volume I

Arizona Department of Education

29

district policies; errors made by school or district personnel regarding test administration
or data entry; or events whose impact could have been foreseen and mitigated by school
or district action were not considered valid appeals.
2. Implementation of the School Improvement Plan. Appeals in this category discussed
how the school was actively, consistently, and reliably implementing the school
improvement plan; that the priorities of the solutions team had been addressed; the school
improvement plan had been revised and updated to address assessed ongoing needs; and
that professional development that supported the targeted goals had been planned and
implemented. If a school provided information highlighting their school improvement
plan, it may have accentuated the other two components on the rubric. However, per
statute, a successful implementation of the school improvement plan alone cannot change
a classification of a school. Additionally, only third-year underperforming schools may
include information about their ASIP in their appeal.
Review of Evidence. Once the appeals progressed through the first tier of the rubric, initial
review, the evidence provided to support the appeal was evaluated. In this tier, three
determinations were possible:
1. Compelling evidence: In this area, the school or district adequately provided information
that led the committee to conclude that, had the circumstance been different, the
achievement profile would have been different as well. If a school had a special
circumstance that affected a certain grade and was able to demonstrate that the specific
grades test scores suffered, the school was deemed to have provided compelling
evidence. Again, if a school provided compelling evidence highlighting their school
improvement plan, it may have accentuated the other two components on the rubric (data
calculations/mitigating factors). However, per statute, a successful implementation of the
school improvement plan alone could not change the classification of a school.
2. Not compelling evidence: Appeals were categorized in this area when they were able to
provide information that a significant issue could have impacted the school’s
performance but did not provide detailed, specific information as to specific outcomes
that hindered the school’s performance.
3. Not applicable evidence: If an appeal was submitted, made it through the initial review,
and presented evidence that was not linked in any way to the performance of the school,
the evidence was deemed not applicable. If the evidence did not directly support the
claim made in the appeal, it was deemed not applicable.
Committee Recommendation. Once the appeal and evidence were reviewed, the committee
arrived at a decision as to the outcome of the appeal. There were three possible outcomes:
1. Appeal granted and AZ LEARNS determination changed. In these cases, the appeal
successfully made it through the initial review and evidentiary stages. It was determined
that the points needed to change classifications would have been earned by the school had
the special circumstance/data discrepancy not occurred. Therefore, the classification for
the school was changed.

Arizona's Accountability System: Volume I

Arizona Department of Education

30

2.

Appeal granted and AZ LEARNS determination remains the same. In these cases,
the appeal successfully made it through the initial review and evidentiary stages.
However, it was determined that the criteria needed to change classifications were not
earned by the school had the special circumstance/data discrepancy not occurred. For
example, a school provided information and evidence that their AYP point was not
accurately included in the calculations and the committee determined the school
provided information to prove they earned the one point set aside for AYP in the AZ
LEARNS formula. However, the school needed 5 points to get from underperforming
to performing. Therefore, the one point earned was not enough to change their
designation, so their determination remained the same.

3.

Appeal Denied. In these cases, the appeal did not successfully make it through the
initial review and evidentiary stages. Therefore, the classification for the school
remained the same.

Step 4. Notification of Result Sent to Schools. Once all appeals were resolved, notifications
were sent to the districts and/or schools that had filed appeals. The contact person of record for
the district/school received an email from Achieve with directions as to how to access appeal
information via the Common Logon when the appeal had been processed. Districts and schools
were notified before the final public release of the AZ LEARNS profiles as to the outcome of the
appeal process. All appeals were final.

Arizona's Accountability System: Volume I

Arizona Department of Education

31

11. AZ LEARNS Achievement Profiles for
Alternative Schools
In 2004, the ADE published profiles for alternative schools for the first time. Alternative
schools are defined as schools that meet the Board-approved definition as schools whose sole
and clearly-stated mission is to serve specific populations of at-risk students. Alternative school
status is granted by application to the ADE. A.R.S. §15-241 makes an allowance for a ―parallel‖
evaluation method for alternative schools. When AZ LEARNS achievement profiles were first
issued in 2002 the Board determined that alternative schools would not receive an achievement
profile using the conventional AZ LEARNS methodology, and that ADE should develop an
alternate method for evaluating these schools.

Definition of an Alternative School
The following is the definition of an alternative school as approved by the Board of Education in
2002. In 2009 there were 114 schools that were granted alternative school status.
1. A school operated by a school district must have adopted a mission statement that clearly
identifies its purpose and intent to serve a specific student population (please see criterion
three) that will benefit from an alternative school setting. A charter school must be
expressly chartered to serve a specific student population that will benefit from an
alternative school setting.
2. The educational program and related student services of the school must match the
mission or charter of the school.
3. The school must intend to serve students exclusively in one or more of the following
categories:
Students with behavioral issues (documented history of disruptive behavior)
Students identified as dropouts
Students in poor academic standing who are either severely behind on academic
credits (more than one year) or have a demonstrated pattern of failing grades
Pregnant and/or parenting students
Adjudicated youth
4. Any school offering secondary instruction for academic credit used to fulfill Arizona
State Board of Education graduation requirements (in part or in full) must offer a diploma
of high school graduation.

Arizona's Accountability System: Volume I

Arizona Department of Education

32

General Process to Produce Achievement Profiles for Alternative Schools
The method for calculating an achievement profile for alternative schools is as follows:
1. AIMS scale score points are calculated using the status point calculation as for all other
schools. Alternative schools do not receive points for improvement or MAP
2. Points based on an alternative school’s dropout rate are calculated as for all other schools.
Starting in 2009, dropout rate will be evaluated for all schools.
3. Alternative schools will only receive labels of performing and underperforming.
4. Schools service both elementary/middle and high school grades will receive a single
profile.
Because of the uncertainty of measurement associated with small sample sizes and the
high stakes of school labels, schools initially determined to be underperforming receive a
―second look.‖ Instead of determining baseline groups based on the mean percent of students
passing AIMS, an alternate baseline group for these schools is determined based on the upper
bound of a 95 percent confidence interval around the mean. If a school initially determined to be
underperforming moves to a higher classification due to the ―second look,‖ that school will
receive a performing label.

Calculation of an Achievement Profile for an Alternative School
The method used to calculate scale score points earned by alternative schools, regardless
of the grades served, for these performance measures are the same as the methods used for other
schools.
A school may receive up to six scale score points for each subject. Due to the small size
of alternative schools, status points are calculated using a multi-year average of scores.
Starting in the current year, scores from previous years are averaged until the total
number of scores is 32. The status points are averaged across grades and then summed
across subjects.
If a school may receive one point based on its dropout rate.
The total points earned by a school are added up and compared to the school
classification scale to determine a school’s preliminary classification.

Arizona's Accountability System: Volume I

Arizona Department of Education

33

Classification Scales for Alternative Schools
The following classification scales are used for all alternative schools.
Table 11.1. Alternative School Classification Cut Points
Scale Points
Underperforming
Performing

Arizona's Accountability System: Volume I

< 6.0
6.0

Arizona Department of Education

34

12. AZ LEARNS Achievement Profiles for K-2
Schools
When AZ LEARNS achievement profiles were first issued in 2002 the Board determined
that K-2 schools would not receive an achievement profile using the conventional AZ LEARNS
methodology, and that ADE should develop an alternate method for evaluating these schools. In
2004, the ADE published profiles for K-2 schools for the first time. K-2 schools are schools that
serve only grades kindergarten through second grade. Since the AIMS is not administered at any
of the grades served by these schools, the AZ LEARNS profiles are based solely on the
performance of the schools’ second graders on the state’s norm-referenced test.
The method of calculating the profile for these schools is straightforward:
1. The average normal curve equivalents (NCE) on the reading and math portions of the test
are calculated for the most current year for a school’s second graders.
2. The average normal curve equivalents for the school are added together, and
3. Compared to a scale to determine the school’s label.
Example. In 2008, the average NCE for second graders in Gila Monster Primary was 52 for math
and 48 for reading. The two averages summed together yield 52 + 48 = 100 points. This is
sufficient for Gila Monster Primary to be a highly performing school.
Table 12.1 provides the performance thresholds for K-2 schools.
Table 12.1. AZ LEARNS Scale for K-2 Schools
Achievement Profile

Points

Underperforming

<70

Performing

70 to 96.9

Highly Performing

97 to 105.9

Excelling

106 or more

Arizona's Accountability System: Volume I

Arizona Department of Education

35

Appendix I: Substantive Appeal Committee Evaluation Rubric for AZ LEARNS

Evaluation Criteria
Data Calculation
Discrepancies
i.e., school attempts to
compare data details with
their data sets and gets
different numbers
Special Circumstances
Outside the Control of
School/District
Administration or
Management
i.e., school indicates
significant teacher attrition;
environmental
issues/events; adverse
testing conditions;
school/community
emergency/crisis

Initial Review
(Please check the applicable option)
Not
ADE data are
Data does not match
applicable
accurate and
that of ADE. School
calculations are submits evidence of
correct.
discrepancies and
provides additional
data.

Review of Evidence
Provided
Compelling
evidence

Comments

Not compelling
evidence
Not applicable
evidence

Not
applicable

Policy/Methodology
Issues
i.e., school disagrees with
use of baseline

Arizona's Accountability System: Volume I

Special
circumstances
that were
outside of the
school’s control,
were not a
substantial
cause of the
overall school
performance.

School had a
situation that was
unavoidable and
outside of the
school’s control and
hindered the test
administration or
student performance.
This situation
resulted in adverse
data for the year(s) in
question.

Compelling
evidence
Not compelling
evidence
Not applicable
evidence

The ADE will not accept/review appeals related to policy/methodology.

Arizona Department of Education

36

Appendix II: Substantive Appeal Rubric for AZ LEARNS
Team Decision AZ LEARNS Substantive Appeal
(Results represent group consensus regarding appeal)
Reason Reviewed
Data Calculation
Discrepancies
Special
Circumstances

Policy/Methodological
Issues

Initial Review
Review of Evidence
N/A
Correct
Data does
N/A
Compelling
data/calculation not match
evidence
N/A

Did not cause
overall
performance

Adverse
result
based on
situation

N/A

Compelling
evidence

Not
compelling
evidence
Not
compelling
evidence

The ADE will not accept/review appeals related to policy/methodology.

Please indicate appropriate response(s) by checking within the box(es) provided.

Committee Recommendation:

Granted

Denied

Final Appeal Decision:

Granted

Denied

Comments:
Appeal Result:

Arizona's Accountability System: Volume I

Arizona Department of Education

37

Appendix III: Substantive Appeal Committee Evaluation Rubric for AZ LEARNS (Failing Schools)

Evaluation Criteria

Initial Review
(Please mark the applicable option)

Data Calculation Discrepancies

Not
applicable

i.e., school compares data details with their
data sets and gets different numbers

Mitigating Factors (incidents, situations,
or events that impact the quality or
quantity of data)

Not
applicable

i.e., school indicates significant teacher
attrition; environmental issues/events;
adverse testing conditions;
school/community emergency/crisis

Implementation of
School Improvement Plan
i.e., school is actively, consistently, and
reliably implementing the school
improvement plan; priorities of the
Solutions Team have been addressed; the
School Improvement Plan has been
revised and updated to address assessed
ongoing needs; professional development
that supports the targeted goals has been
planned and implemented.
Arizona's Accountability System: Volume I

Not
applicable

ADE data are
accurate and
calculations
are correct.

Mitigating
factors that
were outside
of the school’s
control were
not a
substantial
cause of the
overall school
performance.
School
Improvement
Plan was not
implemented
properly
and/or gains
were not
made.

Review of Evidence
Provided

Data does not match
that of ADE. School
submits evidence of
discrepancies and
provides additional
data.



School had a
mitigating factor that
was unavoidable,
outside of the
school’s control, and
hindered test
administration or
student performance.
This resulted in
adverse data for the
year(s) in question.
School Improvement
Plan was properly
implemented and
school is making
gains in growth.
















Comments

Compelling
evidence
Not
compelling
evidence
Not
applicable
evidence
Compelling
evidence
Not
compelling
evidence
Not
applicable
evidence

Compelling
evidence
Not
compelling
evidence
Not
applicable
evidence

Arizona Department of Education

38

Appendix IV: Substantive Appeal Rubric for AZ LEARNS (Failing Schools)
Team Decision AZ LEARNS Substantive Appeal
(Results represent group consensus regarding appeal)

Reason
Reviewed
Data Calculation
Discrepancies

Initial Review


Correct data/
calculation



Data does not
match



N/A



Compelling
evidence



Not
compelling
evidence

N/A



Did not cause
overall
performance



Adverse result
based on
situation



N/A



Compelling
evidence



Not
compelling
evidence

N/A



School
Improvement
Plan not a
factor



School
 N/A
Improvement
Plan
demonstrates
growth
Please indicate appropriate response(s) by checking within the box(es) provided.



Compelling
evidence



Not
compelling
evidence

Mitigating
Factors
Review of School
Improvement
Plan
Implementation



N/A

Review of Evidence





Committee Recommendation:

Maintain AP Label (list label)
Alternative AP Label (list label)

Appeal Comments:

Arizona's Accountability System: Volume I

Arizona Department of Education

39