Introduction to the PARCA Analysis of Spring 2007 Results
on the Alabama Reading and Math Test (ARMT) in Grades 3-8
Alabama tests reading and math in grades 3 through 8 using the Alabama Reading and Math Test, which was derived from the nationally normed Stanford Achievement Test, adjusted to include items that ensure complete coverage of the Alabama curricula for the various grade levels tested and exclude items not applicable to those curricula.
The State of Alabama uses these tests in its accountability system for public schools, and the scoring is calibrated accordingly. Accountability is necessarily related to minimum expected competency, not excellence. We wouldn’t set an “A-Plus” as the only passing grade on a test given to an eighth-grade math class, and identify everything below that level as a failing grade. In the same way, the State creates what we might term a “C” or “passing” grade that represents minimum expected competency on the ARMT tests. Student performance is categorized in four levels:
· Level I = does not meet the grade-level standard,
· Level II = partially meets the grade-level standard,
· Level III = meets the grade-level standard, and
· Level IV = exceeds the grade-level standard.
Large majorities of students score at Levels III and IV, which are considered the passing levels on these tests. For example, in third grade reading, 85% of Alabama students scored at Levels III and IV in the spring 2007 testing. In eighth grade reading, 72% of students scored at Levels III and IV. The results were similar in math and for all grade levels on both tests.
As would be expected, most of the “passing” students score at Level III. The percentages of students scoring at Level IV vary more widely from school system to school system. They provide a better gauge of the high standard of excellence that we should expect in our schools, and they tend to be more consistent with the performance of Alabama students on nationwide tests that are given to samples of students from time to time. For these reasons, we focus our PARCA analysis on Level IV results within and across key student subgroups.
A valuable aspect of the ARMT test results is that they are presented in terms of the percentages of students in various socioeconomic subgroups who score at the various levels on the tests. Historically there have been gaps in the results achieved by students from poverty-level backgrounds, and by Black students, when these two groups are compared with their counterparts from non-poverty and White households. These gaps do not result from differences in students’ capacities; all students are capable of learning at high levels when well taught. The differences, rather, stem from a number of environmental factors that place such students at greater risk of lower academic performance. These differences can be overcome when eliminating the gaps becomes a high priority, as many schools in Alabama are demonstrating today.
The PARCA analysis of each school system’s results on the spring 2007 ARMT tests includes two tables. The Powerpoint Presentation linked to this introduction contains those tables for all school systems in Alabama, with the 67 county systems listed alphabetically, followed by the 64 city systems listed alphabetically.
There are two tables for each school system, each presenting the results for both reading and math tests in grades 3-8. The grade and subject are represented by a combination such as “3R” for third grade reading. The test results are presented in terms of the percentage of students scoring at Level IV on the test.
One table contains results for White students and Black students. The first column under each subgroup heading shows the percentage of test-takers who were in that subgroup. This is useful for those who are interested in comparing systems by the characteristics of the student body. The next two columns show the percentage of students in the school system and statewide who scored at Level IV on each of the ARMT tests. The color-coding in the system column provides a visual way to compare the results for the system with statewide results on each test, based on the categories shown at the top of the page. If the system results were better than the statewide results, the cell is colored green, with darker green representing system results that were 10 or more points above the state. Gray cells are “too close to call,” meaning within plus or minus 1 point of the state percentage. Red cells represent performance below the statewide level, with dark red being 10 or more points below.
The second table for each system shows Poverty and Non-Poverty students side by side, in the same way.
In the green and red colored parts of the table, the comparisons focus on how a school system’s performance compared to the statewide performance for a particular subgroup. The statewide data with which a system is compared are the subgroup’s statewide score. Thus, the performance of Birmingham’s Black students is compared with the statewide performance of Black students, and the performance of Monroe County’s poverty students is compared with the statewide performance of poverty students.
In both tables, the box at the right compares the gap between the two subgroups for the school system, versus the gap between the two subgroups for the state as a whole. Here the focus is on reducing that gap, and the gold color in a cell identifies tests in which the system’s students not only scored above the statewide level for both subgroups compared, but also had a narrower gap between those high-scoring subgroups. If the system’s results for one or both of the subgroups compared were below the statewide results, no gold color was assigned even if the gap was small, since we do not want to highlight gap-closing that results from below-average performance.
Many patterns can be seen in the data when color-coded in this way, and it is our hope that the information is useful to school system administrators interested in focusing on areas where improvements are desired, and on benchmarking other systems that may have implemented practices leading to such improvements. We believe that these data should be used in the school system improvement process and not for labeling some school systems as better than others. There is no school system that is perfect; in fact, there is no perfect organization of any kind. Every school system has strengths and weaknesses at any given point in time, and only by facing the data squarely can it develop improvement plans. Good performance should be celebrated, and weak performance should be focused on, without in either case casting premature praise or blame. In both cases, tomorrow is a new day, and achieving success will bring challenges for every system, no matter what its results from the prior year.