- Mapping State Proficiency Standards onto the NAEP Scales: 2005-2009
- Mapping 2005 State Proficiency Standards onto NAEP Scales
State Analysis Overall
Over the past decade, AIR researchers have worked with the National Center for Education Statistics (NCES) in responding to new and evolving government programs concerned with making comparisons of standards and performance among and across states possible in multiple areas.
The development of state mapping methodology and research done by AIR has made it possible to compare the rigor of each states’ proficiency standards in assessments that are designed on an individual state level. By using NAEP as a common yardstick, AIR’s research allows the level of achievement required for proficient performance in one state to be compared with the level of achievement required in another state.
Inclusion of students with disabilities on the NAEP assessment is another area that varies by state. Research to minimize these differences while increasing participation rates where possible has been undertaken in a multi-pronged approach. This includes taking into account the differing demographics and inclusion policies in each state while researching local decision making processes for participation and implementing a process to determine whether students could participate without their normal state accommodations.
The 2001 No Child Left Behind Act requires states to establish proficiency standards in order to assess whether states were making “adequate yearly progress” on increasing student achievement. There is wide variation among the proficiency standards that states use, as each state’s assessments and benchmarks are unique.
AIR researchers used a rigorous research and evaluation ‘mapping’ methodology that makes it possible for each state to compare the rigor of its standards with other states. By mapping states’ standards onto the NAEP scale, they compared the various proficiency standards used by states with the common metric of NAEP, using 2005, 2007 and 2009 mathematics and reading assessments for grades 4 and 8. The mapping procedure allows the level of achievement required for proficient performance in one state to be compared with the level of achievement required in another state. This offers an approximate way to assess the relative rigor of the states’ standards for proficient performance.
U.S. Secretary of Education Arne Duncan issued a statement about the most recent federal report, released August 10, 2011 by the National Center for Educational Statistics (NCES) and written by experts at AIR. Secretary Duncan said the “study shows that most states that changed their standards in reading and mathematics between 2007 and 2009 increased their rigor. Despite the progress, there is still much room for improvement in providing American students with a rigorous academic education that prepares them for success in the knowledge economy.” Full results can be found in Mapping State Proficiency Standards Onto the NAEP Scales: Variation and Change in State Standards for Reading and Mathematics, 2005-2009.
In 2005, the U.S. Government Accountability Office (GAO) recommended that NAEP “work with the states, particularly those with high exclusion rates, to explore strategies to reduce the number of students with disabilities who are excluded from the NAEP assessment.” The National Center for Educational Statistics (NCES) responded by researching local decision making processes for participation and accommodation of students with disabilities (SD) on NAEP; implementing a process to determine whether students could participate without their normal state accommodations; and improving training of NAEP field staff to clarify inclusion criteria. In addition, NCES conducted research to develop a methodology for measuring state inclusion rates while taking into account the differing demographics and inclusion policies in each state.
The decision about whether a student with disabilities is included in NAEP is made by a school staff member most knowledgeable about the student. A student with disabilities is assumed to be able to participate in NAEP if he or she participated in the state assessment in the selected subject and can participate with accommodations allowed by NAEP. Schools are encouraged to have students with disabilities participate whenever possible. A state with a 90 percent inclusion rate is not necessarily more inclusive than a state with an 80 percent inclusion rate, because students with disabilities may have different characteristics across states. If a state has a higher percentage of severely disabled students, for example, it would be expected to have a lower inclusion rate. Hence, to properly compare the status of inclusion rates across states or to properly measure a state’s change in inclusion rates across time, differences and changes in states’ populations of students with disabilities must be taken into account. To more fully understand the variations in inclusion, researchers examined differences among states by student disability type and severity, along with changes in inclusion over time.