Maintaining the Gold Standard in NAEP Assessment Operations and Development
For more than 20 years, AIR has partnered with the National Center for Education Statistics (NCES) to support the development and administration of the National Assessment of Educational Progress (NAEP), the largest nationally representative and continuous assessment of what students in various grades and age levels know and can do.
NAEP assesses these students in 10 subjects—including reading, mathematics, and science—in both print and digital formats. NAEP also administers questionnaires to students, teachers, and schools to help provide context for the assessment results.
The student achievement data NAEP collects at the national, state, and district levels offer consistent benchmarks for evaluating educational progress and informing education policy and practice across the nation.
Our Work Supporting NAEP
Ensuring Assessments and Questionnaires Are of the Highest Quality for All Participants
With experts in subjects like mathematics, reading, science, civics, and U.S. history education as well as assessment and survey design and platform development, AIR provides NCES with foundational support to ensure that all students and respondents experience high-quality assessments and questionnaires.
- We ensure that all assessment/questionnaire items are valid and reliable measures of the constructs outlined in the NAEP frameworks.
- We integrate best practices in digital assessment content development and provide technical guidance to adapt items from traditional to digital formats.
- We ensure the translatability of all items and provide Spanish translation verification.
- We enhance the participant experience on all NAEP platforms by ensuring adherence to Web Content Accessibility Guidelines (WCAG) and universal design principles.
- We conduct assessment/questionnaire platform testing to ensure all platforms provide a seamless participant experience.
The 2018 NAEP Oral Reading Fluency Study was conducted by NCES to better understand the relationship between oral reading skills and reading comprehension for fourth-grade students.
AIR’s expertise in NAEP assessment development and operations allowed us to act as a key partner to NCES to bring this study of foundational elementary reading skills to life. We applied our skills in content and platform development to develop the study module (writing storyboards, refining audio narrations, and directing instructional videos) and integrate it into the larger NAEP assessment system.
To support the seamless administration of this new assessment, we helped write directions for the assessment, facilitated classroom testing that informed assessment logistics, and conducted platform testing to ensure items function correctly. After the assessment, we helped with activities to ensure student responses were accurately and efficiently scored.
Supporting NCES Throughout the Assessment Development and Administration Process
AIR delivers operational support for NAEP so that all activities are carried out with precision and efficiency. Our services include:
- Advanced statistical analysis: We analyze NAEP assessment/questionnaire data and make recommendations to improve item development and scoring practices.
- Document review: We review NAEP’s operational documentation for comprehensiveness and accuracy. This includes requests to the Office of Management and Budget, scoring memorandums and handbooks, and field staff materials.
- Quality assurance through observation: We observe assessment administrations, scoring activities, and field staff trainings and provide recommendations for improvements. Our observations—and the informal feedback we receive during these observations—have contributed to improvements to platform features and assessment/questionnaire procedures.
- Platform testing and integration. We test assessment/questionnaire platforms as well as assessment coordination and administration platforms to ensure they function properly, integrate with other platforms, and update accurately as changes are made.
- Coordinator support and training: We provide support to assessment coordinators—on NAEP frameworks, specifications, and item development activities—to help them effectively discuss their state or district’s results once they are released.
- Public transparency and accuracy. To support public understanding of the assessments, we confirm the accuracy of all items, scoring guides, and sample student responses available on the NAEP Questions Tool site.
- Strategic leadership: We facilitated a panel that provided strategic direction for NAEP’s technological future and informed future platform monitoring and development efforts to ensure the NAEP platform adheres to current best practices in digital assessment delivery.
Keeping NAEP at the Forefront of Large-Scale Educational Assessment
AIR's commitment to innovation, efficiency, and quality ensures that NAEP assessments remain technically sound, secure, and responsive to the evolving needs of NCES and the broader education community. We develop innovative tools to make key NAEP data available and easily accessible for parents, educators, researchers, and policy makers.
Our team’s deep familiarity with NAEP’s history and evolving requirements allows us to anticipate challenges, identify gaps, and provide actionable recommendations that keep NAEP at the forefront of large-scale assessment.
