Advancing Large-Scale Assessment Through Innovation and Community Building
The National Assessment of Educational Progress (NAEP) has long been considered the gold standard for educational assessment, due in large part to its technical rigor. Under the NAEP Research and Development (R&D) Program, AIR provided the National Center for Education Statistics (NCES) with technical assistance, research and evaluation, and communications support to foster innovation and increased capacity for the NAEP program.
The NAEP R&D program project supported expanding the community of statisticians, psychometricians, assessment specialists, and education researchers who work with NAEP data to support its overall strategic vision to innovate the assessment process, inform a growing audience, and promote the use of NAEP results in a variety of ways.
AIR’s Contributions to the NAEP R&D Community
The NAEP R&D program not only enhanced access to this complex dataset but also fostered a vibrant learning ecosystem where emerging scholars and seasoned analysts could explore, experiment, and contribute to the future of educational measurement, ensuring that the next generation of education researchers is prepared to tackle pressing questions with rigor and creativity.
Our R&D activities focused on three primary areas:
I really enjoyed this training...I learned so much and feel empowered to pursue research related to the NAEP assessment.
- Workshop participant
- Data training workshops, which were hosted by AIR, provided recurring hands-on experience analyzing NAEP data using EdSurvey, a free R package developed by AIR researchers to support NCES’s goal to simplify analysis of NAEP’s complex dataset.
- These workshops brought together national and international quantitative researchers from universities, education associations, and government agencies to deepen their understanding of NAEP and uncover meaningful insights.
By strengthening the research community’s ability to leverage NAEP data, AIR helped drive evidence-based decisions to shape education policy and improve outcomes for students nationwide.
- These workshops brought together national and international quantitative researchers from universities, education associations, and government agencies to deepen their understanding of NAEP and uncover meaningful insights.
- The NAEP Doctoral Student Internship Program gave emerging scholars the chance to work directly with NAEP researchers at AIR and NCES on projects spanning psychometrics, process data, policy-relevant research, data visualization, and artificial intelligence.
- During this annual 10-week program, interns received professional training and mentorship. They also contributed to the development of statistical methods for projects that were assigned based on their skills and interests to address real-world challenges facing education and large-scale assessment.
- AIR supported the interns as they published their findings and presented them at research conferences and various research and professional organizations.
- During this annual 10-week program, interns received professional training and mentorship. They also contributed to the development of statistical methods for projects that were assigned based on their skills and interests to address real-world challenges facing education and large-scale assessment.
- NAEP cooperative agreements funded research to address known challenges and introduce innovative ideas to NAEP. AIR oversaw the solicitation process, coordinated an external expert review panel, distributed funds, established data-sharing agreements, and provided quality assurance.
Amplifying NAEP R&D
Critical to the success of each of these endeavors were the communications and outreach efforts AIR implemented to foster the growth of the NAEP research community.
- Sharing NAEP R&D on social media: AIR created and disseminated written and visual content (including video) for NCES social platforms, connecting NCES’ 7,000 followers with R&D opportunities and cross-promoting with AIR’s 100k+ followers (on LinkedIn alone) for broader distribution.
- Centralizing resources on website: AIR’s technology solutions experts deployed a user-friendly content management system to provide a central source of information for all NAEP R&D activities and collect application materials from participants.
- Driving discussions: AIR hosted the R&D Hub, a bi-weekly blog, on the R&D website to allow deeper dives into work happening across the R&D and broader NAEP community, including work from the Center for Process Data and NAEP Validity Studies Panel. Posts highlighted cooperative agreement awardees, former NAEP interns, and former data training participants, showcasing the impact of the program on their research.
- Engaging the NAEP R&D community: Subscribers to the NAEP R&D program site received email communications on the latest R&D opportunities and news.
