Developing AI Assistants to Improve Access to Education Information
Each year, education agencies produce a wealth of valuable data and statistical information. However, accessing specific information can be a challenge. Users—from researchers to educators to parents—often struggle to find clear answers to seemingly simple questions like, "How does x assessment benefit my child’s school and community?" and "Which school districts experienced the most significant decline in math scores in 2024?"
The answers to these questions are frequently scattered across multiple platforms, files, and formats, making them difficult to locate and synthesize. This creates a critical need for more user-friendly tools that can improve access to education data, especially for users who may not know where to find these data or are not familiar with technical statistical language.
AIR's Work with NAEP
To address these challenges, AIR developed conversational artificial intelligence (AI) assistants with Retrieval-Augmented Generation (RAG) to facilitate access to education data, including data from the National Center for Education Statistics (NCES) and its National Assessment of Educational Progress (NAEP). These AI assistants, which serve as intelligent intermediaries between users and databases, provide clear, accurate answers to users’ questions in natural language. Rather than requiring users to navigate complex databases or technical documentation, they can ask simple questions (e.g., "Are the same school districts selected by NAEP from year to year?") and receive comprehensive synthesized responses drawn from the most current and relevant sources.
Ask NAEP
To help NCES efficiently retrieve and synthesize publicly available NAEP data and respond to user queries, AIR developed Ask NAEP, which uses retrieval-based generative approaches to prepare statistically or factually accurate responses to queries. This prototype chatbot delivered precise responses to queries about NAEP, consolidating information from various sources and formats. AIR demonstrated that using the RAG approach resulted in higher quality responses than using off-the-shelf large language models alone (e.g., GPT 3.5, GPT-4o).
Ask NCES
Following the success of Ask NAEP, NCES asked AIR to develop Ask NCES to serve as a multiagent AI assistant that can respond to user queries on a broader range of NCES survey programs. For Ask NCES, AIR used the same RAG approach to address queries about a wide range of education data available on the NCES website—including data from NAEP, the High School Longitudinal Study, the Digest of Education Statistics, and the Condition of Education—as well as NCES standards and guidelines for statistical reports and restricted-use data licenses.
In the continuously evolving field of generative AI, AIR is leveraging its deep and broad knowledge of education statistics to create AI tools that facilitate rapid user access to accurate statistical information. These tools add to a collection of AIR developed software tools, including EdSurvey and its suite of supplementary R packages, that are designed to facilitate and democratize the use of knowledge related to education statistics.
