Big data has entered the field of education increasingly over the last decade. The most important factor for this increase is the introduction of online/digital learning and assessment environments. In educational assessments, large amounts of data—called “process data”—reflecting the students’ interactions with the assessment and the system in which interactions occur have become available.
AIR, with NCES, is working to maximize the potential of this data to deepen our understanding of student behavior and assessment engagement; to inform decisions on assembly, development, and operations; and to analyze and report data. Since 2017, AIR has disseminated process data knowledge and inquiry to the research community through research studies, presentations, and professional development programs (i.e., training sessions, internship). Most recently, AIR has also supported the release of process data files and supporting documentation, designed to be available and engaging to a broader audience of users.
Process data maximizes the power of assessment results to better understand what the nation’s students know and can do. Through this work, the Center for Process Data has used its explorations of process data to shed light on patterns of students’ test-taking behaviors and to expand current knowledge of students’ use of universal design (UD) features and assessment tool use. Under the Center, process data review processes have also led to high quality data products and discovery of potential data issues which can inform test administration and assembly, as well as item development and test design. All of this work supports NCES goals for process data.
High-level Goals Supporting NCES with Process Data
- Researchers have access to and understand how to use NAEP process data
- NAEP process data are used to support operational processes and decisions
- NAEP process data are used to enhance current NAEP dissemination efforts
- Process data are used to address research issues of interest to NCES and to advance knowledge in the new field of process data research
Key areas of work
- To support rapid data release, yet still be sensitive to NCES’ public commitments, while meeting high quality standards for all data and related files
- Develop data security and quality procedures for Process Data
- Conduct explorations to support operational processes and decisions
- Conduct studies to support validity claims of NAEP assessments
- Conduct studies to gain further insights into students’ NAEP assessment experiences
- Support NCES’s outreach activities for appropriate use of Process Data in research community
- Support NCES’s communications materials for diverse audiences
AIR is looking to expand our work to various subjects and grade levels, and develop tools for making data accessible to all. As part of this effort, we will produce user-friendly reports, blog posts, and briefs for a diverse audience and create a process data community to continue to expand understanding.