Assigning Adaptive NAEP Booklets Based on State Assessment Scores: A Simulation Study of the Impact on Standard Errors
The purpose of this simulation was to assess the improvements in standard errors of estimates that could be expected if students participating in NAEP were pre-assigned to test booklets that were adapted to their level of performance based on their state assessment scores. The simulation assumed that students in extreme quartiles would receive one regular NAEP block and one block adapted to their performance quartile. Students in the middle two quartiles would receive a second "regular" NAEP block. Also considered was an alternative scenario in which adapted blocks were assigned only to the top and bottom deciles of students, rather than to full quartiles. Results from the simulations indicated that assigning low-performing students "easier" test blocks based on their prior test performance could reduce standard errors in measuring these students' ability levels. The reverse, however, was found to be true for "harder" booklets; in this simulation, assigning "harder" test blocks to higher performing students actually increased standard errors. Results were consistent across the four states included in the study.