AIR Education Measurement Experts to Present at the National Council on Measurement in Education’s Annual Meeting

Washington, D.C. – Education measurement experts from the American Institutes for Research (AIR) will participate in the National Council on Measurement in Education’s (NCME) annual meeting, being held April 15-19 at the Intercontinental Chicago in Chicago, IL. The meeting brings together professionals involved in assessment, evaluation, testing and other aspects of educational measurement.

AIR sessions include:

Friday, April 17, 2015

Methods for Comparing NAEP Frameworks to Other Assessments and Standards (Coordinated Session, D7)
2:15 PM - 3:45 PM, Seville Ballroom East, Lobby Level

AIR Session Chair: Markus Broer
AIR Session Discussant: George Bohrnstedt

This set of presentations describes the methodology used for a series of studies comparing the National Assessment of Educational Progress (NAEP) frameworks and item pools with the Trends in International Mathematics and Science Study (TIMSS), the Program for International Student Assessment (PISA), and the Next Generation Science Standards (NGSS).

• Historical Overview of Innovations in NAEP Assessment Framework Comparison Studies
  AIR Presenters: Maria Stephens, Kim Gattis, Teresa Neidorf & Young Yee Kim
  
  This presentation will review the methods used in a series of studies between 2001 and 2013 to compare national and international assessment frameworks and item pools, highlighting the earliest methods used, describing the types of comparisons undertaken (e.g., frameworks, items), and previewing some of the key challenges in this work.
   
• Comparing NAEP and TIMSS Assessment Frameworks in Mathematics and Science
  AIR Presenters: Teresa Neidorf & Kim Gattis
  
  This presentation describes the methods used in a 2012-13 study comparing the mathematics and science frameworks for the 2011 NAEP and Trends in International Mathematics and Science Study (TIMSS) assessments. In addition to quantitative analyses of framework similarity, a qualitative component was introduced to describe similarities and differences in content.
   
• Comparing NAEP and PISA Mathematics Assessment Frameworks and Item Pools
  AIR Presenters: Kim Gattis, Maria Stephens & Young Yee Kim
  
  This presentation describes the methods used in a 2013-14 study comparing the mathematics assessment frameworks and item pools from NAEP 2013 and PISA 2012. It will focus primarily on a new qualitative component, which provided a systematic side-by-side review of the features of NAEP and PISA items.
   
• Comparing NAEP and the Next Generation Science Standards (NGSS)
  AIR Presenters: Teresa Neidorf, Austin Lasseter, Maria Stephens & Kim Gattis
  
  This presentation describes the methods used in a 2014-15 study comparing the NGSS and the NAEP assessment frameworks in science, technology and engineering literacy (TEL), and mathematics.  It will focus on design considerations and methods used to address the multiple dimensions of the NGSS and the three NAEP frameworks.
   

Saturday, April 18, 2015

Assessing the Item Fit in Computerized Adaptive Tests (Electronic Board #11)
4:05 p.m. – 5:05 p.m., Camelot, 3rd Floor, Electronic

AIR Presenters: Yuan Hong, Tao Jiang & Stephan Ahadi

As states start using the immediate score reporting offered by the online assessments, it becomes crucial to maintain a calibrated item pool with pre-equated parameters. Stone’s item fit statistic is used to flag the non-fit items and is generalized to be able to use both MLE and EAP ability estimates.

Examining Erasure Behaviors in Large-Scale Assessment
4:05 p.m. – 6:05 p.m., Adler, 2nd Floor, Paper Session, I7

AIR Presenter: Elizabeth Ayers

Data from a 5th grade reading assessment is used to examine whether aberrant erasure behaviors are associated with irregular item responses from cheating. Erasure analyses using person-fit statistics and modified test-retest methods along with erasure indices show that aberrant erasure behaviors are possibly related to aberrant response patterns from cheating.
 

Sunday, April 19, 2015

Digital Literacy and Performance Gaps in Computer-based Assessments
8:15 a.m. – 10:15 a.m., Toledo, 5th Floor, Coordinated Session, J8

AIR Presenters: Ting Zhang, Young Yee Kim, George Bohrnstedt & Markus Broer

A key condition for the transition will be addressed: digital literacy and performance gaps, presenting the results of various recent mode studies including NAEP grade 8 computer-based assessments (CBAs) in writing and mathematics to investigate the relationship between student CBA performance and their digital literacy, especially their computer familiarity.

More information about the conference may be found at http://ncme.org/.

About AIR

Established in 1946, with headquarters in Washington, D.C., the American Institutes for Research (AIR) is a nonpartisan, not-for-profit organization that conducts behavioral and social science research and delivers technical assistance both domestically and internationally in the areas of health, education, and workforce productivity. For more information, visit www.air.org.

###