Recent projects


Project:

Problem

The objective of this project was to develop specifications for new science and technical test forms (General Science, Auto and Shop Information, Electronics Information, and Mechanical Comprehension) for the ASVAB.

These specifications would be designed to reduce adverse impact against women and minorities while maintaining the levels of fairness and validity associated with the current tests.

Approach


AIR used two approaches to specifying test content for the ASVAB.

The first approach, labeled the training requirements approach, specifies test content in terms of the knowledge and skills necessary for successful performance in technical training courses in the military. To obtain this information, we surveyed training developers and instructors across a wide range of entry-level technical training courses, and used the results of this survey to identify items providing the best match to training requirements.

The second approach, labeled the opportunity to learn approach, specifies test content in terms of the knowledge and skills that examinees are most likely to have been exposed to up to and through high school. This approach emphasizes the demonstrated ability of the high school graduate to learn the different types of knowledge and skill dictated by his or her curriculum and extracurricular activities. Successful learning in this environment is presumptive evidence of the ability to learn similar (technical) content during future entry-level training in the military.

We used a variety of methods to identify the knowledge and skills to which examinees were likely to have been exposed. These included the administration of an "opportunity to learn" survey to high school students and new recruits, and the convening of a panel of high school curriculum experts.

The results of this effort are being used to identify items to which examinees are most likely to have been exposed, regardless of subgroup affiliation.

Results


As a result of our research, the Department of Defense has an enhanced understanding of the impact and utility of including the ASVAB technical subtests in selection classification composites. This understanding includes the gains in predictive power from use of the tests and their contribution to adverse impact and test fairness.