Fundamental Problems in the Measurement of Instructional Processes: Estimating Reasonable Effect Sizes and Conceptualizing What is Important to Measure

Overview

Developing reliable and valid national-level survey measures of instructional processes and other classroom conditions related to learning has proven exceptionally difficult. Extant measures have demonstrated, at best, small links to achievement and most have not been linked statistically to achievement at all. These small links to achievement may reflect, in part, reality: instruction competes with many other factors, including factors outside of the school, for influence on student achievement. However, struggles to link to student achievement reflect also problems with the measures, including a host of validity, reliability, and generalizability problems, as well as underdeveloped conceptualization of what is important to measure about instruction and other classroom conditions. Prior research has documented validity, reliability, and generalizability problems with survey measures of instruction. In this paper, we focus on the size of the impact we can expect from instruction and teachers on student achievement and on the conceptualization of measures of instruction and other classroom conditions related to learning.