The Ohio Department of Education (ODE) recently demonstrated a commitment to incorporating formative assessment strategies into teacher instruction by developing two pilot initiatives: the Ohio Performance Assessment Pilot Project (OPAPP) and the Formative Assessment Middle School (FAMS) Pilot Project. Both programs were funded in part through Race to the Top funds. Through OPAPP’s professional development, teachers learned how to transfer learning tasks and assessment tasks into their classroom instruction. Similarly, FAMS was designed to train middle school teachers of English language arts or mathematics to learn about and implement formative assessment methods and processes in their classrooms. AIR concluded that both OPAPP and FAMS were effective in increasing teachers’ use of formative assessment practices and their ability to use these practices effectively.
Project Summary
American Institutes for Research and Institutional Research Consultants Ltd. conducted a three-year evaluation of two pilot initiatives: the Ohio Performance Assessment Pilot Project and the Formative Assessment Middle School Pilot Project. These initiatives aimed to incorporate formative assessment strategies into teacher instruction.
Project Goals
AIR and Institutional Research Consultants Ltd. conducted a three-year evaluation of the two pilot programs. The objectives of the evaluation were to provide frequent, effective, and actionable feedback regarding program implementation for each of the two pilot programs, allowing ODE to modify the programs as necessary and to document the programs’ outcomes for students, teachers, principals, and coaches, assessing causal relationships if warranted. Three cohorts of students participated in each of the pilot programs.
Findings
AIR concluded that both OPAPP and FAMS were effective in increasing teachers’ use of formative assessment practices and their ability to use these practices effectively. Teachers also generally reported that their use of formative assessment practices had a positive effect on student engagement, although no evidence was found of a positive program impact on student performance.
Methods
The evaluation team used two primary data collection methods:
- Surveys of students, teachers, administrators, and coaches that obtained data on changes over time in perceptions of key aspects of the pilot programs (e.g., student engagement, teacher collaboration) from the broad set of program participants
- A set of site visits, consisting of interviews, student focus groups, and document or artifact collection, that were used to construct case studies capturing the richness and complexity of the program implementation in each cohort
In addition to measuring program implementation, AIR evaluated the impact of the pilot programs on student performance through a comparative interrupted time series analysis, using extant student- and school-level variables. The AIR team used state achievement test data for participating students and schools to test for changes in performance upon introducing OPAPP and FAMS. The team also used extant state data to construct relevant comparison groups for both programs, which strengthened the validity of this approach.