Milwaukee Public Schools Striving Readers Evaluation
As a part of the Striving Readers program supported by the U.S. Department of Education, Milwaukee Public Schools implemented the Read 180 program in five schools in an effort to provide targeted literacy supports to struggling students. AIR's evaluation found that the program had a significant impact on students’ academic performance in reading.
Project Summary
AIR used an experimental design to produce a rigorous estimate of the impact of the Read 180 intervention, a program that provides targeted literacy supports for struggling students in the Milwaukee Public Schools system.
Project Goals
In conducting an evaluation of this effort, AIR used an experimental design to produce a rigorous estimate of the impact of the Read 180 intervention on measures of reading achievement for struggling students. The evaluation team also explored implementation fidelity and the contexts and conditions of implementation that may extend or limit the intervention’s effects.
Findings
The program had a significant impact on students’ academic performance in reading. Students assigned to Read 180 scored approximately 1.8 points higher than control students when controlling for pretest scores and student-level covariates. The Read 180 classroom model was implemented with high fidelity. However, student attendance in Read 180 classrooms was low, and students’ use of the Read 180 instructional software ranged from medium to low levels of implementation. Therefore, nearly all classrooms received a medium rating for fidelity to the classroom model. Teachers reported that prior experience teaching Read 180 and the support of the district Read 180 coordinator were important facilitators for successfully implementing the model.
Methods
To measure program impact on students’ academic performance in reading, AIR analyzed student achievement data collected from the Northwest Evaluation Association Measures of Academic Progress benchmark assessment. AIR also administered a student survey to assess the impact on student engagement and self-efficacy for reading. To measure program fidelity, AIR developed rubrics for assigning implementation fidelity scores for both the professional development component and the Read 180 classroom component. Each rubric was based on data collected from multiple sources, including interviews with Read 180 teachers and building administrators, classroom observations, and extant student- and teacher-level data. Scores were categorized into low, medium, and high levels of implementation for each Read 180 teacher or classroom. Descriptive analyses of implementation data also were conducted to provide context for implementation successes and challenges.