Wraparound Zone Initiative Evaluation

The Massachusetts Department of Elementary and Secondary Education Wraparound Zones (WAZ) Initiative is designed to create coordinated district systems that allow schools to proactively and systematically address the nonacademic needs of students. AIR conducted an evaluation of how well the WAZ Initiative achieved these goals, which found improvements in student behavior, family engagement, and the student referral process, and that students in WAZ schools performed better on the English language arts and mathematics assessment as compared with students in comparison schools.
 

Project Goals

AIR conducted an evaluation of how well the WAZ Initiative achieved its goals. AIR’s research assessed progress on planning, implementation, outcomes, sustainability, and replication related to the initiative’s four priority improvement areas: (1) positive school climate, (2) identification and addressing of student needs, (3) community coalitions, and (4) district systems of support.
 

Findings

Several common themes emerged across the districts and are captured in the following four cross-district findings:

1. Improvements in student behavior, family engagement, and the student referral process were the most commonly cited areas of progress across the WAZ districts.
2. According to interviewees, the key WAZ-driven levers that contributed to their progress were (a) grant-funded staff positions that were deemed essential for moving the work forward and (b) professional development supporting the implementation of social-emotional curricula.
3. Interviewees most commonly reported that family engagement and processes for identifying and addressing student needs were ongoing challenges.
4. All WAZ districts demonstrated positive or partial progress toward the majority, if not all, of the six factors identified in the literature as essential for sustainability.

In addition, the comparative interrupted time series study found the following:

• Students in WAZ schools performed better on the Massachusetts Comprehensive Assessment System English language arts (ELA) and mathematics assessment as compared with students in comparison schools, when considering prior achievement trends. Effects were statistically significant after the first and third years of WAZ implementation, and strongest after the third year.
• The impact of receiving a WAZ grant on academic achievement was greatest in the younger grades.
• For students with limited English proficiency, the impact of WAZ on academic performance was particularly strong, in Years 1 and 3 for ELA and in Year 3 for mathematics.
   

Methods

AIR conducted a mixed-methods formative and summative evaluation of the WAZ program. Data sources included (a) interviews with WAZ district coordinators and other district leaders, WAZ school coordinators and school principals, and a sample of external partners in each WAZ district; (b) in a small sample of schools, focus groups with teachers; (c) surveys of students and staff; and (d) district- and school-level documents related to WAZ planning and implementation. AIR also conducted a comparative interrupted time series study of student outcomes in WAZ schools compared with students in matched comparative non-WAZ schools.