Is ESSA Accountability Playing Out as Intended?
For decades, policymakers have used school accountability systems to stimulate school improvement. Although such policies have shifted over time, the basic premise underlying accountability has changed little: States must measure performance and identify the lowest-performing schools; share information; and, in cooperation with school districts, provide supports. Then schools must work with the districts to develop an improvement plan and implement interventions—all with the goal of improving student outcomes.
This series of steps making up the accountability theory of action seems relatively simple and intuitively should work to promote school improvement. But does accountability lead to improvements in student outcomes? Do all of the steps in this process play out as intended?
Under the Every Student Succeeds Act (ESSA), each state must determine which schools are among the lowest-performing 5% in their state and identify these for Comprehensive Support and Improvement (CSI). For the past five years, a study team from AIR has evaluated the implementation and impact of the CSI designation in three states: California, Florida, and Ohio.
We designed the study to look at each step in the accountability theory of action, aiming to understand what is working and where actual practice may fall short. Here are some of our key findings linked to different steps in the theory of action.
Measuring performance and identifying the lowest-performing schools is the cornerstone of accountability, yet it is far from a perfect science.
Federal law establishes some guidelines regarding how states measure performance and identify schools, but each state has latitude to determine which indicators of school performance to include and how much each will be weighted. The decisions that states make in how they measure performance have potentially important implications for which schools are ultimately identified as low performing.
For example, unlike almost all other states, California holds schools accountable for the out-of-school suspension rate, and this strongly influenced which schools were identified as CSI. And although ESSA was intended to foster accountability systems that integrate multiple measures, our study found that CSI schools tend to be evaluated based on fewer indicators than non-CSI schools are.
Clear, publicly available information that focuses attention on student performance is integral to the change process but can also be a potential stumbling block.
For schools to change behavior in ways that improve outcomes for all students, the accountability system must provide clear, meaningful, and usable information. For CSI schools, the system must convey that student learning is far below expectations and in what ways. Among the states in our study, California reports accountability information through a color-coded system, while Florida provides letter grades (A-F) to communicate school performance. Principals in these two states reported that the accountability designations in their states were clear and understandable.
However, as the measurement of performance and identification of schools has become more complex over time, it becomes more challenging to clearly communicate accountability results in usable ways. We saw this in Ohio, which had a very complex accountability system (with 23 measures) at the time of our study. About half of principals in the state reported that they did not understand how schools were identified as underperforming.
CSI schools appear to get more support than other low-performing schools, but perhaps not at the level anticipated by policymakers, nor at a level sufficient to catalyze change.
Under federal policy, CSI schools should receive supports and resources above and beyond those provided to other schools. ESSA’s policy intent is made explicit through the label assigned to the lowest-performing schools: comprehensive support and improvement.
Our study found that principals of CSI schools were more likely to report receiving support for developing their improvement plans. However, they were no more likely than low-performing non-CSI schools to receive supports in other areas, such as improving quality of professional development for teachers, increasing parent engagement, or identifying standards-aligned curricular materials.
ESSA also earmarks funds specifically for CSI schools. However, analyses of national data demonstrate that CSI schools only receive about $300 more per pupil than other schools, which is not enough to overcome the substantial challenges CSI schools face.
School-level activities to improve student outcomes are not always part of a coherent school improvement process nor informed by research evidence.
At the tail end of the accountability process, schools must implement interventions that are intended to improve student outcomes. Our study shows that some schools appear to be using evidence-based practices in the school improvement process, intentionally focusing their efforts on a handful of potentially high-impact strategies. However, an analysis of principal survey results revealed two sets of schools that lacked a coherent approach to school improvement.
One set of principals reported a major focus on virtually no school improvement activities, suggesting a potential breakdown in the communication of accountability information to school leaders and a lack of urgency to improve. On the other hand, principals at another set of schools took on too much. These principals reported a major focus on an average of 16 school improvement activities, an undertaking that would be unmanageable for even the highest-capacity schools.
Moreover, we found that principals made limited use of the most rigorous evidence on research-based school interventions with principals often relying on recommendations from colleagues from other schools and districts.
Despite potential bumps in the road, there is evidence that accountability systems can sometimes work.
Schools that are identified as low performing do improve student outcomes, which is the end goal of this complex policy. For example, we found that California schools that were flagged for low graduation or college and career readiness improved both sets of outcomes, relative to similarly low-performing schools. While schools identified as CSI did not improve, schools identified as underperforming on specific indicators by a separate state accountability system did demonstrate improvement.
Federal law requires all states to identify CSI schools, but California’s color rating system—green for schools that were doing well on specific indicators, and red for schools performing poorly—seemed more effective at signaling where schools needed to focus their improvement efforts, in contrast with the broad CSI rating.
Realizing Accountability Systems’ Potential
Policymakers have pinned great hopes on school accountability. It is intuitively appealing in theory, yet deceptively complex in practice, rendering it difficult to achieve results at the desired scale. That said, we believe accountability is here to stay and has an important role to play in school improvement. Careful measurement of school performance provides insights to all school leaders, not just those in low-performing schools. State-selected performance indicators signal state priorities to school leaders.
With access to disaggregated student outcomes, education administrators can be better positioned to ensure all students have access to education opportunities. But as our study shows, the accountability process often breaks down at the local level, where school leaders grapple with internal and external challenges that can overwhelm good intentions.
For accountability systems to realize their potential, school leaders need more than the increased pressure, motivation, and urgency that these systems generate. Clear information, systematic supports tailored to their school contexts, and sufficient resources to provide services and implement needed interventions can go a long way on the road to improvement.
