American Institutes for Research Consumer Guide Rates Widely Used Middle and High School Reform Models
Dallas, Tx. – Ten of 18 school improvement models, used in thousands of middle and high schools, demonstrate promising evidence of raising student achievement, according to a first-of-its-kind comprehensive review of research on the models conducted by the American Institutes for Research (AIR).
According to a federally sponsored study released by AIR's Comprehensive School Reform Quality Center (CSRQ), four school improvement models received a "moderate" rating and an additional six received a "limited" rating. The consumer guide was released on Oct. 5, 2006 during a U.S. Chamber of Commerce Education and Workforce Summit in Dallas, Texas.
To produce its ratings the Center reviewed nearly 1,500 research articles, abstracts, and summaries. Most were rejected as not scientifically rigorous enough to count for the consumer guide. Fleischman noted that "some models may be delivering results, but may not yet have had sufficient time to demonstrate their effectiveness through evidence that meets the No Child Left Behind Act's requirement for scientifically based research." However, Fleischman cautioned that "our review makes clear that any model that claims to improve student achievement will be increasingly challenged to demonstrate effectiveness based on rigorous studies."
The four school improvement models that received a "moderate" rating for boosting student achievement were:
- America's Choice School Design (America's Choice), based in Washington, D.C.
- School Development Program, based in New Haven, Conn.
- Success for All-Middle Grades, based in Baltimore, Md.
- Talent Development High School based in Baltimore, Md.
The six with "limited" ratings were:
- Expeditionary Learning, based in Garrison, N.Y.
- First Things First, based in Toms River, N.J.
- Knowledge Is Power Program (KIPP), based in San Francisco, Calif.
- Middle Start, based in New York, N.Y.
- More Effective Schools, based in Kinderhook, N.Y.
- Project GRAD, based in Houston, Tex.
The eight that received "zero" ratings were:
- Accelerated Schools-PLUS, Storrs, Conn.
- ATLAS Communities, Cambridge, Mass.
- Coalition for Essential Schools, Oakland, Calif.
- High Schools That Work, Atlanta, Ga.
- Making Middle Grades Work, Atlanta, Ga.
- Modern Red SchoolHouse, Nashville, Tenn.
- Onward to Excellence II, Portland, Ore.
- Turning Points, Boston, Mass.
With schools nationwide under increased pressure to improve or face sanctions, many low-performing districts have adopted system-wide school improvement models rather than relying on piecemeal reforms that affected only isolated schools or classrooms.
Fleischman noted that, "Our purpose in producing these consumer guides is not to pick winners and losers, but to give the public, the profession and policymakers solid information on which to make judgments about how best to improve schools."
The new guide employs the same standards and procedures that have been used in two previous Center reports, providing ratings based on the model's evidence of effectiveness and quality in five categories:
- Raising student achievement;
- Achieving other outcomes such as attendance, discipline, and graduation rates;
- Involving parents, families and the community;
- Providing a link to research base for the model's design, and
- Offering effective professional development and implementation support.
The Center said the evidence for the second and third categories was sparse, so it gave no ratings in those areas.
On the fourth category, the Center said 14 models had a "very strong" research basis for their approach to whole school improvement. The Coalition for Essential Schools had a "moderately strong" link and ATLAS Communities had evidence of a "limited" link. Two other models, Knowledge is Power Program (KIPP) and Expeditionary Learning, received no rating because they did not participate in a conversation with the Center about the research behind their approaches.
Most models received "strong" or "very strong" ratings for professional development and readiness for implementation.
The Center observed that this review is needed, since it is difficult, even for experienced education researchers, to discern which research studies should carry the most weight. On the key question of whether studies supported a model's impact on student achievement, only 41 of the 190 studies were rigorous enough to count in the ratings. In contrast, the Center had twice as many solid studies on which to base the earlier ratings of elementary school reform models.
After noting that it gave most models strong marks for readiness to implement their reforms, the Center said, "Given the importance of implementation to the success of any whole-school reform, consumers who select models that have low rankings in evidence of effects on student outcomes may still experience success if the models are implemented faithfully."
Read the full report, or individual models reviews.
The Center is funded by the U.S. Department of Education's Office of Elementary and Secondary Education, through a Comprehensive School Reform Quality Initiative Grant.
About AIR
Established in 1946, with headquarters in Washington, D.C., the American Institutes for Research (AIR) is an independent, nonpartisan, not-for-profit organization that conducts behavioral and social science research on important social issues and delivers technical assistance both domestically and internationally in the areas of health, education, and workforce productivity.