Skip to main content
  • About Us
  • Careers
  • Contact

Search form

American Institutes for Research

  • Our Work
    • Education
    • Health
    • International
    • Workforce
    • ALL TOPICS >
  • Our Services
    • Research and Evaluation
    • Technical Assistance
  • Our Experts
  • News & Events

You are here

  • Home
9 May 2016
Blog Post

Can We Stop Treating the NAEP as a Rorschach Test?

NAEP results are Rorschach Tests for policy wonks—a golden chance for free-association policy speculation.

Small fluctuations in average scores on the National Assessment of Educational Progress give rise to big explanations. Whole theories of what’s wrong with American education are built on the thin reed of an average point up or down.

The most recent NAEP results were released earlier this month. Since 2013, the percentage of high school seniors at or above the math Proficient achievement level slipped from 26 percent to 25 percent. In reading, the Proficient slip was 38 percent to 37 percent. The percentage of high school seniors below the Basic achievement level rose slightly from 35 percent to 38 percent in math and from 25 percent to 28 percent in reading.

These small changes have caused the wonkworld to look deeply into the policy inkblot. What does it all mean? Is it poverty? Is the lower high school dropout rate contributing to the lowering of overall scores? Are the tests themselves flawed? Above all—who is to blame?

Actually, the NAEP results are very steady over time. Changes are small and often uneven, and, given all that has happened in education and society in the last 40 years, strikingly consistent. For instance, looking at the NAEP long-term trend results, in 1971, 17 year olds on average scored 285 on the NAEP reading test and in 2015 they scored an average of 287.  The same pattern holds true for math—in 1971 the average was 304, in 2012 it was 306.

When we look at all the average NAEP scores for 9, 13 and 17 year olds from 1971 onward, we see slight increases for 9 and 13 year olds, which is good, but what does it really mean?

Let’s take a deep and thoughtful breath here.

While we shouldn’t overreact to small fluctuations in these NAEP scores, results from the Programme for International Assessment (PISA) reveal we should be worried that American students are not competing well when compared to students in many other countries.   

We should also take seriously other NAEP findings that our high school graduates as a whole are not college ready in math and reading. And for the students who struggle most, the news is not good—an increasing percentage of them are unprepared for college work.     

According to Peggy Carr, acting commissioner of the National Center for Education Statistics, which is responsible for the NAEP data collection: “There is a widening gap between the highest- and lowest- performing students. The students at the lower end are getting worse.”

It appears that our schools are failing our low performing students, or at the least neglecting them. We don’t need NAEP scores to tell us that low-income and minority students most often attend low-performing and chaotic schools. A field trip to almost any one of these schools would inform even the most desk-bound wonk that we live in a world of educational haves and have nots.

Rorschachs are not reality tests; they are meant to reveal unconscious desires and fantasies. Perhaps it’s time for the wonkworld to refresh its sense of reality. Behind the numbers is a school system stratified by race and socio-economic status, and while these factors should not influence opportunities to learn, they do.

The sobering truth is that year after year, our school system underserves and often neglects low income and marginalized children. These inequalities are not random; they are baked into the system. To fix these inequities we need to get over our policy taste for quick fixes and silver bullets.

If we really want to raise the average NAEP scores of all American students we need to create a level playing field where all students attend schools that are adequately financed, where teachers are supported and respected, and where all students have access to up-to-date demanding curriculum supported by a technology infrastructure that promotes lasting learning.

Forget the quibbling over tiny differences in test scores. It’s time to rebuild schools on evidence-based, comprehensive policies that have been shown to work in the real world for all students.

Peter W. Cookson, Jr., is a principal researcher and director of The Equity Project at AIR. 

Related Projects

Project

The Equity Project

Educational opportunity is a cherished part of the American story of upward mobility and social justice. The Equity Project at AIR is committed to building an inclusive and vibrant future through education.

Related Work

30 Nov 2016
Blog Post

Stancavage,_Fran_0028.jpg

TIMSS, PISA, and NAEP: What to Know Before Digging into the Results

In a rare occurrence, PISA, TIMSS, and NAEP assessments are releasing science and math results in the same year. Chances are the results from the various assessments won’t all tell the same story. So what do you need to know to make sense of this bumper crop of assessments? In this latest blog post, George Bohrnstedt and Fran Stancavage offer a quick run-down on how these assessments are similar and different.
Topic: 
Education, International Comparisons in Education, NAEP, STEM

Further Reading

  • Why Did NAEP Scores Drop?
  • NAEP Puzzle: Why Does Academic Progress “Hit The Wall” in High School?
  • Why NAEP Should Go to College
  • Measuring Up in Mathematics: What’s the Right Measure?
  • Three Decades of Education Reform: Are We Still "A Nation at Risk?"
Share

Topic

Education
Equity in Education
NAEP

RESEARCH. EVALUATION. APPLICATION. IMPACT.

About Us

About AIR
Board of Directors
Leadership
Experts
Clients
Contracting with AIR
Contact Us

Our Work

Education
Health
International
Workforce

Client Services

Research and Evaluation
Technical Assistance

News & Events

Careers at AIR


Search form


 

Connecting

FacebookTwitterLinkedinYouTubeInstagram

American Institutes for Research

1400 Crystal Drive, 10th Floor
Arlington, VA 22202-3289
Call: (202) 403-5000
Fax: (202) 403-5000

Copyright © 2021 American Institutes for Research®.  All rights reserved.

  • Privacy Policy
  • Sitemap