Skip to main content
  • About Us
  • Careers
  • Contact

Search form

American Institutes for Research

  • Our Work
    • Education
    • Health
    • International
    • Workforce
    • ALL TOPICS >
  • Our Services
    • Research and Evaluation
    • Technical Assistance
  • Our Experts
  • News & Events

You are here

  • Home
1 Jul 2014
Blog Post

Classroom Observations: Rating the Raters

This is the second of two blog posts about two new studies from AIR researchers and collaborators on the use of classroom observations for teacher evaluation.

Most press coverage about new teacher evaluation systems focuses on student growth (or value-added) measures based on student test scores. But even in districts that use such measures, a teacher’s performance appraisal still depends largely on classroom observations.

In May, the Brown Center on Education Policy at The Brookings Institution released a report presenting new concerns about the classroom observation component of most teacher evaluation systems. In particular, the authors note, “observations conducted by evaluators from outside the building have higher predictive power for value-added scores in the next year than those done by administrators in the building.” The authors attributed this difference to principals’ bias and suggested that evaluation systems either need “unbiased observers from outside the building as a validity check on principal observations, or … training and reliability checks on the principal or other in-building observers.”

My team's latest study supports these suggestions, but bases its recommendations on different evidence.

My colleagues and I analyzed teacher observation data in a large, suburban school district. Twenty-eight principals and 10 peers (other teachers) observed the same teachers, at the same time, in the same classrooms. The principals conducted observations only in their own schools. The peer observers worked across schools and didn’t know the teachers.

Peer observers received observer training; principals did not. (Most of the principals had been conducting in-class observations for years.) The district wanted to know whether peer observers, on average, were more or less lenient than principals when rating other teachers.

Our study showed the district two interesting results:

1. Principals, on average, were more lenient than peer observers when rating teachers. This might suggest that principals want to keep their teachers happy. Or, principals drew on additional information when rating what they saw—rather than rating teachers based on the instructional practices observed.

2. Although principals were more lenient on average, there was a wide range among the 28 principals. The eight most lenient raters were principals, and five of the eight least lenient (or most critical) raters were also principals. This might suggest that principals were using the rating levels of the observation rubric with less consistency than peer observers.

Based on this sample, these findings suggest that through a year of observations, some teachers may have benefited by being assigned to a school with a more lenient principal, and others may have been disadvantaged by being assigned to a school with a less lenient principal.

It’s reasonable for principals to rate teacher classroom performance; principals must know what’s going on in their schools’ classrooms. But given the emerging research and the rising stakes around teacher evaluations, it’s also clear that peer evaluators and principals need careful training in advance and a system to check or calibrate their results as they rate teachers through classroom observations.

David Manzeske is a senior researcher at AIR. He specializes in research on educator effectiveness.

Related Work

1 Jul 2014
Blog Post

Garret Rachel_2300.jpg

Rachel Garrett

The Uncertain Promise of Classroom Observations

How well do classroom observation scores help us understand how much a teacher has added to his or her students’ achievement? As Rachel Garrett describes in this blog post, new research raises questions about the wisdom of basing high-stakes, summative teacher evaluations chiefly on classroom observations.
Topic: 
Education, Teacher Preparation and Performance
1 Oct 2019
Spotlight

Principal-feature-10-08-18-banner.jpg

Image of school principal with students

Building and Supporting Great School Principals and Leaders

Principals impact student achievement through their influence on classroom instruction, organizational conditions, community support, and setting the teaching and learning conditions in schools. Read about what our researchers are learning about leading schools and how they are applying that learning to improve state policy, district programs, and school practices aimed at developing and retaining effective leaders.
Topic: 
Education, Principal Preparation and Performance

Further Reading

  • Performance Feedback for Teachers and Principals Is at Times Unreliable, Even When Using Research-Based Tools and Practices, AIR Study Finds
  • Early Implementation Findings From a Study of Teacher and Principal Performance Measurement and Feedback: Year 1 Report
  • The Uncertain Promise of Classroom Observations
  • The Impact of Providing Performance Feedback to Teachers and Principals
  • Research in Action: Beyond Evaluation- Improving a Teacher Residency Program to Increase Student Achievement
Share

Topic

Education
Teacher Preparation and Performance

RESEARCH. EVALUATION. APPLICATION. IMPACT.

About Us

About AIR
Board of Directors
Leadership
Experts
Clients
Contracting with AIR
Contact Us

Our Work

Education
Health
International
Workforce

Client Services

Research and Evaluation
Technical Assistance

News & Events

Careers at AIR


Search form


 

Connecting

FacebookTwitterLinkedinYouTubeInstagram

American Institutes for Research

1400 Crystal Drive, 10th Floor
Arlington, VA 22202-3289
Call: (202) 403-5000
Fax: (202) 403-5000

Copyright © 2020 American Institutes for Research®.  All rights reserved.

  • Privacy Policy
  • Sitemap