Why Massachusetts Students, the Best in the U.S., Lag Behind Best-in-the-World Students of Hong Kong

Washington, D.C. - Higher expectations for achievement and greater exposure to more difficult and complex mathematics are among the major difference between Hong Kong, home of the world’s top-performing 4th grade math students, and Massachusetts, which is the highest scoring state on the U.S. National Assessment of Education Progress (NAEP), according to a report by the American Institutes for Research (AIR).

While Massachusetts 4th grade students achieved a respectable fourth place when compared with countries taking the 2007 Grade 4 Trends in Mathematics and Science Study (TIMSS-4), Hong Kong students outperformed the Bay State 4th graders in numerous categories.

The Hong Kong performance advantage over Massachusetts was especially large in the percentage of its students achieving at the very highest level. For example, 40 percent of Hong Kong students achieved at the advanced TIMSS level, compared with only 22 percent of Massachusetts students.

To help understand why Hong Kong students outperform Massachusetts students, the AIR study identified differences between the items on Hong Kong’s and Massachusetts’ internal mathematics assessments administered in the spring of grade 3 in 2007 to gather insight into the relative mathematical expectations in Hong Kong and Massachusetts.

The AIR report found that the Hong Kong assessment contained more difficult items, especially in the core areas of numbers and measurement, than the Massachusetts assessment.

“The more rigorous problems on the Hong Kong assessment demonstrate that, even at Grade 3, deep conceptual understanding and the capacity to apply foundational mathematical concepts in multistep, real-world situations can be taught successfully,” said Steven Leinwand, Principal Research Analyst at AIR and co-author of the report.

“Overall, the comparison revealed that each assessment covered similar mathematics topics. However, the Hong Kong assessment required a greater depth of mathematical understanding required to solve many items,” explained Leinwand. “This expectation of deep understanding of math concepts is a likely contributor to Hong Kong’s achievement as the highest performer on TIMMS in the early grades.”

The comparison of the two assessments found that Hong Kong items differed from Massachusetts items in several important ways:

• Hong Kong items were more concentrated in the number and measurement strands (75 percent), compared with Massachusetts (60 percent). A firm understanding of basic number concepts is essential for doing more-advanced work in fractions and algebra. And a solid understanding of measurement concepts is crucial to handling real-world math and learning geometry.
• Hong Kong items were more likely to require students to construct a response (86 percent) than Massachusetts items (29 percent). Constructed-response items tend to be more demanding of students to generate the correct answer by working completely through the problem without the advantage of being able to select a correct answer from a list.
• Items in the Hong Kong assessment were more likely to require more than low computational difficulty (37 percent), compared with Massachusetts items (3 percent). In the numbers domain, where computation is an integral component of the solution, 13 out of 15 (87 percent) of Hong Kong items were of higher computational difficulty, whereas only 1 out of 17 (6 percent) of the Massachusetts items in numbers required more than simple computational skills.
• Hong Kong items were more likely to fall into the moderate or high cognitive complexity category (55 percent) compared with Massachusetts items (34 percent). Performance on higher cognitively complex items is an indicator of the ability to apply mathematical concepts to solving routine and non routine problems.

Researchers sought to identify differences between Hong Kong and Massachusetts in the hope of guiding all U.S. states to reexamine and strengthen their mathematics assessments and to focus on expectations relating to these differences in the early grades, as research has shown a strong correlation between a country’s performance in early grades and later performance in the upper grades.

“If we in the U.S. want our students to perform at the levels of the highest-performing students in the world, we have to reevaluate how we are teaching and testing math concepts from the early grades,” said Leinwand. “One way to do this is to benchmark our assessments and the mathematical expectations they reveal to those of our higher performing global competitors.”

The study, Measuring Up: How the Highest Performing State (Massachusetts) Compares to the Highest Performing Country (Hong Kong) in Grade 3 Mathematics, was funded by the U.S. Department of Education.