
MOSAIC Education Studies

Education Studies
AIR conducted a systematic review and meta-analysis to examine the heterogeneity in mathematics intervention effects by synthesizing 25 years of randomized experiments of interventions designed to improve mathematics achievement in Grades K-12. The goal of this project was to better understand the conditions and contexts for why one study may find that a mathematics intervention works while another study on the same intervention may not find that it works. We developed an interactive web application that allows users to explore the pool of mathematics intervention effects and answer these questions for themselves.
Project Director: Martyna Citkowicz
Project Investigators: Ryan Williams, Jim Lindsay
Subject: Education, Math
Population: Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12)
Project Type: Meta-analysis, Systematic literature review
The What Works Clearinghouse (WWC), the standard bearer for effectiveness research in education, conducts its work through multiple contracts. AIR holds the Statistics, Website, and Training (SWAT) contract, which provides coordination vitally important to the success of the WWC by centrally updating WWC policies and procedures, providing training and technical assistance, conducting quality checks of WWC products, supporting users through a website and Help Desk, and communicating with one voice to WWC contributors, users, and stakeholders.
WWC SWAT-Website
Project Director: Joshua Polanin
Project Investigators:
Subject: Education
Project Type: WWC review
The Effects of College Aid Programs: Systematic Review and Meta-Analysis
The goal of this project is to synthesize the literature on the effects of college aid on student's college success. After conducting a wide-ranging, comprehensive systematic review, the search yielded over 100 studies. The final results are in process and expected to be released in 2021.
Project Director: Joshua Polanin
Project Investigators: Joshua Polanin
Subject: Education
Population: Postsecondary
Project Type: Meta-analysis, Systematic literature review
Systematic Review of College Completion Strategies
AIR is conducting a systematic review of the literature, scanning advising policies and practices that colleges use to improve student outcomes, and conducting a gap analysis that compares the findings from the scan and from the systematic review to identify gaps in the research evidence. The goal of this project is to take stock of what we know about successful strategies to improve college completion and where gaps exists between research and current policy and practice.
Project Director: Amy Feygin
Project Investigators: Amy Feygin
Subject: Education
Population: Postsecondary
Project Type: Systematic literature review
Aligning Adult Education National Leadership Activities Across Content Areas
The goal of this project was to implement a systematic alignment model (SAM) to conduct a systematic review and analysis for improving teaching and learning in adult education and to illustrate how various adult education national activities converge to support high quality teaching and improved learning for low-skilled adults. The objective was to position states to better integrate adult educational reform initiatives, materials, methods, and resources in order to strategically maximize systemic inputs, outputs, and outcomes for low-skilled adults.
Project Director: Anestine E. Hector-Mason
Project Investigators:
Subject: Education
Population: Adults (ages 18+)
Project Type: Systematic literature review
In partnership with the San Mateo County Office of Education, the Glen Price Group, and consultant Elliot Regenstein, AIR supported the development of a California Comprehensive Early Learning Plan. This plan addressed the entire range of policies and programs affecting children from birth to age 5 and was required by the American Recovery and Reinvestment Act's (ARRA's) funds supporting the State Advisory Council on early learning. AIR’s role was to compile data from multiple sources and complete a meta-analysis on the status of children and early learning opportunities in California.
Condition of Children Birth to Age Five and Status of Early Childhood Services in California Synthesis of Recent Research (PDF)
Project Director: Susan K. Muenchow
Project Investigators: Susan Muenchow, Karen Manship
Subject: Education
Population: Early Childhood Education/Pre-Kindergarten
Project Type: Meta-analysis
Models of Professional Development for Mathematics Teachers
The goal of this project was to launch and study an innovative way to examine professional development offered to K-12 mathematics teachers. AIR coded and analyzed descriptive data on mathematics professional development (MPD) designs, and compared similarities and differences across various design models.
Project Director: Kwang S. Yoon
Project Investigators: Kwang S. Yoon
Subject: Education, Math, Professional Development
Population: Adults (ages 18+)
Project Type: Systematic literature review
The Southeast Comprehensive Center (SECC) at AIR was one of 15 regional comprehensive centers funded by the U.S. Department of Education (2012-2019). The center provided training and technical assistance to state education agencies (SEAs) to enable them to assist school districts and schools in the implementation and administration of programs authorized under the Every Student Succeeds Act (ESSA) and the use of research-based information and strategies. The SECC worked closely with SEAs in Alabama, Georgia, Mississippi, North Carolina, and South Carolina to support their efforts to implement, scale up, and sustain initiatives statewide and to lead and support their school districts and schools in improving student outcomes.
Project Director: Beth Howard-Brown
Project Investigators:
Subject: Education, Math, Science, School Improvement
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), Postsecondary
Project Type: ESSA, Knowledge translation, Systematic literature review, Training
Career and Technical Education Research Network
Identification and Counterfactuals for Program Evaluation of Career and Technical Education (September 2020, PDF)
Ready for Causal Research: A National Evaluability Assessment of Career and Technical Education Programs (June 2020, PDF)
The Career and Technical Education (CTE) Research Network at AIR seeks to expand the evidence base on the impact of CTE programs on student outcomes by using research and training to develop the next generation of CTE researchers.
Project Director: Katherine Hughes
Project Investigators: Katherine Hughes
Subject: Education, Career and Technical Education (CTE)
Population: High school (Grades 9 – 12), Postsecondary, Teens (ages 13 – 17), Adults (ages 18+)
Project Type: Meta-analysis, Systematic literature review
AIR conducted two evidence reviews on student outcomes associated with arts instruction. The first review examined the effects of arts integration (i.e., the use of art to enhance instruction in the main academic curricular areas) on students' academic outcomes. The second review examined normal arts instruction (i.e., fine arts, performing arts) on student outcomes. For each review, studies with findings that fell into Every Student Succeeds Act (ESSA) Tiers 1, 2, or 3 were meta-analyzed.
Review of Evidence: Arts Education Through the Lens of ESSA
The Role of Arts Integration and Education in Improving Student Outcomes (PDF)
Project Director: Meredith Ludwig
Project Investigators: Meredith Ludwig, Jim Lindsay
Subject: Education, Art
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12)
Project Type: ESSA, Meta-analysis, Systematic literature review
21st Century Community Learning Centers Measuring Program Success
AIR conducted a literature review to synthesize the results of studies published since 2008 that examined the degree to which afterschool and youth development programs have an impact on participating youth. The goal of this project was to provide the U.S. Department of Education with recommendations for new Government Performance and Results Act (GPRA) measures for the 21st Century Community Learning Centers.
Project Director: Neil Naftzger
Project Investigators: Neil Naftzger
Subject: Education, After School Programs
Population: Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), Economically disadvantaged
Project Type: Systematic literature review
Heinemanns Leveled Literacy Mock WWC Review
AIR staff who are certified What Works Clearinghouse (WWC) reviewers performed preliminary reviews of several literacy programs, including the Heinemann Leveled Literacy Program, prior to the publishers’ submission of the programs for the formal review. Using the WWC standards for review, the AIR reviewers read published journal articles and prepared reports predicting how the product would be rated when subjected to the U.S. Department of Education review process.
Heinemanns Leveled Literacy Mock WWC Review
Project Director: Terry S. Salinger
Subject: Education, English Language Arts (ELA)
Population: Elementary school (Grades 1 – 5)
Project Type: WWC review
AIR partnered with the Boston Foundation and a consortium of school and district leaders across district, charter, and private schools to co-create a model of postsecondary readiness for Boston students. The project team conducted a review of research on postsecondary readiness from the literature as well as interviews with current experts. The result was a report on academic, non-academic (or soft skills), and other factors that predict postsecondary readiness.
College and Career Readiness in Boston: Understanding and Tracking Competencies and Indicators (PDF)
Project Director: Matthew Welch
Project Investigators: Susan Therriault
Subject: Education
Population: High school (Grades 9 – 12), Postsecondary
Project Type: Systematic literature review
Gates Networks for School lmprovement Initiative Evaluation
AIR conducted a systematic review of continuous improvement studies in educational settings. The goal of this project was to understand how researchers in the field have defined continuous improvement, the foci of their continuous improvement efforts, the composition of continuous improvement teams, the processes used to implement continuous improvement, the features that appeared to facilitate implementation, the outcomes, and the process by which continuous improvement is diffused to others.
Project Director: Ryan Eisner, Julie Kochanek
Project Investigators: Mike Garet
Subject: Education, Continuous Improvement
Population: Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), Postsecondary
Project Type: Systematic literature review
AIR is conducting two meta-analyses by synthesizing more than 25 years of empirical research across the world to study the origins of beliefs and motivational processes that could potentially limit the full participation of girls and women in science, technology, engineering, and mathematics (STEM) fields. The goal of this project is to clarify mixed findings about how gender stereotypes about STEM abilities first develop and how these stereotypes relate to gender gaps in STEM by analyzing variation in (a) mean levels of children's gender stereotypes about STEM abilities, and (b) these stereotypes relation to motivational STEM outcomes such as confidence and interests.
The Development of Gender Stereotypes About STEM and
Verbal Abilities: A Preregistered Meta-Analysis Protocol (PDF)
Project Director: David Miller
Project Investigators: David Miller, Courtney Tanenbaum
Subject: Education, Science, Technology, Engineering, and Math (STEM)
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), Children (ages 3 – 12), Teens (ages 13 – 17)
Project Type: Meta-analysis, Systematic literature review
AIR conducted a systematic review and meta-analysis of randomized experiments of interventions directed at classroom practice in Grades K-12. The goal of this project was to understand how responsive a teachers' classroom practice is to intervention and whether particular intervention features were associated with improvements in classroom practice outcomes.
How Responsive Is a Teacher’s Classroom Practice to Intervention? A Meta-Analysis of Randomized Field Studies
Abstract (PDF)
Journal Article (PDF)
Presentation (PDF)
Project Director: Martyna Citkowicz
Project Investigators: Rachel Garrett, Martyna Citkowicz, Ryan Williams
Subject: Education, Professional Development
Population: Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12)
Project Type: Meta-analysis, Systematic literature review
Examining Heterogeneity in English Learner Program Effects with Meta-Analysis
AIR is conducting a systematic review and meta-analysis to examine sources of heterogeneity in English learner program effects. This study will focus on the following broad sources of effect heterogeneity: student, teacher, or school characteristics; program type and program features; outcomes; settings or context; and design and methodological features.
Project Director: Ryan Williams
Project Investigators: Ryan Williams, Rachel Garrett, Joshua Polanin
Subject: Education
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), English learners
Project Type: Meta-analysis, Systematic literature review
The Effects of Increased Learning Time on Student Academic and Nonacademic Outcomes: Findings From a Meta‑Analytic Review
In partnership with the Regional Education Laboratory (REL) Appalachia, AIR conducted a systematic review and meta-analysis of the effects of increased learning time—extending either the school day or school year—on student academic and nonacademic outcomes. Only studies that met What Works Clearinghouse (WWC) standards were included, resulting in a pool of 30 studies.
The Effects of Increased Learning Time on Student Academic and Nonacademic Outcomes
Report (PDF)
Brief (PDF)
Project Director: Jim Lindsay
Project Investigators: Yael Kidron, Jim Lindsay
Subject: Education
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), Children (ages 3 – 12), Teens (ages 13 – 17), Economically disadvantaged
Project Type: Meta-analysis, Systematic literature review
Children’s Access to Print Material and Education-Related Outcomes: Findings From a Meta-Analytic Review
Those favoring continued funding for programs that increase children’s access to print material use findings from four disparate lines of research to support their argument, rather than direct evidence of program impacts. AIR conducted a systematic review and meta-analysis to examine whether the efficacy of such programs is supported by direct evidence.
Children’s Access to Print Material and Education-Related Outcomes: Findings From a Meta-Analytic Review
Report
Brief (PDF)
Project Director: Jim Lindsay
Project Investigators: Jim Lindsay
Subject: Education, English Language Arts (ELA)
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), Infants/toddlers (ages 0 – 3), Children (ages 3 – 12), Teens (ages 13 – 17), English learners, Economically disadvantaged
Project Type: Meta-analysis, Systematic literature review
Forced Displacement and Education
Despite the large number of forcibly displaced people worldwide (more than 79 million as of December 2019), evidence on strategies for including displaced populations into host country national education systems is scarce. To address this evidence gap, the World Bank, the United Nations High Commissioner for Refugees (UNHCR), and the Foreign, Commonwealth & Development Office (FCDO) commissioned a research program to enhance understanding of ways to improve education and learning outcomes for forcibly displaced populations and host communities living in poverty, exclusion, or distress. AIR conducted a rigorous evidence synthesis and intervention mapping to systematically examine the available evidence and current programs in forced displacement contexts, and will conduct case studies to offer an in-depth review of policies and programs that support the inclusion of refugees and internally displaced persons (IDPs) in national education systems.
Project Director: Andrea Coombes
Project Investigators: Thomas de Hoop, Hannah Ring
Subject: Education, Refugees
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12), Postsecondary, Infants/toddlers (ages 0 – 3), Children (ages 3 – 12)
Project Type: Systematic literature review
The Effects of 5E-Based Interventions: Systematic Review and Innovation Through New Meta-Analysis Methodology
AIR is conducting a systematic review and meta-analysis to examine the effectiveness of the 5E (Engagement, Exploration, Explanation, Elaboration, and Evaluation) instructional model on science, technology, engineering, and mathematics (STEM) outcomes of students in Grades PreK-12. The project team also will create an open source, user-friendly data extraction and analysis tool for researchers conducting meta-analyses. The meta-analysis will inform the design of STEM interventions, and the meta-analytic tool will advance the field of systematic review and meta-analysis by streamlining the meta-analysis process and building in state-of-the-art techniques for meta-analysts.
Project Director: Megan Austin
Project Investigators: Joshua Polanin, Ryan Williams
Subject: Education, Science
Population: Early Childhood Education/Pre-Kindergarten, Kindergarten, Elementary school (Grades 1 – 5), Middle school (Grades 6 – 8), High school (Grades 9 – 12)
Project Type: Meta-analysis, Methodology, Systematic literature review