Webinar Series: Systematic Literature Reviews and Meta-Analytic Techniques

In recent years, science, technology, engineering, and mathematics (STEM) education researchers have used systematic review and meta-analysis methods to synthesize research evidence, identify gaps in the literature, develop instructional design guidelines, and inform teaching practice. Systematic literature reviews, however, are often confused with standard literature reviews.

To reduce bias and increase transparency, a systematic literature review includes a precise research question with well-defined inclusion and exclusion criteria, formal literature search methods, and detailed study screening and coding protocols. In contrast, a standard literature review usually lacks formal review protocols and presents the author’s subjective interpretation of research on a given topic. After a systematic review, researchers often use formal statistical techniques known as meta-analysis to rigorously identify overall conclusions and variations in effect.

Join experts from the University of Oregon and the University of Louisville for an AIR-hosted two-part webinar series designed to help researchers become better consumers of systematic reviews and meta-analyses. The series will provide an introduction to systematic reviews, highlight standards for top-quality reviews, and provide resources for learning about more advanced meta-analytic methods. Practical resources and applied STEM education examples will be emphasized.

Both webinars will be interactive, allotting time for audience participation and questions. Participants are encouraged to attend both sessions but may elect to attend just one. AIR's Melissa Rasberry will facilitate the events.

Webinar 2: An Introduction to Meta-Analytic Techniques

September 28, 2020 | 2:30 – 3:45 ET / 1:30 – 2:45 CT / 11:30 – 12:45 PT

In the second part of this series, Drs. Tanner-Smith and Valentine will discuss how researchers can use meta-analytic procedures to combine quantitative data from multiple studies, statistically synthesize findings, and interpret meta-analytic results. They will discuss model choice in meta-analysis, describe the basics of computing and meta-analyzing study effects, illustrate how dependencies in effect sizes can be addressed, and discuss how publication bias tests are used (and misused).

Webinar 1: An Introduction to Systematic Literature Reviews

September 3, 2020 | 1:00 – 2:15 ET / 12:00 – 1:15 CT / 10:00 – 11:15 PT

Dr. Emily Tanner-Smith from the University of Oregon and Dr. Jeff Valentine from the University of Louisville will introduce the logic underlying systematic reviews. They will then share common challenges and provide practical advice for identifying the kinds of research questions that are suitable for systematic review, developing and assessing literature search strategies, and evaluating the credibility of the studies included in a review. Tools and resources will be shared to help the DRK-12 community apply those approaches.

Who should attend?

The webinars are designed for past, current, and aspiring DRK-12 principal investigators and project teams interested in learning more about systematic reviews, applying meta-analytic techniques, and becoming informed consumers of systematic reviews and meta-analyses. The webinars will be geared toward those who do not have significant experience with systematic review methods, although participants who are more familiar with systematic reviews and meta-analysis also may benefit from joining.

Presenters:

• Emily Tanner-Smith, Ph.D., Thomson Professor in the College of Education and Associate Vice President for Research at the University of Oregon
  Dr. Tanner-Smith is an applied research methodologist with expertise in meta-analysis and research synthesis for evidence-based decision-making. She received the Nan Tobler Award from the Society for Prevention Research in honor of her research synthesis contributions to the field. She is a Features Editor for Research Synthesis Methods and an Editorial Board Member for Psychological Bulletin. Dr. Tanner-Smith previously served as a methods and statistical editor for Campbell Systematic Reviews and Systematic Reviews. Her current research is funded by the Institute of Education Sciences, the National Institutes of Health, and the National Institute of Justice.
• Jeff Valentine, Ph.D., Professor and Coordinator, Educational Psychology, Measurement, and Evaluation, College of Education and Human Development, University of Louisville
  Dr. Valentine is a principal investigator for the U.S. Department of Education's What Works Clearinghouse. As an internationally known methodological expert, he has written more than four dozen works that use, explain, or seek to improve the methods of systematic review and meta-analysis. He co-edited (with Harris Cooper and Larry Hedges) the Handbook of Research Synthesis and Meta-Analysis, is an associate editor for Research Synthesis Methods, serves as a statistical editor for the Developmental, Psychosocial, and Learning Problems Group for the Cochrane Collaboration, and is co-chair of the Training Group for the Campbell Collaboration.

Objectives of the Webinars

Following participation, audience members will:

• Understand systematic review and meta-analysis terminology;
• Identify the importance and benefits of systematic reviews;
• Understand key considerations for the literature search, screening, and coding in systematic reviews;
• Understand key considerations when synthesizing evidence using meta-analysis; and
• Consider ways systematic reviews and meta-analytic techniques could further new learning in STEM education.

These webinars are supported by a grant from the National Science Foundation (DRL-1813777). Opinions, findings, and conclusions or recommendations expressed do not necessarily reflect the views of the National Science Foundation.