Developers, implementers, and funders of programs and strategies for education, health, workforce, and related fields need information about what works for whom, and under what conditions, to maximize potential impact. AIR experts in quantitative research methods design and conduct rapid-cycle evaluations to inform local decision-making and continuous improvement.
Our application of rapid-cycle evaluations is a key component of our approach to rigorous research and development, which is guided by the Multiphase Optimization Strategy (MOST) framework. MOST emphasizes the importance of iterative design and development, where rapid-cycle studies can be used to develop and optimize programs through strategic testing to tease out which program components are most effective. Rapid-cycle (and longer-term) evaluations can also assess the effects of the optimized intervention.
Based on MOST, AIR’s research and development approach identifies four distinct phases: development, optimization, evaluation, and scale-up. Across these phases, AIR experts design studies in partnership with developers and practitioners in the field.
AIR’s approach is nimble and innovative, identifying the most rigorous design that is feasible and practical for the situation, and utilizing experimental methods whenever possible. Efficient experimental designs that can be used for rapid-cycle evaluation include A/B testing, factorial experiments, and sequential multiple assignment randomized trials (SMARTs).
Our approach recognizes that valid, reliable, and tailored measures are critical for quick-turnaround studies. AIR content and measurement experts work with partners to identify or develop appropriate measures that are best aligned with the goals of the program or strategy being tested. AIR data scientists have a particular emphasis on unobtrusive measures that do not add burden to stakeholders in the field, such as those based on readily-accessible data and novel measures based on “big data” from web-based programs.