Research and Evaluation
Rigorous research and evaluation are necessary to make informed decisions about policies and interventions that aim to improve people’s lives.
AIR’s research uncovers what works and what does not, and helps document the social and economic benefits of governmental and non-governmental investments.
Our evaluations enable government agencies and other institutions and stakeholders to make evidence-based decisions about the value and utility of policies and practices that directly and profoundly affect people’s lives.
Reliable and thoughtfully designed studies can pave the way for life-changing innovations.
AIR develops, implements, and continuously improves state of the art research designs including large- and small-scale randomized control trials, comparative interrupted time series and cohort comparison studies, evaluations with regression-discontinuity designs, and statistical matching techniques.
To understand program implementation we design and carry out case studies and focus groups and analyze administrative data on program participation, service utilization, fidelity of implementation, and program cost.
Using a wide range of tools and approaches, AIR researchers work with government and private sector clients to develop and improve programs, systems, and processes to measure and optimize organizational performance.
High quality research requires high quality data.
AIR’s survey methodologists use in-depth knowledge of and experience with best practices in statistics, sociology, psychology, political science, economics, computer science, and business management to collect and prepare data for research.
We design and administer small and large-scale surveys in a variety of modes including mail, telephone, in person, and the internet. Our survey methodologists have extensive knowledge of the entire survey lifecycle, including survey sampling, content development, questionnaire design, data collection, data processing, validation, imputation, weighting and variance estimation, non-response bias analysis, and data product creation.
As the survey research practice evolves to include the acquisition of data from administrative sources and other internet-based sources, AIR’s survey methodologists and data scientists are providing research solutions that optimize both primary data collections and data from administrative records. We are committed to implementing innovative methods that uniformly produce cost efficiencies and improve data quality.
AIR researchers specialize in the synthesis of research to inform policy and practice.
We conduct systematic reviews of both published and unpublished research, and design and operate evaluation clearinghouses. This work includes extensive literature searches, review of the quality of research, synthesis of the evidence, statistical meta-analysis to combine data across studies, and communication of the results to the field.
AIR is a leader in the development and application of new psychometric techniques, designed to increase the validity and reliability of our data.
AIR consistently achieves exceptional measurement quality in our high-stakes testing. Our expertise includes Item Response Theory (IRT) modeling, classical item analysis, Generalizability Theory models, exploratory and confirmatory factor analysis, establishing cut scores and performance standards, and developing and implementing efficient pilot- and field-testing procedures.
AIR leads the field in the development and implementation of value-added or student growth measures in education.
We have a robust infrastructure for estimating, evaluating, modifying, and enhancing any statistical model currently in use, and we are not tied to any particular growth or value-added model. In general, our approach is to work with clients through a facilitated process involving stakeholders to review and select a model that best meets their needs. For almost a decade, we have helped clients choose a statistical modeling approach that is appropriate given their available data and policy priorities and help them develop strategies to use and disseminate these sensitive data in an appropriate and productive way.
AIR designs and conducts rapid-cycle studies using innovative methods to accelerate program development and maximize impact.
AIR experts in quantitative research methods design and conduct rapid-cycle evaluations to inform local decision-making and continuous improvement of programs, initiatives, and strategies. The purpose of AIR’s rapid-cycle evaluations is to provide actionable information to developers and stakeholders in the field about what works, for whom, and under what conditions, and to maximize the potential impact.
Our application of rapid-cycle evaluations is a key component of our approach to rigorous research and development, guided by the Multiphase Optimization Strategy (MOST) framework. MOST emphasizes the importance of iterative design and development, where rapid-cycle studies can be used to develop and optimize programs, through strategic testing to tease out which program components are most effective.
Our approach is nimble and flexible. We identify the most rigorous design that is feasible and practical for the situation, utilizing experimental methods whenever possible. Efficient experimental designs that can be used for rapid-cycle evaluation include A/B testing, factorial experiments, and sequential multiple assignment randomized trials (SMARTs).