Department of Labor Evidence to Action Capacity Assessment
The U.S. Department of Labor’s (DOL) Chief Evaluation Office (CEO) oversees periodic assessments of DOL’s capacity to produce and use evidence, with the aim of helping the Department and its agencies identify concrete steps for continuous improvement.
In support of The Foundations for Evidence-Based Policymaking Act of 2018, AIR conducted a strengths-based assessment of DOL’s capacity to use and produce evidence. The assessment included the 16 agencies in DOL’s Strategic Plan and was conducted using a collaborative learning approach that emphasized a continuous improvement mindset and recognized DOL’s progress toward evidence integration.
The primary goals of the assessment were to establish a baseline understanding of DOL’s capacity to produce evidence that the Department and its agencies can use to guide decisions and identify next steps for improving capacity going forward. The assessment also served as an opportunity to educate and engage DOL staff in a more in-depth dialogue about evidence use and production.
Given these goals, AIR administered a staff survey, conducted focus groups with select staff and leadership, and reviewed evidence documents. The data gathered from these activities allowed AIR to identify DOL agencies' strengths in planning, creating, and applying evidence to inform and refine their programs, policies, and operations.
These activities also engaged staff and generated interest in evidence use and production, which ultimately led to the development of the Evidence to Action (E2A) model in collaboration with DOL’s Chief Evaluation Office. The E2A model seeks to bridge the gap between what researchers produce and what practitioners know and do by implementing effective strategies to facilitate evidence use in policymaking and program implementation.
Project Outputs
- Survey and Staff Focus Groups: AIR surveyed staff across 16 DOL agencies and conducted focus groups with DOL staff and leadership to explore a range of related topics including current evidence use and production, barriers to and facilitators of evidence use and production, and training and other supports to help improve capacity.
- Evidence Review: AIR analyzed selected evidence products to understand the coverage, effectiveness, quality, methods, and the independence of those products.
- Capacity Assessment Report: AIR developed a summary of the results from the department-wide survey and the selected evidence review as well as insights from analysis of qualitative data from agency-level focus groups.
- Evidence Capacity Snapshot: AIR developed a high-level, actionable summary of the capacity assessment findings across all 16 agencies.
- Individual Agency Capacity Summaries: AIR developed summaries of the capacity assessment findings for each individual agency included in the assessment.
- Evidence Maturity Model Framework and Discussion Questions: AIR developed an evidence maturity framework and discussion questions to help the Department and each agency consider where they may fall within the maturity model across the areas of evidence practices, capacity, culture, and infrastructure.
- Evidence to Action Model: In collaboration with CEO and in response to the need for higher-quality evidence products, AIR developed the E2A model to guide future evidence creation, translation, and adoption efforts within DOL.
Insights and Impact
The assessment uncovered vital information on patterns of evidence use and production within DOL as well as the potential for growth. It also led to the collaborative development of the E2A model.
