AIR and Mathematica Policy Research (MPR) developed this guide to help you consider evaluation issues likely to arise as you launch ARRA-funded initiatives and other educational reform activities. Many states are already involved in evaluation, so many of the ideas presented here may be familiar. We hope that this guide will provide additional information and straightforward strategies to help you integrate evaluation into your educational reform efforts.

The American Recovery and Reinvestment Act (ARRA) is providing states like yours and districts important opportunities to strengthen efforts to improve educational outcomes for all children. ARRA can support efforts to implement new comprehensive reform initiatives or to enhance or expand existing innovative programs. Four specific reform areas are emphasized:

• Revamping teacher evaluation systems to include measures of student learning
• Ensuring that school leaders and teachers have the professional development and support opportunities they need to be successful
• Creating incentives to place effective teachers in high-need schools and subject areas
• Developing comprehensive support systems to help turn around low-performing schools

Although your primary focus will be on making these reform efforts a success, what you learn from these experiences will help guide future policy. Structuring and implementing ARRA-funded initiatives and other education reforms in evaluation-friendly ways will help you understand the results of your efforts and provide the information you will need for future policy and program decisions. Valuable learning opportunities will be lost if you do not ask these questions and analyze data about your reform programs. Important questions to consider include:

• After the funds are spent, will you know exactly what was done, how wide the scope of participation was, what worked well, and what unexpected implementation problems were encountered?
• How will you know whether your programs and initiatives actually made a difference?
• Will you know which reforms were most effective?

This guide does not address all the ways in which you may be using ARRA and other funds to support educational reforms or all the approaches you could use to evaluate ARRA-funded and other educational initiatives in your state. Instead, the guide aims to help you think about opportunities for building evaluation into your ongoing efforts and define your evaluation priorities.

Throughout the guide, our intent is to provide easy-to-use tools and resources—including tips and checklists—that you can use as part of an evaluation of most of the educational reform strategies that you are likely to initiate. These tips suggest ways in which you can reduce the need for special data collection efforts and make it easier to combine information from various data sources by building on existing data systems.