A Guide for Education Personnel: Evaluating a Program or Intervention
From the Elementary & Middle Schools Technical Assistance Center
We often hear from school and district staff that they love their work, enjoy helping students, and are highly invested in their programs and interventions designed to help students–especially students with disabilities. What these individuals also often say is that one of the most difficult parts of their job is evaluating their programs and interventions. First of all, it can be a daunting task to conduct an evaluation if one has never wrangled with all the technical issues involved in an evaluation before. Often, time and resources are in short supply, and conducting an evaluation may seem like added work without a clear benefit. Moreover, evaluating one’s peers and colleagues, and the program they’ve devoted their professional (and perhaps even person) lives to, can feel awkward and uncomfortable. Given these issues, it’s no wonder that undertaking a well-designed and effective evaluation can be an anxiety producing event!
Despite these issues, evaluations are important, and the benefits that can come from a well-designed and well-executed evaluation far outweigh any challenges.
Often we think of evaluation research happening at the end of a program or intervention’s lifespan in order to determine whether the program worked – program impact. However, an equally important function served by evaluation research is monitoring program implementation. Evaluations of implementation are essential because they help identify problems with program implementation before the program ends, so that changes in programs or interventions can still have an impact. It doesn’t do us a lot of good to talk about results of an intervention if we find out the intervention was not really in place to begin with! You may find cases where the intervention changes a good bit as schools and teachers make it fit their particular circumstances or the needs of their students. Documenting and understanding these changes are important when you start to talk about how the intervention is affecting the problem or situation it was brought in to address. Without this information it may be difficult to replicate elsewhere.
Another important issue related to implementation is good old-fashioned quality control. How do people feel about the quality of how services are being delivered? For example, take a program designed to improve access to computers for children with disabilities. In learning about implementation we should spend some time measuring how satisfied students, parents and teachers are with the program, whether it was responsive to student and teacher needs on a timely basis, and what the challenges have been in implementing the program.
