Approximately 250 million children across the world are not acquiring basic reading and math skills, even though about half of them have spent at least four years in school (UNESCO, 2014). Improvements in students’ learning achievement have lagged behind in low-and middle-income countries, despite significant progress in school enrollment numbers. For several years now, large-scale early grade reading and math assessment data have highlighted worryingly high “zero” scores in reading assessments across the world.

Impact Network’s eSchool 360 program is a multifaceted program comprising an e-learning technology component, ongoing teacher training and professional development, and community ownership among students in community schools in rural Zambia. The e-learning component includes electricity via solar power, projectors, and tablets loaded with materials in the local language that are structured around a curriculum approved by the Zambian government.

The eSchool 360 program represents a promising approach to improving educational outcomes in rural Zambia by incorporating three potentially high-impact components that could create important synergies. Each component could, on its own, have positive impacts on student outcomes by engaging the three main actors in the education system: students, teachers, and parents, respectively. However, combining these elements into a single program may be particularly impactful by working to align all three actors towards improving the educational outcomes of the students. Earlier research has suggested that these complementarities may be substantial with higher impacts from educational technology programs that include a strong focus on pedagogical practices.

AIR will design and implement a two-stage mixed-methods evaluation to measure the effect of Impact Network’s eSchool 360 program on learning outcomes among students in community schools in rural Zambia as well as the cost-effectiveness of any measured impacts. The use of cost-effectiveness analyses will enable us to estimate the cost of achieving certain benefits, such as improvements in learning outcomes. These estimates will in turn guide policy makers in assessing the value for money of investing in the eSchool 360 program. 

To determine the impact of the program, we will use a cluster-randomized controlled trial in which 64 eligible schools were randomly assigned to either receive Impact Network’s eSchool 360 program (“treatment schools”) or not receive the program (“control schools”). The evaluation design starts with quantitative baseline data collection and then follows with quantitative and qualitative data collection to inform the scale-up of the program 1 year and 3 years after the start of the baseline data collection.

We will field a consistent assessment instrument across the midline and endline to measure the impact of the Impact Network program on student outcomes 1 year and 3 years after the introduction of the Impact Network schools. The primary cognitive skills outcomes are aggregate scores on the early grade reading assessment (EGRA), early grade math assessment (EGMA), and Zambian achievement tests, and the secondary outcomes are the EGRA, EGMA, and Zambian achievement test subtasks as well as measures of oral reading fluency. In addition to the EGRA, EGMA, Zambian achievement test, and oral reading fluency measures, we will also estimate the program impacts on school attendance and enrollment, parents’ perceptions of school and education quality, student-level aspirations, and parent-level aspirations. Using these outcome measures will enable us to determine impacts along the causal chain of the theory of change and let us examine the mechanisms underlying the program impacts.