The Zambia eSchool 360 Evaluation Project

Low- and middle-income countries have made significant progress toward placing children into schools, but student learning and achievement are often dreadfully low. Globally, approximately 387 million children of primary school age are not acquiring basic reading and mathematics skills, even though about two-thirds are in school and will complete primary school. Zambia faces many common educational challenges.

Impact Network’s eSchool 360 program represents a promising approach to improving educational outcomes in rural Zambia by incorporating three potentially high-impact components that could create important synergies. Each component could, on its own, have positive impacts on student outcomes by engaging the three main actors in the education system: students, teachers, and parents, respectively. However, combining these elements into a single program may be particularly impactful by working to align all three actors towards improving the educational outcomes of the students. Earlier research has suggested that these complementarities may be substantial with higher impacts from educational technology programs that include a strong focus on pedagogical practices.

The multifaceted program comprises an e-learning technology component, ongoing teacher training and professional development, and community ownership among students in community schools in rural Zambia. The e-learning component includes electricity via solar power, projectors, and tablets loaded with materials in the local language that are structured around a curriculum approved by the Zambian government.
 

The Evaluation

AIR has designed, and is implementing, a two-stage mixed-methods evaluation to measure the effect of Impact Network’s eSchool 360 program on learning outcomes among students in community schools in rural Zambia as well as the cost-effectiveness of any measured impacts. The use of cost-effectiveness analyses will aid in estimating the cost of achieving certain benefits, such as improvements in learning outcomes. These estimates will in turn guide policymakers in assessing the value for money of investing in the eSchool 360 program. 

The evaluation uses a cluster-randomized controlled trial in which 63 eligible schools were randomly assigned to either receive Impact Network’s eSchool 360 program (“treatment schools”) or not receive the program (“control schools”). The evaluation design began with quantitative baseline data collection and then follows with quantitative and qualitative data collection to inform the scale-up of the program one year and three years after the start of the baseline, by measuring aggregate scores on the early grade reading assessment (EGRA), early grade math assessment (EGMA), and Zambian achievement tests, and the secondary outcomes are the EGRA, EGMA, and Zambian achievement test subtasks as well as measures of oral reading fluency.

In addition to the EGRA, EGMA, Zambian achievement test, and oral reading fluency measures, we will also estimate the program impacts on school attendance and enrollment, parents’ perceptions of school and education quality, student-level aspirations, and parent-level aspirations. Using these outcome measures will enable us to determine impacts along the causal chain of the theory of change and let us examine the mechanisms underlying the program impacts.