Over 1.6 billion students were affected by Covid-19 school closures, with an estimated global average of 14 weeks of school lost. This compounded a learning crisis that existed before the pandemic and increased inequality. Tutoring programs that offer intensive, targeted instruction may offer a promising solution that can help students recover learning. A low-cost model that has gained recent attention is tutoring via mobile phone. Estimates show that 70%–90% of households globally own at least one mobile device, suggesting that simple phones can be an effective tool to reach children at scale from a range of backgrounds.
Youth Impact conducted a randomized control trial (RCT) at the outset of the COVID-19 pandemic in Botswana and found that phone tutorials had positive impacts on learning. This study represented some of the first evidence released on learning loss during the pandemic and generated global interest in phone-tutoring as a cost-effective approach to improve learning at scale. .
To build further evidence on these approaches, we joined hands with a coalition of partners - ranging from NGOs to governments to the World bank – to replicate this study through a large-scale, multi-country effort to study the effectiveness of phone tutorials to improve basic learning across a diverse array of global contexts and when delivered by governments. Collectively, this latest set of multi-country RCTs reached over 16,000 children in Kenya, Nepal, India, the Philippines and Uganda and represents some of the quickest, multi-country evidence ever produced in education.
Each of the five RCTs evaluated a pair of simple mobile phone interventions to provide essential tutoring support. This includes short, 20-minute phone tutorials between instructors and households and one-way SMS messages to improve numeracy among primary school children. The program connects students with highly targeted instruction focused on foundational skills according to each child’s learning needs. This pedagogical approach builds on Teaching at the Right Level, a targeted instructional method proven to enhance learning in classroom settings. Below is a summary of key insights from the paper:
Does it work?
We found positive results in each of the five trials. On average, we find a 65 percent increase in the percent of students who learn division - effectively mastering all basic numeracy concepts that we deliver in the course of programming. Results on the effectiveness of SMS messages alone were more mixed. Only trials in the Philippines and Uganda - countries with the longest school closures in the world - showed small increases in learning and then only when there were no other options to access learning. These results prove that simple mobile phones can be highly effective platforms for delivering remote tutoring support and suggest that live phone interactions are necessary to achieve the greatest results in learning.
Is it Cost-Effective?
Phone tutoring is highly cost effective relative to education programming. On average, the programming that we studied in each of the five countries cost USD 12 per child. Cost-effectiveness estimates show the approach delivers the equivalent of nearly four years of high quality education per $100 spent, one of the top 10 most cost-effective education interviews in a recent review.
Can governments effectively implement the approach?
In two countries - Nepal and the Philippines - we tested the approach using government-employed teachers as tutors and working within government systems. In both contexts, teachers proved to be equally effective at delivering the program as NGO-hired facilitators, suggesting that phone tutoring can be delivered and scaled by governments.
What are the Future Policy Implications?
The goal of this research effort was to produce a longer-term public good in the form of knowledge on effective approaches to support learning, both during a host of education emergency responses that are applicable beyond the context of COVID-19. More than 2 billion people live in countries that are vulnerable to shocks that disrupt education. Holidays, rainy seasons, pollution, elections, teacher strikes, conflict, climate shocks, disease, and natural disasters are common scenarios that keep children out of school (GCPEA, Education under Attack 2020). Low-income families are often less equipped to support learning during prolonged closures, as many do not have access to resources that enable learning at home. Our findings on phone tutoring programming suggest that basic mobile devices can be effectively utilized as an education response in emergencies that is low cost, yields high take up across different geographies, and can be targeted to children of varying levels and cultural backgrounds.
Furthermore, the approach could also have relevance during non-disrupted settings. Because of the cheap cost of phone calls, one-on-one phone tutoring programs can be extremely targeted to support the lowest performing students. To this end, Youth Impact has continued to enhance phone tutoring approaches in Botswana, with the goal of realizing its full potential as an addition to in-person classroom interventions.
About the Research and Implementing Coalition
A remarkable coalition of partners made this study possible. Youth Impact provided overall coordination and technical assistance, building on the original proof of concept in Botswana. Implementers involved in this multi-country research effort included a variety of organizations ranging from governments to NGOs.
Nepal: The Ministry of Education Science and Technology, the World Bank, Street Child Nepal and Teach for Nepal
Kenya: NewGlobe
India: Alokit and Global School Leaders
Uganda: Building Tomorrow
The Philippines: The Department of Education and Innovations for Poverty Action
Research partners involved in the project include the University of Oxford, Columbia University, Learning Collider and the Jameel Poverty Action Lab (J-PAL).
Special recognition goes to additional partners who supported this research, including the UBS Optimus Foundation, Jacobs Foundation, Echidna Giving, Mulago Foundation, Stavros Niarchos Foundation, Douglas B. Marshall Jr. Family Foundation, Peter Cundill Foundation.
Key Contacts
Colin Crossley, ConnectEd Program Manager (ccrossley@youth-impact.org)
Karen Clune, Head of Innovation & Development (kclune@youth-impact.org)
Also check out our policy brief for more information.