2024 has been a milestone year for Youth Impact – ten years of growth, ten years of learning, ten years of impact. Earlier this year, we published our 10 Year Impact Report and an interactive online look at our journey over 10 years. These reports share what we did, but not how we did it. Now we want to take the opportunity to share a few lessons learned on the how.
Don’t re-invent, build on what works
This notion is at the heart of our founding – a radically simple proposition: scale what works. Too often there is a temptation to look for a shiny new idea. We didn’t set out to invent a new program. Instead, we set out to scale others’ previously proven ideas. The same applies for us when working with governments. Instead of creating new systems, we take advantage of existing ones. We learnt this quickly: we once had an opportunity to allocate a new, dedicated budget line item to hire trained but unemployed teachers to deliver our programs. While it was a politically salient and exciting proposition, it was hard to make work in practice. We would have had to make changes to the existing and complicated bureaucracy, requiring new forms, procedures, and a host of systems that weren’t in place. So, instead, we leveraged the existing system – teachers already hired, trained, deployed and on payroll – which proved far more effective.
Magical moments in the classroom matter most
As an organization grows, so does the distance between the office, where many of our team members are every day, and the classroom where our work is being delivered. So we have made it our mission to keep students in the classroom at the center of all of our decision-making. Just as our name indicates, we are laser-focused on ensuring youth receive impactful programs. When decisions are stuck, to get them unstuck we often ask “what will benefit young people most?” We encourage all staff to directly engage in fieldwork, including one day per year where every single staff member – from finance to M&E to senior management – delivers our programs to students, to stay grounded and connected to the mission.
The learning never stops
Someone once asked us “when do you stop testing your programs?” Our answer: never – we can always keep improving. In everything we do, evidence is at the foundation, and we continuously learn. Even after conducting our original Randomized Controlled Trials (RCTs) on our ConnectEd program in Botswana and in five other countries which showed the program worked to improve student learning, we continued to iterate and improve the program every term. We found that a small innovation – having a caregiver take over from our facilitator halfway through the phone call and engage more directly with their child – doubled the impact of the program at almost no marginal cost. Imagine if we had stopped testing the program after the initial RCTs? We would have left substantial impact on the table. We don’t just continuously learn formally, we also learn informally. We are known to hold lengthy debriefs after a training, high-level meeting, or school visit. Just hearing the word “debrief” will make any Youth Impacter grab a snack, drink, and a comfortable spot on the couch.
Test well and often
When the 2014 RCT of our health program found mixed results, we decided to drop what wasn’t working and double down on what was working, to optimize the program and make it ready to scale. We wanted to test these changes rigorously, but in a quicker and cheaper way than an RCT. Enter A/B testing, an approach that fits this purpose. A/B tests measure quick indicators of success (such as learning outcomes measured over 30 days), enabling them to be fast and frequent. And they are rigorous: similar to RCTs, by allocating participants randomly to groups, we ensure that any differences between groups captures a causal impact. A/B tests allow us to rapidly iterate; the results of one A/B test are immediately incorporated into the program. We have spent nearly a decade developing A/B testing for the social sector, and have been able to dramatically improve program cost-effectiveness as a result. We now routinely run multiple ongoing A/B tests each term and are supporting other organizations in the sector to run A/B tests too.
Let the data flow
One of our colleagues at Pratham once said that “Data can defy gravity” – it flows up to senior teams in their offices but never flows back down to the implementers in the classroom. Data should flow up and down to bring evidence and practice closer together. In all we do, we try to “right size” our data collection approach, so that it can inform decisions in real time. One example of this is our checkpoint tool in the Teaching at the Right Level program, which is used to target instruction on a regular basis. This paper-based tool uses a simple “problem of the day” to enable teachers to track student learning in real time and plan subsequent lessons accordingly, tailored to the level of the students. The data flows up and it flows down; most importantly, the data can be used immediately in the classroom.
The messenger matters
Our initial health program RCT revealed larger and more significant results (reductions in teen pregnancy) when peer educators delivered the program relative to when teachers did. We have another paper in the works that examines this messenger dynamic closely. Stemming from these lessons, we now focus on leveraging peer educators and have nurtured a partnership with national youth volunteer schemes to do this efficiently and at scale. Over 15 countries in sub-Saharan Africa have similar national service schemes with youth deployed to schools and clinics throughout the country. We have learned to work with these national youth scheme participants, who are often underutilized and overlooked, as a critical complement to our work with government teachers across our health and education programs. Our co-founder, Moitshepi Matsheng, chairs Botswana’s National Youth Council and has led an effort to integrate with these government structures.
Partners and people over geography
Creating change is hard and depends crucially on the people driving it. In our sector, implementation partners are often selected based on their geography. A funding opportunity is announced in a given geography and partners look to see who has reputable and relevant experience there. We’ve found it can work better to prioritize people and partners first. When we ran replication trials for our ConnectEd program in five countries, we selected contexts based on strong partners rather than location. Amazingly, even though we had never previously worked in these five geographies, and the partners and geographies varied substantially – from governments, to the World Bank, to NGOs, and from India to the Philippines to Uganda – all trials got off the ground within a few months and produced strong positive results.
Embrace the scaling snowball
Just like great partners drive impact, they also catalyze scale. It was initially disappointing when we learnt that one of our program “champions” within Government was moving to head up a different region where we weren’t currently implementing. But we decided to “follow the leader” to their new region. We followed their guidance in terms of implementation delivery, school selection, sensitization, and more. Following the leader facilitates and accelerates the scaling process. Swayed by positive results and early adopter regions, other regions follow suit. The scaling process snowballs from one region to the next. This approach of working with changemaker leaders in government who are eager to start (the “early adopters”) and following them, rather than a geography, can propagate a snowball effect.
Seize the moment
Moments of opportunity are rare, but we’ve seen how seemingly inflexible systems can pivot to the need and opportunity at hand. When schools shut down around the world due to COVID-19, we did what we do best – we took action, adapting an in-school numeracy program to a remote intervention requiring only a basic phone so we could reach people even out of school. And we produced rapid results, generating the world’s first evidence on distance education during the pandemic. Seizing these moments can yield high-impact timely results, and also open new paths to pursue into the long run. Many of our partnerships formed during the pandemic have continued to grow since, and this remote intervention became our third program: ConnectEd.
Culture really does eat strategy for breakfast
It’s a well known adage, and one we hold dear. On a given day, you can find our team transitioning from signing MOUs with senior government officials to singing and dancing in the classroom. We often snap our fingers in affirmation at meetings, to encourage any and all contributions. We can’t help but smile when we see our government counterparts snapping along with us. Our government partners and our team members celebrate each others’ family weddings. Every staff member experiences an onboarding session titled “what is good evidence.” We are a tight-knit community committed to the cause. It’s a core part of our how.
10 years ago we were founded to take evidence off the shelf and scale up what works to enable youth to thrive. We’re proud to have done just that, and to have bottled parts of the process too. With our tools now sharpened, we can't wait to see what the next 10 years have in store.
Comments