A/B testing
Pioneering iterative A/B testing in the social sector: rigorous, rapid, and regular
We are a learning organization: we optimize our programs through ongoing rigorous, rapid, and regular testing termly across all countries and programs. These tests act as our in-house learning muscle and help us maximise impact, cost-effectiveness and scalability.
How it works
Iterative A/B testing is a rigorous methodology to optimize programs for cost-effectiveness and scalability. It involves:
→ introducing a targeted variation to an existing program
→ randomly allocating participants to the status quo (A) or the variation (B)
→ comparing changes in outcomes and costs; and
→ implementing the version with stronger cost-effectiveness.
Results from each test inform the design of subsequent tests, enabling continuous, iterative improvement.
Types of test
Cost-reducing
Similar to Jenga, elements are systematically removed to make programs leaner while maintaining impact.
Effectiveness-enhancing
Similar to building with Lego, elements are added to increase program impact at low marginal cost.
Principles: The 3 Rs
Rigorous
Randomized design enables causal inference about program impacts. Multiple groups receive the same program with a targeted variation to test how it can work more effectively, cheaply, and at scale.
Rapid
Results are reported within weeks or months, using short- and mid-term outcomes to inform real-time decisions.
Regular
Testing is embedded within organizational M&E systems and conducted in related, iterative cycles to optimize cost-effectiveness.
Example tests
Four related tests of our remote tutoring program, ConnectEd, conducted as part of an ongoing iterative testing cycle.
Across all 12 tests in this case study, both cost-reducing tests and effectiveness-enhancing tests generated efficiency gains
Working paper
Cheaper (and more effective) by the dozen: Evidence from 12 randomised A/B tests optimising tutoring for scale
Through iterative experimentation, we deliver rigorous, rapid evidence that identifies cost-reducing and effectiveness-enhancing innovations and ultimately makes the ConnectEd program more scalable.
Article
Iterative A/B testing for social impact: Rigorous, rapid, regular
This Stanford Social Innovation Review article focuses on how organizations can close the gap between measurement and implementation, by using A/B testing.
Toolkit
Iterative A/B testing toolkit
A jump-start guide to embedding rigorous, rapid, and regular testing in your organization; this Toolkit distills a decade of field experience into practical steps, best practices, and tools to help any team learn faster and scale impact.
Evidence & tools
Work with us
Building on nearly ten years of A/B testing experience, we now support other organizations interested in integrating the approach. If you are an organization interested in using A/B testing to optimize your programs, read more about the support available here.