Woliso is a beautiful rural area in the Oromia Region of Ethiopia where many families are subsistence farmers and children typically have to walk 30 minutes or more to get to their school, especially in the smaller communities. Between 2002 and 2012, Save the Children invested US$22.5 million (in 2019 dollars) to improve the wellbeing of children and their families that resulted in several life-changing outcomes and impacts, some of them long-lasting. But does that mean it was “worth the investment”? Were our results “good enough” and how do we determine what “good enough” is?
This is no ordinary evaluation, of no ordinary development program. Save the Children’s Child Sponsorship program pools donations from sponsors to benefit whole communities. We invest in activities intended to produce measurable results for children, their families, communities, and the systems and policies that sustain change over time – benefiting both sponsored and non-sponsored children. And we innovate and test new approaches too. That way, the program seeks to achieve lasting social change by addressing the underlying issues including equality, protection, inclusion, and social norms. Worldwide, Save the Children’s Child Sponsorship Programs work with the most marginalized communities in 21 countries, for 10-15 years. This type of Child Sponsorship programming has been operating for more than 100 years, developed and created by Save the Children’s founder, Eglantyne Jebb. But does this type of long-term programming work? How much difference does a Save the Children’s Child Sponsorship Program really make in people’s lives? And how well are the impacts sustained over time?
We wanted to find out the answers to these big, important questions! To understand the impacts of 10-years of programming in Woliso, we commissioned a “retrospective impact evaluation” – i.e. the type of evaluation that looks back into the past and assesses the value of long-term effects, good and bad, that remain from programming that happened a long time ago. In this poor rural part of Africa, from 2002 to 2012, Save the Children had invested in a package of programming: basic education; early childhood care & development; child and family health & nutrition; and adolescent development.
The evaluation team, led by Jane Davidson and Thomaz Chianca from Real Evaluation (realevaluation.com), began their journey back in time in 2019 with help from the Ethiopia Save the Children staff and a local evaluation firm led by Zelalem Adugna Geletu. It was a messy business: not only is Child Sponsorship programmingcomplex and multifaceted, with activities aiming to reach children of all ages, families and communities, but Save the Children’s work in that area had finished up nearly a decade ago. The evaluators faced thin and patchy monitoring data, no secondary outcome data of any use, and they found themselves working in hard-to-reach communities and in very challenging economic and political environments.
But the hurdles were not too big! The evaluators used a range of innovative new evaluative methods and a protocol very different from the usual which, coupled with a large dosage of grit, determination and expertise, led to the production of this fantastic end-product: the final evaluation report, available online here. The visuals are inspirational, catchy, and the accessible writing style makes the reader feel great about themselves for understanding the key messages from such a complex undertaking: there’s no head-scratching!
The report is packed with findings, so this is just a teaser:
1. Returns to investment in Education: This evaluation estimated that having access to the schools built by Save the Children helped ensure, on average, an additional 4.5 years of schooling for the children and adolescents who attended those schools. The estimated additional lifetime earnings for those children/adolescents amounted to US$137.5 million. In other words, for every dollar invested by Save the Children in education there is a potential return of about US$12 in lifetime financial benefits to the (now) young women and men who attended the schools built by Save the Children.
2. Returns to Investment in Water and Sanitation: Save the Children invested about US$4.7 million in building and maintaining water schemes and sanitation facilities. With no primary or secondary data available on the outcomes of this investment, the evaluators turned to a WHO study that calculated a return of US$2.7 to every dollar invested in water and sanitation in sub-Saharan Africa, from: (i) health care savings, (ii) reduced productivity losses, (iii) time saved fetching water, and (iv) premature deaths averted. To avoid overestimation, they discounted a 41% loss for water schemes that were no longer working. They estimated a return of US$7.5 million equivalent in benefits. This equated to an expected value of US$1.59 in benefits for every dollar that Save the Children invested.
3. Ripple effects: A number of positive changes were inspired by SC’s work without having been originally planned or expected as an intentional effect, for example:
– The development of a culture of community mobilization – examples included saving money collectively to resolve pressing issues in the communities and getting better organized to put pressure on government to help address important needs.
– Changes in attitudes towards education, sanitation and hygiene – examples included families valuing sending their children to school, especially the girls, and open defecation being considered a shameful practice.
– Creation of a national school health and nutrition (SHN) policy and plan of action for Ethiopia. This national-level change was inspired by Sponsorship’s successful community-level SHN efforts in Woliso. Sponsorship was invited to sit at the table with the Ethiopian government to help develop this national policy and plan.
With this robust retrospective evaluation in hand, we can now confidently state that this 10-year US$22 million program in Woliso was worth the investment. The program demonstrated multiple, long-lasting, positive changes for children and their communities. That is not to say that Save the Children can claim all credit for the positive impacts. And in fact, if we could affirm that the results were100% due to our work, it would be a failure of the intended Sponsorship model, which is to engage in joint efforts and to be a major catalyst and enabler of change rather than the sole creator. In the same vein, the conclusions in this report do not belong to our evaluators or Save the Children staff alone, but were arrived at through careful consideration of insights from the communities we serve and partners we work with.
MEAL Advisor, Sponsorship
For more information, contact: Sponprograms@savechildren.org