This week, a reader drew my attention to a new study that investigates the education and employment outcomes of child sponsorship.
Wydick, Glewwe, and Rutledge (Journal of Political Economy, Apr2013) survey over 10,000 respondents in Bolivia, Guatemala, India, Kenya, the Philippines, and Uganda in what seems to be the first ever systematic impact evaluation of child sponsorship programs. The data includes formerly sponsored children in communities where Compassion International rolled out its programs between 1980 and 1992, siblings of those sponsored children, and a random sampling of non-sponsored households within both Compassion villages and nearby non-Compassion villages. They find that sponsored children are significantly more likely to remain in school longer and to be gainfully employed, when compared to non-sponsored siblings and non-sponsored children in other households (in both Compassion and non-Compassion villages).
The findings show that Compassion International’s interventions have significant, positive effects on individual lives. The authors conclude that the causal mechanism for these effects is most likely the requirement that all sponsored children participate in a weekly (8 hours)
“intensive after-school program that emphasizes their spiritual, physical, and socio-economic development (395).”
While the study is impressive and provides insights into a massive gap in the literature (seriously – with it being a $3b industry, why is this the first impact study on child sponsorship?), there are a few potential biases that could be influencing the survey outcomes.
Endogeneity is one potential issue that is fairly well addressed and dealt with in this study, making it a good example of how to control for this when conducting a non-randomized experiment.
The more concerning endogeneity issue is the selection of sponsored children. Parents could intentionally select children who are brighter and more likely to succeed in the program. The authors address this somewhat by including data on their siblings and by running estimation models. They find that the oldest sibling below the 12-year threshold was most likely to be selected into the program. However, I am not convinced that this controls entirely for the non-randomness of sponsorship selection.
A second potential endogeneity issue is at the village level. The authors assume that there is a random sampling between their treatment and control villages. However, Compassion’s decision on which villages to operate is hardly random. Here’s a description from their website:
Compassion follows an in-depth process of investigation, analysis and research for God’s direction to expand into new countries and communities. Compassion’s criteria for opening a new country include:
- God’s leading. Each decision about a new country is wrapped in prayer. No country is selected unless the responsible staff are convinced of God’s leading and blessing.
- Need. Compassion works in some of the world’s poorest communities. Local church partners then select the children in their communities who have the deepest need to participate in our program.
- Strong local church partners. Christian leaders and churches must be willing to invite our ministry into their country and competent Christians to staff country offices and lead projects.
- Risk management and legal issues. A country must not have legal barriers to our ministry. There must be a provision for international banking. Our work with the local church to encourage long-term Christian child development also must be understood and accepted.
Thus there could be something entirely unique about the villages within which the treatment (sponsorship) is given. This is somewhat convincingly controlled for in the study using siblings and respondents from non-sponsored households within the treatment village. Although, findings elsewhere have suggested that the effect of not being sponsored within a treatment village can have detrimental impacts on a child’s psyche and create divisions within the community between winners and losers (see Enew 1996; Stalker 1982; Herrell 1986).
The biggest threat to the validity of this study is the survey instrument and its implementation. The authors do not provide a copy of the survey or samples of the questions in the appendix. The actual design of the survey is a bit confusing because in one section the authors state:
“The questionnaire was designed to ask each question sequentially across all siblings by age to avoid focusing on the siblings who had been sponsored by Compassion (406).”
Thus I first thought that the survey was designed to focus on the education and employment of the household regardless of sponsorship and no questions were directly posed about Compassion and/or sponsorship, giving respondents little reason to believe that the survey was about outcomes from the Compassion intervention and making them more likely to respond earnestly. However, then I remembered a statement earlier in the paper that said,
“The survey included an open-ended question asking formerly sponsored children which component of the Compassion program had been most beneficial to them (403).”
So it looks like the authors did target specific questions about Compassion and sponsorship to formerly sponsored respondents. This could have generated a bias in the responses. Formerly sponsored individuals may think that the survey is being administered by Compassion and thus that their negative responses may impede future selection of their children for sponsorship, and/or generally feel pressured to provide positive feedback on the program regardless of actual outcomes.
A second potential issue with the survey is the enumerators. The authors state,
“They were usually recommended by [Compassion] project staff, and were known to be responsible, well-respected community members. They also had been raised in the village and so were knowledgeable about the community, but we eschewed hiring enumerators with formal connections with Compassion to avoid bias in responses (405).”
Thus the authors employed enumerators who had informal connections to Compassion and who were well-known in their communities. Even if these individuals did not have formal connections to the organization, their informal connections could bias responses. The simple fact that they were recommended by Compassion makes their objectivity in performing the survey and the objectivity in the responses they received questionable.
Variations in the Treatment
The final potential issue with this study is the independent variable – a dichotomous, dummy for sponsorship, with 1 being sponsored and zero being non-sponsored. Their findings in a sense capture the impact of sponsorship generally, which appears to have a positive effect on both education and employment. However, individuals were sponsored for various lengths of time and had differing relationships with their sponsors.
“To avoid attrition bias, the sample includes both children who were sponsored for many years as well as children who dropped out relatively early (404).”
Does it matter whether an individual is sponsored for 1 year or 12 years? At present, the authors assume that the duration of the treatment does not matter. The study could be greatly improved by replacing the dichotomous IV with a variable that captures length of years sponsored, with 0 being individuals who were never sponsored. This would greatly enhance the findings in this paper. Right now, all lengths of sponsorship are lumped together, which could be masking temporal treatment effects.
Lastly, as previously reported on this blog, both the BBC and Plan International have reported that relationships with donors can have varying effects on a child’s psyche. Jealousy and anxiety can develop among children and families who do not receive letters or gifts from their sponsors as frequently as other sponsored children in the village. Plan cited this as a potential ethnical concern. However, erratic or seldom contact with sponsors could also impact the outcomes of the sponsorship program, damaging the self-esteem programs that the authors cite as the causal mechanism for Compassion’s success.
The study would be improved by taking into account the frequency and types of contact that the formerly sponsored individuals had with their sponsors to see if this has any bearing on long term educational and employment outcomes.
Overall, despite the study’s shortfalls, Wydick, Glewwe, and Rutledge (2013) attempt to fill an important gap in the literature on child sponsorship as a mechanism for development at the most micro of levels – the individual. They open the door for future research on this topic with other child sponsorship schemes besides Compassion International and its outcomes at other levels like the household and community.
H/T “Rachel” who commented on my previous piece about child sponsorship and alerted me to this exciting and important research.