Evidence-based policy analysis? The strange case of the randomized controlled trials of community-led total sanitation

Our purpose in this paper is to review the findings of 14 randomized controlled trials (RCTs) of community-led total sanitation (CLTS) and recent rural sanitation interventions to assess their usefulness and implications for sanitation policy-making in low- and middle-income countries. The results of the RCT research programme to evaluate CLTS and related sanitation interventions suggest that the magnitude of the treatment effects was much smaller and uncertain than proponents once anticipated. For example, of the ten studies that reported results for reductions in childhood diarrhoea, only three found statistically significant decreases. Surprisingly, the RCT research teams and their funders do not seem to have thought about how their multi-million dollar research agenda would support decision-making on sanitation. Information on the parameters needed for cost-effectiveness analysis or benefit-cost analysis was not collected. However, making reasonable assumptions about the missing information on parameter values, we show that cost-benefit analysis may still 'save' CLTS because small treatment effects may still yield net positive economic benefits if the costs of implementing CLTS programmes are modest. We also discuss the need to move beyond the desire for sanitation policies that are proven to be effective globally, and the importance of focusing on analysis of the local sanitation situation. We describe the data needed to make this shift in policy focus from the global to the local level and stress the importance of interdisciplinary communication between the proponents of RCTs and 'evidence-based policy', and economists who will be responsible for the economic analysis of investments in CLTS and other sanitation interventions. We also argue that the results of these RCTs highlight the importance of coordinating investments in piped water and sanitation with investments in improved housing.