A link to the full paper is here.

This paper is also published as a working paper at the Center for International Development at Harvard University.

Abstract: Developing scalable innovations is a central challenge in development. Organizations that succeed in scaling often prioritize measurement, learning, and evaluation, but how this is done is poorly understood. This paper explores the case of Pratham, one of the largest NGOs in primary education and an exceptional learning organization. While Pratham is renowned for its randomized control trial sequence with J-PAL, we underscore this is just one aspect of a wider array of learning activities. These include an iterative process to continuously refine and protocolize solutions as they scale, the use of on-field qualitative insights and quantitative tracking, open feedback channels between the field, state, and central offices, information exchange among content and evaluation teams, and a nationwide assessment of learning levels (ASER). The RCTs were effective because they were embedded within this broader learning process and culture. Pratham learns at three levels: learning to improve children’s basic skills, learning as an organization about what does and doesn’t work, and fostering learning by others in the system. Its learning capacity is rooted in deep-seated values and a culture of openness, trust, problem-solving, and the freedom to experiment and learn from failure, inculcated and nurtured by its leadership. Throughout, we use the prism of an Adaptive Evaluation to provide a systematic framework for mirroring and understanding Pratham’s organic learning processes (as affirmed by its own leadership). Pratham actively engages with the three main pillars of an Adaptive Evaluation, involving understanding systems, theorizing how to effect change, and iterating its designs. While Pratham’s culture will often not be transferable, the systematic analysis of how Pratham learns can provide a framework for other organizations aspiring to replicate Pratham’s success as a learning institution.