Why Scaling is Hard with Complex Problems: Education in Bihar, India
In many areas of public action, the government is the only way to effect change at scale. Yet governments are better at solving some problems than others. Particularly tough are the "wicked" problems that require changes in how complex systems behave. Education is an example. Let's look at one case, that of Bihar, that illustrates the general issues.
The Government of Bihar has made substantial progress on some educational fronts but not on others. The proportion of children, especially girls, out of school went down significantly between 2006 and 2010, and has been broadly maintained ever since (Figure 1). The share of schools with a separate, usable girls' toilet rose from only 18% in 2010 to 63% in 2012. Yet, progress on learning, that is basic skills in reading and math, has been dismal, and actually worsening, from a bad initial position, between 2008 and 2018 (Figures 2 and 3).
Figure 1: Large reductions in the proportion of boys and girls out of school (Source: ASER 2018)
Figure 2: Dismal outcomes in reading (Source: ASER 2018)
Figure 3: Dismal outcomes in math (Source: ASER 2018)
Bihar is a state of over 100 million people, one of the poorest in India, with in recent decades a reputation for corruption, conflict and profound inequalities across caste, class and gender lines. Yet this pattern of dismal educational outcomes is typical of all India (see the Figures) including in relatively richer and supposedly higher capacity states such as Gujarat and Tamil Nadu. It also reflects a global pattern: governments do a reasonable job at getting children into schools and a very bad job at improving learning, as reflected in the World Bank’s recent World Development Report on education, and the report of the international Education Commission. With a few exceptions, mainly in East Asia, developing countries are experiencing a learning crisis, with children leaving school with levels of skills inadequate for work, and little or no progress on learning improvement.
How do we understand this? First, we can heuristically think of two kinds of problems. There are problems in which what to do is essentially known, both in terms of technical models and alignment with organizational capacities. Building schools, campaigns to get kids into school, constructing girls toilets and instructing principals to ensure they are open, are examples of such problems. Even these, relatively "easy" problems require a political push to get done. The big political change in Bihar came when an election brought Nitish Kumar to be Chief Minister in 2002 on a development platform. Getting children, and especially girls, into school was a key part of this platform.
Second there are the “wicked” or “complex” problems, for which solutions are not known. This is sometimes because a technical model is still to be found. Equally important are cases where there may be a technical solution, but getting it implemented requires an interacting set of behavioral changes within a system. Here the organizational issues constitute the complex problem. It turns out improving learning is one of those problems.
Tackling the second type of problem involves a process of exploration and engagement with the actors within the system. And since the whole point is to do this at scale, that needs to be the orientation from the beginning.
The scaling challenge for such a complex problem almost always has two parts: first, finding an innovation approach that is scaleable in both technical and organizational terms; and second, implementing the scaling itself. What is interesting about the Bihar experience is that the first part was done: this was through a remarkable, ongoing, multi-year process of collaboration and exploration between the government's education system and the non-profit Pratham (the same non-profit that conducts the extraordinary volunteer-based annual survey of education from which all the above information is taken).
Over time, Pratham explored, developed and applied a pedagogy for basic learning that is highly effective, including when delivered by existing teachers in the Bihari school system as well as elsewhere in India, provided it is implemented with fidelity. This is known as "Teaching at the Right Level" (the name describes the essence of the approach). The problem was that even when introduced in schools most teachers didn’t implement it, typically because they were swamped with other curricular or non-curricular activites. So the pedagogy was not implemented with fidelity.
The organizational discovery occurred when a leading bureaucrat in one district, the District Collector, embraced the learning challenge and invited Pratham to advise; he had heard of their work elsewhere in Bihar. They then worked together on how to activate the latent potential within the education system. This involved worked with the frontline education bureaucrats (who supervise schools), to work with principals on the within-school training, organizational and tracking changes that were required of the technical pedagogy. There were substantial gains in learning outcomes of children in participating schools within only a few months.
This looked like a truly scalable innovation precisely because it worked within the education system. Crucially, it needed bureaucratic top-down support to ensure the pedagogic intervention really was implemented by principals and teachers in the schools. Then it really looked like it could go state-wide, when it was taken up in 2013 by the key political actor, Chief Minister Nitish Kumar. He wanted the approach rolled out, and he expected to see learning results. There was indeed a lot of action (including in collaboration with Pratham) and some success in some districts, but, as Figures 2 and 3 illustrate, the results didn’t come through at the level of the whole state. Why? The explanation seems to be a combination of a resistant bureaucratic machinery, multiple problems of coordination, alongside shifting political priorities, as elections came and went.
Two quotes from Pratham's past and present CEO capture well the challenge:
Madhav Chavan: “A motivated state machinery with leadership and consistent policy backing is the key to big systemic changes. NGOs and foundations can be helpful but not without energy from state functionaries”
Rukmini Banerji: "Ensuring that every child has the opportunity to acquire foundational skills in primary school will need substantial changes in the ways that the system currently works. We need to rework what we are doing, why we are doing it, and how we do it, from the policy level to the classroom level."
(Both from the ASER 2018 report)
I believe this will eventually work. Pratham continues to engage with those parts of the education system that want to play. Bureaucrats, principals and teachers have the latent potential, but they are trapped in the current system. Getting the combination of motivation and consistency remains a major political and bureaucratic challenge. An achievement on the activist front is getting learning, as opposed to schooling, belatedly on to the policy agenda. Now the task is working out what alignment from leadership through the whole system means in practice. Meanwhile far too many Bihari, and Indian, and global, children, are leaving school without the skills that they need for work and life.