Ideas & Insights The Missing Middle: Why Is It so Hard for Organizations a...

The Missing Middle: Why Is It so Hard for Organizations at the Base of the Pyramid to Go to Scale?

Working Paper

Nov 12th 2015

A link to the pdf of the full paper is here

By Isabel Guerrero and Michael Walton

This working paper outlines an analytic framework around the scaling up of small organizations at the Base of the Pyramid. This includes small-scale, informal firms, grassroots organizations working for the poor as well as social entrepreneurs looking to provide social goods.

There are different ways of scaling up: (1) through the growth of one organization that reaches new clients/members and new locations, (2) through replication of a product or methodology by other organizations, or (3) through networks working in partnership with other existing organizations that have similar goals. Scaling up through organizational growth is only needed when there are economies of scale, i.e. larger production will bring down average costs. Replication will reach scale by creating a viable ecosystem that allows many organizations to thrive and together have a high impact even while remaining small in size. Networks look to scale impact by connecting with like-minded organizations, which might be doing complementary interventions, to achieve a common goal.

The framework is organized around the interaction between market, government and organizational failures. This is intended as a categorizing of issues, that can then form the basis for diagnosis of which constraints, or failures, are binding, and then the design of strategies for change.

⚬    ⚬    ⚬

1. The Missing Middle

Originally, the concept of the missing middle referred to the observed pattern of firm size in terms of revenue, capital and employment. In developing countries there are many firms that operate in the informal sector and remain small, while creating most of the employment. At the same time there are large firms with substantial number of employees. There is empirical evidence that in many developing countries there is a missing middle with few firms able to grown into medium size in terms of revenue and employment. Most firms remain small and informal. By contrast large firms have access to formal financing and operate under the regulations and rules established for the
private sector.

Figure 1 provides an illustration of how the firm size distribution might look in a perfect world (red curve), compared with the common pattern in developing countries, where the distribution is skewed to the left with most firms in the small size spectrum, with few in the middle and end (blue curve). In his paper on market failures and development, Joseph Stiglitz argues that market failure is more prevalent in less developed countries because of costly and imperfect information that is not ameliorated by non-market institutions (such as internal capital markets within large conglomerates).

On the labor market front, employees working in informal firms, typically lack access to social security and other benefits provided to workers. They will only get these benefits if they are working for firms in the formal economy (small or large). Labor regulations are usually designed for the conditions in large firms and therefore they mostly applied to those. Even then, as one can see in the case of India and Mexico, there is broad evasion and subcontracting to avoid labor regulations by large firms, even by governments.

Fig 1. Firm Distribution With and Without Market, Government and Organizational Failures

Fig 1. Firm Distribution With and Without Market, Government and Organizational Failures

Just as in the case of private enterprises, there is a “missing middle” in the realm of social enterprise activity and public development action. This is represented in Figure 2, where the top of the Pyramid shows service provision by Governments and traditional development aid.

Most of the money spent by governments in developing countries is in large programs that are successful at delivering some services, many using a top-down approach. The challenge for these programs is to reach the poor in the last mile. This is especially true of programs that require discretionary action by local government workers—teachers, nurses, police, extension workers, and so on.

The bottom of the pyramid in such local service provision and action is populated by many small social entrepreneurs, NGOs, and grassroots organizations who are rich in local knowledge and innovation, and who are effective in reaching the poor. Their challenge is to reach sufficient scale to have a larger impact. They tend to remain small and the innovations they have discovered don’t have the impact needed to achieve large-scale change.

Fig 2: The Missing Middle in Development

Fig 2: The Missing Middle in Development

The central idea is that local, small-scale activities have problems scaling up, while organized, large scale activities are weak at reaching the needs of poorer groups and individuals. This is a central development issue since large-scale corporates typically provide employment opportunities for only a fraction of the working population and
government programs often fail to provide decent services for the poor. In areas of social provisioning, where the state is typically a provider—for example in water, sanitation, health, education and safety nets—the issue is of ineffectiveness at the local level. There is often some alternative provision of the services, by the private sector or non-government actors, but of inadequate reach or quality, especially for poor households.

In both areas there is plenty of dynamic grassroots and social entrepreneurs who are involved in informal productive activities, mobilizing and organizing, innovation, providing services, and engaging with local state actors. But this only rarely reaches scale.

The missing middle is a result of the barriers to scaling up pathways for grassroots organization and enterprises at the base of the pyramid. These barriers are due to the interaction between three factors: market failures; government failures; and organizational weaknesses. Market failures- including market power, public goods and externalities, transaction cost and informational problems over contracting- place major hurdles in scaling.

This article will develop how these failures occur and will not cover how they can be solved. We will address these potential resolutions in future articles

⚬    ⚬    ⚬

2. Cascading Failures

There are three categories of reason why: market failures; government failures; and organizational weaknesses. Which failures matter in a particular case is important to how to tackle the scaling problem.

A. Market failures

Many of the classic areas of market failure can impede scaling.

  • Public goods and externalities. Where the benefits of an activity cannot be fully charged for, it will be under-produced by the market. (In economic terms a good is public if one person’s consumption of it doesn’t reduce that of another, and if it is not possible to exclude people from consuming the good.) There are at least three areas where under-production can hurt scale:
    • Ideas and innovations adapted to the needs of grassroots organizations and of private and social entrepreneurs.
    • The absence of the rule of law and adequate regulations is an important constraint to contracting and to protecting property rights of small organizations.
    • Local infrastructure, including markets, roads, electricity, water and sanitation, are local public goods (or “club goods”) that are essential complements to productive and social activities of grassroots organization and entrepreneurs.


  • Information asymmetries. Market exchanges are typically imperfect because of high transactions costs of contracting and incomplete information—for example over the productive potential and effort of an organization. This is particularly problematic at the base of the pyramid for finance and risk management.
    • Finance is thick with failures, including the transactions costs of lending/repayment, and lack of information over borrower creditworthiness. Collateral is a partial “solution” but not usually available to poor households and small enterprises. Micro finance was an institutional innovation that helped solve the repayment issue through group-based incentives, but is intrinsically limited to micro lending.
    • Risk is pervasive for poor households and small scale enterprises, adverse events can be catastrophic, and private insurance mechanisms (e.g. within the community) inadequate. This constitutes both a social need and a restriction on the capacity to risk expansion of productive activities.


  • Different discount rates. Scaling up of social enterprises or civil society organizations, may take years. Private sources of finance often have a relatively short-term financing horizon (and a higher discount rate), because of the demands of their sources of finance (depositors, shareholders), and the risks of having their assets tied up in long-term activities. This interacts with informational concerns over whether an organization is genuinely on the path to long-term, scaled viability. This is why “patient capital” is rare.


  •  Market power. Markets can be uncompetitive because of monopolistic market structures. Large players can use their market power to restrict economic opportunities for small-scale actors (private or social enterprise) or use their monopsony power to pay lower wages to the workers they employ.

Market Failure

B. Government Failures

Market failures provide a classic rationale for government action, in several of the above areas. However, governments also fail, especially in developing countries. The incentives for politicians are often for patronage, vote-buying or provision of “populist” private goods, rather than public good provision. Bureaucracies are often weak, may be corrupt, and not responsive to the needs of small-scale actors.

  • Missing Public Goods. Government provision of public goods could in principle tackle several of the areas that constrain scaling up, but often fail:
    • Local infrastructure is often lacking—especially for the “last mile”— because of weaknesses in implementing agencies, biases toward influential actors, regulatory issues.
    • Social goods are often low quality or non-existent for beneficiaries at the Base of the Pyramid.


  • Inadequate Regulation to offset monopoly power and protect workers and consumers. Regulations are not usually adapted to the needs of small-scale organizations in the informal sector (e.g. restricting activities of street vendors).
    • Provision of the rule of law is typically inadequate, with local police often a source of bribery, and judicial systems slow and costly
    • Regulations are often designed with large firms in mind and are too onerous for small organizations at the BoP.


  • Corruption: Public servants sometimes use their discretionary power as a way of increasing their wealth. The poor are usually the most affected by it because they don’t have access to powerful networks nor institutions that can be a counterweight to the abuse of power.


  • Principal Agent Problem. The combination of centralized decision-making and hierarchical organization in governments, usually results in a failure to reach the last mile. Even if the national government and its ministers support programs for the poor, there are many layers of bureaucracy and levels of government between this and the front-line providers.


  • Different time horizons and discount rates. There are also parallels with the private sector on time horizons. Governments can be impatient—needed quick results to maintain popular support—and donors and foundations are often eager to show results to support their own stakeholders.


  • Transactions, information asymmetries and contracting. Even for well-intentioned governments and donors there is an information problem similar to those in private markets. They have incomplete information on the potential capacity of an organization and don’t know whether financing them will lead to the changes needed for scaling up impact. This might lead to underfinancing of projects with high social value at the BoP or to financing the same organizations again and again, to manage potential risks.

Government Failure

C. Organizational weakness

So organization can be prevented from reaching scale as a result of market and government failures. Sometimes, organizations can solve these failures by internalizing them within their structure. Instead of contracting out, for example, a firm can integrate vertically and solve contracting problems and lack of rule of law. If an organization is large enough, it can more easily finance its capital requirements, through own finance and access to markets, and can even take care of some of its local infrastructure needs(e.g. in electric power generation). This internalization of failures is only possible when working at scale. A large-scale organization, however, brings new problems, associated with the principal agent problems—e.g. around monitoring—and in sustaining the values and practices that may be crucial to success (for private and social enterprises). Innovation is also more difficult from within a large organization, where there are vested interests that want to keep the status quo.

In this section, we will focus on the specific challenges for a small organization to go to scale through growth. As discussed at the end of this note, some of these issues also apply to scaling through replication or partnering with other organizations.

When self-employed workers and households organize, they can solve some of the failures discussed above, through group-based lending or providing some greater market power, as in SEWA’s construction cooperative arranging contracts for members. SEWA and BRAC are also directly providing finance and insurance for their members. Organizations and collective action can also put pressure on governments for changes in policy (as in the successful change in India’s street vendor law) or can influence implementation of government programs (as in the SEWA’s Wash Committees working to activate government water programs.) Sierra Productiva, a farmer organization in Peru, was able to channel innovations to increase rural productivity for their members.

However, there are limitations to typical organizations at the base of the pyramid. Grassroots and social enterprises are often run on informal relations, with committed leaders, strongly shared values and a lot of sweat equity. This can be highly effective at small scale—where shared values are well-understood and continually reinforced, everyone knows what others are doing, and there is space for local innovation. But different structures are needed in large scale activities, and different capabilities needed to support transitional scaling.


As organizations grow, they need to invest in a variety of systems— accounting, human resources, finance, monitoring and evaluation and mechanisms for communicating with outside actors. They need to move from informal ways of running an organization based on personal knowledge and a common history to organizations with formal data systems and supporting technological platforms.

From Stories to Data-based Monitoring and Evaluation

Some grassroots, like SEWA, have been able to break through to scale in terms of numbers, activities and location, through a combination of extraordinary collective organization, values and leadership practices. As they prepare for further scaling up (from 2 million to 4 million members) they face the urgent challenge to develop data systems that can support further scaling, including basic tracking of members and participants, managing programs and surveybased assessment of effects. Their data collection and membership tracking system are no longer functional with such large growth in membership as well as locations. This is both a management and an evaluation issue: SEWA’s leaders need (and want) to know how programs are working, and also the impact of their interventions, to understand what works in improving the lives of its members.

Organizational Transformation and Values

A particular issue in scaling is how to sustain the values and practices that have been essential to the organization’s success, when there is growth in organizational complexity. BRAC is a truly impressive example in terms of organizational scale, but is highly unusual in the way it way it has successfully introduced systems through the organization, while maintaining a core commitment to social ends. An alignment between core values and the new larger organization is full of strategic tensions.

The Visionary Founder Syndrome

Most of the time successful grassroots or social entrepreneurs have been created by amazing leaders that had a vision and the capacity to get others to follow on that vision. Over time as the organizations grow, the leadership requirements shift. A visionary founder is rarely also a great CEO. And many times it is even difficult for the founder to hire and delegate to a capable COO. This will lead to the failure to scale because of not enough delegation, inability to set focus and say no to activities that are not aligned with the strategy as well as difficult succession planning.


Organization Failure

Note that there are a potentially important complementarity between organizational failures and market or government failures. If it is hard for a small-scale organization to successfully undertake the organizational transformations to go to scale—so that only few may be successful. This means it can be entirely rational for potential private or public financiers to be cautious about providing support, especially long-term support. This applies even to a greater degree if funding organizations lack information over the organizational potential and of how to incentivize organizational change in the transition.

⚬    ⚬    ⚬

We have described these organizational failures for scaling up through growth. The same failures apply to scaling up through replication and network with a few additions.

When an organization is scaling up through replication, it can do this by franchising or simply by other organizations copying. But franchising requires systems, data and monitoring capacity. Replication through copying at the BoP usually happens in an informal way, but this brings its own problems of verification of practices and behaviors. Thus even if the scaling path is via entry of small enterprises, effective scaling also requires tackling some of the issues around shifting to formal systems, data management, organizational culture and leadership, if the ecosystem is to be based on financially viable organizations.

As an example of the franchise issue, as SEWA expands to the North East States in India, the new members might not receive the same training on values and practices. The high touch leadership training that was received by members in Gujarat might be difficult to replicate in more distant locations, with different cultural norms. As long as this replication happens without investing in formal systems to standardize, train and monitor, SEWA’s credibility could be at stake.

Similar issues are present when scaling up happens through networks. Many successful examples in one developing country are taken to neighboring countries by members of a network and face different realities that require adaptation. Data, systems, alignment with values and adaptation are key for the network to succeed in a common goal. Partnering across grassroots is specially challenging, as each organization is very focused on urgent day-to-day needs which makes forming alliances difficult.