The World Bank announced that it is discontinuing its its biennial Doing Business Report. The reason is that World Bank insiders, under pressure from national government, leaned on the researchers charged with compiling the report to change their findings–which they did. Details are available in a report: “Investigation of Data Irregularities in Doing Business 2018 and Doing Business 2020 – Investigation Findings and Report to the Board of Executive Directors(September 15, 2021).

Before sketching what happened within the World Bank–a topic admittedly of interest mainly to a small number of insiders–it’s perhaps useful to describe how the Doing Business reports came into existence and what they were trying to do. At least for now you can check out the website of the 2020 Doing Business report for yourself.

Simeon Djankov described the origins of the Doing Business reports in the Winter 2016 issue of the Journal of Economic Perspectives (where I work as Managing Editor). Djankov wrote:

The Doing Business report was first published in 2003 with five sets of indicators for 133 economies. However, the team that created Doing Business had been formed three years earlier, during the writing of the World Development Report 2002: Building Institutions for Markets (World Bank 2001). The focus on the importance of institutions in development was chosen by Joseph Stiglitz, who at the time was the World Bank’s Chief Economist. As a member of the team, I was tasked with authoring the chapters on institutions and firms. At the time, the work by Rafael La Porta, Florencio Lopez de Silanes, Andrei Shleifer, and Robert Vishny on legal origins and various aspects of institutional evolution was generating a great deal of interest. I turned to Shleifer with a request to collaborate on several background papers for the World Development Report. He agreed, on the condition that we used this work as an opportunity to gather and analyze new cross-country datasets on institutions. This is how Doing Business started.

By 2020, the Doing Business report included data for 190 countries on 10 categories: “the processes
for business incorporation, getting a building permit, obtaining an electricity connection, transferring property, getting access to credit, protecting minority investors, paying taxes, engaging in international trade, enforcing contracts, and resolving insolvency.” There was also data collected on regulation of employment and contracting with government, although this data was not an official part of the Doing Business Index–and the idea of an index is perhaps where the trouble starts.

After all, researchers all over the world, both in international institutions like the World Bank or the IMF, as well as in academia and think tanks, do research on these kinds of topics all the time. But the idea of the Doing Business Index was to come up with concrete ways of measuring these 10 categories. Obvious questions arise. For example, say the task is to measure the costs of obtaining, say, an electricity connection in a given country. One cost will just be the cost of the electricity itself, but what about the time and the number of permits required? Doesn’t the reliability of the electricity supply need to be taken into account? Isn’t there likely to be a big difference in a given country between urban and rural areas? Maybe the process and costs will be different by industry, too.

The Doing Business project, to its credit, wanted to be scrupulous about exactly what was being measured. In the case of getting electricity, the 2020 report spells out:

Doing Business records all procedures required for a business to obtain a permanent electricity connection and supply for a standardized warehouse … These procedures include applications and contracts with electricity utilities, all necessary inspections and clearances from the distribution utility as well as from other agencies, and the external and final connection works between the building and the electricity grid. The process of getting an electricity connection is divided into distinct procedures and the study records data for the time and cost to complete each procedure. In addition, Doing Business measures the reliability of supply and transparency of tariffs index … and the price of electricity …

The data for measure all of these dimensions of “getting electricity” was based on survey evidence from local industry experts. As the report notes: “The data on getting electricity is collected through a questionnaire completed by experts in the electricity sector, including electrical engineers, electricians, electrical installation firms, as well as representatives from utility companies and energy regulators, and other public officials involved in this sector. To make the data comparable across economies, several assumptions about the business, the warehouse and the electricity connection are used.”

I hope this brief description of one category of the Doing Business report gives a sense of the ambition and scope of the project. Surveys were going out to multiple industry experts in 190 countries, across these 10 categories. Then the survey answers were being compiled and combined into a single index number for “getting electricity,” and the single index numbers for all 10 categories were being combined into an overall Doing Business index number for every country.

For those putting together the reports, and for those like me reading the reports, it was obvious that the index numbers and ranking were in some sense very broadly informative. However, what was really interesting about the report was that you could drill down into the underlying questions and get a more detailed and granular sense of what was causing a given score to be high or low. You could point out what seemed to be strengths and weaknesses in the business climate of countries. And if you disagreed with a given score, the methodology was clear enough that you could often pinpoint your precise area of disagreement. In other words, Doing Business was trying to take the ideas of “business climate” or “institutions that interact with private business” and make them specific, rather than ethereal and rhetorical.

But for national governments, the overall Doing Business scores felt like a judgement. Governments around the world, including India, Russia, Peru, and others, announced as a policy goal that they would perform better in the Doing Business rankings. Of course, all countries prefer to be above average in their business climate. National governments quarreled with the Doing Business methods and findings.

As one of many issues, the Doing Business rankings often looked at the actual formal rules and regulations. But many companies in practice found ways to circumvent those rules, perhaps with political pull or bribes. A system that functions based on such favoritism may not be a good thing–but if you focus on the formal rules, you may not be capturing how the system actually operates. For an overview of the issues and controversies surrounding the Doing Business indicators, a useful starting point is the two-paper symposium in the Spring 2015 issue on Doing Business in the Journal of Economic Perspectives:

For the 2018 Doing Business report, the focus of the outside report is on China. There was deep concern at the World Bank that China might reduce its financial commitments to the Bank, with phrases like could be in “very deep trouble” being tossed around. Overall, China had been ranked #78 in the previous Doing Business report, and China’s government was complaining to the top leaders at the bank that this ranking was too low, and didn’t reflect progress China had made. The underlying data did show that China had made substantial progress–but it also showed that a number of other countries ranked near China had also made even more substantial progress. Thus, in terms of rankings, China was scheduled to drop.

Shenanigans followed. The 2018 report was about to be published, with China having a ranking of #85. Top leadership of the Bank then pulled the report just before publication, and starting asking for some way of recalculating China’s numbers. The report documents in painful detail how the pressure was exerted, how options were proposed and then discarded (like, say, using data for Macao to measure business conditions in China), and eventually, how China got its #78 ranking back.

For the next cycle of the Doing Business report scheduled for 2020, a similar cycle occurred, this time with Saudi Arabia as the protagonist. For example, early data ranked Jordan higher than Saudi Arabia as a reformer in the Middle East region. Again, political pressure was brought to bear on the researchers doing the work. Again, the numbers got altered. A similar pattern also happened for Azerbaijan.

One finishes the outside report wondering how many other national government were negotiating with the World Bank over their scores. Once such doubts are not just rumors, but backed by outside investigators, there may not have been much choice but to end the report, at least for a time. One might also argue that given the imperfections of the report–the difficulties and idiosyncracies of how it defined terms, gathered information, and combined that information into rankings–perhaps the loss is not an enormous one.

On the side, the demise of Doing Business seems unfortunate to me, but then, I’m a person for whom more data is pretty much always better. Yes, the rankings were imperfect, but having no one attempting to systematically measure and compare across these categories, so that there is no information at all, doesn’t seem to me an overall gain. More troubling, the saga of the Doing Business report reveals how national governments will push back against research findings and statistics they don’t like. This story shows how researchers were pressured to knuckle under, and did. When that happens, it casts a shadow over research not just at the World Bank, but everywhere.