At first glance, it may appear that the 2021 Nobel prize in economics (more formally the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2021) was given for two rather separate contributions. The award announcements says that one-half of the award is given to David Card “for his empirical contributions to labour economics,” while the other half is given jointly to Joshua D. Angrist and Guido W. Imbens “for their methodological contributions to the analysis of causal relationships.” (It must have been an interesting if slightly farcical conversation that led to splitting the prize 1/2, 1/4, 1/4, rather than 1/3, 1/3, 1/3, but of such stuff are committee decisions made.)
But the underlying connection between the three co-winners is obvious enough if you read the background materials released by the Nobel committee. Each year, the committee produces a highly readable “Popular science” explanation of the prize, this year titled “Natural experiments help answer important questions,” along with more specialized and longer “Scientific background” paper, this year titled “Answering Causal Questions Using Observational Data.”
As a starting point for understanding the contribution here, it’s useful to begin with an old familiar warning from every introductory statistics class that “correlation does not imply causation.” If two variables A and B are correlated with each other, it’s possible that A causes B, or B causes A, or both A and B are being affected by some unnamed set of factors C, or even that if there is a human tendency to look for patterns in data, but when you just look at enough enough combinations of variables, some of them will be correlated with each other by random chance, with no actual connection between them at all. Back when I was first learning statistics in the late 1970s, it was common for these warnings all to be mentioned in the intro econometrics class, but as a practical matter, we then went right back to calculating correlations.
Scientists often seek to demonstrated causation with controlled experiments. In one famous example, when Louis Pasteur developed a vaccine for sheep anthrax back in 1881, he took a herd of 100 sheep, vaccinated half of them, and then exposed the herd to anthrax. The unvaccinated sheep died and the vaccinated lived. But people aren’t sheep. So if an economist wants to address a question like whether a higher minimum wage causes unemployment (not is correlated with it!) or whether a surge of immigration leads to higher unemployment in the native-born population (not is correlated with it!), how is it possible to use real-world observational data in a way that lets a social science researcher draw conclusions about causality? To put it another way, do real-world events sometimes create a kind of “natural experiment” for researchers to study?
The Nobel committee describes some of the most prominent studies built on this “natural experiment” foundation. For example, consider the question of whether education raises one’s level of income. Yes, it’s of course true that there is a correlation between higher levels of education and income, but perhaps there are underlying persistence characteristics–say, persistence or rule-following or an ability to work with others–that are correlated both with higher education and with higher income. How can we look at real-world data and obtain a causal estimate? The Nobel committee explains.
Joshua Angrist and his colleague Alan Krueger (now deceased) showed how this could be done in a landmark article. In the US, children can leave school when they turn 16 or 17, depending on the state where they go to school. Because all children who are born in a particular calendar year start school on the same date, children who are born early in the year can leave school sooner than children born later in the year. When Angrist and Krueger compared people born in the first and fourth quarters of the year, they saw that the first group had, on average, spent less time in education. People born in the first quarter also had lower incomes than those born in the fourth quarter. As adults they thus had both less education and lower incomes than those born late in the year. Because chance decides exactly when a person is born, Angrist and Krueger were able to use this natural experiment to establish a causal relationship showing that more education leads to higher earnings: the effect of an additional year of education on income was nine per cent.
This study essentially uses the idea that people born in the fourth quarter of a given year are not fundamentally different than those born in the first quarter of the next year, and so the date when you are allowed to leave school in effect created a random separation of these two equivalent groups–logically similar to Pasteur’s vaccine. Researchers began to focus on other situations with the characteristics of a “natural experiment.”
For example, many government programs have an eligibility cut-off, and one can reasonably believe that those who are just barely on one side of the cutoff are pretty much the same as those who are just barely on the other side of the cut-off, with the cutoff itself randomly dividing these two groups. Thus, comparing those barely included from those barely excluded can allow for a causal estimate of the effect of the program.
Other programs are based on an element of randomness: for example, in many cities if the spots in desirable charter high schools are oversubscribed, there is a lottery for who gets admitted. In Oregon a few years back, the state decided to expand Medicaid coverage but because of limited funds, the new benefit was given out by lottery. Sometimes when a new program is implemented, it is rolled out in some areas before others–and those early-adopting areas may occur more-or-less at random. When economists and other social scientists hear about randomization in a program, they start thinking about whether it might serve as a natural experiment to provide evidence about causality.
In other situations, there can be an event or policy choice that works like a natural experiment. The Nobel committee describes a prominent study of immigration done by David Card:
A unique event in the history of the US gave rise to a natural experiment, which David Card used to investigate how immigration affects the labour market. In April 1980, Fidel Castro unexpectedly allowed all Cubans who wished to leave the country to do so. Between May and September, 125,000 Cubans emigrated to the US. Many of them settled in Miami, which entailed an increase in the Miami labour force of around seven per cent. To examine how this huge influx of workers affected the labour market in Miami, David Card compared the wage and employment trends in Miami with the evolution of
wages and employment in four comparison cities. Despite the enormous increase in labour supply, Card found no negative effects for Miami residents with low levels of education. Wages did not fall and unemployment did not increase relative to the other
cities. This study generated large amounts of new empirical work, and we now have a better understanding of the effects of immigration. For example, follow-up studies have shown that increased immigration has a positive effect on income for many groups who were born in the country, while people who immigrated at an earlier time are negatively affected. One explanation for this is that the natives switch to jobs that require good native language skills, and where they do not have to compete with immigrants for jobs.
In effect, the Cuban boatlift of 1980 was a natural experiment, addressing the question: “What would happen if an enormous number of unskilled immigrants arrived suddenly and without much warning in a major US city?” But once you start thinking along these lines, you can consider a variety of other events as natural experiments, too.
The natural experiment examples I have mentioned here are relatively straightforward. But as with many Nobel prizes, the award is given largely because the early work spawned a vast array of follow-up work and altered how economists think about these issues. Even in these relatively clear-cut cases, detailed and multi-faceted arguments have followed about exactly what can be inferred, or not, from looking at the data in different ways. The Nobel committee also describes the “natural experiment’ approach as “quasi-experimental.”
Together the work by this year’s Laureates laid the ground for the design-based approach, which has drastically changed how empirical research is conducted over the past 30 years. … Quasi-experimental variation can come from the many experiments provided by nature, administrative borders, institutional rules, and policy changes. The design-based approach features a clear statement of the assumptions used to identify the causal effect and
validation of these identifying assumptions.
To put it another way, when you hear economists say that a variable is “associated” or “correlated” with another variable, they mean something quite different from when they claim to have found a causal effect. The old statement that “correlation does not equal causation” is now taken with gimlet-eyed earnestness.