Month: July 2021

What paths can high school students take in accumulating hard and soft skills so that they can make the transition to a career and job? The main answer in US society is “go to college.” But for a large share of high school graduates, being told that they now need to attend several more years of classes is not what they want to hear.

Historically, a common alternative career path was that companies hired young workers who had little to offer other than energy and flexibility, and then trained and promoted those workers. Unions often played a role in advocating and supporting this training, too. But in the last few decades, it seems that a lot of companies have exited the job training business. Their general sense is that young adults aren’t likely to stay with the company, so in effect, you are training them for their next employer. Instead, better just to require that new hires already have experience.

The earn-and-learn career path tries to steer between these extremes. Yes, it involves additional learning, because that’s what 21st century jobs are like, but it seeks to have that learning take place more in the workplace than in the classroom. Also, instead of paying to learn, you get paid while learning. On the other side, firms that participate in this kind of training don’t need to take on the entire responsibility and cost of doing so.

Annelies Goger sketches this framework in “Desegregating work and learning through ‘earn-and-learn’ models” (Brookings Institution, December 9, 2020). She points out the gigantic difference in public support for higher education vs. public support for an earn-and-learn approach.

The earn-and-learn programs under the public workforce system—authorized under the Workforce Innovation and Opportunity Act (WIOA)—are underused and hard to scale. Publicly funded job training options are tiny overall compared to investments in traditional public higher education or classroom-based job training. Funding for public higher education was $385 billion in 2017-18, compared to about $14 billion for employment services and training across 43 programs. The net result is that higher education is the main provider of publicly funded training for most Americans, and most of the $14 billion for employment services and training goes to services (most of which isn’t training) for special populations such as veterans and people with disabilities.

This difference in public support is even more stark when you recognize that those with a college education are likely to end up with  higher average incomes during their lives, so that we are doing more to subsidize the training of the relatively high earners of the future than we are to subsidize the training of the middle- and lower-level earners.

What do the earn-and-learn programs look like in practice? Here’s a graphic:

I won’t try to go through these choices one at a time: for present purposes, the salient fact is that they are all small in size. As long-time readers know, I’m a fan of a dramatic expansion of apprenticeships (for example, here, here, here, and here). But as Goger writes:

For example, the U.S. had roughly 238,000 new registered apprentices in 2018. However, if the U.S. had the same share of new apprentices per capita as Germany, we would have 2 million new apprentices per year; if we had the same share as the United Kingdom or Switzerland, that number would be 3 million.

One infuriating aspect of the US health care industry is the high administrative costs. Here’s a description from David Scheinker, Barak D. Richman, Arnold Milstein, and Kevin A. Schulman in “Reducing administrative costs in US health care: Assessing single payer and its alternatives” (Health Services Research, published online March 31, 2021, not yet assigned to an issue). They write (footnotes omitted):

The transaction cost of paying for services with a commercial credit card is approximately 2% of the total cost, whereas Tseng (2018) calculated that it is 14.5% when providers bill insurance companies for physician services. A similar percentage is consumed for hospital billing, and approximately an additional 15% is retained by commercial insurers for claims processing and other costs under the Affordable Care Act.

Health care administrative costs in the United States are higher than in other rich nations. Estimates suggest over $265 billion of annual spending is wasted due to administrative complexity, yet the substantial literature on wasted health care spending offers little discussion of what drives these costs or how to reduce them. … The most common proposal to reduce the transaction costs of paying for health care has been to advocate for a single-payer “Medicare-for- All” model that nationalizes Medicare fee-for-service coverage, with all of that program’s complexity.

There are at least three possibilities for making a meaningful reduction in administrative spending in the US health care sector: a single payer system, a set of rules that would require standardization and simplification of health care contracts, and a national health care automated clearinghouse for payments to health care providers. Let’s mull them over in turn.

Perhaps the main difficulty with seeking to reduce US health care administrative costs by the use of a single-payer system is that, for better or worse, there seems to be approximately zero political momentum for the US to adopt such a system.

However, an additional concern is that single-payer systems come in many different models. Yes, if there is a single payer for all health care expenses who just sends out checks to all health care providers, administrative costs will lower. But given the very high that the US health-care system is about one-fifth of the entire US economy, and rising, even a single-payer system will need to have various controls over what prices are being charged for what services and what services are covered. Most single-payer systems around the world are combined with overall limits on hospital spending, physician fees, and limits on technology. In other words, part of the reason for lower administrative fees in single-payer systems around the world is that the choices available in what services to offer and what to charge have already been tightly limited.

In an essay on “Reducing Administrative Costs in U.S. Health Care,” David M. Cutler acknowledges that a single-payer system would certainly eliminate some administrative costs, like the costs sales and marketing, as well as the profits, of private insurance companies (March 2020, Hamilton Project, . However, if a US single-payer system did not also include limits on fees and available services, administrative costs will remain substantial. Cutler writes (citations omitted for readability)

Perhaps the major issue in single-payer health care is the trade-off between administrative costs and other ways of controlling use. Many single-payer proposals envision lower administrative costs such as those in the Canadian system. Such a system has other important features, however, that are necessary to offset administrative spending reductions. For example, Canada has very tight restrictions on technology acquisition; there are only one-quarter the number of MRIs per capita in Canada as there are in the United States. Similarly, there is a budget for hospitals and a fee schedule for physicians, where fees are set so that a total spending target is met. The ability to do this in the United States is in some doubt … If the United States does not implement the type of budget and technology regulation that other countries have, the impact of single-payer health care on administrative costs is less certain.

Given the political and practical difficulties of tackling health care administrative costs via a single payer system, Scheinker, Richman, Milstein, and Schulman go through a modelling exercise where they try to separate out the main billing and insurance-related (BIR) costs across “three components of BIR costs (fixed costs, per-visit clinical documentation variable costs, and per-visit nonclinical documentation variable costs) associated with five types of visits (primary care, emergency department visits, inpatient stays, ambulatory surgery, and inpatient surgery).” They then consider a list of ways of standardizing and simplifying current billing procedures, together with how these might happen in a single-payer or in a multi-payer health care system. They conclude:  

Our model estimates that national BIR [billing and insurance-related] costs are reduced between 33% and 53% in Medicare-for-All style single-payer models and between 27% and 63% in various multi-payer models. Under a wide range of assumptions and sensitivity analyses, standardizing contracts generates larger savings with less variance than savings from single-payer strategies. … Although moving toward a single-payer system will reduce BIR costs, certain reforms to payer-provider contracts could generate at least as many administrative cost savings without radically reforming the entire health system. BIR costs can be meaningfully reduced without abandoning a multi-payer system.

I lack the expertise in health care billing systems to offer any useful insight into the realism of their claims, but for what it’s worth, all of the authors are affiliated with the Clinical Excellence Research Center at Stanford’s medical school.

In the essay mentioned above, David Cutler proposes that these kinds of changes might be implemented via a national health care automated clearinghouse for payments to health care providers: that is, all health care providers would submit their bills to the clearinghouse, and all insurers would pay the bills as submitted through the clearinghouse. Cutler makes the argument this way:

I propose that Congress establish a clearinghouse for bill submission … More than 6 billion medical claims are filed annually in the United States. While almost all of these claims are filed electronically, the system is not as efficient as it could be. The issue is sometimes posed as the need for a single claim form, but that is not correct. The HIPAA legislation of 1996 required standardized claims forms and that has now been achieved. Nevertheless, the system still has some limitations. To begin, the information required by different payers can be different, even with the same form. For example, one insurer might require special revenue codes for particular specialties that are different from other insurers. In other cases, the physician’s specialty may differ across insurers (medicine vs. gastroenterology, for example), which could necessitate a different set of codes. Or the codes given for claim denial may differ across insurers. And still other insurers are not required to use these standardized forms (e.g., workers’ compensation and auto insurers). In addition, many insurers require attachments to claims, and these attachments are generally not standardized. An attachment might involve a certificate of medical necessity, a discharge summary, or details of a lab report … [O]nly 20 percent of claims attachments are standardized. One of the primary reasons why attachments are not standardized is that a federal standard has not yet been named by the Department of Health and Human Services as required under HIPAA and the ACA …

Cutler goes on to point out opportunities for streamlining issues like getting prior authorization for procedures and for documenting quality control. He suggests that any health care providers involved with government payments could be required to send their bills through the national system, and those who do not wish to submit payments through the system would be charged a modest fee for doing so.

Making these kinds of changes to health care billing and administration, whether it is done through a national clearinghouse or in some other way, isn’t a deep conceptual problem or one involving complex tradeoffs. In a well-functioning political system, it seems to me like the kind of detailed, nuts-and-bolts problem that could be addressed by detailed, nut-and-bolts politicians and administrators beavering away at the task on a bipartisan basis until it’s done.

I have been the Managing Editor of the Journal of Economic Perspectives since the first issue in Summer 1987. The JEP is published by the American Economic Association, which decided about a decade ago–to my delight–that the journal would be freely available on-line, from the current issue all the way back to the first issue. You can download individual articles or the entire issue, and it is available in various e-reader formats, too. Here, I’ll start with the Table of Contents for the just-released Summer 2021 issue, which in the Taylor household is known as issue #137. Below that are abstracts and direct links for all of the papers. I will probably blog more specifically about some of the papers in the next week or two, as well.

_________________________________________________________

Symposium on COVID-19

“Effects of the COVID-19 Recession on the US Labor Market: Occupation, Family, and Gender,” by Stefania Albanesi and Jiyeon Kim

The economic crisis associated with the emergence of the novel corona virus is unlike standard recessions. Demand for workers in high contact and inflexible service occupations has declined while parental supply of labor has been reduced by lack of access to reliable child care and in-person schooling options. This has led to a substantial and persistent drop in employment and labor force participation for women, who are typically less affected by recessions than men. We examine real-time data on employment, unemployment, labor force participation and gross job flows to document the impact of the pandemic by occupation, gender and family status. We also discuss the potential long-term implications of this crisis, including the role of automation in depressing the recovery of employment for the worst hit service occupations.

Full-Text Access | Supplementary Materials

“The Great Unequalizer: Initial Health Effects of COVID-19 in the United States,” by Marcella Alsan, Amitabh Chandra and Kosali Simon

We measure inequities from the COVID-19 pandemic on mortality and hospitalizations in the United States during the early months of the outbreak. We discuss challenges in measuring health outcomes and health inequality, some of which are specific to COVID-19 and others that complicate attribution during most large health shocks. As in past epidemics, preexisting biological and social vulnerabilities profoundly influenced the distribution of disease. In addition to the elderly, Hispanic, Black and Native American communities were disproportionately affected by the virus, particularly when assessed using the years of potential life lost metric. We provide a conceptual framework and initial empirical analysis that seek to shed light on contributors to pandemic-related health inequality, and we suggest areas for future research.

Full-Text Access | Supplementary Materials

“Tracking the Pandemic in Real Time: Administrative Micro Data in Business Cycles Enters the Spotlight,” by Joseph Vavra

In this paper I discuss the increasingly prominent role of administrative micro data in macroeconomics research. This type of data proved important for interpreting the causes and consequences of the Great Recession, and it has played a crucial role in shaping economists’ understanding of the COVID-19 pandemic in near real-time. I discuss a number of specific insights from this research while also illustrating some of the broader opportunities and challenges of working with administrative data.

Full-Text Access | Supplementary Materials

Symposium on the Washington Consensus Revisited

“Some Thoughts on the Washington Consensus and Subsequent Global Development Experience,” by Michael Spence

This paper discusses the Washington Consensus, its origins, and its insights in terms of subsequent development experience in a broad range of countries. I continue to find that when properly interpreted as a guide to the formulation of country-specific development strategies, the Washington Consensus has withstood the test of time quite well. In my view, subsequent experience, especially in Asia, reveals a number of places where a shift in emphasis would be warranted. Finally, I try to identify some misuses of the Washington Consensus and suggest that it was vulnerable to misuse due to the absence of an accompanying and explicit development model.

Full-Text Access | Supplementary Materials

“The Baker Hypothesis: Stabilization, Structural Reforms, and Economic Growth,” by Anusha Chari, Peter Blair Henry and Hector Reyes

In 1985, James A. Baker III’s “Program for Sustained Growth” proposed a set of economic policy reforms including, inflation stabilization, trade liberalization, greater openness to foreign investment, and privatization, that he believed would lead to faster growth in countries then known as the Third World, but now categorized as emerging and developing economies (EMDEs). A country-specific, time-series assessment of the reform process reveals three clear facts. First, in the ten-year period after stabilizing high inflation, the average growth rate of real GDP in EMDEs is 2.6 percentage points higher than in the prior ten-year period. Second, the corresponding growth increase for trade liberalization episodes is 2.66 percentage points. Third, in the decade after opening their capital markets to foreign equity investment, the spread between EMDEs average cost of equity capital and that of the US declines by 240 basis points. The impact of privatization is less straightforward to assess, but taken together, the three central facts of reform provide empirical support for the Baker Hypothesis and suggest a simple neoclassical interpretation of the unprecedented increase in growth that has taken place in EMDEs since the early 1990s.

Full-Text Access | Supplementary Materials

“Washington Consensus in Latin America: From Raw Model to Straw Man,” by Ilan Goldfajn, Lorenza Martínez and Rodrigo O. Valdés

We take stock of three decades of a love-hate relationship between Latin American policies and the Washington Consensus, reviewing its implementation, national debate, and outcomes. Using regional data and case studies of Brazil, Chile, and Mexico, we discuss the various degrees of the Washington Consensus implementation and evaluate performance. We find mixed results: macroeconomic stability is much improved, but economic growth has been heterogeneous and generally disappointing, despite improvement relative to the 1980s. We discuss the risk that the region could revert parts of the Washington Consensus reforms, which are necessary building blocks for a new agenda more focused on social integration, a fairer and just society, and environmentally sustainable growth based on better education.

Full-Text Access | Supplementary Materials

“Washington Consensus Reforms and Lessons for Economic Performance in Sub-Saharan Africa,” by Belinda Archibong, Brahima Coulibaly and Ngozi Okonjo-Iweala

Over three decades after market-oriented structural reforms termed “Washington Consensus” policies were first implemented, we revisit the evidence on policy adoption and the effects of these policies on socio-economic performance in sub-Saharan African countries. We focus on three key ubiquitous reform policies around privatization, fiscal discipline, and trade openness and document significant improvements in economic performance for reformers over the past two decades. Following initial declines in per capita economic growth over the 1980s and 1990s, reform adopters experienced notable increases in per capita real GDP growth in the post–2000 period. We complement aggregate analysis with four country case studies that highlight important lessons for effective reform. Notably, the ability to implement pro-poor policies alongside market-oriented reforms played a central role in successful policy performance.

Full-Text Access | Supplementary Materials

Symposium on Statistical Significance

“Statistical Significance, p-Values, and the Reporting of Uncertainty,” by Guido W. Imbens

The use of statistical significance and p-values has become a matter of substantial controversy in various fields using statistical methods. This has gone as far as some journals banning the use of indicators for statistical significance, or even any reports of p-values, and, in one case, any mention of confidence intervals. I discuss three of the issues that have led to these often-heated debates. First, I argue that in many cases, p-values and indicators of statistical significance do not answer the questions of primary interest. Such questions typically involve making (recommendations on) decisions under uncertainty. In that case, point estimates and measures of uncertainty in the form of confidence intervals or even better, Bayesian intervals, are often more informative summary statistics. In fact, in that case, the presence or absence of statistical significance is essentially irrelevant, and including them in the discussion may confuse the matter at hand. Second, I argue that there are also cases where testing null hypotheses is a natural goal and where p-values are reasonable and appropriate summary statistics. I conclude that banning them in general is counterproductive. Third, I discuss that the overemphasis in empirical work on statistical significance has led to abuse of p-values in the form of p-hacking and publication bias. The use of pre-analysis plans and replication studies, in combination with lowering the emphasis on statistical significance may help address these problems.

Full-Text Access | Supplementary Materials

“Of Forking Paths and Tied Hands: Selective Publication of Findings, and What Economists Should Do about It,” by Maximilian Kasy

A key challenge for interpreting published empirical research is the fact that published findings might be selected by researchers or by journals. Selection might be based on criteria such as significance, consistency with theory, or the surprisingness of findings or their plausibility. Selection leads to biased estimates, reduced coverage of confidence intervals, and distorted posterior beliefs. I review methods for detecting and quantifying selection based on the distribution of p-values, systematic replication studies, and meta-studies. I then discuss the conflicting recommendations regarding selection resulting from alternative objectives, in particular, the validity of inference versus the relevance of findings for decision-makers. Based on this discussion, I consider various reform proposals, such as deemphasizing significance, pre-analysis plans, journals for null results and replication studies, and a functionally differentiated publication system. In conclusion, I argue that we need alternative foundations of statistics that go beyond the single-agent model of decision theory.

Full-Text Access | Supplementary Materials

“Evidence on Research Transparency in Economics,” by Edward Miguel

A decade ago, the term “research transparency” was not on economists’ radar screen, but in a few short years a scholarly movement has emerged to bring new open science practices, tools and norms into the mainstream of our discipline. The goal of this article is to lay out the evidence on the adoption of these approaches—in three specific areas: open data, pre-registration and pre-analysis plans, and journal policies—and, more tentatively, begin to assess their impacts on the quality and credibility of economics research. The evidence to date indicates that economics (and related quantitative social science fields) are in a period of rapid transition toward new transparency-enhancing norms. While solid data on the benefits of these practices in economics is still limited, in part due to their relatively recent adoption, there is growing reason to believe that critics’ worst fears regarding onerous adoption costs have not been realized. Finally, the article presents a set of frontier questions and potential innovations.

Full-Text Access | Supplementary Materials

Articles and Features

“Why Is Growth in Developing Countries So Hard to Measure?” by Noam Angrist, Pinelopi Koujianou Goldberg and Dean Jolliffe

Occasional widely publicized controversies have led to the perception that growth statistics from developing countries are not to be trusted. Based on the comparison of several data sources and analysis of novel IMF audit data, we find no support for the view that growth is on average measured less accurately or manipulated more in developing than in developed countries. While developing countries face many challenges in measuring growth, so do higher-income countries, especially those with complex and sometimes rapidly changing economic structures. However, we find consistently higher dispersion of growth estimates from developing countries, lending support to the view that classical measurement error is more problematic in poorer countries and that a few outliers may have had a disproportionate effect on (mis)measurement perceptions. We identify several measurement challenges that are specific to poorer countries, namely limited statistical capacity, the use of outdated data and methods, the large share of the agricultural sector, the informal economy, and limited price data. We show that growth measurement based on the System of National Accounts (SNA) can be improved if supplemented with information from other data sources (for example, satellite-based data on vegetation yields) that address some of the limitations of SNA.

Full-Text Access | Supplementary Materials

“Retrospectives: James Buchanan: Clubs and Alternative Welfare Economics,” by Alain Marciano

James Buchanan wrote “An Economic Theory of Clubs” and invented clubs to support a form of welfare economics in which there is no social welfare function (SWF) and individual utility functions cannot be “read” by external observers. Clubs were a means to allow the implementation of individualized prices for public goods and services and to allow each individual to pay exactly the amount he wants to pay. He developed this project to answer and counter Paul Samuelson’s analysis of public goods, in which social welfare functions play a crucial role. Buchanan and Samuelson disagreed over the allocation of the costs of the public good to each individual. To Buchanan, it was by relying on individual’s preferences. To Samuelson, by using a SWF. Buchanan’s clubs are thus foreign and incompatible with the traditional Samuelson-style public economics in which they are used.

Full-Text Access | Supplementary Materials

“Recommendations for Further Reading,” by Timothy Taylor

Full-Text Access | Supplementary Materials

If you don’t know about the Odebrecht case, which the the US Department of Justice called “the largest foreign bribery case in history,” Nicolás Campos, Eduardo Engel, Ronald D. Fischer, and Alexander Galetovic tell the story and offer some insights in “The Ways of Corruption in Infrastructure: Lessons from the Odebrecht Case” in the Spring 2021 issue of the Journal of Economic Perspectives (where I work as Managing Editor). They describe the company’s rise and fall this way:

In 2010, the Swiss business school IMD chose Odebrecht, a Brazilian conglomerate, as the world’s best family business. Odebrecht was chosen for the excellent performance of its companies, its continuous growth, and its social and environmental responsibility. Sales had quintupled between 2005 and 2009,
and Odebrecht had become Latin America’s largest engineering and construction company and ranked 18th worldwide among international contractors (Engineering News-Record Magazine 2009).

By 2015, however, Odebrecht chief executive Marcelo Odebrecht had been arrested on corruption charges. Nine months later he was sentenced to more than 19 years in prison. The Odebrecht case, as it came to be known, involved bribe payments in ten countries in Latin America and two countries in Africa. Deltan Dallagnol, lead prosecutor in Brazil, commented (as reported by Pressly 2018): “The Odebrecht case leaves you speechless. This case implicated almost one-third of Brazil’s senators and almost half of all Brazil’s governors. A single company paid bribes to 415 politicians and 26 political parties in Brazil. It makes the Watergate scandal look like a bunch of kids playing in a sandbox.”

These bribery cases mainly involved bidding on large government infrastructure projects. involved US Department of Justice became involved because many of the bribes were paid through US banks. Here are a few of the details that jumped out at me.

1. The sheer brazenness of the bribery was, in its own way, impressive. the Odebrecht company had an actual division devoted to bribery, with standardized reporting and procedures. The JEP authors write:

By 2006, bribery at Odebrecht had become so institutionalized that the company created the Division of Structured Operations (DSO), a stand-alone department dedicated to corruption. According to the plea agreement between the Odebrecht chief executive officer Marcelo Odebrecht and the US Department of Justice, the DSO specialized in buying influence through legal and illegal contributions to political campaigns and also in paying bribes to public officials and politicians. Within the DSO, three full-time executives and four experienced assistants were responsible for paying bribes to foreign accounts. Bribe payments followed a clear organizational flow. A contract manager would deal with potential bribe recipients—public officials and politicians—and reported to the country manager. The country manager could approve small bribes paid with local funds. Larger bribes were vetted by an executive reporting directly to the Odebrecht chief executive officer who often made the final decision.

2. Odebrecht wasn’t a case of huge bribes being paid and huge company profits as a result. In this sense, the pre-existing rules on public bidding did limit the most egregious forms of corruption. Indeed, the authors suggest that the size of the bribes was pretty much equal to the size of additional profits. Thus, this seems like a case of opportunistic bribes paid at key points, but more with the goal of dramatically expanding the size of Odebrecht (as noted above, the company sales quintupled from 2005-2009), rather than increasing its profit margins.

3. The countries where the bribery took place typically had rules to try to ensure fair bidding for public projects. But as it turned out, these rules often had some key vulnerabilities. One was that the rules often had conditions that firms could only bid if the company had certain technical and business qualifications. This sounds reasonable enough–but it meant that when the qualifications had a degree of subjectivity, was sometimes possible to bribe the public officials who were looking at these qualifications, so that they would disqualify some other potential bidders. But perhaps the main vulnerability was that when a company won the project, it could then later re-open negotiations for higher payments, or additional payments for expanding the scale of the work, and so on. Thus, a firm could bid low on a project, but then bribe the re-negotiators later in the process.

The public policy lessons here–applicable around the world–are to make the rules about bidding on public projects as objective as possible, and to have a high degree of openness, including the possibility of putting additional work out for open bids, when contract renegotiations come up.

There isn’t likely to be one magic bullet that addresses all the issues related to carbon emissions in the atmosphere. Instead, you take your partial solutions where you can find them, and call it progress. Could geothermal energy be one of those partial solutions? Eli Dourado offers a useful overview of the state of play in “Harnessing the Heat Beneath our Feet” (PERC Reports, Summer 2021).

Geothermal energy finds ways to harness heat from the earth’s core and turn it into electricity. In some ways, this is old technology. As Dourado writes:

Humans have produced electricity from the earth’s subsurface heat since 1904, when Italians first harnessed geothermal steam at Larderello, in Tuscany. Today, the same site produces enough power for 10 million Italian households, about 10 percent of the world’s geothermal electricity. … Conventional geothermal technology is only deployed at sites where subsurface heat makes itself evident through visible features like hot springs, geysers, and fumaroles. The main geothermal field at Larderello is called Valle del Diavolo—Valley of the Devil—because it contains springs of boiling water.

Thus, the question for geothermal for some time has been whether it was limited to the few locations where the heat was literally bubbling up to the earth’s surface, or whether it was possible, in a substantially larger number of locations, to reach down below the earth’s surface. It turns out that some of the underground drilling technologies used in fracking can also be used, in a different way, to set up geothermal sources of electricity. One option is what Dourado called “conventional geothermal”:

Conventional geothermal wells are technically hydrothermal—they work by extracting steam from a production well. Typically this steam flows upward through hot porous rock, acquiring heat energy along the way, but then gets trapped under impermeable caprock. Placing a production well where the steam is trapped gives it only one way to go—up the well, where, at the surface, it can power a turbine to produce electricity. A second well, called an injection well, is used to put water back into the system, without which the supply of steam would eventually dry up and lose pressure. Producing hydrothermal energy is pretty simple, and it would be very cheap at scale, but it requires this subsurface configuration—hot porous rock topped with impermeable capstone—to work.

The current research is to find ways of generating geothermal electricity in a wider range of subsurface configurations. As one example, there are “closed-loop” designs where the news drilling technologies first go down, and then go horizontally underground. The “working fluids” that are injected to generate steam are recaptured and re-injected. Start-up companies and academic researchers are are trying out different approaches. It turns out that in a US context, most of the sites that might be suitable for geothermal electricity are in the western part of the country, and thus, given the enormous amounts of western land still owned by the federal government, the willingness of the feds to allow development of geothermal resources on federal lands is likely to make a big difference.

Another recent overview of geothermal is “Can Geothermal Power Play a Key Role in the Energy Transition?” by Jim Robbins in Yale Environment 360, an online magazine published by the Yale School of the Environment (December 22, 2020). Robbins offers an example of geothermal energy–albeit for heating rather than electricity–from Boise, Idaho:

A river of hot water flows some 3,000 feet beneath Boise, Idaho. And since 1983 the city has been using that water to directly heat homes, businesses, and institutions, including the four floors of city hall — all told, about a third of the downtown. It’s the largest geothermal heating system in the country. Boise didn’t need to drill to access the resource. The 177-degree Fahrenheit water rises to the surface in a geological fault in the foothills outside of town. It’s a renewable energy dream. Heating the 6 million square feet in the geothermally warmed buildings costs about $1,000 a month for the electricity to pump it. (The total annual cost for depreciation, maintenance, personnel, and repair of the city’s district heating system is about $750,000.)

Of course, most places aren’t located above an accessible river of steaming hot water. But the warmth of geothermal can be used for “district heating” in some cases; in Iceland, which sits on abundant geothermal resources, about 90% of all homes are heated with geothermal energy. In addition, geothermal for electricity is already in wider use in the US than you might think. Robbins notes:

Even though geothermal is barely on the alternative energy radar, the U.S. already produces 3.7 gigawatts (GW) of geothermal electricity, enough to power more than 1 million homes. It’s the world’s leading producer — primarily in central California and western Nevada. California has 43 operating geothermal generating plants, and is about to build two more.

There’s also a 2019 report from the Geothermal Technologies Office at the US Department of Energy. Robbins describes a central finding of the report this way:

And a 2019 U.S. Department of Energy (DOE) report — GeoVision: Harnessing the Heat Beneath Our Feet — refers to the “enormous untapped potential for geothermal.” By overcoming technical and financial barriers, the report says, generating electricity through geothermal methods could increase 26-fold by 2050, providing 8.5 percent of the United States’ electricity, as well as direct heat.

Geothermal certainly isn’t going to address climate change issues all by itself. But unlike intermittent sources of carbon-free energy like solar and wind, geothermal does have one great advantage: It is always on.

In early July, the Gallup Poll carried out an annual survey in which people are asked about their confidence in various institutions. Here are some of the results, as reported at the Gallup website by Jeffrey M. Jones, “In U.S., Black Confidence in Police Recovers From 2020 Low” (July 14, 2021) and by Megan Brenan, “Americans’ Confidence in Major U.S. Institutions Dips” (July 14, 2021).

This figure shows the share of people who express “A great deal/Quite a lot of confidence” in each of these institutions. The overall percentage of approval is on the far right, and the breakdown by white, black, and Hispanic is shown by the dots.

For me, figures like this lead to lots of inner conversations, and I will spare you most of that. But since I’ve been reading a fair amount about policing lately, here are a few thoughts:

1. It was interesting to me that while whites were the group expressing the most confidence for the top few items on the list, Hispanics were expressing the most confidence for many of the items lower on the list. The one institution in which blacks expressed the greatest confidence was “Television News.”
2. The extremely low levels of confidence expressed for Congress, the media, big business and big labor, and other areas is worth some reflection
3. The biggest gap between confidence of whites and blacks appears for the police.
4. The survey asks separately about “The Police” and “The criminal justice system.” The confidence level in the police is far higher for every group than the confidence in the rest of the criminal justice system.
5. Despite a year of intense controversy over the police, they still rank well above many of these other categories in terms of public confidence.
6. A separate figure shows the confidence in police for blacks and whites over time. It’s interesting to note that the confidence of blacks in this area was already declining in the 214-2019 time frame, and that after a very sharp decline in 2020, there has been something of a bounceback in 2021.

Here’s one more figure, this one showing a breakdown of the same categories by political party.

Republicans are vastly more confident in the police, organized religion, the military, and small business. Democrats are vastly more confident in the presidency, newspapers and television news, public schools, and organized labor. The lack of approval for Congress, the criminal justice system, banks, and big business is largely bipartisan.

It’s hard to get data on the investment patterns of the very wealthy. Many surveys are intended to cover the entire population, from to bottom, so they don’t offer many data points for looking at the behavior of the top 1% or the top 0.1%. In addition, any survey about economic facts, like personal wealth, is only as good as the memories and willingness-to-disclose of those taking the survey.

Thus, one of the hot topics in economics research is finding ways to access “administrative” data–which refers to data that was collected for other purposes, but in an appropriately anonymous form can be made available to researchers. For example, researchers looking at income inequality have found ways to used appropriately anonymized income tax data. For wealth data, Cynthia Mei Balloch and Julian Richers found a source of such data to address the question of “Asset Allocation and Returns in the Portfolios of the Wealthy” (presented at the 2021 Conference on Research in Income and Wealth held at the Summer Institute of the National Bureau of Economic Research, July 19-20, 2021). Just in case this isn’t yet broad knowledge in the economics community, I’ll also add that the NBER holds many workshops, methods lectures, and mini-conferences during its Summer Institute, and hours of material of of top research economists presenting their current work is available at the NBER YouTube page).

Here’s how Balloch and Richers describe their data:

[W]e use anonymized portfolio-level data from Addepar, a leading technology provider for the wealth management industry. Addepar provides an advanced financial reporting and analysis software platform for private wealth advisors. These advisors range in scale from single family offices to large wealth management firms with thousands of individual advisors and client portfolios. Advisors use the platform to get a comprehensive picture of asset holdings and returns across different asset and sub-asset classes, ranging from standard equity and fixed income investments to private equity, real estate and collectibles. While individual investors can access their own account data directly, advisors are the primary users of the software. These include family offices, private wealth advisors at banks, and
other advisors. Across 373 managing firms, we observe over 50,000 client portfolios on the platform, each representing an individual household. The range of total holdings ranges from the mid-six
figures to multi-billion dollar portfolios, with an average total size of portfolios of 16.8 million
(median 1.3 million) at the end of 2019. By this time, there are close to 1 trillion in assets
recorded on the platform …

With this data, the authors are observing changes in market values over time; for example, they can see both realized and unrealized capital gains. Because the wealth managers want to know about all aspects of health, this data also includes information on wealth held in private businesses and in real estate. Again, this data is the actual investments of the wealthy, not what the wealthy say when they fill out surveys about their wealth.

What are some of the main patterns that emerge?

One is that as wealth increases, people are more likely to put a larger chunk of their money in “alternative assets,” which is a category that refers to special funds like private equity funds or hedge funds. A second pattern is that as wealth increases, the average rate of return goes up, but so does the level of risk:

Among investors with less than three million in assets under management, the average return is 4.38 percent, while for investors with more than 100 million in wealth, the average return increases to 6.37 percent. This pattern of increasing return is mirrored in the standard deviation of returns, which rises from 13.9 percent among the least wealthy investors to 19.8 percent at the top of the wealth distribution.

Among other kinds of investments–bonds, stocks, mutual funds, exchange-traded funds–the returns on assets for the wealthy are basically the same as they are for everyone else, after adjusting for the risk of each kind of investment. However, the returns that the ultra-wealthy get from hedge funds and private equity funds are substantially higher than the returns that the less-wealthy get from these categories of investments. This pattern suggests that the ultra-wealthy have access to some combination of better money managers and better investment opportunities than the rest of us.

Those familiar with patterns of how college and universities have invested their endowments in the last few decades will recognize this pattern (for discussion, see “Some Snapshots of University Endowments,” July 22, 2019). The big kids like Yale, Harvard, and others crowded into various categories of alternative investments back in the 1990s, and have made outsized returns in doing in the last few decades. Indeed, the enormous endowments that the wealthiest universities have built up are just as much (or even more) due to canny investment teams as they are to big donors. However, universities and colleges with smaller endowments who attempted to follow the same pattern have generally not been as successful.

Of course, the follow-up question is whether there might be a way to give smaller-wealth investors a chance to benefit from these higher-return alternative investments.

A number of financial markets let investors make predictions about the future path of prices and risks: for example, those who buy a stock are thinking that the price will rise. There are also “futures” markets, which allow someone to promised to deliver a certain item at a certain prices at a given date in the future. Some futures markets are based on financial prices like the price of the Standard & Poor’s stock market index; others are based on prices of physical items like oil, gold, wheat, soybeans, and many others.

When economists refer to a “prediction market,” they have in mind a market where instead buying and selling based on expectations of future prices, the buying and selling happens based on expectations of future events. One of the best-known examples is the Iowa Electronic Markets, where you can place small-sized bets on the outcome of elections and related events. Rising or falling prices in this market can then be used as a measure of the likelihood of who will win the election.

But back in the early 2000s, the idea of making more widespread use of prediction markets took a serious public relations hit. Justin Wolfers and Eric Zitzewitz tell the story in their article “Prediction Markets” in the Spring 2004 issue Journal of Economic Perspectives (where I work as Managing Editor). They wrote:

In July 2003, press reports began to surface of a project within the Defense Advanced Research Projects Agency (DARPA), a research think tank within the Department of Defense, to establish a Policy Analysis Market that would allow trading in various forms of geopolitical risk. Proposed contracts were based on indices of economic health, civil stability, military disposition, conflict indicators and potentially even specific events. For example, contracts might have been based on questions like “How fast will the non-oil output of Egypt grow next year?” or “Will the U.S. military withdraw from country A in two years or less?” Moreover, the exchange would have offered combinations of contracts, perhaps combining an economic event and a political event. The concept was to discover whether trading in such contracts could help to predict future events and how connections between events were perceived. However, a political uproar followed. Critics savaged DARPA for proposing “terrorism futures,” and rather than spend political capital defending a tiny program, the proposal was dropped.

After that experience, while prediction markets have continued to exist, they have mainly been in small and limited forms. For example, the Iowa Electronic Markets is limited in size as an experimental market designed specifically for research and teaching purposes. It has become fairly common for big companies like Google and Ford to run “internal” prediction markets, where those inside the company can place bets on issues like whether project deadlines or sales targets will be met; it often turns out that the feedback from the internal market is a useful corrective to the promises from managers that everything is going just fine. The Hollywood Stock Exchange lets you place a bet on what the sales totals of a movie will be in the weeks after it is released. Of course, Americans can also bet on various outcomes using better markets in other countries–or just watch those market to see what messages they are sending.

But now there are some stirrings that US prediction markets never completely went away, and may be ready to rise again. Mary Brooks an Paul Rosenzweig tell the story in “Let’s Bet on the Next Big Policy Crisis—No, Really” (Lawfare blog, July 13, 2021). They point to some new academic experimentation beyond the long-running Iowa Electronic Markets:

More recently, Georgetown University built out its own crowd-forecasting platform—which is not strictly prediction market but rather a way of surveying and pooling expert opinions—specifically for geopolitical futures. Similarly, Metaculus offers a platform for a quasi-prediction market, in which the currency of exchange is prestige points, and anyone can submit a question for inclusion in the market.

They point out that use of internal corporate prediction markets has been rising, and indeed, the there is a market for those with top-secret clearance in the US intelligence community:

[T]here is significant demand for internal corporate prediction markets and crowd-forecasting. Google, Ford, Yahoo, Hewlett-Packard, Eli Lilly and a number of other prominent corporations have operated or continue to operate a corporate market. Some of their questions may delve into geopolitics, but in most cases employees bet on subjects such as whether deadlines will be met, what products will take off and what earnings statements will be. …

For example, in 2010, the intelligence community started a prediction market for top-secret-cleared government employees on its classified networks. From 2011 to 2015, the Intelligence Advanced Research Projects Activity (IARPA)—the intelligence-minded sister of DARPA—ran the Aggregative Contingent Estimation (ACE). ACE was a project designed to “dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts … [by means of] techniques that elicit, weight, and combine the judgments of many intelligence analysts.” Today, IARPA still runs the Hybrid Forecasting Competition, which “develop[s] and test[s] hybrid geopolitical forecasting systems.”

But perhaps most interesting, there is now a company called Kalshi, approved by US government regulators, which will allow trading on the outcome of events. Brooks and Rosenzeweig write:

Just this past week, a prediction market that operates as a true financial exchange opened its digital doors. Kalshi—a San Francisco-based startup currently operating in beta—is the first fully regulated (CFTC-approved) prediction market. Because Kalshi is regulated, more significant amounts of money can be wagered than in many other markets, enabling them to build out a new asset class of events futures. The implications for this are obvious: An asset class like this could serve as an alternative or a supplement to more traditional insurance, allowing companies and individuals to hedge against crop failures, cyberattacks or floods.

It’s of course easy to raise concerns about prediction markets. Will they allow some people to benefit when an unpleasant event occurs? Yes, but so do any number of completely legal investments one can make in other financial markets (like those related to stock prices of insurance companies). Can they be gamed by investors? While a group of investors can certainly drive the price in one direction or another, there is the ultimate question of whether the event actually happens or not. Those who seek to drive the prediction market price to strange places need to be prepared to lose money.

No one says the market prediction is perfect: in sports betting, for example, the favorite doesn’t always win. But prediction markets provide a way of bringing together inputs of information and belief from a wide variety of people, rather than relying on other imperfect methods of prediction like listening to insider experts, outside experts, or polling data. If you think the chance of a future event embodied in the prediction market prices is wrong, that’s fine; are you willing to back your belief with actual cash? If you feel queasy about doing so, perhaps you should reconsider how strongly you hold that belief.

Michael Chui and Anna Bernasek of the McKinsey Global Institute have a half-hour interview with Daron Acemoglu. “Forward Thinking on technology and political economy with Daron Acemoglu” (July 14, 2021 audio and an edited transcript available). Acemoglu has focused for decades on the idea the idea of “directed technological change”–that is, the idea that the direction of technology is not a random event determined by scientists, but is to some extent a response to the incentives of what areas are being investigated by researchers and the incentives for firms and entrepreneurs in applying new ideas in the economy.

In Acemoglu’s view, economists of the past too often treated “technology” as a general all-purpose ingredient that tended to raise productivity of workers. In contrast, Acemoglu points out that technology often has quite particular effects. He says:

And if you look at the way that economists think about technology, it’s this latent variable that makes you just more productive. But very few technologies actually do that. Electricity didn’t make workers more productive. It made some functions in factories more feasible, and some few items more productive. A hammer doesn’t make you more productive in everything. It makes you just more productive in one single, simple task—hammering a nail. And many technologies don’t even do that. The example of spinning and weaving machinery that I gave, or the factory system, or, more recently, databases, software, robots, numerically controlled machines, they are mostly about replacing workers in certain tasks that they used to perform. …

It may benefit some workers more than others. It may well be that computers augment educated workers more than high school dropouts, so inequality can increase. But at the end you shouldn’t see the high school dropouts lose out. Their real wages shouldn’t decline. And the real wages of workers shouldn’t decline. But, in fact, one of the striking but very robust features of the last 40 years of economic development in the US and the UK has been that many groups, especially low-education or middle-education men, have actually seen their earnings fall, some groups by as much as 25 percent, in real terms, since 1980. Phenomenal. This isn’t the American dream.

In the traditional economics approach, this is a nuisance that we often sweep under the carpet. ,,, [I]it is something that doesn’t really fit into this technology as augmenting framework. But when technology, at least in part, is about automation, replacing, displacing workers from their tasks, then this happens quite often. You can have productivity improvements—capital benefits, firms benefit, but workers, especially some types of workers, all workers overall can lose out in real terms. …

[O]nce you go to this micro level, then the direction of technology, the future of technology looked at through the perspective of what type of technologies we’re going to build on, that becomes much richer and much more interesting. It’s not just whether we’re going to increase the productivity of skilled workers versus unskilled workers, both of which benefit all of them since they are complementary. It’s more like, are we going to completely give up on unskilled workers? Are we going to try to replace them? Are we going to try to replace humans? Are we going to create new tasks for humans? How are you going to use the AI platform? All of these questions about the direction of technology become much more alive, and then also the productivity implications become much more interesting.

Acemoglu points out that from the 1950s through the 1970s, the US economy had a high rate of technological change and productivity growth with a pretty stable distribution of income. But since the 1980s, technological change and productivity growth have been accompanied by a more unequal distribution of income. What changed? Acemoglu argues that the overall effects of technological change will be determined by factors like whether it gives rise to new sectors of the economy with opportunities for displaced workers of the present and future workers, and whether the technologies generate large gains in productivity (think of large-scale mechanization of agriculture and how it raised output per acre) or if it just allows replacing workers with machines. In Acemoglu’s view, too much of the innovation surrounding modern automation, robotics and artificial intelligence is about replacing existing workers, rather than empowering new industries.

The details of an appropriate policy response here are hard to enunciate, and Acemoglu (wisely) shies away from offering granular recommendation. But he does offer these general thoughts:

One is we have to free ourselves from the excessive obsession on automation. It is true in the area of AI. It’s true in other areas, too. [In] our current business community, for a variety of reasons, some of it is cost cutting, some of it is because where the technology leaders in Silicon Valley have sort of set the agenda, some of it is because government policies are just too focused on automating everything.

Instead, we have to come back to a world in which we put as much effort in increasing human productivity, both in the tasks that they already produce, but also creating new tasks in entertainment, in healthcare. There are so many new things that we can do, especially with AI, but some of it with just our existing technologies, some of it with virtual reality or augmented reality. There are many, many things ranging from judgment, social skills, flexibility, creativity, that humans are so much better at than machines.

But we’re not empowering them right now. That’s the first leg. That second leg is that we also have to pull back from using AI as a method of control. And again, that’s about how we use the AI technology. Do we use it to empower individuals? To be better communicators, better masters of their own choices and data? Be able to sort of understand the veracity or the liability of different types of information? Or do we develop these tools in the hands of platforms so that the platforms themselves are doing all of that thinking and all of that direction for the individuals? I think that those two are very different futures as well.

For a more detailed version of Acemoglu’s argument, a useful starting point is his essay with Pascual Restrepo, “Automation and New Tasks: How Technology Displaces and Reinstates Labor,” in the Spring 2019 issue of the Journal of Economic Perspectives (where I work as Managing Editor). Basically, they suggest a framework in which automation can have three possible effects on the tasks that are involved in doing a job: a displacement effect, when automation replaces a task previously done by a worker; a productivity effect in which the higher productivity from automation taking over certain tasks leads to more buying power in the economy, creating jobs in other sectors; and a reinstatement effect, when new technology reshuffles the production process in a way that leads to new tasks that will be done by labor. They then apply this framework to US data.

That JEP symposium includes three other papers:

• “Artificial Intelligence: The Ambiguous Labor Market Impact of Automating Prediction, by Ajay Agrawal, Joshua S. Gans and Avi Goldfarb
• “`Automation’ of Manufacturing in the Late Nineteenth Century: The Hand and Machine Labor Study” by Jeremy Atack, Robert A. Margo and Paul W. Rhode
• “The Rise of Robots in China,” by Hong Cheng, Ruixue Jia, Dandan Li and Hongbin Li

Behavioral economics is the combination of insights from psychology with economic behavior. A usual starting point for models of economic actors is that while they will sometimes make wrong decisions, they will not make the same wrong decisions over and over. To put it another way, they will continually be trying to avoid what they now perceive as the errors of the past, while also committing a range of mistakes.

But psychological research suggests that in certain settings, many people will make the same mistake over and over. For example, many people have personal or economic habits (say, exercise, or eating healthier, or quitting smoking, or saving more) that they would like to change. They know with some part of their mind that if they don’t make a change, they are likely to regret it in the future. Nonetheless, they keep deciding they will start the desired change tomorrow, rather than today. There are many of these biases that, while one can learn to overcome them, seem built into cognition. People often do a poor job of thinking about risks that have a low probability of happening. Behavioral economics looks at the outcome of economic situations where some or many people have these biases.

For those who want to learn what behavioral economics is all about, a good starting point is The Behavioral Economics Guide 2021, edited by Alain Samson. It’s from the behavioraleconomics.com website, which serves as an online hub for people interested in the topic. I recommend the volume for at several purposes.

There’s a 30+ page glossary of behavioral science concepts, for those who would like a little help with the lingo, starting with “action bias” and ending with the “zero price effect.” Here’s “action bias:”

Some core ideas in behavioral economics focus on people’s propensity to do nothing, as evident in default bias and status quo bias. Inaction may be due to a number of factors, including inertia or anticipated regret. However, sometimes people have an impulse to act in order to gain a sense of control over a situation and eliminate a problem. This has been termed the action bias (Patt & Zeckhauser, 2000). For example, a person may opt for a medica treatment rather than a no-treatment alternative, even though clinical trials have not supported the treatment’s effectiveness. Action bias is particularly likely to occur if we do something for others or others expect us to act (see social norm), as illustrated by the tendency for soccer goal keepers to jump to left or right on penalty kicks, even though statistically they would be better off if they just stayed in the middle of the goal (Bar-Eli et al., 2007). Action bias may also be more likely among overconfident individuals or if a person has experienced prior negative outcomes (Zeelenberg et al., 2002), where subsequent inaction would be a failure to do something to improve the situation.

Here’s the definition of “zero price effect:”

The zero price effect suggests that traditional cost-benefits models cannot account for the psychological effect of getting something for free. A linear model assumes that changes in cost are the same at all price levels and benefits stay the same. As a result, a decrease in price will make a good equally more or less attractive at all price points. The zero price model, on the other hand, suggests that there will be an increase in a good’s intrinsic value when the price is reduced to zero (Shampanier et al., 2007). Free goods have extra pulling power, as a reduction in price from $1 to zero is more powerful than a reduction from $2 to $1. This is particularly true for hedonic products—things that give us pleasure or enjoyment (e.g. Hossain & Saini, 2015). A core psychological explanation for the zero price effect has been the affect heuristic, whereby options that have no downside (no cost) trigger a more positive affective response.

If you have a drop of social scientist blood in your veins, descriptions like this will start your pulse pounding. Do these effects really exist? In what context? How would you measure them? Are these effects intertwined with a different or perhaps broader effect? If you need concrete examples, the bulk of the report is 15 short and readable summaries of recent studies in the area, with examples concerning prevention of gender-based violence, banking and insurance, sustainable agriculture, career coaching, and more.

One theme that emerges from several papers in the volume is that behavioral economics effects are often deeply rooted in a particular context: that is, you can’t just grab an item from the glossary, plug it into your life or business or organization, and assume you know how it will work. For example, Florian Bauer and Manuel Wätjen write about their research in “Tired of Behavioral Economics? How to Prevent
the Hype Around Behavioral Economics From Turning Into Disillusionment.” They write:

Applying the behavioral economics effects found in academic experiments to marketing is becoming more and more popular. However, there is increasing evidence that copy-and-pasting academic effects does not achieve the desired effects in real life. This article aims to show that this is not because customers are becoming wise to nudges or that behavioral economics does not work at all, but because the application of behavioral economics typically ignores the contextual aspects of the actual decision to be influenced. Herein, we present a framework that considers these aspects and helps develop more effective behavioral interventions in marketing, pricing, and sales.

John A. List makes an argument that is similar in tone but broader in his introduction to the volume, “The Voltage Effect in Behavioral Economics.” List points out that it is fairly common for someone to latch on to an academic study in behavior economics, but then are disappointed when it doesn’t seem to “scale up” to a real-world context. List writes:

Indeed, most of us think that scalable ideas have some ‘silver bullet’ feature, i.e., some quality that bestows a ‘can’t miss’ appeal. That kind of thinking is fundamentally wrong. There is no single quality that distinguishes ideas that have the potential to succeed at scale with those that do not do so. In this manner, moving from an initial research study to one that will have an attractive benefit cost profile at scale is much more complex than most imagine. And, in most cases, scaling produces a voltage drop—the original BE [behavioral economics] insights lose considerable voltage when scaled. The problem, ex ante, is determining whether (and why) that voltage drop will occur. … What this lesson inherently means is that scaling, in the end, is a weakest link problem: the endeavor is only as strong as the weakest link in the chain.

In other words, the connection from an academic study to a real-world application involves a number of links in a chain. List lays out five of them:

1. Infererence. Perhaps the academic study you looked at was a “false positive”–that is, the result won’t hold up in other similar studies. List suggests that before believing an effect is real, one should look for “three or four well-powered independent replications of the original finding.”
2. Representativeness of the population. If a study was done on a group of college sophomores, or retirees, or people with a certain medical condition, you can’t necessarily assume that it apply well on other groups, like working-age adults or high school dropouts.
3. Representativeness of the situation. For example, a small-scale program with a small group of dedicated and trained participants may not scale up well to a larger general population group that is less dedicated and less well-trained.
4. Spillovers and general equilibrium effects of scaling. Say that I start a program in a certain area to teach a highly desirable skill to some workers. Those workers get much higher pay as a result. So then I expand the program very substantially to teach this skill to many more workers. The original group did well in part because it was a small group, and the skill was still scarce. But at some point, as the skill becomes common in that are, the rewards will be lower.
5. Marginal cost considerations. As program expands, there are two possibilities. One is that there are economies of scale: that is, as the program covers more people, average cost per person falls. For example, a web-based program that can be expanded to cover many more people might have this property. However, the other possibility is that as the program covers more people, average cost per person rises. This can happen if the program needs some specific and particular skills that may be hard to get: for example, perhaps I can train a small group of teachers who volunteer to be part of my new specific curriculum, but as I try to train bigger and bigger groups, it gets harder.

List draws on Tolstoy to summarizes his thinking. At the beginning of Anna Karenina, Tolstoy famously wrote: “All happy families are alike; each unhappy family is unhappy in its own way.” List echoes: “[A]ll successfully scaled ideas are alike; all unsuccessfully scaled ideas fail in their own way.” I would just add that from this perspective, one of the advantages of behavioral economics is that it helps to drag social science generalities down into the realness of the particular.