The new artificial intelligence technologies are getting a lot of buzz. How are they likely to be used and how will they affect productivity. It seems to me obviously too early to know, but just the right time to start thinking concretely about plausible outcomes. In that spirit, Martin Neil Baily, Erik Brynjolfsson, and Anton Korinek discuss “Machines of mind: The case for an AI-powered productivity boom” (Brookings Institution, May 10, 2023).
The authors focus on what they call “foundation models,” which are “vast systems based on deep neural networks that have been trained on massive amounts of data and can then be adapted to perform a wide range of different tasks. ” Examples include “large language models” like ChatGPT (from Open AI), Bard (from Google), and Claude (from Anthropic). “But generative AI is not limited to text: in recent years, we have also seen generative AI systems that can create images, such as Midjourney, Stable Diffusion or DALL-E, and more recently multi-modal systems that combine text, images, video, audio and even robotic functions. “
Evidence is accumulating about how these technologies will affect actual jobs. The unifying theme here is saving time: that is, just as I save time when I can download articles while sitting at my desk, rather than walking through library stacks and making photocopies, lots of existing jobs can be done more quickly with the new technologies. Some examples:
There is an emerging literature that estimates the productivity effects of AI on specific occupations or tasks. Kalliamvakou (2022) finds that software engineers can code up to twice as fast using a tool called Codex, based on the previous version of the large language model GPT-3. That’s a transformative effect. Noy and Zhang (2023) find that many writing tasks can also be completed twice as fast and Korinek (2023) estimates, based on 25 use cases for language models, that economists can be 10-20% more productive using large language models.
But can these gains in specific tasks translate into significant gains in a real-world setting? The answer appears to be yes. Brynjolfsson, Li, and Raymond (2023) show that call center operators became 14% more productive when they used the technology, with the gains of over 30% for the least experienced workers. What’s more, customer sentiment was higher when interacting with operators using generative AI as an aid, and perhaps as a result, employee attrition was lower. The system appears to create value by capturing and conveying some of the tacit organizational knowledge about how to solve problems and please customers that previously was learned only via on-the-job experience.
It’s easy enough to run across other examples. Two MIT researchers, Shakked Noy and Whitney Zhang, have a working paper up called “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence” (MIT working paper, March 2, 2023).
We examine the productivity effects of a generative artificial intelligence technology—the
assistive chatbot ChatGPT—in the context of mid-level professional writing tasks. In a
preregistered online experiment, we assign occupation-specific, incentivized writing tasks
to 444 college-educated professionals, and randomly expose half of them to ChatGPT.
Our results show that ChatGPT substantially raises average productivity: time taken
decreases by 0.8 SDs and output quality rises by 0.4 SDs. Inequality between workers
decreases, as ChatGPT compresses the productivity distribution by benefiting low-ability
workers more. ChatGPT mostly substitutes for worker effort rather than complementing
worker skills, and restructures tasks towards idea-generation and editing and away from
rough-drafting. Exposure to ChatGPT increases job satisfaction and self-efficacy and
heightens both concern and excitement about automation technologies.
There have been studies for a few years now suggesting that use of AI technologies can help doctors to more accurate diagnoses. A recent study along these lines that caught my eye is “Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum,” by John W. Ayers, Adam Poliak, Mark Dredze, et al. (JAMA, April 28, 2023). From the abstract:
In this cross-sectional study of 195 randomly drawn patient questions from a social media forum, a team of licensed health care professionals compared physician’s and chatbot’s responses to patient’s questions asked publicly on a public social media forum. The chatbot responses were preferred over physician responses and rated significantly higher for both quality and empathy.
AI systems are also pushing research forward more rapidly. Here’s an article from Steven Rosenbush in the Wall Street Journal (“Biologists Say Deep Learning Is Revolutionizing Pace of Innovation,” March 22, 2023).
A milestone in computational biology was announced last July, when Alphabet Inc.’s DeepMind Technologies subsidiary announced that its AlphaFold2 AI system had been used to predict the three-dimensional structure of nearly all proteins known to science, essentially solving a problem that researchers had been trying to crack for the past 50 years. On March 16, Facebook-parent Meta Platforms Inc. said its research arm, Meta AI, had used its new AI-based computer program known as ESMFold to create a public atlas of 617 million predicted proteins. Like OpenAI’s ChatGTP, the Meta tool employs a large language model, which can predict text from a few letters or words.
What about overall effects? Tyna Elaoundou (who works for Open AI), Sam Manning, Pamela Mishkin, and Daniel Rock have a working paper called “GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models (arXiv working papers, March 23, 2023). They write:
We investigate the potential implications of large language models (LLMs), such as generative Pretrained Transformers (GPTs), on the U.S. labor market, focusing on the increased capabilities arising from LLM-powered software compared to LLMs on their own. Using a new rubric, we assess occupations based on their alignment with LLM capabilities, integrating both human expertise and GPT-4 classifications. Our findings reveal that around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs, while approximately 19% of workers may see at least 50% of their tasks impacted. …
The projected effects span all wage levels, with higher-income jobs potentially facing greater exposure to LLM capabilities and LLM-powered software. Significantly, these impacts are not restricted to industries with higher recent productivity growth. Our analysis suggests that, with access to an LLM, about 15% of all worker tasks in the US could be completed significantly faster at the same level of quality. When incorporating software and tooling built on top of LLMs, this share increases to between 47 and 56% of all tasks.
In a similar spirit, a much-cited report from two Goldman Sachs analysts, Joseph Briggs and Devesh Kodnani, considers ” The Potentially Large Effects of Artificial Intelligence on Economic Growth” (March 26, 2023, not directly at the Goldman Sachs site but available on the web if you hunt for it). They write:
We estimate that generative AI could raise annual US labor productivity growth by just under 1½pp over a 10-year period following widespread adoption, although the boost to labor productivity growth could be much smaller or larger depending on the difficulty level of tasks AI will be able to perform and how many jobs are ultimately automated. The boost to global labor productivity could also be economically significant, and we estimate that AI could eventually increase annual global GDP by 7%. Although the impact of AI will ultimately depend on its capability and adoption timeline, this estimate highlights the enormous economic potential of generative AI if it delivers on its promise.
Again, it seems to me too early to trust any specific estimates here. But several themes of this line of research seem especially salient to me.
First, these are practical discussions of how the new technologies can help workers in various jobs. Thus, they help us stop thinking about the new AU technologies as the embodiment of bad science fiction movies (and in fairness, a few good science fiction movies, too!), and instead to think about practical realities. These technologies are not about being taken over by sentient robots. They are about humans being able to do their work more quickly.
Second, many of these studies have an interesting theme that they tend to help the lesser-skilled worker in any occupation by more. The better-skilled workers often have already developed their own shortcut and information sources and methods, and are drawing on their greater mental database of past experiences. The AI tools often help other workers catch up.
Third, we apparently are doomed to replay, one more time, one of the long-standing public dramas of new technologies: that there is only a fixed amount of work to do, and if existing workers can do it faster, then the available jobs will shrink dramatically, leading to mass poverty. This fear has been manifested many times in the past. Some of the examples I’ve collected over time include: worries from the US Secretary of Labor about automation and job loss in 1927; fear of robotics and automation in 1940; the US government commission on the dangers of automation and job loss in 1964; and when Nobel laureate Wassily Leontief predicted in the early 1980s how automation would lead to mass unemployment. A few years back I linked to an essay by Leslie Willcocks called “Robo-Apocalypse Cancelled,” going through reasons why predictions of a technology-driven economic disaster never quite seem to happen.
But big picture, think about all the technological changes of the last two decades–heck, over the past two centuries. Surely, if technological advances and automation were likely to lead to mass unemployment, we would already have arrived at a world where only 10% or fewer of adults have jobs? But instead, needing many fewer workers for jobs like growing wheat, lighting streetlights, filling out accounting ledgers by hand, operating telephone switchboards, making a ton of steel, and so on and so on have opened the way for new occupations to arise. I see no compelling reason why this time and this technology should be different.