Economists have a knee-jerk negative reaction to proposals that are phrased in terms of “free,” whether its “free” school lunch, or “free” health care, or “free” housing, or “free” tuition. In each case, the issue isn’t whether the program is a good idea or not. It might be. The issue is that just because something is provided at no cost to a user doesn’t mean it is actually “free,” only that the costs are being paid in some other way. Thus, “free” programs need to be evaluated, like any others, based on who receives benefits and pays costs, not on a pretense that saying “free” solves the problem.

Back in 2015, Oregon passes legislation for the “Oregon Promise,” which said the state would pay the average cost of Oregon community college tuition for Oregon high school graduates. The same legislation required that a state agency called the Higher Education Coordinating Commission produce a report evaluating the Oregon Promise every other year, and the 2022 report is now out.

The hope of the Oregon Promise, of course, was that it would encourage greater college attendance, especially among those from families with lower incomes or from backgrounds traditionally under-represented in colleges. Here’s the summary of the findings:

We found that in the first two years of the program, enrollment rates rose, but declined in the last four years, especially due to the impact of the pandemic on college enrollment. The initial implementation of the Oregon Promise was associated with a clear increase in enrollment at the colleges, and early enthusiasm and attention to the program seemed to realize the program’s goals of opening the door to postsecondary education and training wider. After six years, these early increases have not been sustained, as community college and statewide college-going rates are lower than prior to the Oregon Promise program.

We did not find evidence to suggest the Oregon Promise is associated with reducing equity gaps in college-going rates. Racial/ethnic gaps in college-going rates were similar before and after the implementation of the program, at least until the pandemic. Although college-going rose for Black/African American and for Latino/a/x/Hispanic graduates increased, this increase started before the Oregon Promise and therefore is not attributable to the Oregon Promise program. Gaps in college-going rates by geography and gender widened since the program began. …

We found that Oregon Promise recipients are generally representative of their high school graduating class, though they are somewhat more likely to be women and Latino/a/x/Hispanic. Additionally, students with the Oregon Promise are more likely to be from low-income backgrounds and from urban areas. We also noted that because of the last dollar structure of the program, the vast majority of Oregon Promise dollars go to students from middle- and upper-income backgrounds, even though nearly half of the students are from low-income backgrounds. …

By design, the program maximizes federal financial aid coming into Oregon for those who receive the Oregon Promise grant. … For students, the program slightly reduces the percentage of students facing unaffordable costs. Nonetheless, nearly two out every five students receiving the Oregon Promise still cannot meet the expected cost of attendance at their college even with the grant, and almost two-thirds of students from low-income backgrounds cannot meet the cost of attendance even with the grant. …

We found no association between the Oregon Promise program and the number of terms enrolled or credits earned among recent high school graduates. In addition, we found no lasting increases in completion rates coinciding with implementation of the Oregon Promise program, though the number of cohorts and years available to assess this question are still limited.

One can of course raise a number of possibilities here. Maybe it was the pandemic that hurt the program–although little bump in community college attendance after the passage of the program was already over by 2020. Indeed, it looks in some of the more detailed data as if the bump in community college attendance was because of a drop in attendance from other Oregon four-year colleges; in other words, the program caused a few students to shift from four-year to the now “free” two-year institutions.

Maybe a bigger Oregon Promise is needed, going beyond tuition and also governing books, living expenses, and other costs? It’s notable that the program had a “top-up” design: that is, you first qualify for federal assistance for low-income families via programs like Pell Grants, and then the Oregon Promise tops up the rest. But the result of this approach was that most of the funding for the Oregon Promise ended up flowing to middle- and high-income families, who weren’t eligible for low-income aid. Maybe a Promise that gave more generous aid to those with low-incomes, and didn’t also subsidize those from middle- and high income families, would make more sense?

Or it may be that a high level of mentoring and advising that continues into the community college year is just as or more important to nontraditional or low-income students than the financial aid itself. After all, getting admitted to community college but then not having a sense of what classes to take, how to do the work, and where to go when you are having troubles may not help much. Or maybe the Oregon Promise should have requirements that high school students take certain classes or have a certain GPA or test scores to be eligible?

The details of program design matter. There are now a variety of “Promise”-style programs: the Kalamazoo Promise, the Pittsburgh Promise, and similar programs across 24 states. Laura W. Perna, Jeremy Wright-Kim, and Elaine W. Leigh look at some differences in design in “Is a College Promise Program an Effective Use of Resources? Understanding the Implications of Program Design and Resource Investments for Equity and Efficiency” (AERA Open, 6: 4, October-December 2020). They write:

Estimates of net benefits for the Kalamazoo Promise and Pittsburgh Promise are likely not transferable, as these programs differ from others in ways that may influence program outcomes, characteristics of recipients, and costs. The Kalamazoo Promise requires students to attend district schools from kindergarten through high school graduation to be eligible for the maximum financial award, does not reduce the award by other grant aid (i.e., “first dollar”), and allows students to use the award at public 4-year institutions across the state (Bartik et al., 2016). Comparatively, the Tennessee Promise, for example, requires students to apply as high school seniors, provides a financial award that is reduced by other grant aid (i.e., “last dollar”), and limits use of the award to attendance at community and technical colleges (House & Dell, 2020Meehan et al., 2019).

The Oregon Higher Education Coordinating Commission obviously isn’t the final word here, but this is how they sum up the existing research on Promise-style programs:

Whether and how College Promise programs affect access to and success in college has been of national interest. Across the country, these programs cover tuition, but they differ in both scope and design. Regarding scope, some apply to a specific college, others apply only to high school graduates in a specific school district, and still others to multiple public institutions for high school graduates statewide. Regarding eligibility, requirements vary around student residency, high school grade averages, application materials and fees, enrollment levels, and income limits.

In the initial years of these various programs, evaluations found increased college enrollment associated with both local and statewide College Promise programs. However, more recent research has found that College Promise programs do not consistently sustain these increases in enrollment, citing differences in eligibility requirements. Programs that have eligibility requirements consistent with students who are most likely to go to college have not produced long lasting enrollment increases.

Though the relative newness of College Promise programs limits the research on college completion, recent studies suggest that student supports (e.g., advising, mentoring, and other educational supports) are an important intervening factor. Programs with more minimal eligibility requirements in particular did not demonstrate increases in postsecondary credential attainment without additional support resources. Prior research has shown limited impacts of College Promise on equity in college access and success. Programs with eligibility requirements that are consistent with the characteristics of those already likely to attend college maintain existing inequities. Those structured as last-dollar programs show minimal to no improvements in equity.

There’s an old saying that states are the “laboratories of democracy,” the place where you can try things out and see what works. After all, you can learn from your failed experiments, too.