This post was co-written with Yasuyuki Motoyama and Jared Konczal.
A few weeks ago, a new ranking was published measuring “opportunity” on a state-by-state basis. Vermont came out on top as the friendliest state for upward mobility, scoring well on both the Community and Education metrics.
Someone from Vermont e-mailed us here at the Kauffman Foundation, wondering how this could be in light of Vermont’s demographics. And indeed, the state’s demographic direction does not scream opportunity. By median age, Vermont is four years older than the nation as a whole. Today, residents over the age of 65 comprise 15 percent of the population—by 2030, it will be 20 percent and rising. This person observed many young people abandoning the state, but Vermont still has a high rate of self-employment—maybe the report envisioned the opportunities created for everyone who stays?
Checking other state rankings, it’s difficult to get a grasp on how Vermont, or any other state, truly measures up. In a ranking of personal and economic freedom, Vermont ranks 43rd overall, with black marks for regulatory restrictions. On another ranking of top states “for business” (otherwise undefined), Vermont comes in 39th. Yet it ranks third in quality of life and fourth in education in that report, two qualities that would seem to be pretty important for business growth. In fact, if one sorts these “top states for business” rankings using a second dimension, education, a strange picture emerges. Of the top 10 states for education, the highest rank achieved overall for business friendliness is 17th (Wisconsin).
We don’t mean to pick on Vermont—it scores well in several categories of the State New Economy Index from the Information Technology and Innovation Foundation (ITIF)—and was the first state in the nation to adopt digital firm formation (creating and operating your company in the cloud), something the Kauffman Foundation has helped promote.
But these state (or city or metropolitan) rankings are generally not helpful—and in some cases, are quite unhelpful—in gauging a state’s economic position. At worst, because they masquerade as policy-relevant, they can distort policy discussions and decisions.
We recently assembled a set of indicators specific to entrepreneurship and innovation and ran 1,000 simulations, randomly changing the weights that different indicators received (many rankings assign different weights). Across these simulations, 22 different states could claim to be a “top 10” state depending on the variation. Five different states could claim the mantle of “number one” in entrepreneurship and innovation: Colorado, Oregon, Florida, New Hampshire, and Montana.
These are not differences of degree, either—in many instances, depending on the importance assigned to different indicators, states can zoom up or down the rankings. Depending on what they want to see, policymakers and promoters and others can, quite literally, create whatever ranking they choose.
The reasons for this rankings-o-matic madness aren’t hard to identify. For one thing, even though any given ranking purports to be based on data, the data themselves are always shaped by
