Amazon Web Services (AWS), the first “public cloud” offering third-party data storage and compute services, launched in March 2006, and by 2012, there were multiple credible competitors. The hype was strong. Back then, cloud evangelists were predicting that essentially everything in the enterprise data center would migrate to one of just a few public clouds. And despite the public cloud’s incredible advantages—unlimited capacity and scale, ubiquitous access, and massive redundancy—this hasn’t happened.
That’s because the cloud has an Achilles’ heel: latency. And as a result, there’s an entirely new battle brewing at the metro edge of national data networks, giving players who moved late towards the public cloud an opportunity to dominate a giant emerging market.
The rise of the edge
Public cloud facilities are enormous, so to build them affordably, Amazon (NASDAQ: [[ticker:AMZN]]), Google (NASDAQ: [[ticker:GOOG]]), and Microsoft (NASDAQ: [[ticker:MSFT]]) place them in sparsely populated areas with inexpensive land that’s often near cheap, renewable power sources. The data centers can be hundreds, even thousands, of miles from customer locations and, at those distances, even the speed of light isn’t fast enough to provide the rapid response that users and applications require.
To overcome this latency, compute and storage operations are moving closer to the user at the network edge. And having participated in the early evolution of the cloud with my prior company, CloudSwitch, I can say with confidence that the edge is developing much more quickly than the cloud did.
But building an edge data center is not anything like building a “hyperscale” cloud facility. Urban real estate is expensive, and resources such as space and power are often tightly constrained. The public cloud giants are not the favorites to win in this new technology race.
As Gartner analyst Thomas Bittman wrote in 2017, “Cloud computing providers are really good at managing their standardized, centralized, massively scaled data centers and control software. But the technologies for the edge will be completely different, much more dynamic, much more evolutionary and competitive.”
“Colos” and “telcos” well positioned for the edge land grab
Given the high cost of real estate around large cities, the computing colocation and telecommunications players that were late to the battle for the cloud earlier this decade are now in the strongest position to take the lead on the edge. That’s because they’re already sitting on the technological equivalent of beachfront property.
Highly connected colocation providers who were unable to achieve the massive scale required to compete for the cloud market have spent the last decade building out hundreds of tightly interconnected data centers near population hubs around the globe. And those telecommunications carriers who couldn’t catch up technologically to the current cloud giants now have a huge advantage, thanks to the thousands of highly connected facilities they already operate in the midst of large cities.
That’s not to say there’s no room for upstarts, however. Growing, young companies such as Vapor IO and EdgeConneX are building extremely compact, micro data centers.
All of these players recognize they can’t build the edge computing market on their own. Vapor IO, for instance, has created the Kinetic Edge Alliance to bring together established and emerging technology vendors to increase edge capabilities. And the mobile telecommunications technology vendor Ericsson has an internal startup called Edge Gravity which has brought together more than 80 global carriers to create an edge services network. The edge ecosystem is already well underway.
The emerging data layer
But as is often the case in the early days of new technologies, not much attention has been paid to the data layer, and it’s not an easy problem to solve at the edge. These facilities will be processing, aggregating, and backhauling gargantuan amounts of data from enterprise operations, internet-connected sensors, self-driving cars, and a host of use cases we’ve not yet even imagined. But there’s certainly not enough room or electricity on the edge to deploy traditional storage boxes to handle it all.
A new edge data center in Foxborough, MA, at the base of an SBA Communications wireless tower points to a possible solution. SBA is collaborating with Packet, which provides a bare metal cloud (i.e. dedicated, single-tenant servers that aren’t shared with other customers), and my company, ClearSky Data, which provides the data layer.
By leveraging our service network, 100 terabytes of on-premises storage can represent 1 petabyte of usable storage, a tenfold gain that reduces the amount of space required by 90 percent. We’re able to accomplish this by using an intelligent caching algorithm that only retains “hot” data locally, which is the most frequently used information and typically makes up about 10 percent of the total data set. “Warm” data—information that’s not likely to be used as often, but probably will be accessed within a couple of months—is cached in a location close enough to provide sub-millisecond latency. And the master copy of the full data set lives in a backing cloud, which could be hundreds of miles away. But because hot and warm data are cached inside or very close to the edge data center, we can overcome the cloud’s inherent latency issues.
Data management and access aren’t the only challenges that will require a new approach at the edge. However, that won’t stop many incumbent storage and IT infrastructure vendors from slapping a “ready for the edge” marketing label on their products and services without doing any of the additional work to justify it. Given how rapidly the edge is developing, there’s an even greater risk for “edge washing” than there was for “cloud washing” earlier this decade. So, buyer beware. There’s a lot more to the edge than standing up equipment in a colocation facility.
Despite the many challenges, however, the emerging edge infrastructure is already starting to transform the cloud’s promise into a reality. Enterprises, for instance, are decommissioning on-premises infrastructure as they move workloads to hybrid services that incorporate the edge and the cloud. It’s still early, but the edge is maturing faster than many people think, and as it unlocks the full potential of the cloud, we will see an unprecedented technological transformation.
