The performance requirements of AI and other emerging applications is driving data center technology to smaller, edge facilities...
Data centers are expanding to the network edge to meet demand by artificial intelligence and other applications requiring fast response times not available from traditional data center architectures.
The problem with traditional architectures is their centralized framework. Data often travels hundreds of miles from the edge to computers, then back again. That’s fine when you’re dealing with email, Google, Facebook and other applications delivered via the cloud. Human brains are slow computers, unable to register the lag time between, say, clicking on an email message in a browser and the message opening.
But AI and other emerging applications — Internet of things (IoT), cloud-based gaming, virtual reality — require much faster network response times, otherwise known as “latency.” That means data center processing must move to the network edge. Edge computing can take place in small data centers, roughly the size of shipping containers, rather than the warehouse-sized edifices that currently power the cloud.
Startups such as EdgeMicro and Vapor.io are deploying these “mini data centers.”
Data center operators can still use their traditional structures, with fast networks and other hardware and software required to ensure the speedy response times needed for edge applications.
And edge data centers can reside on enterprise premises, or in exotic locations such as mines, ships and oilfields.
“The number one thing [driving edge computing] is the amount of data being created outside the data center,” said Patrick Moorhead, president and principal analyst of Moor Insights & Strategy. The numbers of connected sensors will reach 1 trillion by 2024, primarily driven by smart cities and video.
Latency isn’t the only problem requiring edge computing. “It’s cost — that’s really driving edge computing,” Moorhead said. “Every time you bring data in [to a data center] you pay someone money. The Internet is not free.” Internet providers charge for bandwidth, and cloud providers, like Amazon Web Services (AWS), have “egress charges” for moving data in and out of their clouds.
Organizations need computing at the edge in applications where they can’t access connectivity: in a ship, down a mine shaft or on an oilfield. Meanwhile, a growing list of privacy regulations requires on-site data processing in some applications, especially healthcare.
“If you’re a hospital you’re flat-out not allowed to send any data into the cloud,” Moorhead said. And even if it were allowed, bandwidth costs would make moving much of that data, particularly diagnostic images, prohibitively expensive.
Operators look to the edge
Digital Realty is one of the world’s largest data centers operators, differentiating themselves via a global platform and infrastructure ranging from massive multi-megawatt facilities to individual cages and racks. The company has 267 data centers worldwide, in 20 countries.
Edge locations require a new kind of infrastructure. “By no means does it look like a traditional data center,” Digital Realty CTO Chris Sharp said. “The size is much smaller, with workloads requiring a lot of power density and interconnection density.” These mini data centers need to be lights-out, with no operations staff on-site. and multitenant support. Dense fiber connectivity back to the core cloud infrastructure also is a must.
Digital Realty is in early stages of deploying mini data centers, with prototypes in Chicago, Atlanta and Dallas, in partnership with startup Vapor IO.
Mini data centers aren’t the only option. In many locations, edge applications can run inside a conventional data center and still achieve the 5-msec latency needed for rapid application response times, said Russell Shriver, Digital Realty’s director global service innovation. “For a lot of enterprises that are looking for an edge in major metro areas, that’s going to be more than sufficient for their needs,” he said.
The mini data center remains an emerging market. Indeed, Digital Realty competitor Equinix sees its existing facilities as serving edge needs for its service provider and enterprise customers, said Jim Poole, Equinix vice president of global business development. “Equinix, as it exists, is the edge,” Poole asserted.
Equinix has more than 190 data centers in more than 44 major metropolitan areas worldwide. Much of the U.S. is already a10-msec round-trip access time over fiber from applications sitting in clusters of Equinix data centers. That latency covers 80 percent of the U.S. population. Before building new, mini data centers, companies and service providers are looking to maximize deployment of edge applications within that existing infrastructure.
Wireless creates bottleneck
While Equinix can achieve low latency over fiber, edge applications require wireless as well, and wireless remains a bottleneck for AI and other emerging edge applications. Current 4G wireless latency is 40 msec at best, and the average is between 60 to 120 msec, Poole said.
5G promises to slash latency. Hence, service providers are partnering with hyper-scale cloud service providers to take leverage improved performance. AWS and Verizon, for example, are teaming to connect an AWS data center in downtown Los Angeles to Verizon’s radio access network (RAN) tower complex in the city. The project demonstrates they can create a sub-10 msec latency zone around the metro area, Poole said. That, in turn, could generate demand for mini data centers. “But until we fix this particular problem, nobody’s going to spend the incremental capital,” he said.
Additional, 5G “network slicing” capabilities will make it possible to deploy private, wireless networks for added control and security, Poole added.
For now, mini data centers are a promising technology lacking scale. According to Poole, “The reason you don’t see people running around making big announcements of deploying hundreds and hundreds of these mini data centers is that people don’t see the business case yet.”
Nonetheless, Equinix does see use cases for modular data centers — not necessarily on the edge, but as a means to enter emerging markets where, for now, it makes little sense to build a $100 million data center.
AI is generating big demand for edge computing, according to Kaladhar Voruganti, an Equinix senior fellow,
AI applications include two primary workloads: training and inferencing. Training is what it sounds like — teaching an AI model how to solve a problem. This process often involves organizing petabytes of data.
“Usually you need a lot of compute,” Voruganti said. Training runs on power-hungry GPUs, with each fully loaded rack consuming up to 30 to 40 kilowatts. Training generally needs to run in a big data center to satisfy power requirements, as well as privacy and regulatory concerns in some applications.
Digital Realty has partnered with Nvidia to provide the hardware vendor’s GPUs in colocation servers.
Once models are trained, the next step is inference, a process where the model applies what it has learned in training and puts it to work in a production application. Inference requires much less data crunching, and can run in a rapidly deployed Docker or other software container at the network edge — in a smart phone, a Tesla or mini- or metro data centers.
“You might train it in the big cloud, and run the application and do the inference right on the factory floor, or Walmart, or the gas station,” analyst Patrick Moorhead said
These sorts of AI applications can be used in a variety of cases. For example, an airline company might use “digital twins” for predictive maintenance. Or, as the economy opens up from the Covid-19 pandemic, a business could use AI to run heat-mapping and facial recognition to identify people entering a facility who might be infected.
Other applications requiring edge compute (and frequently using AI) include gaming, IoT, smart factories, shipping and logistics. Additionally, retail technologies require edge computing to deliver needed responsiveness.
Moorhead sees particular demand for edge data centers in retail. A “store of the future” like Amazon Go has hundreds of cameras, and likewise Walmart uses video to track customers. “They’re driving the heck out of the need for this,” he said.
Smart city planners are looking to use AI and other edge applications to promote health and safety, track infrastructure maintenance needs and manage traffic.
Other demands will come from transportation — including much-hyped self-driving cars — along with advanced manufacturing and visual inspection of products. The energy industry is also driving demand, particularly for remote inspection.
Special hardware requirements
Edge AI applications typically use flash storage for high performance, said Equinix’s Voruganti.Those applications also require a high degree of networking connectivity, both from the device to the edge and to the data center. Links also are required between application components that might be running in different locations on the network, Poole said. “They need to have low latency between components and domains,” he added.
Edge computing also needs to be rugged, for deployment in locations such as elevators, public transit turnstiles and mining equipment, Moorhead said. Shipboard computers must be salt-water resistant.
Edge computing also presents physical security challenges. Conventional hyper-scale data centers have near-military-grade security, but an edge data center in a rural area, unguarded, is susceptible to break-ins, or even an attacker carrying an entire remote data center off in a truck.
And the winners are….
Hyperclouds, enterprise vendors, telecom providers and data center operators all look like winners at the edge. “AWS is the big mothership. It’s slowly but surely fielding a credible edge offering,” Moorhead said.
The public cloud giant unveiled AWS Outposts, a hardware rack running its infrastructure software — the same infrastructure run in an AWS data center. Outposts can run on-premise, on the edge, or in a data center. AWS Snowball, an edge computing device, provides computing, memory and storage for remote environments such ships. Another Amazon offering called Wavelength is an edge device aimed at carriers, putting computer closer to the edge for 5G deployments.
On the software side, alternatives include AWS IoT Greengrass, an operating system that connects IoT devices to the cloud. Meanwhile, public cloud rival Microsoft provides its Azure cloud IoT-for-edge services while VMware also provides edge services. Moorhead said VMware is “surprisingly competitive in this space.”
Google, the other major public cloud vendor, has been a bit of a laggard, but is stepping up with its Anthos services for distributed cloud applications.
Meanwhile, IT infrastructure giants like Dell, Hewlett Packard Enterprise (edge servers) and Cisco (IoT networking) have an advantage since edge computing requires a big ecosystem, with tissue connecting on-premises infrastructure and the cloud, Moorhead said.
Emerging edge data centers vendors like Vapor IO also have an opportunity to redefine old technology, Moorhead reckons. “There have been data centers on the edge for 50 years. Any Walmart has a raised floor and a data center. If you go into a gas station or McDonald’s they have a server on the wall,” he said. “Where Vapor IO is really leaning-in is adding compute close to the network, specifically the 5G network.”
Telco central offices also can be repurposed as mini data centers, creating opportunities for carriers. “A typical neighborhood has a cement bunker with analog lines and a bunch of racks in it,” Moorhead said. “They’re almost empty now. They have a lot of power. They’re industrial strength — literally a cement bunker that would be hard to break into — and they have the power and cooling.”
Equinix and Digital Reality assert they are well positioned at the edge given their strengths as global data center and network operators. “You can’t defy physics,” said Poole. “The telcos will do well — they can create access on the local level. There is no way to get around that.”
He adds, “Data center companies such as Equinix that have a highly distributed footprint will do well because we are where the applications are today.”
Adds Digital Realty’s Sharp: “You need a global platform or you will have a hard time being successful. Customers are very cautious about doing deals with point providers in single markets. If you’re not truly invested and have the wherewithal to support a global environment, you’re not going to win.
“Customers don’t want to manage 10 or 15 vendors to roll out an infrastructure,” Sharp adds.