Edge computing is rapidly gaining momentum. Last week’s Linux Foundation news on the launch of LF Edge is one more sign of accelerating growth at the edge. What’s driving this sudden intense interest? It’s more than just the sheer number of edge devices or mounting concerns around interoperability and security. Pushing compute resources closer to the edge can save the cloud and networks from waves of ephemeral data.
The concept of edge computing certainly isn’t new. In years BC (“before connectivity”), there were plenty of edge computing boxes. These featured single-board computers designed to specifications such as VMEbus, STDbus, PC/104, CompactPCI, AdvancedTCA and many others. Processing happened at the edge as data arrived. Ethernet connections made administration and results-sharing easier, but very few architectures relied on shipping all the data into the network.
Of course, there was one big exception: a business model pulling all the data into the network. Telecom providers aggregated traffic from the edge into their core networks. As data traffic joined voice traffic, core networks grew with monstrous bandwidth and increased services. Core networks interconnected. Data traffic rose even faster, pushing the network into a cloud architecture. In years AD (“after distribution”), everything from the edge is now connecting to the cloud.
Much of the increase in data traffic came from the shift to video content. Users expect higher resolution video programming with the same lag-free performance. Streaming networks and quality of service (QoS) implementations have made great strides in content delivery. 5G infrastructure rollouts in progress will keep mobile a strong downstream contender.
What about upstream applications such as the IoT? Nearly the same terminology applies there. Devices gather data on the edge and transmit it to the cloud through a gateway. There are two key differences with most IoT applications, however. Real-time data may be lost if the streaming connection into the cloud is interrupted. Also, the fallacy of low bandwidth does not work at scale if all the data is transmitted. At some point—perhaps ten thousand, a hundred thousand, or a million or more sensors—an aggregate bandwidth problem develops.
Just as in the BC age, it’s time to rethink shipping all that IoT data upstream, instead taking steps at the edge. Cisco Systems has studied data trends extensively, producing its white paper, “Cisco Global Cloud Index: Forecast and Methodology, 2016-2021.” The company points to the IoT ushering in the “yottabyte era” very soon. Cisco estimates that 220ZB of data was generated in 2016 and projects that by 2021, there will be nearly 850ZB from people, machines and things combined.
Now for the interesting part: Cisco also estimates that as much as 90 percent of that data is ephemeral, short-lived data not useful to save. Even if only 10 percent of data is useful, it will swamp projected data center traffic by an estimated factor of four in 2021. Here’s Figure 24 from the Cisco white paper:
Edge computing, or its close cousin fog computing, allows two things to happen: It can pre-process ingested data, ironing out protocols and format differences such that everything arriving at the cloud is immediately ready for algorithms. Any savings in the cloud from not shuffling data before computation are valuable, increasing processing and network bandwidth availability. Parts of an algorithm could easily be farmed out toward the edge, perhaps decimating or filtering data before shipping it upstream for heavy analysis.
A bigger use for edge computing may be using algorithms to avoid shipping ephemeral data upstream altogether. Using generative models at the edge could make a huge difference in identifying useful data. Most of us have heard that no human can sit and watch hours and hours of surveillance video from multiple sources looking for one moment of a problem developing. Consider another example with an EKG monitor attached to a smartphone: While the patient is at home, at rest and unstressed, the EKG is in a perfectly normal range as learned by the application. Doctors are alerted when readings slip out of normal range, with raw data transmitted for observation if required. A similar example is waiting for a motor to fail while monitoring its operation.
Both the LF Edge initiative and work happening in the OpenFog Consortium are creating specifications for edge computing. These initiatives are aimed at interoperable, secure architectures where cooperative processing mines data. If items of interest can be extracted efficiently from useful and ephemeral data at the edge, loads on the cloud and networks would be lighter. Edge computing will likely detect exceptions more quickly, bringing people in to look at what is going on faster. The result will be more scalable IoT applications with better chances of delivering value from all that data.
