HotEdge '18 Workshop Program

Machine learning has changed the computing paradigm. Products today are built with machine intelligence as a central attribute, and consumers are beginning to expect near-human interaction with the appliances they use. However, much of the deep learning revolution has been limited to the cloud. Recently, several machine learning packages based on edge devices have been announced which aim to offload the computing to the edges. However, little research has been done to evaluate these packages on the edges, making it difficult for end users to select an appropriate pair of software and hardware. In this paper, we make a performance comparison of several state-of-the-art machine learning packages on the edges, including TensorFlow, Caffe2, MXNet, PyTorch, and TensorFlow Lite. We focus on evaluating the latency, memory footprint, and energy of these tools with two popular types of neural networks on different edge devices. This evaluation not only provides a reference to select appropriate combinations of hardware and software packages for end users but also points out possible future directions to optimize packages for developers.

In this paper, we propose a novel mobile deep inference platform, MODI, that delivers good inference performance. MODI improves deep learning powered mobile applications performance with optimizations in three complementary aspects. First, MODI provides a number of models and dynamically selects the best one during runtime. Second, MODI extends the set of models each mobile application can use by storing high quality models at the edge servers. Third, MODI manages a centralized model repository and periodically updates models at edge locations, ensuring up-to-date models for mobile applications without incurring high network latency. Our evaluation demonstrates the feasibility of trading off inference accuracy for improved inference speed, as well as the acceptable performance of edge-based inference.

Multi-camera real-time vision at the Edge is facilitated by low-latency distributed data stores. In this paper, we take the position that latency criticality in the challenging operating conditions at the Edge can only be attained through application specific designs incorporating autonomous computing techniques. In our initial prototype, we implement a key-value Edge data store that autonomously monitors run-time conditions to maintain latency-criticality of one class of data (feature vectors), while sacrificing the latency and accuracy of another class of data (keyframes). Early results show a median latency improvement of 84.8% over non-autonomous operation, for videos with large scene dynamics, and operational conditions of intermittent wireless channel interference.

Low-latency demands for cellular networks have attracted much attention. Mobile edge computing (MEC), which deploys a cloud computing platform at the edge closer to mobile users, has been introduced as an enabler of low-latency performance in 4G and 5G networks. In this paper, we propose an MEC platform deployment solution in 4G LTE networks using a middlebox approach. It is standard-compliant and transparent to existing cellular network components, so they need not be modified. The MEC middlebox sits on the S1 interface, which connects an LTE base station to its core network, and does traffic filtering, manipulation and forwarding. It enables the MEC service for mobile users by hosting application servers. Such middlebox approach can save deployment cost and be easy to install. It is different from other studies that require modifications on base stations or/and core networks. We have confirmed its viability through a prototype based on the OpenAirInterface cellular platform.

As more sensors and actuators are deployed in industrial manufacturing, the industry requires a new production system architecture that offers better real-time and network transmission performance. Yet cloud computing (based on a centralized datacenter) is limited in its possibilities, because it suffers from heavy bandwidth costs and lengthy time delays. As a solution, we propose an industrial robot system based on edge computing. Here, we present its three-layer architecture in detail: the cloud, edge, and physical resource layers. Initially, we deploy an edge node near the data sources, to integrate various devices’ interfaces and acts as a raw data filter. Then, we apply the proposed system on the robotic welding of the membrane wall cell. Finally, we test the system by conducting an experiment. The results demonstrate the system’s feasibility and prove that the system yields better real-time and network transmission performance than a cloud-based scenario.

Powerful edge compute frameworks address the issue of latency for IoT data processing at the edge. However, continuous application layer WAN streaming to core for consolidated deep analysis and learning consumes excessive bandwidth and becomes the bottleneck for responsiveness at the core. A state-of-the-art storage or hyperconverged system, on the other hand, advertises compelling in-built features like WAN efficient data protection and delta replication, global unified management, space and bandwidth saving through inline data compression and deduplication. Traditional storage semantics and services, however, are built for data at rest while edge analytics prioritizes responsiveness by processing and moving streaming data at an application layer. In this paper, we propose to enable streaming of IoT data transparently through storage replication. Based on this foundation, we further present light-weight storage plugins to reduce IoT data transfer by detecting and translating semantic redundancies to a deduplication friendly form. Our early results demonstrate that (a) leveraging storage to take responsibility of streaming data in an application consistent way results in efficient data transfer (b) real-world IoT time-series datasets exhibit a high degree of similarity which can be detected to reduce data transfer from edge (c) video streams for autonomous cars, the transfer of which cannot be reduced enough using traditional video compression or storage deduplication techniques, have significant semantic redundancy. Collectively, advancing research in this direction paves the way to enhance the versatility of state-of-the-art infrastructure for optimized edge computing.

Due to the strong potential of edge computing, service providers are aware of its excellent market opportunity. In this paper, we examine new pricing plans for edge-computing services that jointly consider communication and computing costs in mobile devices and edge servers. In our proposed model, users can freely choose to use: 1) a pay-as-you-go payment for communication and computing costs, 2) an upfront data plan for unlimited communication volume, 3) an upfront computing plan for unlimited computing workload, or 4) a combo plan for unlimited communication volume and computing workload. Based on this, we design a smart plan-purchasing scheme, namely smart online reservation (SOR) scheme for users and prove it achieves the best possible competitive ratio among all possible online deterministic schemes. With SOR adopted by users, we develop pricing plan guidelines for service providers to achieve profitability by motivating users to purchase the combo plan. We provide trace-driven simulations to verify the advantage of SOR and identify how different prices influence user behavior.

Extracting value from insights on unstructured data on the Internet of Things and Humans is a major trend in capitalizing on digitization. To date, the design space for doing AI inference on the edge has been highly binary: either consuming cloud-based inference services through edge APIs or running full-fledged deep models on edge devices. In this paper, we break this design space duality by proposing the Semantic Cache, an approach that blends best-of-breed features of the extreme ends of the current design space. Early evaluation results on a first prototype implementation of our semantic cache service on object classification tasks shows tremendous inference latency reduction, when compared to cloud-only inference, and high potential in scoring adequate accuracy for a plurality of AI use-cases.

The increasing adoption of the Internet of Things (IoT) paradigm in structuring services for “smart cities” has led to an explosion of data generated, ingested, processed, and stored in the system. Traditional time-series databases being Cloud-based are not suitable for meeting the low-latency requirement of such geo-distributed smart services. We propose DataFog, a geo-distributed data-management platform at the edge of the network to cater to the needs of smart services for the IoT age. We identify key challenges towards building such a database over a widely geo-distributed and heterogeneous edge computing environment. Preliminary evaluations show the performance potential of DataFog in comparison to state-of-the-art distributed data-stores. DataFog is the first system, to the best of the authors’ knowledge, that is meant for data-management at the network edge.