Having demonstrated the scalability and energy efficiency of PRESTO, we next evaluate its adaptation to query and data dynamics. In general, adaptation only changes what the sensor does for future data and not for past data. Our experiments evaluate adaptation for queries that request data from the recent past (one hour).
|
In our first experiment, we run PRESTO for 12 hours. Every two hours,
we vary the mean of the distribution of query precision requirements
thereby varying the query error tolerance. The proxy tracks
the mean of the query distribution and notifies the sensor if the mean
changes by more than a pre-defined threshold, in our case, 10. Figure
6(a) shows the adaptation to the query distribution
changes. Explicit feedback from the proxy to each sensor enables the
system to vary the 
corresponding to the changes in query
precision requirements. From the figure, we can see that there is a
spike in average query latency and the energy cost every time the
query confidence requirements become tighter. This results in greater
query miss rate and hence more pulls as shown in
Figure 6(a). However, after a short period, the proxy
provides feedback to the sensor to change the pushing threshold, which
decreases the query miss rate and consequently, the average
latency. The opposite effect is seen when the query precision
requirements reduce, such as at the 360 minute mark in
Figure 6(a). As can be seen, the query miss rate reduces
dramatically since the model at the proxy is too precise. After a
while, the proxy provides feedback to the sensors to increase the push
threshold and to lower the push rate. A few queries result in pulls as
a consequence, but the overall energy requirements of the system
remains low. In comparison with a non-adaptive version of PRESTO that
kept a fixed , our adaptive version reduces latency by more
than 50%.
In our second experiment, we demonstrate the benefits of adaptation to data dynamics. PRESTO adapts to data dynamics by model retraining, as described in Section 5. We use a four day dataset, and at the end of each day, the proxy retrains the model based on the pushes from the sensor for the previous day, and provides feedback of the new model parameters to the sensor. Our result is shown in Figure 6(b). For instance, on day three, the data pattern changes considerably and the communication cost increases since the model does not follow the old patterns. However, at the end of the third day, the PRESTO proxy retrains the model and send the new parameters to the sensors. As a result, the model accuracy improves on the second day and reduces communication. The figure also shows that the model retraining reduces pushes by as much as 30% as compared to no retraining.
While most of our experiments involved the use of temperature traces as a substitute of live temperature sampling, we conducted a number of experiments with a live outdoor deployment of PRESTO using one proxy and four sensors. These experiments corroborate our findings from the trace-driven testbed experiments. The result of one such experiment is shown in Figure 6(c). The figure shows that, over a period of three days, as the model adapts via retraining, the frequency of pulls as well as the total frequency of pushes and pulls falls.
Summary: Feedback from the proxy enables PRESTO to adapt to both data as well as query dynamics. We demonstrate that the query-adaptive version of PRESTO reduces latency by 50%, and the data-adaptive version reduces the number of messages by as much as 30% compared to their non-adaptive counterparts.
