Soft quorums: A high availability solution for service oriented stream systems

Abstract

Large-scale information gathering becomes more and more common with the increasing popularity of smartphones, GPS, social networks, and sensor networks. Services based on this real-time data are the logical next step. Service Oriented Stream Systems (SOSS) have a focus on one-time ad hoc queries as opposed to continuous queries. High availability is crucial in these services. However, data replication has inherent costs, which are particularly burdensome for high rate, often overloaded, SOSS. To provide high availability and to cope with the problem of overloading the system, we propose a mechanism called soft quorums. Soft quorums provide high availability of data, a tradeoff between query result accuracy and performance, and adaptation to dynamic data/query stream rates. Finally, we conduct a comprehensive experimental study using real-world and synthetic datasets.