Cache and priority queue based approximation technique for a stream of similarity search queries

Abstract

Content-based similarity search techniques have been employed in a variety of today applications. In our work, we aim at the scenario when the similarity search is applied in the context of stream processing. In particular, there is a stream of query objects which need to be evaluated. Our goal is to be able to cope with the rate of incoming query objects (i.e., to reach sufficient throughput) and, at the same time, to preserve the quality of the obtained results at high levels. We propose an approximation technique for the similarity search which combines the probability of an indexed object to be a part of a query result and the time needed to examine the object. We are able to achieve better trade-off between the efficiency (processing time) and the quality (precision) of the similarity search compared to traditional priority queue based approximation techniques.