Optimizing storage intensive vision applications to device capacity

Abstract

Computer vision applications today run on a wide range of mobile devices. Even though these devices are becoming more ubiquitous and general purpose, we continue to see a whole spectrum of processing and storage capabilities within this class. Moreover, even as the processing and storage capacity of devices are increasing, the complexity of vision solutions and the variety of use cases create greater demands on these resources. This requires appropriate adaptation of the mobile vision applications with minimal changes in the algorithm or implementation. In this work, we focus on optimizing the memory usage for storage intensive vision applications. In this paper, we propose a framework to configure memory requirements of vision applications. We start from a gold standard desktop application, and reduce the size for a given the memory constraint. We formulate the storage optimization problem as mixed integer programming (mip) based optimization to select the most relevant subset of data to be retained. For large data sets, we use a greedy approximate solution which is empirically comparable to the optimal mip solution. We demonstrate the method in two different use cases: (a) Instance retrieval task where an image of a query object is looked up for instant recognition/annotation, and (b) Augmented reality where computational requirement is minimized by rendering and storing precomputed views. In both the cases, we show that our method allows a reduction in storage by almost 5× with no significant performance loss.