(Sync|Async)+ MPI search engines

Abstract

We propose a parallel MPI search engine that is capable of automatically switching between asynchronous message passing and bulk-synchronous message passing modes of operation. When the observed query traffic is small or moderate the standard multiple managers/workers thread based model of message passing is applied for processing the queries. However, when the query traffic increases a roundrobin based approach is applied in order to prevent from unstable behavior coming from queries demanding the use of a large amount of resources in computation, communication and disk accesses. This is achieved by (i) a suitable object-oriented multi-threaded MPI software design and (ii) an "atomic" organization of the query processing which allows the use of a novel control strategy that decides the proper mode of operation. © Springer-Verlag Berlin Heidelberg 2007.