A result verification scheme for mapReduce having untrusted participants

Abstract

MapReduce framework is a widely accepted solution for performing data intensive computations efficiently. The master node prepares the input to be distributed among multiple mappers which distribute the reduced task to the reducers. Reducers perform identical set of computations on the reduced data independently. If any one of the reducers works maliciously and does not produce results as desired by the end-user, a significant error in the final output can be observed. Many other distributed computing platforms also face the same problem due to the malicious participants. The problem for MapReduce must be solved keeping into account the data intensive nature of the computations carried out by MapReduce. MapReduce does not provide any mechanism to detect such Lazy Cheating Attacks by a computation provider. In this paper, we propose a generalized defense to this type of attack on statistical computations. The solution does not involve redundant computations on the data to prove the worker malicious. Implementation results on Hadoop show the detection rate of such cheating behavior by the proposed scheme. The accompanying theoretical analysis proves that the solution does not noticeably affect the timeliness and accuracy of the original service.