MRSG – a MapReduce simulator over SimGrid

Kolberg, Wagner; Marcos, Pedro de Botelho; Anjos, Julio Cesar Santos dos; Miyazaki, Alexandre Kenta Salgueiro; Geyer, Claudio Fernando Resin; Arantes, Luciana Bezerra


MapReduce is a parallel programming model to process large datasets, and it was inspired by the Map and Reduce primitives from functional languages. Its first implementation was designed to run on large clusters of homogeneous machines. Though, in the last years, the model was ported to different types of environments, such as desktop grid and volunteer computing. To obtain a good performance in these environments, however, it is necessary to adapt some framework mechanisms, such as scheduling and data distribution algorithms. In this paper we present the MRSG simulator, which reproduces the MapReduce work-flow on top of the SimGrid simulation toolkit, and provides an API to implement and evaluate these new algorithms and policies for MapReduce. To evaluate the simulator, we compared its behavior against a real Hadoop MapReduce deployment. The results show an important similarity between the simulated and real executions.

Show full item record


Files in this item

This item appears in the following Collection(s)


  • C3 - Artigos Publicados em Periódicos