On the feasibility of byzantine fault-tolerant mapreduce in clouds-of-clouds

Miguel Correia, Pedro Costa, Marcelo Pasin, Alysson Bessani, Fernando Ramos, Paulo Verissimo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

MapReduce is a framework for processing large data sets largely used in cloud computing. MapReduce implementations like Hadoop can tolerate crashes and file corruptions, but there is evidence that general arbitrary faults do occur and can affect the correctness of job executions. Furthermore, many individual cloud outages have been reported, raising concerns about depending on a single cloud. We present a MapReduce runtime that tolerates arbitrary faults and runs in a set of clouds at a reasonable cost in terms of computation and execution time. The main challenge is to avoid sending through the internet the huge amount of data that would normally be exchanged between map and reduce tasks. © 2012 IEEE.
Original languageEnglish (US)
Title of host publicationProceedings of the IEEE Symposium on Reliable Distributed Systems
Pages448-453
Number of pages6
DOIs
StatePublished - Dec 1 2012
Externally publishedYes

Fingerprint

Dive into the research topics of 'On the feasibility of byzantine fault-tolerant mapreduce in clouds-of-clouds'. Together they form a unique fingerprint.

Cite this