A map-task view Generation strategy Based on Rough Set Theory

Yiyi Xu, Peihe Tang and ZeKun

Abstract

When MapReduce process mass-data,it highly abstracted parallel computing process on large clusters into two functions (Map and Reduce). so pre-organization input dataset and generating Map task view is a key step for processing.. In this paper, iterative reduction on the existing complex, large-scale task set based on rough set knowledge, get sub views equivalence class task after the update, calculate the optimal Properties based on the set with minimal time overhead, according to the optimal attribute set to delete redundant view, Finally the task combination view for parallel processing obtained after optimized,. Simulation results show that, compared with the reduction before optimization, MapReduce algorithm avoids unnecessary complexity in dealing with the same task, the running time and efficiency are better promotion, show the effectiveness of the method.