81761869-a4fa-464b-af54-a15097b8b9d520220308062600219naun:naunmdt@crossref.orgMDT DepositInternational Journal of Computers and Communications2074-129410.46300/91013http://www.naun.org/cms.action?id=30503820223820221610.46300/91013.2022.16https://npublications.com/journals/cc/2022.phpUsing Intermediate Data of Map Reduce for Faster ExecutionShah PratikPrakashSchool of Computing Science and Engineering VIT University – Chennai Campus Chennai, IndiaPattabiramanV.School of Computing Science and Engineering VIT University – Chennai Campus Chennai, IndiaData of any kind structured, unstructured or semistructured is generated in large quantity around the globe in various domains. These datasets are stored on multiple nodes in a cluster. MapReduce framework has emerged as the most efficient technique and easy to use for parallel processing of distributed data. This paper proposes a new methodology for mapreduce framework workflow. The proposed methodology provides a way to process raw data in such a way that it requires less processing time to generate the required result. The methodology stores intermediate data which is generated between map and reduce phase and re-used as input to mapreduce. The paper presents methodology which focuses on improving the data reusability, scalability and efficiency of the mapreduce framework for large data analysis. MongoDB 2.4.2 is used to demonstrate the experimental work to show how we can store and reuse intermediate data as a part of mapreduce to improve the processing of large datasets.38202238202220264https://npublications.com/journals/cc/2022/a082012-004(2022).pdf10.46300/91013.2022.16.4https://npublications.com/journals/cc/2022/a082012-004(2022).pdf10.1007/s11227-012-0792-8A. Espinosa, P. Hernandez, J.C. Moure, J. Protasio and A. Ripoll . “Analysis and improvement of map-reduce data distribution in mapping applications”. Published online: 8 June 2012 JSupercomput (2012) 62:1305-1317. 10.1145/1967422.1967427Diana Moise, Thi-Thu-Lan Trieu, Gabriel Antoniu and Luc Bougé. “Optimizing Intermediate Data Management in MapReduce Computations”. CloudCP „11 April 10, 2011 Salzburg, Austria ACM 978-1-4503-0727-7/11/04. 10.14778/2168651.2168659Iman Elghandour and Ashraf Aboulnaga. “ReStore: Reusing Results of MapReduce jobs”. The 38th International Conference on Very Large Data Bases, August 27th 31st 2012, Istanbul, Turkey. Proceedings of the VLDB Endowment, Vol. 5, No. 6. J. Dean and S. Ghemawat. “MapReduce: Simplified data processing on large clusters”. In Proc. OSDI, pages 137-150. 2004. 10.1109/fpt.2009.5377629Qiang Liu, Tim Todman, Wayne Luk and George A. Constantinides. “Automatic Optimisation of MapReduce Designs by Geometric Programming”. FPT 2009 IEEE. 10.11591/closer.v2i1.1508Rabi Prasad Padhy. “Big Data Processing with Hadoop-MapReduce in Cloud Systems”. International Journal of Cloud Computing and Services Science (IJ-CLOSER) Vol.2, No.1, February 2013, pp. 16~27 ISSN: 2089-3337. 10.1109/icde.2012.104Raman Grover and Michael J. Carey. “Extending Map-Reduce for Efficient Predicate-Based Sampling”. 2012 IEEE 28th International Conference on Data Engineering. 10.1109/syscon.2012.6189510Ruxandra Burtica, Eleonora Maria Mocanu, Mugurel Ionut, Andreica, and , Nicolae Ţăpuş. “Practical application and evaluation of no-SQL databases in Cloud Computing”. 2012 IEEE. This paper is under research grants PD_240/2010 (AATOMMS - contract no. 33/28.07.2010) from the PN II - RU program and ID_1679/2008 (contract no. 736/2009) from the PN II - IDEI program.