MURAL - Maynooth University Research Archive Library



    CodHoop: A system for optimizing big data processing


    Asad, Zakia and Asad Rehman Chaudhry, Mohammad and Malone, David (2015) CodHoop: A system for optimizing big data processing. In: 2015 Annual IEEE Systems Conference (SysCon) Proceedings, 13-16 April 2015, Vancouver, BC.

    [img]
    Preview
    Download (262kB) | Preview


    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...



    Add this article to your Mendeley library


    Abstract

    The rise of the cloud and distributed data-intensive (“Big Data”) applications puts pressure on data center networks due to the movement of massive volumes of data. This paper proposes CodHoop a system employing network coding techniques, specifically index coding, as a means of dynamically-controlled reduction in volume of communication. Using Hadoop as a representative of this class of applications, a motivating use-case is presented. The proof-of-concept implementation results exhibit an average advantage of 31% compared to vanilla Hadoop implementation which depending on use-case translates to 31% less energy utilization of the equipment, 31% more jobs that run simultaneously, or to a 31% decrease in job completion time.

    Item Type: Conference or Workshop Item (Paper)
    Keywords: CodHoop; Big Data processing; cloud applications; distributed data-intensive applications; data center networks; Hadoop; proof-of-concept implementation;
    Academic Unit: Faculty of Science and Engineering > Mathematics and Statistics
    Faculty of Science and Engineering > Research Institutes > Hamilton Institute
    Item ID: 10066
    Depositing User: Dr. David Malone
    Date Deposited: 05 Oct 2018 13:34
    Refereed: Yes
    Funders: Science Foundation Ireland (SFI)
    URI:

      Repository Staff Only(login required)

      View Item Item control page

      Downloads

      Downloads per month over past year