Header menu link for other important links
X
GraphIVE: Heterogeneity-Aware Adaptive Graph Partitioning in GraphLab
Published in Institute of Electrical and Electronics Engineers Inc.
2015
Volume: 2015-May
   
Pages: 95 - 103
Abstract
GraphLab, distributed graph-processing framework, has found multiple applications in data-mining. Its scalability makes it the perfect choice for running graph algorithms on large data. The current scheduler in GraphLab splits the graph based on various partitioning strategies. These strategies split the graph into approximately equal parts, which is suited for homogeneous clusters, but is liable to perform poorly in the presence of heterogeneity. A number of challenges arise when the nodes differ in memory and processing power. We show that memory in particular can be a severe bottleneck, even leading to the termination of certain jobs. We determine the extent to which the current scheduler can handle heterogeneity. We further propose GraphIVE (Graph Processing In Varied Environments), a capability-aware graph partitioning policy for GraphLab applications. Moreover, GraphIVE continously tries to reach optimum performance via hill climbing. We describe how GraphIVE reduces the communication overhead by reducing the replication factor of vertices. We implemented a prototype of GraphIVE and present the preliminary results. GraphIVE significantly improves the execution time of jobs. The results also show how it allows for seamless graph processing on a heterogeneous cluster. © 2014 IEEE.
About the journal
JournalData powered by TypesetProceedings of the International Conference on Parallel Processing Workshops
PublisherData powered by TypesetInstitute of Electrical and Electronics Engineers Inc.
ISSN15302016
Open AccessNo
Concepts (12)
  •  related image
    Data mining
  •  related image
    Graphic methods
  •  related image
    Scheduling
  •  related image
    Communication overheads
  •  related image
    Heterogeneity
  •  related image
    HETEROGENEOUS CLUSTERS
  •  related image
    Hill climbing
  •  related image
    Multiple applications
  •  related image
    PARTITIONING STRATEGIES
  •  related image
    REPLICATION FACTORS
  •  related image
    VERTEX-CUT
  •  related image
    Graph theory