Parallel clustering algorithm for large-scale biological data sets

PLoS One. 2014 Apr 4;9(4):e91315. doi: 10.1371/journal.pone.0091315. eCollection 2014.

Abstract

Backgrounds: Recent explosion of biological data brings a great challenge for the traditional clustering algorithms. With increasing scale of data sets, much larger memory and longer runtime are required for the cluster identification problems. The affinity propagation algorithm outperforms many other classical clustering algorithms and is widely applied into the biological researches. However, the time and space complexity become a great bottleneck when handling the large-scale data sets. Moreover, the similarity matrix, whose constructing procedure takes long runtime, is required before running the affinity propagation algorithm, since the algorithm clusters data sets based on the similarities between data pairs.

Methods: Two types of parallel architectures are proposed in this paper to accelerate the similarity matrix constructing procedure and the affinity propagation algorithm. The memory-shared architecture is used to construct the similarity matrix, and the distributed system is taken for the affinity propagation algorithm, because of its large memory size and great computing capacity. An appropriate way of data partition and reduction is designed in our method, in order to minimize the global communication cost among processes.

Result: A speedup of 100 is gained with 128 cores. The runtime is reduced from serval hours to a few seconds, which indicates that parallel algorithm is capable of handling large-scale data sets effectively. The parallel affinity propagation also achieves a good performance when clustering large-scale gene data (microarray) and detecting families in large protein superfamilies.

Publication types

  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Algorithms*
  • Cluster Analysis*
  • Computational Biology / methods*
  • Databases, Protein / statistics & numerical data
  • Datasets as Topic / statistics & numerical data*
  • Gene Expression Profiling / statistics & numerical data
  • High-Throughput Screening Assays / statistics & numerical data
  • Humans
  • Microarray Analysis / statistics & numerical data
  • Multigene Family
  • Proteins / analysis
  • Proteins / metabolism
  • Proteomics

Substances

  • Proteins

Grants and funding

This research is partially supported by the Specialized Research Fund for the Doctoral Program of Higher Education [SRFDP 20113108120022], the Key Project of Science and Technology Commission of Shanghai Municipality [No. 11510500300], and the Major Research Plan of NSFC [No. 91330116]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.