Parallelization of Genome Sequence Data Pre-Processing on Big Data and HPC Framework


KIPS Transactions on Computer and Communication Systems, Vol. 8, No. 10, pp. 231-238, Oct. 2019
10.3745/KTCCS.2019.8.10.231,   PDF Download:  
Keywords: Genome Sequence Data Preprocessing, NGS, Big data, Hadoop, HPC, parallelization
Abstract

Analyzing next-generation genome sequencing data in a conventional way using single server may take several tens of hours depending on the data size. However, in order to cope with emergency situations where the results need to be known within a few hours, it is required to improve the performance of a single genome analysis. In this paper, we propose a parallelized method for pre-processing genome sequence data which can reduce the analysis time by utilizing the big data technology and the high- performance computing cluster which is connected to the high-speed network and shares the parallel file system. For the reliability of analytical data, we have chosen a strategy to parallelize the existing analytical tools and algorithms to the new environment. Parallelized processing, data distribution, and parallel merging techniques have been developed and performance improvements have been confirmed through experiments.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
E. Byun, J. Kwak, J. Mun, "Parallelization of Genome Sequence Data Pre-Processing on Big Data and HPC Framework," KIPS Transactions on Computer and Communication Systems, vol. 8, no. 10, pp. 231-238, 2019. DOI: 10.3745/KTCCS.2019.8.10.231.

[ACM Style]
Eun-Kyu Byun, Jae-Hyuck Kwak, and Jihyeob Mun. 2019. Parallelization of Genome Sequence Data Pre-Processing on Big Data and HPC Framework. KIPS Transactions on Computer and Communication Systems, 8, 10, (2019), 231-238. DOI: 10.3745/KTCCS.2019.8.10.231.