Abstract
Conference Title: 2014 IEEE Long Island Systems, Applications and Technology Conference (LISAT) Conference Start Date: 2014, May 2 Conference End Date: 2014, May 2 Conference Location: Farmingdale, NY, USA Explosion of biological data due to large-scale genomic research and advances in high throughput data generation tools result in massive distributed datasets. Analysis of such large non-relational, heterogeneous, and distributed datasets is emerging challenge in data driven biomedical industries. Highly complex biological data require unconventional computational approaches and knowledge-based solutions. Distributed datasets need to be reduced to smaller datasets that can be efficiently queried. Since genomic and biological data is generated in large volume and is stored in geographically diverse locations, distributed computing on multiple clusters, our objective here is to assess the feasibility of using Cloud based platform to analyze genomic big data. In this paper we present an enhanced Hadoop architecture to reduce computation by utilizing "Common Features" before performing redundant computation. The enhanced Hadoop architecture allows the jobs to share these common features among them. The common features describe the contents of data in blocks and can be used to determine DataNodes that store the required data. [PUBLICATION ABSTRACT]