The team of experts compared data needs of genomics with three of the biggest players in big data: astronomy, Twitter and YouTube. They projected growth in each area through the year 2025 and found that genomics is poised to be a leader in data acquisition, storage, distribution and analysis.
The team’s assessment is published in the journal PLOS Biology.
«As
«Astronomy, Twitter and YouTube represent three diverse domains that generate and use a huge amount of data, albeit with huge differences in computing needs. The diversity of these three forms of Big Data provides an excellent framework for comparative analyses with genomics," he said.
Like YouTube and Twitter, genomics data are highly distributed, coming from many different sources. However, both Twitter and YouTube have standard formats for their entries, while genomic data can assume many different formats, making sharing and storing more complex.
The authors estimate that the genomics information so far, from sequencing different organisms and a number of humans, has produced data on the petabyte scale (a petabyte is a million gigabytes). However, over the last decade, genomic sequencing data doubled about every seven months, and will grow at an even faster rate as personal genome sequencing becomes more widespread. The researchers estimate that by 2025, genomics data will explode to the exabyte scale — billions of gigabytes. This surpasses even YouTube, the current title holder among the domains studied for most data stored.
Yet the sequences are only one element of genomic data.
«The DNA sequence in itself is not particularly useful for realizing all the great possibilities that genomics technology promises," said
The need for complex analysis is similar to astronomy, but with an important difference, the authors say. Astronomy generates vast amounts of data but incorporates several processing technologies at the time of data collection, requiring less time and computational power later on. The researchers suggest that integrating similar processing methods could cut down on the storage needs for genomic data as well. But there’s a catch: The whole genome may offer insights not yet anticipated, as new understandings may emerge as more people are sequenced.
«In the future, we may have to take the hard decision of storing only the processed form and not the original, and that, too, in heavily compressed forms, to drastically reduce the storage needs," Sinha said.
The authors urge new technology development to handle the expected explosive growth in genomics data beyond what is predicted for social media and astronomy.
«Genomics will soon pose some of the most severe computational challenges that we have ever experienced," Robinson said. «If genomics is to realize the promise of having a transformative positive impact on medicine, agriculture, energy production and our understanding of life itself, there must be dramatic innovations in computing. Now is the time to start.»
Editor’s note: To reach Gene Robinson, call
To reach Saurabh Sinha, call
http://news.illinois.edu/news/15/0707genomics_data_generobinson_saurabhsinha.html
