Big data: Science in a petabyte era

Abstract

In Nature this week, features and opinion pieces on one of the most daunting challenges facing modern science: how to cope with the flood of data now being generated. [Editorial/Introduction p. 1] A petabyte is a lot of memory, however you say it - a quadrillion, 1015, or tens of thousands of trillions of bytes. But that is the currency of 'big data'. We visited the Sanger Institute's supercomputing centre, and its petabyte of capacity. [News Feature p. 16] Wikipedia's success shows how well the 'wiki' concept of open-access editing can work. It could work too as a way of coping with the data flows of modern biology. [News Feature p. 22] The world’s leading search engine is ten this month. Eleven years ago few would have predicted Google’s domination: undaunted we ask scientists and business people to try to predict the next big thing, a Google for the petabyte era. [Special Report p. 8] Digital data are easily shared, and just as easily wiped or lost. The problem of keeping on-line data accessible is especially difficult for the smaller lab. [Commentary p. 28] In Books & Arts, Felice Frankel and Rosalind Reid champion the cause of data visualization as a way of finding meaning in an otherwise daunting data stream. [Books & Arts p. 30] From the 1700s to the mid 1950s, most 'computers' were human. Best known were the 'Harvard computers', a group of women working from the 1880s until the 1940s, at the Harvard College Observatory. Employed to classify stars captured on millions of photographic plates, some of the ‘computers’ made significant contributions to science. [Essay p. 36] Online databases are a vital outlet for publishing the data being produced by biological research. But the data need to be properly organized. This is the role of the biocurator, but as a team of authors from 15 of the world’s major online research resources explains, biocuration is now sadly neglected. [Feature p. 47] An aspect of the data boom with a political dimension is the environment: how much data to collect, how much money to spend. [Party of One p. 15] For 'Big data' online, go to www.nature.com/news/specials/bigdata/ and to www.nature.com/podcast.