NOAA Open Data Dissemination (formerly NOAA Big Data Project / Program)

The National Oceanic and Atmospheric Administration (NOAA) research to operations (R2O) experiment called the Big Data Project (BDP) was envisioned as a scalable approach for disseminating exponentially increasing NOAA observation, model, and research datasets to the public using commercial cloud services. At the start of the project, during the concept development phase, it was unclear how the specifics might work so a spiral development approach was adopted. It was expected that the number of data sets would increase, and the data extent would grow to cover complete records of some holdings, and that format experimentation would be needed to determine optimal cloud offerings. This dissemination model would require a new way for the BDP and NOAA to engage with end-users, who could range from large enterprises to small businesses and individuals. The BDP was expected to change the game-not just by reaching a broad and diverse set of users but by encouraging new ones. As Dr. Kathy Sullivan, former NOAA Administrator under whom the BDP began, noted, “The agency’s aim is to ‘spur innovation’ and to explore how to create a ’global economic return on investment” (Konkel, 2015). This Chapter describes the journey of BDP as it developed, transitioned and evolved from an experiment to an operational enterprise function for NOAA, now known as NOAA Open Data Dissemination (NODD). Obstacles to the Public’s Use of NOAA Environmental Data NOAA’s mission is to understand and predict changes in climate, weather, oceans, and coasts, to share that knowledge and information with others, and to conserve and manage coastal and marine ecosystems and resources. The agency takes seriously the need for communication of NOAA’s research, data, and information for use by the Nation’s businesses and communities to allow preparation, response and resilience to sudden or prolonged changes in our natural systems. This includes climate predictions and projections; weather and water reports, forecasts and warnings; nautical charts and navigational information; and the continuous delivery of a range of Earth observations and scientific data sets for use by public, private, and academic sectors (NOAA About our agency, 2021).