MCP: FAST Climate Science

The FAST project is addressing a number of challenges related to the use of Big Data in climate science. The large volumes of data are increasing the turnover time for analysis and hypothesis testing. Researchers are also throwing away valuable model data that could provide useful insights because of the perception that data storage volumes are limited by cost and/or availability. In this project, we are developing a Big Data platform support and tools to eliminate data triaging and to vastly reduce the time required from asking questions to getting answers, thus improving researcher productivity.

 

Another area of work is to explore Deep learning on gridded climatological reanalysis datato derive an alternative way of parameterizing the effect of radiation on the climate system in Earth System Models. Radiation calculations from first principles are one of the most costly andthis new approach has the potential to reduce long simulation times on 10s of thousands of CPUs, saving large amounts of energy.

 

The final area of work we are addressing is the sharing of climate science data. We have worked on peer-to-peer technology for the efficient sharing of multi-TB datasets between Hadoop clusters. We currently host a number of large datasets that are available for researchers globally to discover and download with a few mouse clicks.

 

This work is available at www.hops.site. Our primary source code repository is available on Github at: https://github.com/ClimateFAST

 

People Involved

Pis: Gunilla Svensson*, Rodrigo Caballero* and Jim Dowling#

 

Postdocs, students: Ying Liu*, Joy Monteiro* and Lars Kroll#

 

* Stockholm University

# KTH