Molecular: Distributed computing simulation

The amount of compute resources has grown vastly in the recent years, however they are underutilized but there are many problems that can put them into good use. As an example within the field of molecular dynamics, adaptive sampling algorithms such as Markov State modeling or Free energy calculations constitute of many short(100-1000) simulations to gather statistics followed by iterations of adaptive sampling in order to guide simulations for the coming iteration. Although it is a simple workflow to define, these type of problems can easily utilize thousands of cores and generate massive amounts of data and require something more than a queue. Copernicus is a platform that enables large scale computing to be defined as a workflow.

The platform will take care of the task breakdown, distribute it to available compute resources, all in a secure and fault tolerant manner.

Its overlay P2P network can utilize a wide variety of heterogeneus compute resources such as desktops,clusters and cloud com- pute instances and automatic resource awareness makes sure to to use the best resources for the defined job.

As a proof of concept the folding of the vil- lin headpiece was performed by combining molecular simulations with Markov state modelling for kinetic clustering and statisti- cal model building. This combination made it possible to identify the native folded state without any prior knowledge within 46 hours utilizing a total of 5736 cores on a Cray XT6 and an AMD Istanbul cluster. By being able to combine simulations with statistical mod- el building parallelization was achieved on a fine grained level and on an algorithmical level resulting in much stronger scaling. The Copernicus platform is built in a general manner and its plugin architecture makes it possible to enable any executable to be utilized in a workflow making it ideal for any type of large scale statistical computations and data processing.

Project link