CSIRO's GPU cluster being installed at the Data Centre.

Speeding up science: CSIRO's CPU-GPU supercomputer

CSIRO's latest supercomputer cluster is among the world's first to combine traditional CPUs with more powerful graphics processing units or GPUs, providing a world class computational and simulation science facility to advance priority CSIRO science.

• 19 November 2010 | Updated 14 October 2011

• CSIRO's GPU cluster
• GPU technology
• Driving computational and simulation science
• CSIRO CUDA Research Centre
• E-research agenda

Scientific research is constantly generating vast quantities of highly complicated data. Handling and processing this information requires advances in data management, analytics, computing and collaboration tools. 

Graphics processing units (GPUs) have typically been used to render complex graphics in computer games. However they can also be used to accelerate scientific computing by multi-tasking on hundreds of computing cores.

Using GPUs to analyse large and complex research data sets is a relatively new global trend in computing.

CSIRO aims to be at the forefront of GPU super computing for scientific and industrial research. Our investment will enable the technology to be applied across CSIRO from desktop GPU workstations to our GPU supercomputer cluster, helping to solve big data challenges in many research fields from bioscience, image analysis, fluid dynamics modelling and environmental science.

CSIRO's GPU cluster

CSIRO’s GPU cluster was the first of its kind in Australia combining traditional central processing units (CPUs) with graphics processing units (GPUs) when it launched in 2009.

Running dual Linux and Windows applications it caters for the needs of scientists who use different operating systems for running software codes and analysing data from experiments.

A recent collaboration with Microsoft to optimise the Windows HPC operating system on the cluster has yielded a performance of 75.3 Teraflops in double precision running the Linpack benchmark test.

It is currently one of the fastest supercomputers running a Windows HPC operating system in the world.

Ranking in the TOP500 list of the world's fastest and most powerful supercomputers in 2010 and 2011, the GPU supercomputer is also ranked highly on the Green500 list - a ranking determined by energy efficiency, making it the most energy efficient supercomputer in Australia.

The GPU cluster has:

• 128 Dual Xeon E5462 Compute Nodes (i.e. a total of 1024 2.8GHz compute cores) with 16 GB or 32 GB of RAM, 500 GB SATA storage and DDR InfiniBand interconnect
• 64 NVIDIA Tesla S2050s (256 GPUs with a total of 114 688 streaming processor cores)
• 144 port DDR InfiniBand Switch
• 80 Terabyte Hitachi NAS file system.

It was supplied by Xenon Systems of Melbourne and is located at a data centre in Canberra, Australia.

GPU technology

Watch this video about the CSIRO GPU cluster.

The unique feature of the CSIRO GPU cluster is the use of NVIDIA GPU technology to deliver outstanding computational performance at low cost and low energy demand.

Compared to the latest quad-core CPUs, the Tesla C2050s inside the CSIRO GPU cluster deliver the equivalent supercomputing performance at 1/10th the cost and 1/20th the power consumption.

The GPU technology can be accessed using the CUDA parallel computing architecture [external link] or using new compiler technology released by the Portland Group [external link].

CSIRO science applications have already seen 6-200x speedups on NVIDIA GPUs.

Driving computational and simulation science

The CSIRO GPU supercomputer supports computational and simulation science research across CSIRO.

The technology is used to support CSIRO research in the areas of:

• computational biology
• climate and weather
• multi-scale modelling
• computational fluid dynamics
• computational chemistry
• astronomy and astrophysics
• computational imaging and visualisation
• advanced materials modelling
• computational geosciences.
  

CSIRO projects being undertaken using the cluster include:

• high-content analysis of nerve cell images for medical research and drug discovery
• deconvolving (un-blurring) three-dimensional (3D) images from astronomy, microscopy and medical imaging
• reconstructing large 3D computer tomography (CT) images from medicine and materials science and developing specialised software to run on the Australian Synchrotron
• quantifying uncertainty in complex environmental models
• simulating in high resolution biomechanical processes like the movements of a person swimming.

CSIRO CUDA Research Centre

In 2010 CSIRO was selected as part of NVIDIA’s prestigious CUDA Research Centre Program [external link]. We join a community of research institutions that are embracing GPU computing across multiple research fields including; Johns Hopkins University in the Unites States, Nanyan University in Singapore, Technical University of Ostrava in the Czech Republic, and SINTEF (Norway).

As a CUDA Research Centre, CSIRO gains access to the latest developments in GPU technology and is part of a wider community of organisations with GPU facilities, sharing information and ideas.

E-research agenda

E-research underpins the future delivery of great science.

Modern scientific research is increasingly generating vast quantities of highly complicated data. Making the most of the rich information it contains is the key to success.

Handling and interrogating this information requires advances in data management, computing and collaboration tools – of which the CSIRO GPU Supercomputer is one.

Our e-research agenda is supporting CSIRO’s Transformational Capability Platforms, building our capability to sustain and accelerate the delivery of solutions for national challenges.

The Platforms are:

• computational and simulation science
• transformational biology
• sensors and sensor networks
• advanced materials.

To find out more about CSIRO's Central Processing Unit (CPU)-GPU supercomputer cluster and its capabilities contact: