The digital universe that lives inside a supercomputer

The institute has been building digital universes inside supercomputers for a decade.

Its current supercomputer, known as the Cosmology Machine or COSMA 4,  has about 3,000 processor cores and more than 13 TB of RAM. It is seven times faster than their previous supercomputer, COSMA3, and was built by high-performance computer specialist OCF and IBM.

Storage presents a headache for the organisation, - unsurprising perhaps given a computer model of the effect of dark matter on galaxy formation can produce 100TB of data in a single run. Having filled up two thirds of its 612TB storage that came with Cosma 4 the institute recently upgraded to 1.1PB. This shot shows IBM System x iDataPlex server installed racks.

Overall Cosma 4 relies on 220 IBM iDataPlex dx360 M3 servers, with a total of 2640 2.67 GHz Intel X5650 cores and 13.2TB of RAM. It also uses two IBM x 3850 machines, with 32 2GHz 7550 cores and 512MB of RAM each.

Photo: Durham University

When studying something as vast as the universe, computer models provide one of the few ways that researchers can test their theories about the origins of the cosmos.

Frenk said: "Unlike other sciences it is very difficult to 'test' theories on the Universe. Brain power alone is not enough to calculate the complex algorithms.

"Computing power today is changing the way in which we do science. In my own discipline, which is cosmology and astrophysics we are completely reliant on computing, which has opened up completely new methods of looking at the universe."

This is a shot of IBM System x iDataPlex server cabling.

Photo: Durham University

"The level of detail in the models we use is continually increasing, so there's a perpetual need for more processor power and data storage capacity," said Dr Lydia Heck, senior computer manager at Durham University.

"Just one of our new research projects will consume the whole processing power of COSMA4 for half of a year. 288 extra cores are very welcome because we have other smaller projects, which can be serviced alongside the major research."

The size and complexity of the galaxies that can be simulated by the institute is limited by the memory of the current generation of 64-bit supercomputers, and the time it takes them to process data.

To work within these limitations computer models are simplified, such as by losing some of the detail of what is being modelled - for instance, choosing to model the formation of millions of stars at a time rather than trying to model the formation of each one of the hundreds of billions of stars within a galaxy.

To refine the models, researchers use a process of elimination, comparing the characteristics of galaxies produced by the computer model with observations of galaxies taken by astronomers to see if the computer model is a good fit for real life.

This shot shows Cosma 4's storage in a rack.

Photo: Durham University

A computer model showing the structure of the universe one billion years after the Big Bang, the green swirls represent dark matter and the circles represent growing galaxies.

The institute is focusing on two main projects, one to model galaxy formation from a time soon after the Big Bang until present day, the second to model the gas outside of galaxies, particularly in clusters and groups of galaxies.

The ultimate goal of this modelling is to produce galaxies whose characteristics match measurements of real life galaxies taken by astronomers, and then use those models to help understand the processes that are important in producing these galaxies.

The simulations produced by the institute have modelled certain structures in the universe with a high degree of accuracy – such as the strings of diffuse matter that exist between galaxies – which researchers say indicates that their models of galaxy formation are on the right track.

Photo: Sarah Noble and Vicky Greener, department of physics, Durham University

The 1.1PB storage cluster will enable up to 50 researchers, working collaboratively, to access and review data simultaneously. Researchers will also be able to access data up to seven years old through a single management system, the IBM General Parallel File System (GPFS).

"Our researchers are sitting on a lot of data," said Heck. "With hierarchical storage management and using the full capabilities of IBM GPFS, we can learn to manage data better. Storage is too expensive to use it poorly."

This shot shows the tray design of the IBM system storage DCS3700, which provides easy access to the discs.

Photo: Durham University