IBM Roadrunner
From Wikipedia, the free encyclopedia
| IBM Roadrunner | |
|---|---|
 Roadrunner components |
|
| Dates | operational 2008, final completion 2009 |
| Sponsors | IBM,  United States |
| Operators | National Nuclear Security Administration, United States |
| Location | Los Alamos National Laboratory, United States |
| Architecture | 12,960 IBM PowerXCell 8i CPUs, 6,480 AMD Opteron dual-core processors, Infiniband, Linux |
| Power | 2.35 MW |
| Space | 296 racks, 6,000 sq ft (560 m2) |
| Memory | 103.6 TiB |
| Speed | 1.7 petaflops (peak) |
| Cost | US$133M |
| Ranking | TOP500: 1, June 2008 |
| Purpose | Modeling the decay of the U.S. nuclear arsenal. |
| Legacy | First TOP500 Linpack sustained 1.0 petaflops, May 25, 2008 |
| Web site | http://www.lanl.gov/roadrunner/ |
Roadrunner is a supercomputer built by IBM at the Los Alamos National Laboratory in New Mexico, USA. Currently the world's fastest computer, the US$133-million Roadrunner is designed for a peak performance of 1.7 petaflops, achieving 1.026 on May 25, 2008,[1][2][3] and to be the world's first TOP500 Linpack sustained 1.0 petaflops system. It is a one-of-a-kind supercomputer, built from commodity parts, with many novel design features.
Contents |
[edit] Overview
IBM built the computer for the U.S. Department of Energy's (DOE) National Nuclear Security Administration.[4][5] It is a hybrid design with 12,960 IBM PowerXCell[6] 8i CPUs and 6,480 AMD Opteron dual-core processors[7] in specially designed server blades connected by Infiniband. The Roadrunner uses Red Hat Enterprise Linux along with Fedora as its operating systems and is managed with xCAT distributed computing software. It also uses the Open MPI Message Passing Interface implementation.[8]
Roadrunner occupies approximately 6,000 square feet (560 m2)[9] and became operational in 2008.
The DOE plans to use the computer for simulating how nuclear materials age in order to predict whether the USA's aging arsenal of nuclear weapons is safe and reliable. Other uses for the Roadrunner include the sciences, financial, automotive and aerospace industries.
[edit] Hybrid design
Roadrunner differs from many contemporary supercomputers in that it is a hybrid system, using two different processor architectures. Usually supercomputers only use one, since such a design is easier to design and program for. To realise the full potential of Roadrunner, all software will have to be written specially for this hybrid architecture. The hybrid design consists of dual-core Opteron server processors manufactured by AMD utilizing the standard AMD64 architecture. Attached to each Opteron core is a Cell processor manufactured by IBM using Power Architecture technology. As a supercomputer, the Roadrunner is considered an Opteron cluster with Cell accelerators, as each node consists of a Cell attached to an Opteron core and the Opterons to each other.[10]
[edit] Development
Roadrunner has been in development since 2002, and went online in 2006. Due to its novel design and complexity it was constructed in three phases and became fully operational in 2008.
[edit] Phase 1
The first phase of the Roadrunner was building a standard (albeit quite large) Opteron based cluster, while evaluating the feasibility to further construct and program the future hybrid version. This Phase 1 Roadrunner reached 71 teraflops and has been in full operation at Los Alamos National Laboratory doing advanced weapons simulations since 2006. Even if Roadrunner had not been greenlit for Phase 2, the Phase 1 form would still be a formidable supercomputer and would have ranked, at its time, in the top 10 list of the world's fastest computers.
[edit] Phase 2
Phase 2 known as “AAIS” (Advanced Architecture Initial System) included building a small hybrid version of the finished system using an older version of the Cell processor. This phase was used to build prototype applications for the hybrid architecture. It went online in January 2007.
[edit] Phase 3
The goal of Phase 3 was to reach sustained performance in excess of 1 petaflops. Additional Opteron nodes and new PowerXCell processors were added to the design. These PowerXCell processors are five times as powerful as the Cell processors used in Phase 2. It was built to full scale at IBM’s Poughkeepsie, New York facility, where it broke the 1 petaflops barrier during its fourth attempt on May 25, 2008. The complete system was moved to its permanent location in New Mexico in the summer of 2008, where fine tuning of the applications will continue until final completion in the latter stages of 2009.
[edit] Technical specification
 |
This section may require cleanup to meet Wikipedia's quality standards. |
[edit] Processors
Roadrunner is unique for its hybrid design using two different models of processors.
[edit] Opteron
AMD Opteron 2210, running at 1.8 GHz. These are processors with two general purpose cores each. Opterons are used both in the computational nodes feeding the Cells with useful data and in the system operations and communication nodes passing data between computing nodes and helping the operators running the system. Roadrunner has a total of 6912 Opteron processors (6480 computation, 432 operation), for a total of (12960+864) 13824 cores.
[edit] PowerXCell
IBM PowerXCell 8i, running at 3.2 GHz. These processors have one general purpose core (PPE), and eight special performance cores (SPE) for floating point operations. Roadrunner has a total of 12,960 PowerXCell processors, with 12,960 PPE cores and 103,680 SPE cores, for a total of 116,640 cores.
[edit] Number of cores
On the Top500 list, Roadrunner is said to have 122,400 cores. It is important to know which core is counted.
- 13,824 Opteron cores + 116,640 Cell cores = 130,464 cores for both the computing nodes and the operation nodes.
This is a number larger than the one mentioned on Top500. It turns out that the Roadrunner only used 17 Connected Units while doing the LINPACK benchmark, and it was not counting the cores in the operations and communication nodes (they didn't run the benchmark).[11]
- 6,120 Opteron (2 cores) + 12,240 PowerXCell 8i (9 cores) = 122,400 cores
[edit] TriBlade
Logically, a TriBlade consists of two dual-core Opterons with 16 GB RAM and four PowerXCell 8i CPUs with 16 GB Cell RAM.[7]
Physically, a TriBlade consists of one LS21 Opteron blade, an expansion blade, and two QS22 Cell blades. The LS21 has two 1.8 GHz dual-core Opterons with 16 GB memory for the whole blade, providing 8GB for each CPU. Each QS22 has two PowerXCell 8i CPUs, running at 3.2 GHz and 8GB memory, which makes 4 GB for each CPU. The expansion blade connects the two QS22 via four PCIe x8 links to the LS21, two links for each QS22. It also provides outside connectivity via an Infiniband 4x DDR adapter. This makes a total width of four slots for a single TriBlade. Three TriBlades fit into one BladeCenter H chassis.
[edit] Connected Unit (CU)
A Connected Unit is 60 BladeCenter H full of TriBlades, that is 180 TriBlades. All TriBlades are connected to a 288-port Voltaire ISR2012 Infiniband switch. Each CU also has access to the Panasas file system through twelve System x3755 servers.[7].
CU system information:[7].
- 360 dual-core Opterons with 2.88 TiB RAM.
- 720 PowerXCell 8i cores with 2.88 TiB RAM.
- 12 System x3755 with dual 10-GBit Ethernet each.
- 288-port Voltaire ISR2012 switch with 192 Infiniband 4x DDR links (180 TriBlades and twelve I/O nodes).
[edit] Roadrunner cluster
The final cluster is made up of 18 connected units, which are connected via eight additional (second-stage) Infiniband ISR2012 switches. Each CU is connected through twelve uplinks for each second-stage switch, that makes a total of 96 uplink connections.[7]
Overall system information:[7]
- 6,480 Opteron processors with 51.8 TiB RAM (in 3,240 LS21 blades)
- 12,960 Cell processors with 51.8 TiB RAM (in 6,480 QS22 blades)
- 216 System x3755 I/O nodes
- 26 288-port ISR2012 Infiniband 4x DDR switches
- 296 racks
- 2.35 MW power[12]
[edit] See also
[edit] References
- ^ Sharon Gaudin (2008-06-09). "IBM's Roadrunner smashes 4-minute mile of supercomputing". Computerworld. http://www.computerworld.com/action/article.do?command=viewArticleBasic&taxonomyName=hardware&articleId=9095318&taxonomyId=12&intsrc=kc_top. Retrieved on 2008-06-10.
- ^ "Military supercomputer sets record - CNET News.com". http://news.cnet.com/Military-supercomputer-sets-record/2100-1010_3-6241145.html?tag=nefd.top.
- ^ "Supercomputer sets petaflop pace". BBC. 2008-06-09. http://news.bbc.co.uk/1/hi/technology/7443557.stm. Retrieved on 2008-06-09.
- ^ "IBM to Build World's First Cell Broadband Engine Based Supercomputer". IBM. 2006-09-06. http://www-03.ibm.com/press/us/en/pressrelease/20210.wss. Retrieved on 2008-05-31.
- ^ "IBM Selected to Build New DOE Supercomputer". NNSA. 2006-09-06. http://nnsa.energy.gov/news/1015.htm. Retrieved on 2008-05-31.
- ^ International Supercomputing Conference to Host First Panel Discussion on Breaking the Petaflop/s Barrier
- ^ a b c d e f "RR Seminar - System Overview" (PDF). Los Alamos National Laboratory. 2008-03-13. http://www.lanl.gov/orgs/hpc/roadrunner/pdfs/Koch%20-%20Roadrunner%20Overview/RR%20Seminar%20-%20System%20Overview.pdf. Retrieved on 2008-05-31.
- ^ Jeff Squyres. "Open MPI: 10^15 Flops Can't Be Wrong" (PDF). Open MPI. http://www.open-mpi.org/papers/sc-2008/jsquyres-cisco-booth-talk-1up.pdf. Retrieved on 2008-11-22.
- ^ "Los Alamos computer breaks petaflop barrier". IBM. 2008-06-09. http://www.ibm.com/news/us/en/2008/06/2008_06_09.html. Retrieved on 2008-06-12.
- ^ Kevin J. Barker, Kei Davis, Adolfy Hoisie, Darren J. Kerbyson, Mike Lang, Scott Pakin, Jose C. Sancho. "Entering the Petaflop Era: The Architecture and Performance of Roadrunner" (PDF). http://www.c3.lanl.gov/~kei/mypubbib/papers/SC08:Roadrunner.pdf. Retrieved on 2008-11-22.
- ^ "Roadrunner: Science, Cell and a Petaflop/s" (PDF). Los Alamos National Laboratory and IBM. 2008-06-18. http://versi.edu.au/downloads/SuperCGrice.pdf. Retrieved on 2008-09-22.
- ^ "TOP500 List - June 2008". TOP500 Supercomputer Sites. 2008-06-15. http://www.top500.org/list/2008/06/100. Retrieved on 2008-08-12.
[edit] External links
- "Los Alamos National Laboratory Roadrunner Home Page". Los Alamos National Laboratory. 2007-03-30. http://www.lanl.gov/roadrunner. Retrieved on 2008-05-31.
- "In Pictures: A look inside what may be the world's fastest supercomputer". Computerworld. 2008-05-13. http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9085021. Retrieved on 2008-05-31.
