News

Terascale Computing System Installed at Pittsburgh Supercomputing Center. The most powerful computer system in the world for open research is u p and running.

PITTSBURGH — The Terascale Computing System (TCS), the most powerfu l system in the world committed to unclassified research, is installed on s chedule. Developed and implemented by the Pittsburgh Supercomputing Center in collaboration with Compaq Computer Corporation, with funding from the Na tional Science Foundation, the TCS provides computational capability to sci entists and engineers nationwide. They will use it in many areas of researc h that have wide social impact, including earthquake modeling, storm-scale weather forecasting, global climate change, and protein genomics, modeling that's integral to the development of new drug therapies.

A joint project of Carnegie Mellon University, the University of Pitt sburgh and Westinghouse Electric Company, the Pittsburgh Supercomputing Cen ter (PSC) has deployed the TCS to fill a gap in U.S. basic research capabil ity highlighted in a 1999 presidential report. Terascale means computationa l power beyond a "teraflop" — a trillion calculations per second. With pe ak capability of six teraflops, the new system is now by far the most power ful available as an open resource for researchers attacking a wide range of problems.

"The TCS system at Pittsburgh Supercomputing Center continues a histo ry of National Science Foundation support for high-performance computing," said Robert Borchers, director of NSF's Division of Advanced Computational Infrastructure and Research. "Through the NSF's Partnerships for Advanced C omputational Infrastructure (PACI) program, this system will increase long- term, fundamental research across all science and engineering disciplines."

"In scale alone, the TCS pushes beyond where open-resource supercompu ting technology has been before or would have gone without the NSF PACI pro gram," said PSC scientific directors Michael Levine and Ralph Roskies in a joint statement. "Compaq committed themselves to the success of the TCS and worked side-by-side with us to make it available on schedule. With storage capacity that's 100,000 times that of most PCs and with 10 million times t he communications capability, this system brings significant new research c apability to bear on many important problems. While the immediate, direct b eneficiaries will be academic scientists, the benefits will flow to the cou ntry as a whole, in practical ways we can't forecast."

"Compaq is proud to join with the Pittsburgh Supercomputing Center an d the National Science Foundation in delivering TCS," said Bill Blake, Comp aq's Vice President of High Performance Technical Computing. "And we're exc ited that Compaq's AlphaServer SC supercomputing architecture will make suc h a significant contribution in opening new frontiers in scientific computi ng."

The TCS represents an unprecedented synthesis of "off-the-shelf" comp onents integrated with an advanced interconnect — from Quadrics Supercomp uters World — and other technologies to provide a very large-scale system for scientific computing. It comprises 3,000 Compaq Alpha EV68 microproces sors, housed in 750 four-processor AlphaServer systems running Tru64 UNIX. The latest evolution of the widely used Alpha microchip technology, the EV6 8 has peak floating-point capability of two gigaflops (two billion calculat ions per second).

Along with six teraflops of processing power, the TCS features 3.0 te rabytes of memory, high-bandwidth, low-latency interconnections and remarka ble capabilities for large-scale data handling, including the ability to wr ite the entire memory to disk in under 40 seconds. This extremely short sys tem-write time, developed through PSC systems and software engineering, is critical to efficient checkpointing, needed to preserve research data in th e event of component failure.

Preparation for the TCS began in October 2000 with installation of a 256-processor prototype system. In August 2001, the first of the new AlphaS erver systems arrived at the PSC computer room at Westinghouse Energy Cente r in Monroeville, Pennsylvania. System components came in multiple deliveri es from Compaq facilities in Texas and Scotland. An on-site team of Compaq, PSC and Westinghouse engineers and technicians — supported by expert tea ms at Compaq locations in the United States, Bristol, England and Galway, I reland — worked aggressively to meet the Oct. 1 installation date.

"PSC's success in deploying this unprecedented, very large-scale syst em right on time is a fine achievement," said Jared L. Cohon, president of Carnegie Mellon University. "This is another important chapter in the cente r's outstanding record of providing the nation's scientists with the most a dvanced computational tools. This world-class computing system reflects Pit tsburgh's international leadership in technology development and is a key c omponent of our region's technology future."

"This computing system is an important advance in assuring the contin uation of our nation's leadership in basic research," said University of Pi ttsburgh Chancellor Mark A. Nordenberg. "There is growing national and inte rnational recognition that university research is at the heart of most comm ercial innovation and much of our recent economic prosperity. And it is sig nificant for our region that Western Pennsylvania is the home of this great national resource — attesting to the strength of our institutions. When great research universities such as Pitt and CMU partner with industry -- i n this case Westinghouse and Compaq — and the federal government, we are poised to achieve the next great breakthroughs of this new century."

"PSC is to be congratulated on bringing this powerful new technology into being," said Charlie Pryor, president and CEO of Westinghouse Electric Company. "Westinghouse is proud to add its internationally recognized expe rtise in management excellence and technology leadership to the team. Once again, the Pittsburgh Supercomputing Center has demonstrated its leadership in high-performance computing."

The TCS installation marks the first operation of AlphaServer SC, the system software that ties AlphaServer systems together, on this scale and the first large-scale, multi-level Quadrics switch structure that supports thousands of processors while achieving sustained operation across the syst em. Standard benchmark software has measured system performance over three teraflops. The TCS will next go through a period of "friendly user" testing , and by early 2002 it will become available to researchers nationwide thro ugh the peer-review process of the NSF PACI program.

PSC and Compaq collaborated on numerous machine enhancements to impro ve the performance of the TCS, changes that range from the disk controller and file system to wiring optimizations. By careful site planning and redes ign of the AlphaServer configurations, PSC engineers reduced the distance b etween processors, thereby also reducing cabling and minimizing network lat ency.

Total TCS floor space is roughly that of a basketball court. It uses 14 miles of high-bandwidth interconnect cable to maintain communication amo ng its 3,000 processors. Another seven miles of serial, copper cable and a mile of fiber-optic cable provide for data handling.

The TCS requires 664 kilowatts of power, enough to power 500 homes. I t produces heat equivalent to burning 169 pounds of coal an hour, much of w hich is used in heating the Westinghouse Energy Center. To cool the compute r room, more than 600 feet of eight-inch cooling pipe, weighing 12 tons, ci rculate up to 900 gallons of water per minute, and twelve 30-ton air-handli ng units provide cooling capacity equivalent to 375 room air conditioners.

For background information, see http://www.psc.edu/publicinfo/terasca le/bigiron.html

Compaq and AlphaServer are trademarks of Compaq Information Technolog ies Group, L.P.

The Pittsburgh Supercomputing Center was established in 1986 and is s upported by several federal agencies, the Commonwealth of Pennsylvania and private industry.