Professor develops software to run super collider systems

Professor develops software to run super collider systemsMarch 20, 2002

Jae Oh, assistant professor of electrical engineering and computer science in the L.C. Smith College of Engineering and Computer Science, is helping to show some of America’s top physicists the importance of computer science in the study of matter and anti-matter asymmetries. Oh is part of a team, funded by the National Science Foundation, that is developing very large-scale, real-time embedded operating systems software that will run the trigger and data acquisition computer systems of the Tevatron Collider that is located in the Fermilab National Laboratory in Batavia, Ill. The Tevatron Collider is America’s highest-energy particle accelerator.

“The physicists at Fermilab study the fundamental nature of matter and energy using Tevatron,” Oh says. “What they found was that so much data is generated in this four-mile long accelerator that it could not be efficiently and reliably collected by existing ordinary computer systems, and they called in the computer guys. The hardware group made a proposal for funding a computer system that would directly extract data from the accelerator and process it. The hardware group then realized that they needed specially designed computer operating systems software that would run the system efficiently and reliably, and brought in our group.”

Oh is part of a research group called the BTeV Real Time Embedded Systems Research Group. The group comprises physicists, computer scientists and electrical engineers from SU, Vanderbilt University, the University of Pittsburgh, Fermilab and the University of Illinois, Urbana-Champaign. Paul Sheldon of Vanderbilt University is the project’s principal investigator.

The team received a $4.98 million grant from the NSF to develop an advanced computer system capable of scanning terabytes (thousands of billions of bytes) of information produced by the detector in a new high-energy physics experiment, called BTeV.

Through BTeV, new detectors are being built for the Tevatron Collider to examine subatomic particles, called B particles, that have a property physicists think can explain why the universe is filled with ordinary matter rather than anti-matter. Beams of protons and antiprotons enter from left and right to collide at the center of the detector, producing showers of hundreds of millions of subatomic particles per second that are recorded by the detector. The detector will stand 16 feet high, be 78 feet long, weigh 1,400 tons and have 30 million detector channels.

BTeV’s goal is to assemble as many as 10,000 parallel computers and make them work together dependably and consistently in the triggering and DAQ system despite incorporating different kinds of computers with different tasks. The BTeV trigger will be challenged to reconstruct 15 million particle events per second, and to use that reconstruction data in deciding which events to keep for further analysis. It will be further challenged to perform the reconstructions around the clock while spotting and correcting any problems that arise.

“Creating usable software for this type of real-time embedded system will require research into solutions of general problems in the fields of computer science and engineering,” says Oh. “We plan to approach these problems in a way that is general, and to produce methodologies and tools that can be applied to many scientific and commercial problems.”

The classes of systems targeted by this research include those embedded in environments, like BTeV, that produce very large data streams that must be processed in real-time using data-dependent computation strategies. These systems require ultra high performance, necessitating parallel hardware architectures, which in the case of BTeV is composed of a mix of thousands of commodity processors, special purpose processors such as Digital Signal Processors (DSPs) and specialized hardware such as Field Programmable Gate Arrays (FPGAs), all connected by very high-speed networks. The systems must be dynamically reconfigurable to allow maximum performance from the available and potentially changing resources. The systems must be highly available, since the environments produce the data streams continuously over a long period of time, and interesting phenomena important to the analysis are rare and could occur at any time.

BTeV has been approved by Fermilab management and is expected to be constructed over the next five to six years to run in conjunction with the Fermilab Tevatron Collider. The data-taking phase of the experiment is expected to last at least five years.

Oh says he’s excited to have the opportunity to work on the NSF project with an outstanding group of researchers and his graduate research assistants at SU. The computer science graduate students working with Oh are doctoral student Leland Hovey and master’s student Madhura Tamhankar.

“I’m very lucky to be working with some outstanding people and now that the project is shaping up, it gets more exciting all the time,” he says. “To my knowledge, there is no computer operating system in the world like the one we are building for the BTeV trigger system.”