Julian J. Bunn, Harvey B. Newman, Richard P. Wilkinson, Caltech
The GIOD (Globally Interconnected Object Databases) Project, a joint effort between Caltech, CERN and Hewlett Packard Corporation, is investigating the use of WAN-distributed Object Databases and Mass Storage systems for LHC data. The Project began in June 1997, and is due to complete in June 1999. In this paper, we discuss the results so far obtained, and describe the plans for the coming year.
The Project was proposed in late 1996 by Caltech’s HEP department and CERN’s Information Technology Division, and was agreed to by all parties in March 1997. Hardware infrastructure from Hewlett Packard was already funded and in place at Caltech’s Center for Advanced Computing Research (CACR).
The GIOD Project’s aims are to evaluate and test plausible LHC Computing Models . as a function of the bandwidth expected to be available in local, regional and wide area networks, . with various data storage and access methods, . taking account of performance, scalability and limitations of the ODBMS
We are making use of several "leading edge" hardware and software systems, including the 256 CPU Caltech HP Exemplar (~ 0.1 TIPS), the HPSS from IBM, the Objectivity/DB ODBMS, and various high speed Local Area and Wide Area networks.
The Exemplar machine runs the SPP-UX Unix operating system (binary compatible with HP-UX), and contains PA8000 processors interconnected with a toroidal topology, linked to 64 Gbytes of global shared memory. The CPUs will be replaced in 1999 by 64 IA-64 chips. Other hardware of note is an HP model C200 workstation equipped with a 155 Mbit/sec ATM link directly fiber attached to the Exemplar. The HPSS system includes a 10 TByte tape robot with SSA disk buffers, linked to the Exemplar via HiPPI. These systems have connections to CERN (over a 4 Mbit/sec trans-Atlantic link) and to an Exemplar peer system at the San Diego Supercomputer Center (over a "CENIC" OC12 link). The Objectivity/DB ODBMS is licensed both for the Exemplar and all the GIOD workstations.
In the first part of the paper we present results from scaling tests of Objectivity/DB, wide area database replication from CERN to Caltech, use of the HPSS NFS interface to store the database, and tests of the Exemplar/Objectivity installation with up to two hundred database clients. We also compare test results between the Objectivity and Versant ODBMS.
In the second half of the paper we describe a prototype reconstruction and analysis application for the CMS experiments’ Tracker and Electromagnetic Calorimeter detectors. This has been implemented using Objectivity’s C++ and Java bindings. The application incorporates methods that allow an Objectivity federated database to be populated with "raw" data contained in ZEBRA-format files that have previously been produced by production runs on the Exemplar of the GEANT3-based CMSIM program. The application then performs track reconstruction and ECAL clustering to build new physics analysis objects, including electrons, photons and jets, which are in turn placed in the database. We show results from populating and using a database with many tens of thousands of Higgs to two photon and background events.
Finally, we describe the use of simulation tools to explore the characteristics of various LHC computing models. We show results obtained from both the SoDA and ModNet tools using input parameters derived from ODBMS client traffic measurements on our dedicated ATM link.