PPDG Quarterly Report, Caltech March->June 2002

 

1) Eric Aslakson

Eric's activities during this period were divided between developing software for physics analysis and designing software for production activites related to scheduling. In the analysis software area, Eric completed work on a Root-Tree to SQL database insert program. He generalized and extended code supplied by Krzysztof Nienartowicz, of the IT group at CERN, to use Microsoft SqlServer as well as Oracle databases. He also extended the modules to map arbitrary Root group names into SQL table names, enabling conversion of (almost) arbitrary Root trees. Due to design requirements that the code run unattended, Eric also converted batched database inserts to ODBC function call inserts. This allows more robust error recovery for unattended inserts. This ODBC layer also facilitates pluggable SQL database usage. He then started proving ODBC drivers on Linux for Oracle and SQLServer. He also began work on a standard template library based objectification of SQL result sets. This layer will facilitate front end caching and easy tranport of SQL results sets optimized for high latency network connections. This software will plug into Clarens for use in Grid Analysis Environment prototype. In the production area, Eric continued to participate developments in the PPDG monitoring area by participating in the very few phone conferences occuring. He continued design of the Execution Priority Manager, a module working in concert with the DAG processors and grid nodes to attempt a guarantee of throughput promises for production activities. This work will continue in August when he will be working with the relevant groups at Fermilab to implement this module.

2) Edwin Soedarmadji

In this reporting period, Edwin concentrated on the tasks : (1) reverse-engineering a portion of JetMet analysis code produced by Pal Hidas while at FNAL (2) writing programs that convert JetMet PAW ntuple into raw text (3) building a SQLServer database and (4) import the converted ntuples into the database. These tasks were recently completed and now serve as the foundation for the next set of tasks. The SQLServer database will be used in conjunction with an Oracle database at CERN (under construction) to serve out JetMet analysis data in a more efficient and granular way. Currently Edwin is working on a set of web services that would allow queries to be submitted and executed against the Jetmet database by a client software.

3) Saima Iqbal

Saima's main task is the evaluation of Oracle9i for the CMS Event store. Oracle9i implements an Object Relational Model. The object type maps closely to the class mechanisms found in C++ and Java, as a result the object based code that is already in use by CMS would not need be changed or rewritten. Only an SQL wrapper is required to store and access data from Oracle database. Saima's evaluation started by investigating the Object Relational features of Oracle. She measured the overheads on the objects after storing them in the Oracle9i database. For these overhead measurements simple objects, embedded objects and objects with REF were created in the form of Object-Tables. These object tables were loaded with simple data from vectors and hits. In the Object-Table each C++ object corresponds to one row and each attribute maps to the column in the row of the table. LOADING AND ACCESS OF TAGS DATA FROM ORACLE 9i DATABASE: As a follow up activity Saima looked at the proposed architecture for a CMS event store using Oracle 9i and its related tools. A small-scale data warehouse prototype for CMS TAG data was developed by her. To evaluate the performance of Oracle for the persistency of Tags data the Java classes TagLoader.class and TagAccess.class developed at Caltech for SQLServer were converted for Oracle. Preliminary results show a data warehouse ingestion speed of 25 MB/s. The tuning of this process will improve over time. ORACLE 9i WEB APPLICATION: In the next step (loading and accessing the Tags Data) an Oracle base Web Application was developed by Saima. After the comparative study of web applications developed by using Microsoft FrontPage, Perl, CGI (Common Gateway Interface) and Java, the web application was designed in Java/JDBC (using Java Server Pages). It is platform independent and its byte codes can easily be run on any machine providing a JVM (Java Virtual Machine). It has an open structure to avoid restricting users to specific hardware and software configurations. It does not require the Oracle Client installation. The demo of this web application was presented by Saima during CMS Week June-2002.   ARCHITECTECTURE TO USE ORACLE 9i IN GRID ENVIRONMENT: A proposed architecture makes use of Oracle 9i data warehousing in a Grid environment, enabling hybrid data access with Oracle9i and Microsoft SQL server. For this purpose Saima began work on loading the Jetmet Ntuples version 2.06 in the Oracle Server at CERN.

4) Suresh Man Singh

Suresh's main activities during the period were focused on upgrading hardware and software for Caltech's AMD cluster in the USCMS grid testbed. He upgraded the RAID software of the Network Attached Storage (NAS) unit of the cluster. The Linux kernels of all the cluster nodes were upgraded to 2.4.18 allowing improved performance. Various security related fixes or upgrades were made to vulnerable packages. These included the Apache web server, openssh 3.4 (secured shell), openssl, python etc. The time protocol NTP was upgraded on all the nodes and the head node was synchronized to CDT - as required by the distributed Monte-carlo Production (MOP). Suresh installed both VDT 1.1 server and client on the cluster's head node (citcms.cacr.caltech.edu) which includes Globus 2.0. As part of the host, service and user certificate migration process from Globus signing to DOE Science Grid certificates, he obtained the DOESG signed host certificate for the cluster's head node and installed it. This head node was registered with University of Florida's GIIS MDS server and can be queried by Globus grid-info-search command. Suresh is in the process of upgrading the cluster's local batch scheduler package to the most recent release of Condor to 6.4. This will be used for distributed MOP production by providing a hook from the Globus jobmanager. At the moment only the "FORK" jobmanager of Globus is active.

5) Conrad Steenberg

Summary: Work on the Clarens web services infrastructure continues, with improvements in robustness and functionality. Client functionality was expanded, with a ROOT remote file access method and an ftp-like command like file download client recently being the main highlights. The Clarens project has a new development home where a file repository (CVS), web page, and mailing list functionality is available in order to enable wider collaboration. Work started on developing and deploying Virtual Organization (VO) management tools to ease Grid site security management. 1.Presentations - "Clarens remote analysis enabling environment"CMS Tutorial week, May 1 - 4, UC San Diego - "LDAP CA Authorization management" PPDG/Griphyn All hands meeting, April 23 - 25, Argonne NL, 2.Project management The Clarens project home page was moved to http://clarens.sourceforge.net which provides managed web hosting to open source projects. A project mangement interface can be found at http://sourceforge.net/projects/clarens/, with access to bug tracking, a versioned file repository, and mailing lists. 3.Tool development and software changes - Clarens services added Added file download and browsing service added. Since most forms of data are still stored in files, this service is a logical basic compoenent of Clarens. The service streams data from a file to the client in raw form, or provides file listings for the directory structure of the file system. User files may be published from a special directory created in the user's login directory, and accessed using the well known tilde notation (~user/). - Service description publication Automatically generated descriptions and API documentation is provided using the system.Listmethods, system.methodHelp, and system.methodSignature API calls - Clarens client side Two clients were implemented for the file access service, a command-line ftp-like interface, as well as enabling the ROOT analysis package to access remote files tranparently - VO Management In order to manage authorization in the Grid computing model, the concept of Virtual Organizations was developed (by others). This involves placing users in different groups or categories and managing access to certain resources on the Grid based on these groups, instead of on an indivudual basis, thereby reducing the load on site administrators. To aid this process a tool to manage VOs was developed that provides a graphical user interface to the user, group and certificate information, as opposed to the cumbersome collection of scripts that are currently used. This tool has a web page at

http://heppc22.hep.caltech.edu/groupman.

6) Vladimir Litvin

Vladimir organised the production of 50k events for forward jet-tagging mechanism studies, and 25K events for calibration studies. He made a presentation at ACAT2002. He prepared analysis software for July CMS milestone in the Egamma group. He analysed five Level1 collections for the DAQ TDR, and five full datasets for the pixel isolation studies. An importer for data files from CERN has been installed, and 0.8TBytes of data moved from CERN and analysed.

7) Julian Bunn

Planning for demonstrations at iGrid2002 and SC2002 is advancing, with discussions centering around which components of the Grid Enabled Analysis Environment will be ready, and which need to be "glued" together to make a sensible application. For iGrid2002 we are collaborating with Bill Allcock and others in the Globus team at Argonne. The intention is to show a ROOT front end client on the show floor, which talks to Clarens servers at Caltech, Argonne, CERN and possibly UCLA and Rio. GridFTP will be used to move large data files generated on the server by Clarens to the client machine. We want to show a truly heterogeneous GAE at iGrid2002 and SC2002, with Oracle9i storing JETMET data at CERN, SQLServer storing a complementary set of JETMET data at Caltech, and then perhaps more data in PostgreSQL or MySQL at UCLA and Rio. Bandwidth usage will need to be very heavy in order to interest participants at the conference, so we are adopting a somewhat artificial analysis scheme which involves the use of GridFTP to move files across the WAN. Perhaps at SC2002 we will use Clarens instead of GridFTP, if we can reproduce the streaming performance of GridFTP on the special striped servers being provided by the Globus team. A particular item of work is on developing a database layer to which the Clarens server talks in SQL, and which is responsible for returning a vector of physics objects. Alternatively this layer would generate a file containing the objects, which may then be transferred using GridFTP. We are committed to using the Globus authentication methods as part of the demonstrations, and as part of the GAE being developed. We will use DoE Science Grid certificates, which are being obtained for the servers and users involved. As far as hardware for the demonstrations is concerned, we have spent some time analysing the market for the best motherboard to use to maximise the Gbit NIC performance of a server. With help from Ian Fisk (UCSD) we have ordered two powerful servers based on the SuperMicro P4DP6 board, each equipped with dual SysKonnect SK9843 optical NICs. Tests of the network performance in back-to-back LAN configurations are proceeding. We also have acquired Dlink Gbit NICs for testing. A further server has been ordered, based on the SuperMicro P4DP8-G2 motherboard, featuring on-board dual Gbit NICs (copper). This server will also be tested. Finally, once a suitable configuration has been proven to allow sustained high bandwidth transfers in the LAN, we will purchase several for placement at CERN, Caltech and at the STARLIGHT in Chicago. Work on optimising the WAN performance of the GAE is an important aspect of our activities, and we have started a very fruitful collaboration with Steven Low's (Caltech) group on Adaptive Queue Managament for TCP, which promises much better performance for large data transfers in the WAN. This work is being partially funded by a successful proposal we submitted to NSF.