Overview of GAE Status, and Suggestion for Florida Participation, Julian Bunn, 10/10/2002:

Hello again Haifeng,

It seems we are well on the way to having a setup at Florida which will
allow us to incorporate your Tier2 in the general GAE work and 
specifically the demo for SC2002.

I think it's important to clarify the GAE plans so that we avoid you
repeating some of the work already done by Eric and Edwin in this area.

What follows is perhaps of interest to others in CMS, so I am copying
Harvey, Rick, Lothar and Claudio.

The GAE plan is to "Grid Enable" analysis tools being used in CMS.
Currently we are targetting the ROOT analysis tool. We will shortly begin work on
adding Iguana into the scheme.

We have a particular desire to make use of RDBMS to store analysis (GAE)
data, as well as native file formats such as ROOT files.

So we have accumulated a large quantity of JETMET data into a form that
can be easily loaded into a SQL database. The databases we have targetted up
to now are Oracle and Microsoft SQLServer.

We have developed the Clarens system to mediate the "conversation"
between the analysis client (ROOT/Iguana/?) and the data. The Clarens system (as
you know) runs on the server, and talks the lingua franca of the client. So
if you are using a ROOT client, Clarens makes available to you ROOT files
on the server. For Iguana it may well be an OpenInventor scene graph files.

The Clarens system supports Grid-based authentication methods. The long
term goal is to develop inter-Clarens communications that "broker" a client
request for analysis data between the available data repositories on the CMS
Grid.

One interesting technology choice is the layer between Clarens and the
data in the SQL database. There are at least two approaches which show
promise:

1) Clarens obtains the data by interacting with a Web Service that
itself talks to the database. In this model, Clarens takes a user request for
data and passes it to one or more Web Services, which then extract the
required data from the SQL database, massage it into the required format, and
return pointers to the created file(s) which Clarens then communicates to the
client. We have working, or almost working, implementations of this method using
SQL Server with .NET Web Services, and Oracle with Java based Web
Services. Edwin and Saima Iqbal have been working in this area.

2) Clarens obtains the data by interacting with an OTL-based software
layer that talks directly to the database. This OTL-based software creates the
required data files on the fly, allowing the data to be streamed by
Clarens to the client, as it is created, or allowing it to be sent to the client
once the data files are fully written. We have an almost complete
implementation running with SQL Server, and porting to Oracle is underway. Eric is
responsible for these developments.

Your work at Florida thus fits very nicely into the above scheme.
Initially, the target should be to populate a database (we suggest PostGreSQL, but
this is not essential) with all or a subset of the JETMET data that we are
using at Caltech (in SQLServer) or CERN (in Oracle). Then to implement the
Clarens server (which I believe you already have installed). Then to work with
Eric and Edwin on the database interface. This should initially involve
installing the software which we already have, and getting it working.

At that point, we can slice up an interesting and significant portion of
the pending work for you to work on. A suggestion is that you work on the
coordination of query requests between Clarens servers at the various
sites, and amalgamating the data files produced by the Web Services or OTL
software.

Another very useful line of work is to make a Java-only version of
Clarens. This would make an installation of Clarens on various hardware/OS
platforms much more straightforward.

In conclusion, we should probably arrange a telecon between you, Eric,
Edwin, me and anyone else who is interested sometime soon, to mull over the
above.

In the meantime, please let us know what aspects interest you the most
and the direction you would like to take.

With best wishes,

Julian