SLD Staging system on the SLACVX cluster

Introduction

Staging is a mechanism by which tape data is first copied to disk and then read from disk by the user. The advantage of this over directly reading the tape is that
  1. The tape drive is only tied up during the relatively fast tape-to-disk copy, not for the entire duration of the users analysis.
  2. If the data is to be read many times it only needs to be copied to disk the first time around.

The SLD staging system was designed as a general purpose staging system for VMS. Although in principle the system can be used to stage any type of tape data, currently the only supported way to use the system is for staging input JAZZDATA files via IDA.

The staging system maintains a large pool of disk space where the staged data is stored. Data is kept on disk as long as space if available, but when space is needed for new files, the least recently used files are deleted from disk automatically.

How to use the staging system

To read files with the staging system from IDA all you have to do is use the IDA OPENTAPE command. Two extra qualifiers control staging:
STAGE
Tells IDA you want to read the file using the STAGING system rather than directly reading the tape.
WAIT
Tells IDA you want to wait for the data to be put on disk. If you do not specify WAIT when running IDA interactively and the data is not already on disk, the the job to copy the data to disk is submitted, but the OPENTAPE command is then terminated with an error. If you specify WAIT then the OPENTAPE command will wait until the data has been copied to disk before returning.

When running in batch WAIT is the default.

Data is actually copied to disk by a batch job submitted under the userid SLDSTAGE. You will be able to see such jobs from time-to-time in the batch system, they can be idenified by the fact that they are always named STAGE_XXXXXX, where XXXXX is the tape name (e.g.QW1234). If for any reason that input staging job fails, you might need to look at the log files for the staging job. This can be found as

   STAGE$LOG:STAGE_XXXXXX.LOG

Other Useful Commands

The STAGE/QUERY command can be used to check the status of staged files. You can also use WWW to query the status of the staging system.

The STAGE/CLEAR command can be used to clear files out of the staging area.

Technical Details

Read the source code (in STAGE_$SRC).

Known Problems

None known.

Further Reading

For more information you could try reading the design specification.
TonyJ
Last Updated: