Implementation Details: ducs_arch full
When the archivist issues the command to make a full copy of the current
DUCS, the copy work occurs locally at each DUCS host, the SLAC copies are
made from the current SLAC DUCS, the VU copies are made from the current
VU DUCS. This brings in the risk that if the current DUCS is different
at the various DUCS hosts, the archive copies will come out different.
One way around this problem would be to instead copy the code to all hosts
from the current SLAC DUCS, however the size of the full DUCS, 0.5
GB, makes this impractical. Another way around this problem would be to
ship tapes to all hosts from the current SLAC DUCS, however this is
a cumbersome procedure. Instead, the archive system relies on a series
of checks to insure that the current DUCS at the non-SLAC hosts match the
current DUCS at SLAC.
The implementation of a full DUCS copy is as follows:
- DUCS is disabled to prevent any new updates from occurring during measuring
and copying procedures.
Currently, there is no actual confirmation that this DUCS disable action
succeeded, however such confirmation will be added once the DUCS hub moves
to vms.
- DUCS directories for non-SLAC hosts are compared against the corresponding
directories at SLAC.
At all hosts, a full listing of DUCS files is obtained, including file
names, file types, file versions and file sizes. The most recent copies
of these listings are maintained in the unix DUCS account's main directory.
The filename of each listing is formed from the host name and model, for
example: SLDA6_VAX.CONTENTS.
Directory listings for non-SLAC hosts are then checked against the
corresponding SLAC listings. If any difference is found in file specifications
or sizes, the copy procedure is halted for all hosts. In such cases, a
listing of the offending differences is left in the unix DUCS account's
main directory with a file name such as VUPACB_AXP_SLDA6_AXP.DIFF.
The .dir files themselves are excluded from these measurements.
Such files may not match exactly between one host and another, since the
deletion of files within a directory may leave wasted space in the .dir
file. Such activity tends to leave .dir files at non-SLAC hosts
better optimized than those at SLAC. Certain other files are also excluded
(the list is specified in produtil:ducs_arch.com).
It is not required that file dates match. These date can legitimately
be out of synch if files have been re-sent from SLAC to clear up earlier
problems.
- The database is queried to obtain the next major version number.
- Current available disk space is measured on the archive disks at all
hosts. The system will refuse to perform copies at any hosts if it does
not measure sufficient available disk space
at all hosts.
- The database is queried to find out how much disk space should be available
in the SLAC archives. This information is provided to the archivist as
a cross-check to tell if the archive disk has been misused (occupied by
code not part of the archive system). The estimated disk space should closely
match the actual measured disk space. However only the actual measured
disk space is used to decide whether to proceed with the copy.
In the past, space conflicts with non-archive system code have been the
primary cause of archive copy failures. A full 4 GB disk is supposed to
be reserved for the DUCS archives at SLAC.
- The DUCS code is copied at all hosts. For a full DUCS copy, this can
take up to half an hour per host. The same files that were excluded from
measuring are excluded from copying.
- Confirmation that the copy completed successfully is obtained by scanning
the remote shell login's output for the line: DUCS_ARCH STATUS: SUCCESS
- The new archive copy is then measured at all hosts by checking the
total number of files and the grand total size. An error message is given
if any copy does not match the corresponding original SLAC DUCS version.
- With measuring and copying now complete, DUCS is re-enabled.
Currently, there is no actual confirmation that this DUCS enable action
succeeded, however such confirmation will be added once the DUCS hub moves
to vms.
- The database is queried to find a tape already being used by the archive
system that has sufficient room left to fit the new archive version.
To account for slightly unpredictable expansion of code as it is copied
to tape (measured by experience to be around a 14 percent expansion),
the system assumes each tape to be only 80 percent of its real length.
Thus the system treats these 800 MB tapes as if they were only 640 MB
(which equals 1.25 million VMS blocks).
The system attempts to find the fullest tape that will still fit the
given archive version.
Tapes that already contain AXP code will only be used for more AXP code.
Tapes that already contain VAX code will only be used for more VAX code.
- If there is no tape already being used by the archive system that has
sufficient room to left to fit the new archive version,
a fresh tape is obtained.
- The DUCS code is then copied from the archive disk to the archive tape
.
- The database is updated to reflect the new major version.
Joseph
Perl
24 September 1997
