Offline Checklist
The goal of the offline checklist is to monitor
the data quality promptly as the production processing wends it way from
the online tape to full reconstruction. Data problems are to be reported
ASAP to the processing overseer, (currently Erez Etzion) or one of the
production processing crew (Richard Dubois, Joe Perl, Gary Bower, and Karen
Heidenreich) as appropriate.
That follows may seem to be a checklist of numbers to compare to standards
and make a simple binary decision on data quality. Rather you should use
the detailed questions to get yourself oriented and warmed up to use your
physics skills to think about the run output and look for problems. If
data quality was just a matter of some binary decisions based on some numbers
we would just have the computer do it and we wouldn't need you!
You are responsible for any stages of run processing that finish during
your shift period. The easiest way to determine this is to use the links
from the checklist bellow. In the date range on the "Links" page enter
your shift days and it will list all processing that has finished during
that range. Thus, if your shift runs from the 3rd through the 5th you should
have a final look on the 6th since the range is midnight to midnight California
time. If some of the runs are 'catch-up' processing on older missed runs,
concentrate on the runs from the day/previous day that are most time critical
to report on.
Readme file
See the
OFFSITE
README file for the latest temporary kludges to the monitoring. It
will be updated regularly as problems come and go.
Processing Stages
Once the end-run button is pressed in the online, the acquisition (ACQ)
data tape is released for further processing in the SLACVX cluster.
The SLDPM server is notified that the tape is ready for processing.
It copies the tape to disk, followed by running a PASS1 filter, followed
by a full reconstruction and PASS2 filter.
Relevant Pages from the STATUS DISPLAY System
-
Page 12: Offline
-
gives details on the last run processed.
-
Page 14: Offline
Zs
-
includes plot of the number of SLD Reconstructed Zs.
-
page 15: Processing
-
gives list offline jobs currently queued or running.
Tape copy (ACQCOPY)
This stage performs two functions. It copies the input tape to disk for
effective access in subsequent steps. The second function is to monitor
the trigger records for inconsistencies. The run monitor is notified if
any are found.
ACQCOPY Duties
The only duty here is to determine whether trigger errors
(
example ) have been flagged, and notify the run coordinator if there
have.
For each run there is a file, called ASrun# SDATCHK
(
example ). Look at this file to see if any trigger errors have been
flagged. These are the ones to notify the run coordinator of.
Also examine ASrun#.STATS ( example ) to
see if any subsystem data is missing.
Do
it (Check tape copy/trigger)
Filter (Pass 1) duties
-
Check the Filter stat. files.
(
example )
-
Compare SLD Z's with Luminosity 'precise' bhabhas. They should be about
1:2. Look up the online SLC Zs they should agree approximately with the
SLD Zs.
-
Polarization these days should be between 70% and 80%.
-
If running conditions are good the bad polarization fraction should be
essentially zero.
-
If running conditions are good the CDC should be on basically 100%..
-
All diagonal elements of the filter and trigger correlation matrices should
be non-zero (for a reasonable number of input events).
-
Precise/total bhabhas ratio should be ???.
-
Check Pass 1 Histograms. Most of these histograms are
for diagnostics in the filter job itself. Of interest for monitoring are
the polarization and CDC HV-on plots. We would like a nice gaussian for
POL and a spike at 10 layers for the CDC.
-
The ID=QPOL polarization histogram should be gaussian with few low P measurements.
-
The ID=DCHV CDC HV layers plot should show 10 layers on most of the time.
Do
it (Check Filter, pass1 reconstruction)
Recon (pass 2) duties
-
Examine Recon stat. files
(
example )
-
The z at IP should be small. We are looking for RF shifts in SLC that cause
a 5 cm shift (units here are cm).
-
Pol errors should be zero.
-
L/R asymmetries should be flagged if more than 4 sigma from zero.
-
The number of Z passing the 'Energy' filter should also be 1:2 with the
LUM bhabhas (from the Filter STATS file).
-
All diagonal elements of the filter correlation matrix should be non-zero
(for a reason number of input events).
-
Examine recon histograms
Most of the RECON histograms are the same ones used for the RTH online
monitoring system, and undergo the same statistical comparison to the 'expected'
distributions. Histograms that fail this comparison are flagged for the
monitor's further investigation. (Not implemented yet).
-
Plot ID=DCHD should show a flattish shape in the barrel with maybe a 20-30%
rise to the endcaps.
-
Plot ID=DTFD should show a flat phi.
-
Plot ID=TZND should show a sharp spike near zero.
-
Plot ID=DERD should show a smooth efficiency for putting hits on tracks
vs CDC layer number. A dip in layers 0 and 1 are expected.
-
Check the Event Displays
Both raw and reconstructed data from the KAL and DC systems are displayed
for a sample of hadron candidates. The sample is selected from events that
pass either hadron filter; it shoots to randomly select about 5 events;
if there are less, all are displayed.
Each event gets three views: the first is a 3-view with KAL cluster
hits and CDC tracks shown; the 2nd is a blowup of the CDC with tracks and
vectored hits; the 3rd shows a blowup of the VXD and CCD hits on tracks.
These are just there for visual verification that the detector is OK;
no event classification is called for!
Look for lots of extra vectored hits in the tracking or missing layers.
Make sure CCD hits are being associated to CDC tracks.
Do
it (Check pass 2 Reconstruction )
ZCHKDAY
This stage runs ZXFIND on all the PASS2 output events for a given day.
It looks at the primary vertex position, gamma conversions and K-shorts,
and track muliplicities and VXD linking efficiency. Default parameters
are used in ZXFIND.
Output diagnostics from this stage include statistics
files ( example ) :
-
IP positions should be within errors of previous days' values.
-
Ratio of gamma conversions at VXD vs CDC inner wall should be around 2.
-
Should be about 0.15 gamma conversions/Z
-
Should be about 0.20 V/Z
-
The V mass should be with errors of 0.498 and width around 6 MeV.
-
The total track multiplicity should be around 18. VXD inefficiency around
4%.
And Histograms
-
Plots XPRI, YPRI and ZPRI should be single peaked gaussians. Plots XVSN,
YVSN and ZVSN should show no time dependence.
-
Plots VEP, VECT and VEPH should show no areas of inefficiency.
-
Peruse the other plots to see if they match your physics intuition.
Links
to ZCHKDAY Output
creates a table of links from which you can browse statistics and histogram
files from the previous day's ZCHKDAY job.
Unanalyzed Runs
-
Unanalyzed
Runs
-
shows information on runs never seen by the offline, runs currently being
analyzed and runs with analysis in suspended state. The SLC estimate for
the Z count is given.
Offline Shift Report
An example.
Quantities that need to be verified are:
-
polarization magnitude & error rate
-
SLC luminosity estimates agree with SLD LUM & Z rates
-
Beam spot z centered
-
Event Displays show no diseases
-
RTH histograms check out or have been reported
-
CDC HV-on fraction is high
The last three items, plus an explicit overall evaluation, are asked for
in the report. Other problems should be reported in the comments section.
-
Generate
an Offline Shift Report
-
prompts you for some basic information then generates the report form.
-
Old
Offline Shift Reports
Erez Etzion
Last Updated: