|
| [ Intro | Pubs | WebTables | Workbook | Software | SLACVX | Search ] |
| [ WebTables : Help | Maintenance ] |
A few general comments about WTINTERMED.COM are given here. For more details, study the extensive comments in the exec itself.
The Intermediate Exec is the intermediary that handles all of the mechanics of plotting and tabulating.
Since several users may be using the Run Information System almost simultaneously, unique filenames must be constructed for each user. Filenames are made unique by including a string that is composed of the job start time in hundredths of a second. The resulting integer is the * in the above filenames.
The Intermediate Exec is designed to run either on an Alpha or a Vax.
http://www-sld.slac.stanford.edu/fastbin/wtintermed ...
http://www-sld.slac.stanford.edu/htbin/wtintermed ...
The Intermediate Exec begins by unpacking the data from the URL and
creating some filenames for temporary files and output files.
It is able to use the standard CGI_SYMBOLS executable to unpack the symbols.
It does not use the special SLD_SYMBOLS version needed by WTFORMREAD.COM,
the Interface Form Reading Exec.
Like all of the executables called by WTINTERMED.COM, CGI_SYMBOLS comes in two flavors,
Alpha and Vax.
One can set the Oracle Interface Exec to return debug output rather than
normal output by prefacing the formtitle in the URL
with the string "DB3" or "DB4".
Normal table or plot output will be interrupted, but useful debug output
will then be found as disk$scr0:[sldwww]oo*.tmp
Both options turn tracing on in WTQUERY.
The datanames part of the URL is parsed to create a number of strings for input to the Oracle Interface Exec, for page, plot and column headers and for input to the Column Math Engine, COLMATH.EXE.
Various quantities are indexed both by COLMATH input column and by COLMATH output column. The two indices differ when COLMATH is asked to divide one column by another and output the result.
For each dataname, the system searches for an appropriate glossary file. If such a file is found, the comment lines at the top of the file yield information for use in titles, rounding, error bar handling and default y scale limits. If no such glossary file is found, we still allow the dataname but we have to use the dataname instead of a proper title string, and we have to rely on defaults for rounding and so forth.
It is an important feature of the Run Information System that it does not insist that the dataname be in DATANAMES.DAT and therefore have a glossary file. Any dataname in one of the Oracle tables defined in the DATANAMES.DAT file will be allowed. This means that one never needs to rush to update DATANAMES.DAT just because someone wants to see another part of an already supported Oracle table.
Datanames of datatype LINK have some special handling. The set of datanames passed to Oracle in the string data_for_oracle does not end up containing the same datanames as came in to the Intermediate Exec. For each of these LINK datanames, a second dataname is to be found in the additional parts of the datatype statement. This second dataname, rather than the one fed in to the Intermediate Exec, is what is to be passed to Oracle. The Intermediate Exec will then build a column of output by substiting the resulting Oracle data into a predetermined pattern (also contained in the additional parts of the datatype statement). Details are given later in this section.
Another task that is performed during the datanames parsing is the decision of what datanames, if any, to omit to create PLOT output. The current plotting engine can only handle plots of one quantity versus another. The x axis must contain a number or a date. The y axis must contain a number.
We still carry the complete set of datanames in the URL so that the user can toggle between seeing output as a TABLE and seeing it as a PLOT. The omissions affect the PLOT output, but if the user then selects, TABULATE, they still see everything they initially selected.
The actual Oracle query is not done by the Intermediate Exec. It is done by a query engine that lives on Unix, WTQUERY, the Oracle Interface Exec.
The Intermediate Exec packs all of the query input into a URL almost the same as
the one by which the Intermediate Exec itself was called.
(That URL is described in detail in the document:
Constructing Plot and Table URLs by Hand).
The only difference is that, for the new URL:
The new URL is called using the "vmsproxy" system developed by
Tony Johnson.
Like all of the executables called by WTINTERMED.COM, VMSPROXY comes in two flavors,
Alpha and Vax.
The data returned by the Oracle Interface Exec is trapped in an output file of the form OO*.TMP (for Oracle Output) by a temporary reassignment of SYS$OUTPUT. It, and all of the other temporary files, is kept on disk at least five minuites, enough time to allow subsequent parts of the process to use the file. Even then, the file is not deleted at the end of WTINTERMED but is only deleted the next time WTINTERMED is called. This has the effect of leaving the temporary files around for debugging purposes.
The Intermediate Exec next does some of the work involved in creating the resubmission form. It calls the Range Layout Exec, WTRANGEWRITE.COM, and the Qualifier Layout Exec, WTQUALWRITE.COM. The resubmission form is initialized with the same values as the original submission.
For TABLE output, the entire resubmission form layout is completed before the table is outputted. This is done because, for some long tables, the user may choose to stop their browser before the table is finished. By completing the form layout before starting the table output, users who stop their browser in this way will still have a usable resubmission form.
For PLOT output, the resubmission form is left incomplete until the plot output is complete so that the YMIN and YMAX input areas can be placed beneath the plot.
The output from the Oracle Interface Exec, OO*.TMP, is then read and merged with any non-Oracle columns of data (datatype CONSTANT).
A column of data is merged into the Oracle output if there was a dataname of datatype CONSTANT. The constant is the dataname name itself.
If the datatype was DATE, the column of data must be converted from Oracle date format to Topdraw date format. This format is integer month followed by decimal point followed by integer day. For example, 31-JAN becomes 1.31.
The new set of columns ends up in a file of the form CI*.TMP (for Column math Input).
The next step in forming the plots or tables is to handle any math that needs
to be performed on the columns of data.
This occurs when the user has requested ratios or when errors must be
calculated.
Since DCL cannot perform math on real numbers,
this math processing is done in a C program,
the Column Math Engine.
Like all of the executables called by WTINTERMED.COM, COLMATH comes in two flavors,
Alpha and Vax.
The C program reads the columns of data from CI*.TMP, performs the necessary calculations, and outputs the data to another temporary file, CO*.TMP (for Column math Output).
The program currently only handles only division and Poisson errors, but could be expanded to handle other mathematical operations. In many cases, it is easiest to instead have Oracle do the math. This can be done by encoding the operations in the SELECT line of the appropriate DATANAMES.DAT block.
Data is sent through the column math engine regardless of whether there is any work to actually be performed on a given column. This takes no significant time and allows for simplified program flow in WTINTERMED.COM. (Columns of data for which no operation needs to be performed are handled as strings, written back out just as then came in.)
If the user has requested a plot, the output from the Column Math Engine is converted to a file suitable for input to Topdraw, TI*.TD (for Topdraw Input).
If there are any gaps in the list of runs in a Plot by Run, extra work must be done to bring the histogram down to YMIN for missing runs. Otherwise, Topdraw draws a misleading stair-step in the missing run range. It uses the previous run's value for the first half of the gap and the next run's value for the second half of the gap. It would be good to include a similar fix for gaps in the list of dates in a Plot by Date, but date manipulation is too ugly to handle.
A special version of Topdraw is called
to do the actual plotting.
Its output is a file of the form TO*.GIF (for Topdraw Output).
This file is displayed as an inline image.
Like all of the executables called by WTINTERMED.COM, TOPDRAW comes in two flavors,
Alpha and Vax.
If the user has requested a table or sum, the output from the Column Math Engine is converted to a table. Whenever possible, the table headers are links to glossary files.
As mentioned earlier, both datatypes VMSFILE and LINK can result in output that is a link to a file on disk.
The Intermediate Exec then goes on to check whether the given file is on disk, has been shelved (by the Hierarchical Storage Manager) or is missing.
If the Oracle output for a given column was the word "v-a-r-i-o-u-s", the word "varous" is written out with html markup that makes it a link to the relevant part of the WebTables help file.
The final step of table output is to decode various special character sequences
that are described earlier in the section on DATANAMES.DAT.
The order of operations in this sequence is important.
To see a breakdown of where time is being spent in table and plot creation, change the show_timing line at the top of WTINTERMED.COM to
show_timing = "Y"
A first step in debugging the Intermediate Exec is to study the various temporary and output files mentioned above. All of these files are preserved for at least the last five minutes (subsequent queries trigger a delete/before=-:05).
For more information on debugging the Intermediate Interface Exec, see To Debug a COM File Running on the WWW-SLD Web Server.