Workbook for SLD Offline Users - Compile, Link and Run

The previous section of the Offline Users Workbook showed you how to write Prepmort code. Next you need to learn how to incorporate your new code into an IDA job. This just involves learning one new command, EXPEDITE, that you issue from the DCL prompt. This one command works both to compile your code and to link your code into the overall structure of IDA (using something called "shareables").

Users who are just looking for quick hints on how to convert older jobs to run under the new system may want to look at another document, Converting from BUILD and SANECTRL to Using Shareables.

This section of the workbook teaches the use of the EXPEDITE command, provides useful background information, and has you compile, link and run an example Prepmort routine.

The EXPEDITE command was created in 1995. Users who learned to compile and link SLD code earlier were taught the commands PREPMORT and MAKESHR. Information on those commands has now been moved to subsections at the ends of the discussion of Compile and Link. New users should learn EXPEDITE rather than these older commands. It does the same work in a more user-friendly way.

Up to now, we have not had to make much distinction between running on a VAX and running on an Alpha. But now that you are learning about compilation and linking, you need to pay attention to which of these two types of machines you are using. There is no difference between the commands that you will use on the two different machines, but you cannot change from one kind of machine to the other between compiling, linking and running.

the output from the Alpha compiler can only be linked on the Alpha
the output from the Alpha linker can only be run on the Alpha
the output from the VAX compiler can only be linked on the VAX
the output from the VAX linker can only be run on the VAX

These differences arise because, even though the VAX and the Alpha use a common operating system, VMS, they have very different machine languages (the VAX is a Complex Instruction Set machine, the Alpha is a Reduced Instruction Set machine).

The example code you will use in this section is the Prepmort you saw in Writing Code: Making Histograms from Prepmort. You will compile it, link it and then run it from IDA to create and fill two histograms.

Is Your Account Correctly Configured?

Before you go on, go through the following procedure to confirm that your VMS account is correctly configured to compile and link SLD Offline code.

Check that the Logical Name `DUCS$WORK` is Correctly Defined

To compile and link SLD Offline code, you need to have the logical name DUCS$WORK correctly defined. New users are generally given accounts that have this logical name correctly defined. Users who have had their VMS accounts for a long time before using SLD Offline code may not have this logical name correctly defined.

Type SHOW LOGICAL DUCS$WORK

If the result is something like

     %SHOW-S-NOTRAN, no translation for logical name DUCS$WORK

you are missing this logical name.

If the result does not point to a subdirectory ending in .DUCSWORK on your scratch disk, you have this logical name defined badly.

If this logical name is missing or is incorrect, fix it now as follows:

Create a subdirectory of your scratch disk called .DUCSWORK. To do this, use the command CREATE/DIR.
Study the example LOGIN.COM in
```
     PRODUTIL:LOGIN.COM
```
Copy the line that begins
```
     $!Define/Nolog Ducs$Work $scr0:[Tony.ducswork]
```
to a similar area in your LOGIN.COM. Remove the ! to uncomment the line and change the directly specification to your own DUCSWORK directory.
Either run your LOGIN.COM or type the DEFINE statement by hand to make the new definition take effect immediately.

You can actually set the DUCS$WORK logical to any directory that you want. But be very careful before using any other directory than the one suggested above. The commands that link your code into IDA use the DUCS$WORK directory to do various file manipulations. They need plenty of space to work in and they erase all of the files from this directory before they start.

Any other code you have in this directory will be erased.

Check that You Have Enough Disk Quota

The IDA jobs in the previous sections of this Workbook have not required a very large disk quota. But from here on, you will need more space. If you are not sure how much disk quota you have, check it now with the command SHOW QUOTA.

A reasonable quota for people writing Prepmort for SLD is 20,000 blocks. You will probably also be OK with 10,000, but anything less will cause you unnecessary hardships. Now is the time to demand more. If necessary, point out to your system manager that 20,000 blocks is only 10 MBytes and should therefore cost less than $10. Repeat to them the words, "disk space is cheap, disk space is cheap."

Compile using the EXPEDITE Command

You can do the following work from any directory, but the job will be easier to understand if you start with a directory that is empty.

SET DEFAULT to that directory.

Copy the example Prepmort program to that directory.

   > COPY SLACVX::DISK$SLD_FAC0:[SLDWWW.WORKBOOK]WBPREP1.PREPMORT *

Take a look at the file so that you understand what it is supposed to do.

Notice that the header section of the file contains the statement SECTION: USER. This declares what DUCS code section this file will belong to. The USER section is a special DUCS section reserved for user code.

Before you can compile this code, you need to issue a DUCS command to tell DUCS where you want to keep code for this DUCS section. Issue the following command to tell DUCS you want to keep USER code in your current directory:

   > DUCS TEST USER *

You can now compile the routine using the EXPEDITE command.

   > EXPEDITE WBPREP1.PREPMORT

You should get the following response:


   Action--> PREPHELP WBPREP1.PREPMORT TESTUSER
   Action--> @DUCSUDUC:DUCSLIB TESTUSER:USER WBPREP1.OBJ
   %LIBRAR-S-REPLACED, module WBPREP1 inserted in your_directory:USER.OLB

The first line tells you that the compiler was invoked. The last two lines tell you that the resulting object file was copied into a USER object library. This is a library where any compiled code from the USER section will be kept.

If you then do a DIR, you should have four files in your directory:

   WBPREP1.MORTFORT;1      WBPREP1.PREPMORT;1
   USER.OLB;1              USERSHR.VEC;1

The WBPREP1.MORTFORT file is an intermediate step from the compilation process. It is left behind for use by the debugger (to be discussed later in this section of the Workbook).
The USER.OLB file is the object library where all code in the USER section will be kept. It currently just contains your WBPREP1 routine, but if you were to then compile some other routine in the USER section, this routine would be added to this same USER.OLB.
The USERSHR.VEC file is a set of instructions which are used by the linker when you go on to the next part of the Compile, Link and Run process. This VEC file is created for you automatically only when you compile code in the USER section. When you compile code in other DUCS sections, you make this VEC file by hand, generally just copying and modifying a version already in DUCS.

To see how the above example would be handled using the older compile command, PREPMORT, see the detailed section, Compile using the Old Command, PREPMORT.

The EXPEDITE command works both for code written in any of several different languages, PREPMORT, MORTRAN, FORTRAN or C. It will invoke whatever compiler is appropriate for the given language.

For further details on the user of EXPEDITE, see the detailed section, the EXPEDITE Command.

If your project involves compiling routines from many different DUCS sections, you will get additional helpful features by running EXPEDITE within the overall framework of the PROJECT system described in another section of this Workbook.

Link using the EXPEDITE Command

You need to run one more EXPEDITE command to make the code that is in your object library available to IDA. This command is EXPEDITE USERSHR.VEC.

The command reads instructions from your USERSHR.VEC and from another file already in DUCS called USERSHR.OPT. These instructions cause EXPEDITE to turn the compiled code from your USER.OLB into something called a "Shareable Image" suitable for execution. This process is called "linking."

Before we go on, here is a little background on shareable images.

Background on Shareable Images

Before code can be run, all of the object code needs to be converted into executable code. The object code is collected together and information is added about how each routine can reference the other routines and system functions it needs to call.

In many software environments, all of the object code is linked together into a single "Executable Image," a large piece of machine language code containing all of the routines and stored on disk. At run time, this one large executable image is loaded into main memory and then executed. Any user who changes any routines needs to have their own whole copy of this executable image on disk.

The SLD Offline software environment involves too much code to conveniently run in this way. There are so many thousands of routines involved that linking them all together is very slow and the resulting executable image is too large to conveniently fit on disk. Also, this one large executable image takes a long time to load into main memory and consumes a large amount of memory.

To avoid these problems, the SLD Offline instead uses a linking concept called "Shareable Images." The executable image is stored in about fifty small pieces rather than in one large piece. At run time, these smaller executable images are loaded into main memory only as they are needed.

When you start IDA, just a few pieces of the system are loaded into main memory. Other pieces of the executable image, such as the piece containing the MFIT Minuit Fitting Package, are only loaded later and only if you actually try to use them. Since no one IDA session actually uses all of the pieces of the SLD Offline software system, the individual user avoids wasting time and space loading unnecessary code.

Each shareable image contains information about what other shareable images it needs to have loaded with it. These other shareable images are loaded automatically as they are needed.

Shareable images not only save loading time and loading space, they also save linking time and disk space. When you make a new routine or make a change to a routine, you only need to relink a single one of the shareable images. And you only need to use up your disk space to store those shareable images that you have changed. The rest of the shareable image files are just taken from DUCS, from a common place where all users can share them.

Shareable images are sometimes referred to just as "Shareables." They basically contain a piece of the overall executable image plus a set of instructions on how they connect to the other shareable images.

An Example Link

The same command that you used to compile your code can also be used to link it into a shareable image. EXPEDITE looks at the file type to determine whether to compile or link.

When you compiled WBPREP1.PREPMORT, the file USERSHR.VEC was made for you. To link the code from your USER object library into a shareable:

   > EXPEDITE USERSHR.VEC

Keep in mind that if the compilation was performed on an Alpha, the link must be performed on an Alpha. If the compilation was performed on a VAX, the link must be performed on a VAX.

You should get the following response:


Action--> @DUCSUDUC:UBLDSHR USERSHR USER TEST
Action--> COPY USERSHR.EXE TESTUSER:

The first line tells you that the linker was invoked. The second line tells you that the resulting shareable image file was copied from the DUCS$WORK directory into your USER directory.

You should have two new files in your directory:

The USERSHR.EXE file is your shareable image.
The USERSHR.SETUP_COM file defines some logical names needed by IDA to recognize your shareable image. We will discuss more about the SETUP_COM file a little later in this section of the Workbook.

To see how the above example would be handled using the older link command, MAKESHR, see the detailed section, Link using the Old Command, MAKESHR.

Routines written in the languages C, FORTRAN and MORTRAN can be linked the same way as Prepmort routines.

For further details on the use of EXPEDITE, see the detailed section, the EXPEDITE Command.

In the above example, the .VEC file you needed was generated for you when you ran EXPEDITE to compile your code. As was mentioned, this is only done for you when your code is in the USER DUCS section. You also benefitted from an .OPT file that was already in DUCS.

There is a lot more to .VEC and .OPT files that was not covered here. You will need to learn more about these files when you want to add code to DUCS sections other than USER or when you need to have your code call certain other routines already in DUCS. You may encounter error messages about "routine not found" when you link or run more advanced shareables. At that point, you will find more help in the detailed Workbook section, OPT and VEC Files.

Run

If you now just start up IDA, you will find that you can call your Prepmort routine. You use the External command that you learned about in IDA Continued.

Start up IDA and test the routine as follows:

   Ida>  EXTERNAL WBPREP1
   Ida>  OPENTAPE READ REC94_MDST STAGE WAIT

   Ida>  DEF EVANAL
   Ida>    CALL WBPREP1
   Ida>  ENDDEF

   Ida>  GO 50

   Ida>  HCAT

When IDA reads the command EXTERNAL WBPREP1, it notes that it does not yet have such a routine loaded. It then follows a trail of logical names left behind by MAKESHR and determines that such a routine can be found in your shareable image WBPREP1.EXE. It then loads your shareable image.

Since the routine WBPREP1 creates two histograms, the HCAT should give the result:

   ID     Range        Bins  Plot Under  Over   Title
   101(    0,  400)      40   908     0     0    Transverse Momentum (PT)
   102(    0, 2000)      40    -1    -1    -1    PT versus Tan(lambda)
      (   -2,   78)

You can then view these histograms using the familiar IDA command HOUT.

As was mentioned above, you may encounter error messages about "routine not found" when you link or run more advanced shareables. At that point, you will find more help in the detailed Workbook section, OPT and VEC Files.

Users who are just looking for quick hints on how to convert older jobs to run under the new system may want to look at another document, Converting from BUILD and SANECTRL to Using Shareables.

The SETUP_COM file

The logical names that tell IDA how to load your new Prepmort routine only exist for the current VMS session. That is, IDA will only accept the command EXTERNAL WBPREP1 if it is run in the same session that you used for the EXPEDITE USERSHR.VEC command.

If you need to use the shareable from any other session including batch jobs, you must first restore any special DUCS settings you had and then run the SETUP_COM file that was created by EXPEDITE USERSHR.VEC.

As an example, try starting IDA from another session without running this SETUP_COM file. When you type EXTERNAL WBPREP1, IDA will respond:

   %UCALL-F-NOTFOUND Routine WBPREP1 cannot be found in current image
   IDA error executing EXTERNAL (4)

Now exit IDA and issue the following two DCL command:

   > DUCS TEST USER *
   > @USERSHR.SETUP_COM

The first command is the one you did earlier to tell DUCS where you are keeping your USER code. The second command recreates some logical names that were originally created for you by the command EXPEDITE USERSHR.VEC.

Once you have done those two steps, all subsequent IDA jobs from this session will accept EXTERNAL WBPREP1.

When you submit a job to batch, include this SETUP_COM by calling it from the main COM file.

For example, the following COM file could be submitted to batch to set up and run our IDA job:

   $DUCS TEST USER *
   $@USERSHR.SETUP_COM
   $IDA
    EXTERNAL WBPREP1
    OPENTAPE READ REC94_MDST STAGE WAIT

    DEF EVANAL
       CALL WBPREP1
    ENDDEF

    GO 50

    HOUT ALL POSTSCR
    QQUIT

Remember to include a CHAR=VAX or CHAR=ALPHA option in your SUBMIT command. This is necessary to make the job run on the same kind of machine that you compiled and linked your code on.

For complicated projects, where you have modifications to several shareables and there are many SETUP_COM files involved, the PROJECT command can help keep things straight. It is discussed in another section of this Workbook.

Installing New Code to DUCS

Eventually you will have code that you have written that will be of use to the rest of the collaboration. At that point, you can arrange to have the code installed into DUCS. Your code then goes out over the Internet to all of the computer sites in the SLD collaboration.

Each DUCS Section has at least one person charged with maintaining that subset of the SLD code. You can generally find out who they are by checking the list of people charged with responding to bug complaints. This is in a document called List of SLD GNATS Maintainers.

Most new code fits into one of the existing DUCS sections. In rare cases it may be considered worthwhile to create a whole new DUCS section.

When new code is installed into DUCS, it typically goes first to the relevant "development" area. Here it becomes available to any user who explicitly asks for it (who issues the relevant DUCS DEV command). Users who do not ask for it do not see the new code.

Once the code has been tested by a variety of people, it can be moved to the relevant "production" area. Only a few people have privileges to move code to production. If the person who first installed the code to development does not have these privileges, they will know who does.

Do not hold your useful code back where only you can use it. If it seems of general use to the group, put it out into DUCS.

Back to Workbook Front Page

Joseph Perl
15 September 1995