DARTS Loading Task

Background

With the introduction of the DARTS system a program is needed to handle sending PAM and other types of data files to be loaded into DARTs and receive loading responses back. This documents a task to perform those operations.

This task is based on the standard external data manager (ext.dat.man) file structure. The standard file structure is:

<data_type>/ready/<test_cell_1>
                 /<test_cell_2>
                 /...
<data_type>/hold/<test_cell_1>
                /<test_cell_2>
                /...
<data_type>/complete/<test_cell_1>
                    /<test_cell_2>
                    /...

Each test cell directory will then contain input or response files for that test cell. Test cells are responsible for getting files to the appropriate location under the source "ready" directory.

This program only handles data for a single site and a single data type. Files to be sent to DARTS should go into the appropriate test cell directory under the "ready" directory. Files here will be sent to DARTS for processing. After being sent a file will be moved from "ready" to the appropriate test cell directory under "hold".

The "hold" directory will contain files sent to DARTS and awaiting a response back from DARTS. If there is a file ending with .err but otherwise with the same name as a file in this directory, this file is an error message from DARTS indicating a loading problem. So files under the "hold" directory should be monitored for files not getting responses or files getting error responses.

If DARTS replies with a "success" response, the file in "hold" will be moved to the "complete" directory. Files in "complete" should be cleaned up from time to time.

Managing Load Processing

There are 2 scripts involved in managing this task.

start_darts_loading <property_file> - This script will start this task into operation for a specific configuration.
stop_darts_loading <property_file> - This script will terminate this task.

Each data type sent to DARTS will have its own instance of this program running and its own configuration file. For example PAM will have 3 instances of this running, one for data points, one for headers and one for MUA data files.

This process is controlled by a configuration properties file. A documented sample should be included in the package containing this writeup. See it for further details.

Note To Windows Installers

The data directories should use the backslash (\) character instead of the forward slash character(/).
The stop_darts_loading script is not available for Windows.