Wide Field and Planetary Camera 2 Instrument Handbook for Cycle 12

8.7 On-The-Fly Reprocessing Systems

---

The On-The-Fly Calibration (OTFC) system, publicly released in Dec. 1999, calibrated data at the time a user requested data from the archive. The advantages to using OTFC included the automatic application of improved calibration files and switches, use of most recent calibration software (allowing for rapid access to improved algorithms, new capabilities, and software fixes), and correction of header keywords if needed. An additional benefit is that only the uncalibrated data needs to be stored in the archive.

The On-The-Fly Reprocessing (OTFR) system replaced OTFC on May 16, 2001. The change will be transparent to most HST archive users. Requests for data will be submitted as usual via StarView or WWW; raw and freshly-calibrated data will be delivered. There will be no need to explicitly ask for OTFR: all requests for WFPC2 data will be handled by the OTFR system.

The primary difference between the two systems is that OTFR begins earlier in the data path. It uses the original telemetry files ("POD" files) received from Goddard Space Flight Center and performs all pipeline processing steps; OTFC performed only the last pipeline processing step (calibration), on raw files retrieved from the archive. An overview of the data flow for both systems is summarized in the table below. The benefits of the new OTFR system encompass the benefits in the OTFC system; in addition, OTFR data will need fewer header corrections (most problems will be fixed as part of the pre-calibration pipeline processing) and the system as a whole will require significantly less maintenance effort than OTFC.

UCHCOORD and OTFR.

Improved knowledge of the detector plate scales and chip rotations, as well as changes in reference pixel locations, have resulted in periodic changes to the pointing parameters, especially early in the instrument's lifetime. These header parameters, which define the mapping between the pixel and world coordinate systems, can be updated using the STSDAS task UCHCOORD. The keywords affected include the reference pixel locations (crpix*), the values of the world coordinate system at the reference location (crval*), the partial derivatives of the world coordinate system with respect to the pixel coordinates (cd*), and the orientation of the chip (orientat).

Prior to OTFR (released to the public on May 16, 2001), observers requiring the most up-to-date pointing information in their science image headers ran UCHCOORD on their calibrated images. The new OTFR system, however, automatically calculates the best values for these parameters at the time the data are requested so there is no need to run UCHCOORD on freshly-processed OTFR data. OTFR data that have not been recently retrieved (i.e. have been sitting on disk or tape for some time) or pre-OTFR data may benefit from an update; any version of UCHCOORD may be used to update pre-OTFR data, but note that only the June 2001 or later version of UCHCOORD should be run on data processed through OTFR in order to avoid additional, un-needed (and inappropriate) corrections. The version of UCHCOORD released after June 2001 correctly handles both old data and new OTFR data by detecting and using the new keyword PROCTIME which been added to OTFR data to record when the system processed the data.

Table 8.1: Comparison of Dataflow in On-The-Fly Systems

OTFC	OTFR
Request for data is submitted to the archive via StarView or WWW interface; archive responds with acknowledgement email.	Same as OTFC.
Raw files are retrieved from the HST archive and passed to the OTFC system. For WFPC2, the raw files include d0, q0, q1, x0, and trl files.	POD file (original telemetry file) is retrieved from HST archive and passed to OTFR system. For WFPC2, there is typically 1 POD file for each image. Pre-calibration OPUS processing is performed: data partitioning, data editing, and generic conversion; these steps generate the raw files (d0, q0, q1, x0, and trl files).
Any problems in the header keywords are fixed by special lookup table.	Same as OTFC although the OPUS pre-calibration processing will have fixed the majority of keyword problems automatically (i.e., significantly fewer header corrections required in OTFR).
The best calibration files & switches are determined by separate standalone task, and header keywords updated accordingly.	Not needed. The best calibration files & switches are set by the pre-calibration OPUS code (generic conversion).
Images are calibrated by STSDAS calxxx module and sent back to the archive system.	Same as OTFC.
Archive delivers raw + calibrated data and emails completion notification to the requestor.	Same as OTFC.