On-The-Fly Calibration/Reprocessing Systems
The On-The-Fly Calibration (OTFC) system, publicly released in Dec. 1999,
calibrated data at the time a user requested data from the Archive. The advantages
to using OTFC included the automatic application of improved calibration files and
switches, use of most recent calibration software (allowing for rapid access to
improved algorithms, new capabilities, and software fixes), and correction of
header keywords if needed. An additional benefit is that only the uncalibrated
data needs to be stored in the Archive.
The On-The-Fly Reprocessing (OTFR) system replaced OTFC on May 16, 2001. The
change is transparent to HST Archive users. Requests for data are submitted as usual
via Starview or WWW; raw and freshly-calibrated data are delivered. There is no need
to explicitly ask for OTFR: all requests for WFPC2 data are handled by the OTFR system.
The primary difference between the two systems is that OTFR begins earlier in the
data path. It uses the original telemetry files ("POD" files) received from
Goddard Space Flight Center and performs all pipeline processing steps; OTFC performed
only the last pipeline processing step (calibration), on raw files retrieved from the
Archive. An overview of the data flow for both systems is summarized in the table below.
The benefits of the new OTFR system encompass the benefits in the OTFC system; in
addition, OTFR data will need fewer header fixups (most problems will be fixed
as part of the pre-calibration pipeline processing) and the system as a whole will
require signficantly less maintenance effort than OTFC.
NOTE: do not run STSDAS task uchcoord (Feb 1996 version or earlier)
on OTFR data.
UCHCOORD and OTFR. In the near-term, observers should not run the STSDAS task UCHCOORD
(Feb. 1996 version or earlier) on data processed through the new OTFR system. This task
updates the header keywords which define the mapping between the pixel and world coordinate
systems. The keywords include the reference pixel locations (crpix*), the values of the
world coordinate system at the reference location (crval*), the partial derivatives of
the world coordinate system with respect to the pixel coordinates (cd*), and the
orientation (orientat). Data processed through OTFR will already have all necessary
corrections performed, and it is possible that UCHCOORD would make additional, un-needed
(and inappropriate) corrections.
To address this issue, a new header keyword, PROCTIME, has been added to OTFR data
to record when the system processed the data. If the keyword PROCTIME is present in the
header, indicating that the data came from OTFR, UCHCCORD need not be run, and should
not be run until the new software version is released. This new version of UCHCOORD
will be part of the summer 2001 STSDAS release; it will correctly handle both old data
and new OTFR data by detecting and using the PROCTIME keyword. UCHCOORD can, of course,
still be run on any of the older OTFC or tape data observers already have (i.e., images
without the PROCTIME keyword). WFPC2 observers with old data are still advised to execute
the STSDAS task UCHCOORD to update the position-related group header keywords.
Comparison of Dataflow in On-The-Fly Systems
|
OTFC
|
OTFR
|
|
Request for data is submitted to the Archive via Starview or WWW interface;
Archive responds with acknowledgement email.
|
Same as OTFC.
|
|
Raw files are retrieved from the HST Archive and passed to the OTFC system.
For WFPC2, the raw files include d0, q0, q1, x0, and trl files.
|
POD file (original telemetry file) is retrieved from HST Archive
nd passed to OTFR system. For WFPC2, there is typically 1 POD file for each image.
Pre-calibration OPUS processing is performed: data partitioning, data editing,
and generic conversion; these steps generate the raw files (d0, q0, q1, x0, and trl files).
|
|
Any problems in the header keywords are fixed by special lookup table.
|
Same as OTFC although the OPUS pre-calibration processing
will have fixed the majority of keyword problems automatically
(i.e., significantly fewer header fixups required in OTFR).
|
|
The best calibration files & switches are
determined by separate standalone task, and header keywords updated accordingly.
|
Not needed. The best calibration files &
switches are set by the pre-calibration OPUS code (generic conversion).
|
|
Images are calibrated by STSDAS calxxx module
and sent back to the Archive system.
|
Same as OTFC.
|
|
Archive delivers raw + calibrated data and emails
completion notification to the requestor.
|
Same as OTFC.
|
|
 |
|
