Science data taken in orbit by WFC3 are received from the Space Telescope Data Capture Facility and sent to the STScI OPUS pipeline, where the data are unpacked, keyword values are extracted from the telemetry stream, and the science data reformatted and repackaged into raw (uncalibrated) FITS files by the Generic Conversion process (see
Section 1.1.1
of the
Introduction to the HST Data Handbooks).
All WFC3 science data products are two-dimensional images that are stored in Multi-Extension FITS format files. For each exposure taken with WFC3, there is one FITS file with a unique 9-character rootname followed by a 3-character suffix:
rootname_xxx.fits. The rootname identifies the observation and the suffix denotes what type of file it is (see
Chapter 5
of the
Introduction to the HST Data Handbooks for more details on
HST file names).
|
•
|
An exposure is a single image file, the atomic unit of HST data.
|
|
•
|
A dataset is a collection of files having a common rootname.
|
|
•
|
A sub-product is a dataset created by combining a subset of the exposures in an association.
|
|
•
|
A product is a dataset created by combining sub-products of an association.
|
The initial input files to the calibration program calwf3 are the raw files (
raw) from Generic Conversion and the association (
asn) table, if applicable, for the complete observation set.
For UVIS images, a temporary file, with the suffix “_blv_tmp”, is created once bias levels are subtracted and the overscan regions are trimmed. This file will be renamed with the “
_flt” suffix after the standard calibrations (flat-fielding, dark subtraction, etc.) are complete. The “
_blv_tmp” files serve as input for cosmic ray rejection, if required. For UVIS
CR-SPLIT and
REPEAT-OBS exposures, a temporary CR-combined image (
crj_tmp) is created and then renamed with the “
_crj” suffix once basic calibrations of that image are complete.
Processing of IR exposures results in an intermediate MultiAccum (ima) file, which is a file that has had all calibrations applied (dark subtraction, linearity correction, flat-fielding, etc.) to all of the individual readouts of the IR exposure. A final step in
calwf3 processing of IR exposures produces a combined image from the individual readouts, which is stored in an
flt output product file.
MultiDrizzle is used to correct all WFC3 images for geometric distortion, whether they are taken as single exposures or as part of an association. For
CR-SPLIT and
REPEAT-OBS,
MultiDrizzle supersedes the
calwf3 cosmic-ray rejection processing and uses the individual
flt files directly as input, performing cosmic-ray rejection in the process of producing the final drizzled image from multiple exposures (see
Table 2.2). This has significant advantages in cases where small numbers of
CR-SPLIT images were obtained at a small number of different dither positions, because
MultiDrizzle will use all the information from all the
flt files to produce the best cosmic-ray rejection. The resulting drizzled images should generally be useful for science, although subsequent reprocessing off-line may be desirable in some cases to optimize the data for specific scientific applications.
Table 2.2:
The calwf3 and
MultiDrizzle input and output products.
Association tables are useful for keeping track of the complex set of relationships that can exist between exposures taken with WFC3, especially with
REPEAT-OBS,
CR-SPLIT, and dithered exposures, for both the UVIS and IR channels. Images taken at a given dither position may be additionally
CR-SPLIT into multiple exposures (e.g., UVIS observations). In these cases, associations are built to describe how each exposure relates to the desired final product. As a result, WFC3 association tables can be used to create one or more science products from the input exposures, just like ACS associations. The relationships defined in the association tables determine how far through the calibration pipeline the exposures are processed and when the calibrated exposures get combined into sub-products for further calibration.
The format of WFC3 association tables closely resembles the ACS and NICMOS association format, with three primary columns:
MEMNAME, MEMTYPE, and
MEMPRSNT. The column
MEMNAME gives the name of each exposure making up the association and output product name(s). The column
MEMTYPE specifies the role that the file has in the association. WFC3 uses the same set of
MEMTYPES as ACS to provide support for multiple products. These
MEMTYPES are summarized in
Table 2.3.
A sample association table for a two-position dithered observation with CR-SPLIT=2 is presented in
Table 2.4. This example shows how both
MEMNAME and
MEMTYPE are used to associate input and output products. The
MEMTYPE for each component of the first
CR-SPLIT exposure,
IxxxxxECQ and
IxxxxxEGQ, are given the type
EXP-CR1. The sub-product
Ixxxxx011 is designated in the table with a
MEMTYPE of
PROD-CR1. The last digit of the product filename corresponds to the output product number in the
MEMTYPE. A designation of zero for the last digit in the filename is reserved for the dither-combined product.
The column MEMPRSNT indicates whether a given file already exists. For example, if cosmic ray rejection has not yet been performed by
calwf3, the
PROD-CRn files will have a
MEMPRSNT value of “no”. The sample association table in
Table 2.4 shows the values of
MEMPRSNT prior to
calwf3 processing.
Each task used by calwf3 creates messages during processing that describe the progress of the calibration and are sent to STDOUT. In calibration pipelines written for other
HST instruments, trailer files were created by simply redirecting the STDOUT to a file. Because multiple output files can be produced in a single run of
calwf3, creating trailer files presents a unique challenge. Each task within
calwf3 must decide which trailer file should be appended with comments and automatically open, populate, and close each trailer file.
calwf3 will
always overwrite information in trailer files from previous runs of
calwf3 while preserving any comments generated by Generic Conversion. This ensures that the trailer files accurately reflect the most recent processing performed. The string “
CALWF3BEG” will mark the first comment added to the trailer file. If a trailer file already exists,
calwf3 will search for this string to determine where to append processing comments. If it is not found, the string will be written at the end of the file and all comments will follow. Thus any comments from previous processing are overwritten and only the most current calibrations are recorded.
As each image is processed, an accompanying trailer file with the “.trl” suffix will be created. Further processing with
calwf3 will concatenate all trailer files associated with an output product into a single file. Additional messages will then be appended to this concatenated file. This duplicates some information across multiple trailer files but ensures that for any product processed within the pipeline, the trailer file will contain processing comments from all the input files.
Linking trailer files together can result in multiple occurrences of the “
CALWF3BEG” string. Only the first, however, determines where
calwf3 will begin overwriting comments if an observation is reprocessed.
The support files contain information about the observation and engineering data from the instrument and spacecraft that was recorded at the time of the observation. A support file can have multiple FITS image extensions within the same file. Each extension holds an integer (16-bit) image containing the data that populates the
_spt.fits header keyword values.
