E.1 The STScI Reduction and Calibration Pipeline

In this appendix, we summarize the basic reductions and calibrations that are performed in the STScI WFC3 pipeline. The material in this appendix is intended to provide only enough background to develop robust observing proposals. The WFC3 Data Handbook provides more detailed information needed for analyzing your data.

Science data taken by WFC3 are received from the Space Telescope Data Capture Facility and sent to the STScI OPUS pipeline, where the data are unpacked, keyword values are extracted from the telemetry stream, and the science data reformatted and repackaged into raw (uncalibrated) FITS files by the generic conversion process. All WFC3 science data products are two-dimensional images that are stored in FITS image-extension files. Like ACS and STIS images, WFC3 UVIS channel exposures are stored as triplets of FITS image extensions, consisting of science (SCI), error (ERR), and data quality (DQ) arrays. There is one triplet of image extensions for each CCD chip used in an exposure. Full-frame exposures, using both chips, therefore have two triplets of SCI, ERR, and DQ extensions in a single FITS file. UVIS subarray exposures, which use only one CCD chip, have a single triplet of extensions in their FITS files.

WFC3 IR channel exposures use the NICMOS file structure, which are quintuplets of FITS image extensions, consisting of science (SCI), error (ERR), data quality (DQ), number of samples (SAMP), and integration time (TIME) arrays. There is one quintuplet of extensions for each of the non-destructive detector readouts that make up an IR exposure. Using the maximum number of readouts (16) in an IR exposure therefore results in a single FITS file containing a total of 80 image extensions.

The raw, uncalibrated FITS files are processed through calwf3, the software task that calibrates the data for individual exposures, producing calibrated FITS files. Exposures that are obtained as part of an associated set, such as dithered images, have calwf3 calibration applied to the individual exposures before being processed as a set for the purpose of image combination. All calibrated images will be processed further with the STScI AstroDrizzle software, for the purpose of removing geometric distortions from individual exposures and for combining associated exposures.

The FITS file name suffixes given to WFC3 raw and calibrated data products are described in Table E.1 and closely mimic the suffixes used by ACS and NICMOS. The initial input files to calwf3 are the raw (RAW) files from generic conversion and the association (ASN) table, if applicable, for the complete observation set. For WFC3/UVIS images, a temporary file, with the suffix “BLV_TMP,” is created by calwf3 once bias levels have been subtracted and the overscan regions trimmed. This file is renamed using the “FLT” suffix after the remaining standard calibrations (dark subtraction, flat fielding, etc.) have been completed. For exposures taken as part of a UVIS CR-SPLIT or REPEAT-OBS set, a parallel set of processing is performed, using the BLV_TMP files as input to an image combination and cosmic ray rejection routine. The resulting CR-combined image, with a temporary file name suffix of “CRJ_TMP”, then receives the remaining standard calibrations, after which it is renamed using the “CRJ” suffix.

Table E.1: WFC3 File Name Suffixes.

File Suffix	Description	Units
_RAW	Raw data	DN
_ASN	Association file for observation set
_SPT	Telemetry and engineering data
_TRL	Trailer file with processing log
_BLV_TMP	Bias subtracted individual UVIS exposure	DN
_CRJ_TMP	Uncalibrated, CR-rejected combined UVIS image	DN
_IMA	Calibrated intermediate IR exposure	e–/s
_FLT	Calibrated individual exposure	e– (UVIS) e–/s (IR)
_CRJ	Calibrated, CR-rejected, combined UVIS image	e–
_DRZ	Calibrated, geometrically-corrected, dither-combined image	e–/s

Processing of WFC3/IR exposures results in an intermediate MULTIACCUM (IMA) file, which is a file that has had all calibrations applied (dark subtraction, linearity correction, flat fielding, etc.) to all of the individual readouts of the IR exposure. A final step in calwf3 processing of WFC3/IR exposures produces a combined image from the individual readouts, which is stored in an FLT output product file.

The UVIS processing portion of calwf3 is based on calacs calibrations of ACS/WFC CCD images, while the calwf3 IR processing is very similar to calnica processing of NICMOS images. Calwf3 performs the following basic science data calibrations:

•	Bad pixel flagging

•	Bias level subtraction (UVIS); Reference pixel subtraction (IR)

•	Bias image subtraction (UVIS); Zero-read subtraction (IR)

•	Dark current subtraction

•	Non-linearity correction

•	Flat field correction and gain calibration

•	Shutter shading correction (UVIS only)

•	Up-the-ramp fitting (IR only)

•	Photometric calibration (keyword updates only)

•	CR-SPLIT/REPEAT-OBS image combination

As noted in the list above, the details of some calibration steps differ for UVIS and IR exposures, while others do not apply at all. The process of bias subtraction, in particular, differs for UVIS and IR exposures. The UVIS channel CCDs include regions of overscan, which are used for measuring and subtracting the overall bias level from each CCD exposure. A bias reference image is also subtracted from each science exposure to remove spatial variations in the bias. For IR exposures, the reference pixels located around the perimeter of the detector are used to track and remove changes in the overall bias level between readouts, while the image from the initial (“zeroth”) readout of the exposure is subtracted from all subsequent readouts to remove spatial bias structure.

UVIS shutter shading correction is in principle only necessary for very short duration exposures. Note, however, that testing has shown that the shading amounts to only a 0.2-0.3% variation across the field and therefore this step is normally not applied.

Up-the-ramp fitting is applied to IR exposures to determine a final signal rate for each pixel in the image. This process not only determines the best-fit rate from the individual readouts of the exposure, but also detects and removes effects due to cosmic-ray hits. This process is also capable of recovering a useful signal for pixels that go into saturation during the exposure by using only the non-saturated readouts to compute the fit.

WFC3 grism observations are handled in a special way by the pipeline. Grism observations require a special flat-fielding procedure, where the flat-field value for each pixel is based on the wavelength of the detected signal. Calwf3 processing of grism images therefore uses an “identity” flat-field reference image (an image filled with values of 1.0 at each pixel), which allows for the gain calibration part of the flat-fielding step to still be applied without actually flat-fielding the science image. A separate software package, aXe, which was developed at ST-ECF and previously used for processing NICMOS and ACS spectral observations, is used to extract and calibrate one-dimensional spectra from WFC3 grism exposures (see Section 8.5). The aXe software is available in STSDAS via PyRAF. The aXe software is used to locate and extract spectra of individual sources from calibrated images and performs wavelength calibration, background subtraction, flat fielding, and absolute flux calibration for the extracted spectra.

Table E.2 shows the values assigned to pixels in the DQ arrays of calibrated images, which indicate anomalous conditions and are frequently used in downstream processes to reject a pixel value. If more than one data quality condition applies to a pixel, the sum of the values is used. Note that some flag values have different meanings for UVIS and IR images.

Table E.2: WFC3 Data Quality Flags.

FLAG Value	Data Quality Condition
	UVIS	IR
0	OK	OK
1	Reed-Solomon decoding error	Reed-Solomon decoding error
2	Data replaced by fill value	Data replaced by fill value
4	Bad detector pixel	Bad detector pixel
8	(unused)	Unstable in zero-read
16	Hot pixel	Hot pixel
32	CTE tail	Unstable photometric response
64	Warm pixel	Warm pixel
128	Bad pixel in bias	Bad reference pixel
256	Full- well saturation	Full-well saturation
512	Bad or uncertain flat value	Bad or uncertain flat value
1024	Charge trap	(unused)
2048	A-to-D saturation	Signal in zero-read
4096	Cosmic ray rejected by AstroDrizzle	Cosmic ray rejected by AstroDrizzle
8192	Cosmic ray detected during CR-SPLIT or REPEAT-OBS combination	Cosmic ray detected during up-the-ramp fitting
16384	Pixel affected by ghost or crosstalk	Pixel affected by crosstalk