HST Data Handbook for WFPC2

3.3 Standard Pipeline Calibration

---

Each calibration step, and the keyword switches used to turn the step on or off, is described in detail in the following sections. These steps are performed in the following order:

1. Flag static bad pixels.
2. Perform analog-to-digital (A/D) correction.
3. Subtract bias level.
4. Subtract bias image.
5. Scale and subtract dark image for exposures longer than 10 seconds.
6. Multiply by flatfield image.
7. Apply shutter shading correction to exposures of less than 10 seconds.
8. Calculate photometry keywords and update calibrated science image header accordingly (does not affect pixel values).
9. Calculate histograms.
10.  Generate final science data quality file.

3.3.1 Calibration Files

Table 3.3 lists the types and related suffixes of the WFPC2 reference files used in calwp2. Most suffixes have the form rNh/rNd, where N is a number that identifies the step in which the file is used. The associated data quality file, if it exists, has the suffix bNh/bNd. The rootname of a reference file is based on the year, date, and time that the file was delivered to the Calibration Data Base System (CDBS).

Table 3.3: WFPC2 Calibration Reference Files

Suffix	Reference File
r0h, r0d	Static mask
r1h, r1d	Analog-to-digital look-up table
r2h, r2d, b2h, b2d	Bias
r3h, r3d, b3h, b3d	Dark frame
r4h, r4d, b4h, b4d	Flatfield
r5h, r5d	Shutter shading
c3t	Photometry table (generated, not required)

 

The file names and history of all WFPC2 reference files in CDBS (and retrievable from the HST Archive) are summarized in the WWW Reference File Memo, updated with each new delivery. Any CDBS file is available for retrieval through the HST Data Archive (see chapter 1 of the HST Introduction). Some older, alternative reference files generated by the WFPC2 IDT are listed in the IDT Reference File Memo, also available on the web. All of the installed reference files contain HISTORY keywords at the end of the header that can be viewed using the imhead task or a standard editor. These HISTORY keywords provide detailed information about how the reference file was generated and when it was installed into the database.

There are history records at the bottom of the header file of the calibrated data as well as each of the calibration reference file headers. These history comments sometimes contain important information regarding the reference files used to calibrate the data in the pipeline.

3.3.2 Calibration Steps

Static Mask Application

Header Switch: MASKCORR

Header Keywords Updated: MASKCORR

Reference File: MASKFILE

The static mask reference file (.r0h/.r0d) contains a map of the known bad pixels and columns; the mask reference filename is recorded in the MASKFILE header keyword in the science header images (.d0h and.c0h). If this correction is requested, (MASKCORR=PERFORM), the mask is included in the calibration output data quality files; the science data themselves are not changed in any way. The STSDAS task wfixup can be used on the final calibrated science image (.c0h/.c0d) to interpolate across pixels flagged as bad in the final data quality file (.c1h).

A/D Correction

Header Switch: ATODCORR

Header Keywords Updated: ATODCORR

Reference File: ATODFILE

The analog-to-digital (A/D) converter takes the observed charge in each pixel in the CCD and converts it to a digital number. Two settings, or gains, of the A/D are used on WFPC2. The first converts a charge of approximately seven electrons to a single count (called a Data Number or DN), and the second converts a charge of approximately 14 electrons to a DN, also referred to as "gain 15" for historical reasons. The precise gain conversion values for each chip are listed in the WFPC2 Instrument Handbook.

A/D converters work by comparing the observed charge with a reference and act mathematically as a "floor" function. However, these devices are not perfect and some values are reported more (or less) frequently than they would be by a perfect device; although the "true" DN values can never be recovered, one can statistically adjust for this systematic bias. Fortunately, the WFPC2 A/D converters are relatively well-behaved and the correction is small (the largest correction is about 1.8 to 2.0 DN for bit 12, i.e., 2048)

The A/D fix up is applied when the ATODCORR keyword is set to PERFORM; the calibration file (lookup table) used to correct for the A/D errors has the suffix.r1h. The file has four groups, one for each detector; each group has 4096 columns (i.e., all possible DN values for a 12 bit detector) and two rows. The first row contains a -1 in the first pixel, followed by the Bay 3 temperature values in the subsequent pixels; the next row (each row corresponds to one temperature) contains the A-to-D conversion corrections for each DN value¹. The example below illustrates determining the A-to-D fix up for a DN value of 450, in WF2, for gain 7, assuming the ATODFILE has already been retrieved from the Archive. Note that the correction for a given DN value is found in the DN+1'th pixel (i.e., [DN+1,2]).

cl> listpix dbu1405iu.r1h[2][451,2] 1. 450.1722

 

Bias Level Removal

Header Switch: BLEVCORR

Header Keywords Updated: BLEVCORR

Group Header Keywords Updated: DEZERO, BIASEVEN, BIASODD

Reference File: BLEVFILE, BLEVDFIL

The charges that are in each pixel sit on top of an electronic pedestal, or "bias", designed to keep the A/D levels consistently above zero. The mean level of the bias must be determined empirically using the extended register (overscan) pixels which do not view the sky. The values of these pixels are placed in the extracted engineering files (.x0h/.x0d). The overscan area used to calculate the mean bias levels is [9:14,10:790], with BIASODD being determined from columns 10, 12, and 14 and a BIASEVEN being determined from columns 9, 11, and 13 (this counter-intuitive nomenclature is due to an offset in the.x0h file; even and odd are correctly oriented with respect to the data file columns). In observations taken before March 8, 1994, the pipeline used a larger part of the overscan region, resulting in occasional oversubtraction of the bias level and possibly large negative pixel values. Separate even and odd bias levels were extracted in data taken and/or processed after May 4, 1994. Note that any data retrieved through the OTFR system will have the correct bias levels determined and removed. The keyword BLEVCORR controls the subtraction of the bias in calwp2.

Bias Image Subtraction

Header Switch: BIASCORR

Header Keywords Updated: BIASCORR

Reference File: BIASFILE, BIASDFIL

The value of the bias pedestal can vary with position across the chip. Therefore, once the mean bias level correction has been completed, the pipeline looks at the keyword BIASCORR. If it is set to PERFORM, then a bias image (.r2h/.r2d) is subtracted from the data to remove any position-dependent bias pattern. The bias reference file is generated by stacking a large set of A/D and bias-level corrected zero-length exposures. The correction consists of subtracting the bias reference file from the observation; any bad pixels flagged in the data quality file for the bias (.b2h/.b2h) are also flagged in the science image data quality file (.c1h/.c1d).

Dark Image Subtraction

Header Switch: DARKCORR

Header Keywords Updated: DARKCORR

Reference File: DARKFILE, DARKDFIL

A dark correction is required to account for the thermally-induced dark current as well as a glow (see section 4.3.2) from the field flattening lens. The dark reference file (.r3h/.r3d) is generated from a combination of a superdark image (a stack of typically 120 dark frames²) and warm pixels identified from a smaller stack of individual dark frames (five, as of ~1996). Prior to stacking, each dark frame is examined and regions affected by image anomalies, such as CTE residual images (see figure 4.2), are masked out. If a dark correction is requested (DARKCORR=PERFORM), the dark reference file, which was normalized to one second, is scaled by the DARKTIME keyword value and subtracted from the observation. By default, DARKCORR is set to "PERFORM" for all exposures longer than 10 seconds, and set to "OMIT" for shorter exposures.

Flatfield Multiplication

Header Switch: FLATCORR

Header Keywords Updated: FLATCORR

Reference File: FLATFILE, FLATDFIL

The number of electrons generated in a given pixel by a star of a given magnitude depends upon the quantum efficiency of that individual pixel as well as on any large scale vignetting of the field-of-view caused by the telescope and camera optics. To correct for these variations, the science image is multiplied by an inverse flatfield file. WFPC2 flatfields are currently generated from a combination of on-orbit data (so-called Earthflats, images of the bright Earth) and pre-launch ground data. The on-orbit data allow a determination of the large-scale illumination pattern while the pre-launch data are used to determine the pixel-to-pixel response function. The application of the flatfield file (extension .r4h) is controlled by the keyword FLATCORR.

Shutter Shading Correction

Header Switch: SHADCORR

Header Keywords Updated: SHADCORR

Reference File: SHADFILE

The finite velocity of the shutter produces uneven illumination across the field of view (thus the term "shutter shading"), resulting in a position-dependent exposure time. The effect is only significant, however, for exposures of a few seconds or less. The shutter shading calibration is applied by default to all exposures less than ten seconds. It has the form of an additive correction, scaled to the appropriate exposure time, that varies spatially across the detectors. The keyword switch is SHADCORR, and the shutter shading file name is stored in the keyword, SHADFILE.

Creation of Photometry Keywords

Header Switch: DOPHOTOM

Header Keywords Updated: DOPHOTOM, PHOTTAB

Group Header Keywords Updated: PHOTMODE, PHOTFLAM, PHOTZPT, PHOTPLAM, PHOTBW

Reference File: GRAPHTAB, COMPTAB

Photometry keywords, which provide the conversion from calibrated counts (DN) to astronomical magnitude, are computed by calwp2 using the STSDAS package synphot. (More information on synphot can be found in this document, and in the Synphot User's Guide, which is available via the web.) The keyword switch for this step is DOPHOTOM, and the reference file keywords are GRAPHTAB and COMPTAB. Note that the science data (.c0h/.c0d) pixel values are not changed as a result of performing DOPHOTOM, the data remain in units of DN (data number); calwp2 merely computes the photometric parameters and populates the appropriate header keywords.

The photometric keywords that are computed are listed in figure 3.3 below; the first two are in the ASCII header (both .d0h and .c0h), while the last five keywords are group parameters (use the IRAF tasks imheader, hselect, or hedit to examine the group keywords-see chapter 2 and chapter 3 of the HST Introduction for more details).

Figure 3.3: Photometry Keyword Descriptions
 

The calwp2 task constructs the PHOTMODE based on the specific image characteristics, which are given in the header keywords (i.e., INSTRUMENT, DETECTOR, ATODGAIN, FILTNAM1, FILTNAM2 and LRFWAVE); the throughput table for that PHOTMODE is then generated based upon the paths and files defined in the GRAPH and COMP tables stored in the HST Calibration Data Base System (CDBS). That path, i.e., the input files used to generate the final throughput table for the image, is recorded in the HISTORY comments of the calibrated science image (.c0h), and may also be viewed by running the STSDAS task showfiles.

The example below steps through both options; please refer to the Synphot Handbook for more details concerning the synphot tasks and the Photometric Tables section for suggestions on how to download the most recent synphot tables. Note that in the example below, the individual throughput files that are used to create the final image throughput table are: HST OTA, WFPC2 optics, filter (F606W), detector quantum efficiency table, a-to-d gain, and flatfield. Observers may notice that currently, the flatfield tables wfpc2_flat* (invoked by the "cal" in the photmode) are set to 1 though this could of course change with future WFPC2 flatfield and synphot table updates.

The final throughput table created for the image is saved in a STSDAS table with extension of ".c3t"; calwp2 computes the photometric parameters from the .c3t table and updates the group keyword values accordingly.

wf> hsel u5kl0101r.c0h[2] $I,instrume,detector,atodgain,filtnam1,filtnam2 yes u5kl0101r.c0h[2] WFPC2 2 7.0 F606W wf> hsel u5kl0101r.c0h[2] photmode,photflam,photplam,photbw,photzpt yes WFPC2,2,A2D7,F606W,,CAL 1.842431E-18 6004.956 637.824 -21.1 cl> imhead u5kl0101r.c0h long+ | match HISTORY ...[some HISTORY lines removed to save space] HISTORY The following throughput tables were used: HISTORY crotacomp$hst_ota_007_syn.fits, crwfpc2comp$wfpc2_optics_006_syn.fits, HISTORY crwfpc2comp$wfpc2_f606w_006_syn.fits, HISTORY crwfpc2comp$wfpc2_dqewfc2_005_syn.fits, HISTORY crwfpc2comp$wfpc2_a2d7wf2_004_syn.fits, HISTORY crwfpc2comp$wfpc2_flatwf2_003_syn.fits wf> showfiles "WFPC2,2,A2D7,F606W,,CAL" #Throughput table names: crotacomp$hst_ota_007_syn.fits crwfpc2comp$wfpc2_optics_006_syn.fits crwfpc2comp$wfpc2_f606w_006_syn.fits crwfpc2comp$wfpc2_dqewfc2_005_syn.fits crwfpc2comp$wfpc2_a2d7wf2_004_syn.fits crwfpc2comp$wfpc2_flatwf2_003_syn.fits wfl> bandpar "WFPC2,2,A2D7,F606W,,CAL" photlist=all # OBSMODE URESP PIVWV BANDW WFPC2,2,A2D7,F606W,,CAL 1.8424E-18 6004.9 637.82 # OBSMODE FWHM WPEAK TPEAK WFPC2,2,A2D7,F606W,,CAL 1502. 6188.8 0.020972 # OBSMODE AVGWV QTLAM EQUVW WFPC2,2,A2D7,F606W,,CAL 6038.9 0.0066095 39.467 # OBSMODE RECTW EMFLX REFWAVE WFPC2,2,A2D7,F606W,,CAL 1881.8 3.6564E-15 6038.9 # OBSMODE TLAMBDA WFPC2,2,A2D7,F606W,,CAL 0.019887

 

It is not necessary to rerun calwp2 and/or re-retrieve data via OTFR to merely recompute the photometric parameters; these may be obtained by running the STSDAS Synphot task bandpar directly, as shown in the example above.

Histogram Creation

Header Switch: DOHISTOS

Header Keywords Updated: DOHISTOS

Reference File: none

This step will create a multigroup image (.c2h/.c2d) with one group for each group in the calibrated data file. Each group contains a three-line image where the first row is a histogram of the raw data values, the second row is a histogram of the A/D corrected data, and the third row is a histogram of the final calibrated science data. This operation is controlled by the keyword DOHISTOS; the default is to skip this step.

Data Quality File Creation

By performing a bitwise logical OR, the calwp2 software combines the raw data quality file (.q0h, .q1h) with the static pixel mask (.r0h) and the data quality files for bias, dark, and flatfield reference files (.b2h, .b3h, .b4h) in order to generate the calibrated science data quality file (.c1h). This step is always performed; even if no *CORR keyword is set (i.e., if MASKCORR, BLEVCORR, BIASCORR, etc. were all set to OMIT), a .c1h file would still be generated, though it would not contain much useful information. The flag values used are defined below; by convention, DQF pixel values of zero (0) designate good pixels. The final calibrated data quality file (.c1h) may be examined, for example, using SAOimage, ximtool, or imexamine, to identify which pixels may be bad in the science image.

Table 3.4: WFPC2 Data Quality Flag Values

Flag Value	Description
0	Good pixel
1	Reed-Solomon decoding error. This pixel is part of a packet of data in which one or more pixels may have been corrupted during transmission.
2	Calibration file defect-set if pixel flagged in any calibration file. Includes charge transfer traps identified in static pixel mask file (.r0h).
4	Permanent camera defect. Static defects are maintained in the CDBS database and flag problems such as blocked columns and dead pixels. (Not currently used.)
8	A/D converter saturation. The actual signal is unrecoverable but known to exceed the A/D full-scale signal (4095).1
16	Missing data. The pixel was lost during readout or transmission. (Not currently used.)
32	Bad pixel that does not fall into above categories.
128	Permanent charge trap. (Not currently used.)
256	Questionable pixel. A pixel lying above a charge trap which may be affected by the trap.
512	Unrepaired warm pixel.
1024	Repaired warm pixel.

1 Calibrated saturated pixels may have values significantly lower than 4095 due to bias subtraction and flatfielding. In general, data values above 3500 DN are likely saturated.

 
¹ On-orbit tests have shown that for WFPC2, the conversion values have remained constant; therefore, the A-to-D reference file contains only one temperature and one set (row) of conversion values.

² A dark frame is a long exposure taken with the shutter closed; each individual dark has the standard calibration corrections applied (ATODCORR, BLEVCORR, and BIASCORR).