Space Telescope Science Institute
WFC3 Data Handbook 2.1 May 2011
help@stsci.edu
Table of Contents Previous Next Index Print


WFC3 Data Handbook > Chapter 3: WFC3 Data Calibration > 3.7 Manual Recalibration of WFC3 Data

3.7
3.7.1
Retrieving Software and Input Data Files
If observers decide to recalibrate their WFC3 on their own, the following must be available on their system:
calwf3 software from the STSDAS hst_calib.wfc3 package
MultiDrizzle and PyDrizzle from the STSDAS dither package
PyRAF, to run MultiDrizzle and PyDrizzle, obtained from STScI
Uncalibrated (raw) data files from the HDA
Association (asn) table, if needed
synphot data set, if needed for photometric calculations
The most recent version of STSDAS, which includes the wfc3 package required for reprocessing, can be downloaded from: http://www.stsci.edu/institute/software_hardware/stsdas
Setting up “iref
Before any recalibration can be done, the user’s local directory containing the calibration reference files must be defined for the software tasks. For WFC3, this directory is referred to as “iref”. The raw image headers already contain the appropriate keywords that list the reference file names that were assigned during STScI pipeline processing. The user must simply define the location of the “iref” directory in the Unix environment.
If done from the command line, this setup must be done in the same window in which IRAF (or PyRAF) will be started. Setting “iref” from within IRAF will not work, even though subsequently typing “show iref” would suggest it might. For convenience, this setup command can be added to your .setenv file, so that the iref environment variable will always be defined.
Selecting Calibration Switches
When retrieving data from the HDA, OTFR uses the latest available calibration reference files by default. In order to use non-default reference files, manual recalibration is required. The calibration reference file keywords will need to be updated manually in the raw data files with the desired file names before running calwf3. In addition, the user can choose to change which calibration steps are performed by calwf3 by resetting the values of the calibration switch keywords. These keywords are listed in Table 3.8 along with their default values as used in the STScI pipeline. To change the values of any of the keyword switches, use a FITS keyword editor, such as the IRAF hedit task:
Table 3.8: Calibration switch and default settings.
DEFAULT = PERFORM
DEFAULT = OMIT
DEFAULT = PERFORM
CCD Overscan Region Subtraction (UVIS) or Reference Pixel Bias Correction (IR)
DEFAULT = PERFORM
DEFAULT = PERFORM
DEFAULT = OMIT
UVIS: If CRSPLIT or NRPTEXP > 1 then PERFORM’, else OMIT
IR: DEFAULT = PERFORM
DEFAULT = PERFORM
DEFAULT = PERFORM
DEFAULT = PERFORM
DEFAULT = PERFORM
DEFAULT = OMIT
DEFAULT = PERFORM
DEFAULT = PERFORM
If NRPTEXP > 1 then PERFORM, else OMIT
Full calibration of individual exposures in an association
DEFAULT = PERFORM
DEFAULT =PERFORM
Set Up to Run the PHOTCORR Step
calwf3 does not alter the units of the pixels in the image when calculating photometric information. Instead it calculates and writes the inverse sensitivity conversion factors (PHOTFLAM and PHOTFNU) and the ST magnitude scale zero point (PHOTZPT) into header keywords in the calibrated data files. Refer to subsections on PHOTCORR in Section 3.4.2 (UVIS) and Section 3.4.3 (IR) for more information.
To compute values for the photometric keywords during the PHOTCORR step, calwf3 uses the STSDAS synthetic photometry package, synphot, which requires accessing two reference files, GRAPHTAB and COMPTAB, which are included in the synphot data set. This data set must be installed on the user’s system if this step is to be performed during recalibration (see Section 4.5.1 of the Introduction to the HST Data Handbooks). In order for calwf3 to access the synphot files, environment variables pointing to the local synphot directories must be defined as follows:
Bypassing the PHOTCORR Step
The synphot data set contains numerous files that are updated on a regular basis, making it cumbersome for the user to maintain. A simple alternative is to set the PHOTCORR calibration switch to “OMIT” in the primary header of the raw file. This avoids the need for downloading and maintaining your own copy of the synphot data files. The user may then simply copy the photometric keyword values from the previously calibrated data files into the raw file header and then run calwf3, skipping the PHOTCORR step. This is shown in the examples in 3.7.2.
Speed of Pipeline Processing
Reprocessing WFC3 UVIS and IR datasets can stress some computing platforms because of the potentially large data volume and CPU-intensive calculations. Great care has been taken to minimize the memory requirements of the pipeline software. Line-by-line I/O used during UVIS processing is particularly useful when more than one image is operated on at a time, such as during flat-field application or combining images. Unfortunately, this places an extra burden on the I/O capabilities of the computer. calwf3 requires up to 130MB of memory to process a full-frame UVIS image and up to 250MB for an IR exposure containing a full set of 16 non-destructive reads. MultiDrizzle requires up to 400MB.
Timing tests for processing WFC3 datasets using calwf3 are given in Table 3.9. Geometric correction or dither-combining using MultiDrizzle will take extra time, because these are performed separately. The CPU usage column reports the amount of time the CPU was active and reflects the amount of time waiting for disk I/O. WFC3 observers should keep these requirements in mind when securing computing resources for data processing.
Table 3.9: Timing tests for running calwf3 alone (no drizzling).
Dell Optiplex GX280, P4 3.2GHz CPU, 1GB RAM, SATA (SCSI) HD
UVIS Assoc. CR-SPLIT=2 (full-frame, unbinned)
3.7.2
This section presents several examples of calwf3 reprocessing. The boxes show commands and output to the screen. The lines beginning with the “iraf>” symbol, indicate commands typed into IRAF or PyRAF. Lines with no symbol indicate output from IRAF.
Example 1: Reprocessing a Single Exposure
The following example uses hypothetical UVIS observations of a stellar cluster, observed with the F814W filter. The exposures are CR-SPLIT into two exposures of 20 seconds each. The association table for this observation is i8bt07020_asn.fits. Typing “tprint i8bt07020_asn.fits” reveals the rootnames of the individual exposures: i8bt07oyq and i8bt07ozq.
For the purposes of this first example, assume that the observer desires to reprocess only one of these exposures. This example illustrates the steps required to reprocess a single exposure after changing the bias reference file from the default value to a file specified by the user.
1.
We assume here that the user has followed the instructions in the section See “Setting up “iref”” . We also assume that calibration reference files were obtained as part of the HDA request, and are stored in the local “iref” directory (i.e., “/mydisk/myiref/”). In this example, the directory “/mydisk/” contains the pipeline data.
2.
To see what bias reference file is currently in use, use hedit. Note that the period used after the third argument is required.
3.
Next, edit the primary image header of the raw file to reflect the name of the new bias file.
4.
Now set the PHOTCORR processing step to “OMIT” and copy the photometric keyword values from the previously calibrated image to the raw image. Notice that the PHOTCORR keyword resides in the primary header of the FITS file, while the remaining PHOT* keywords are located in the SCI image extension headers (see Tables 2.7 and 2.8). Alternately, the user may keep track of these numbers in any other preferred manner. Most users will only require knowledge of the PHOTFLAM or PHOTFNU keywords for photometric calibration. Setting PHOTCORR=OMIT allows users to skip this synphot-based calibration step (see See “Bypassing the PHOTCORR Step” for more information).
5.
Within the directory containing the pipeline products (“/mydisk/”), create the subdirectory “recal”. Copy the raw file to this subdirectory, and “cd” to it.
6.
Finally, load the package stsdas.hst_calib.wfc3 and run calwf3 on the single raw exposure.
The product will be a single calibrated image with the “_flt” file name suffix.
Example 2: Reprocessing Multiple Exposures within an Association
This example uses the same data from Example 1 and illustrates the steps required to reprocess a WFC3 association after changing the bias reference file from the default value to a file specified by the user. The steps required are similar to the previous example, with a few modifications. IRAF output comments that are similar to Example 1 have been omitted.
1.
2.
3.
Edit the primary image header of all the raw images to reflect the name of the new bias file.
4.
Set the PHOTCORR processing step to “OMIT” and copy the photometric keywords from the calibrated to the raw file (see See “Bypassing the PHOTCORR Step” for more information).
5.
Within the directory containing the pipeline products (“/mydisk/”), create the subdirectory “recal”, copy the raw and asn files to this subdirectory, and “cd” to this directory.
6.
Finally, run calwf3 on the image association.
Note: If this command is executed in the same directory in which you have run the previous example, then one of the flt files will already exist and calwf3 will not overwrite existing images. Either delete the existing flt file, move it to a separate directory, or rename it.
The products will be two separate calibrated flt images (i8bt07oyq_flt.fits, i8bt07oyq_flt.fits) and a single CR-combined crj image (i8bt07021_crj.fits).
Example 3: Reprocessing Images taken as part of a Dither Pattern
The following example uses IR images that are part of a 2-point line dither pattern. This example illustrates the steps required to reprocess images that are part of a dither pattern using a non-default dark reference file. The steps are similar to Example 2, but the format of the association and the data products are unique.
1.
2.
3.
Edit the primary image header of all the raw files to reflect the name of the new dark reference file.
4.
Set the PHOTCORR processing step to “OMIT” and copy the photometric keywords from the calibrated to the raw file (see See “Bypassing the PHOTCORR Step” for more information).
5.
Within the directory containing the pipeline products (“/mydisk/”), create the subdirectory “recal”, copy the raw and asn files to this subdirectory, and “cd” to this directory.
6.
Finally, run calwf3 on the image association.
The output products will be two separate calibrated datasets, consisting of ima and flt files for each of the input images. In subsequent processing (see Chapter 4 for details), MultiDrizzle can be used to combine the two flt files into a single drz image (i8e654010_drz.fits).
Example 4: Reprocessing IR data to reject data from entire readouts
The following example is for a hypothetical IR exposure that has some number of individual readouts affected by an anomaly, such as scattered Earth light. In this example we reprocess the raw data using calwf3 after flagging all the pixels in the last 3 readouts of the exposure, so that the data from those readouts is not used in the ramp fitting process (CRCORR step). A convenient data quality flag value to use is 256, which causes the ramp fitting step to ignore any flagged reads as if the data were saturated.
Raw WFC3 FITS files contain null DQ arrays, so it is not possible to directly edit the pixel values in the DQ extensions of the raw files. Instead, the DQ extension keyword “pixvalue” is modified, which will cause all pixels within the DQ extensions to take on that value when the raw files are read into calwf3. The modified raw image file is then processed normally with calwf3.
1.
Edit the value of the “pixvalue” keyword in the DQ extensions corresponding to the last 3 readouts of the exposure.
2.
 

Table of Contents Previous Next Index Print