Space Telescope Science Institute
WFC3 Data Handbook v. 3.0
help@stsci.edu
Table of Contents Previous Next PDF


WFC3 Data Handbook > Chapter 3: WFC3 Data Calibration > 3.7 Manual Recalibration of WFC3 Data

3.7
3.7.1 Requirements for Manual Recalibration
Retrieving Software and Input Data Files
If observers decide to recalibrate their WFC3 on their own, the following must be available on their system:
calwf3 software from the HSTCAL wfc3 package
AstroDrizzlefrom the stsci_python drizzlepac package
PyRAF if the user wishes to use this environment, but it’s not required.
Uncalibrated data and calibration reference files can be obtained from the HDA:
http://archive.stsci.edu/hst/search.php
The most recent version of HSTCAL, which includes the wfc3 package required for reprocessing, can be downloaded from:
http://www.stsci.edu/resources/software_hardware/stsdas
The most recent version of HSTCAL, which includes the wfc3 package required for reprocessing, can be downloaded from:
http://www.stsci.edu/resources/software_hardware/stsdas
Setting up ‘iref’
Before any recalibration can be done, the user’s local directory containing the calibration reference files must be defined for the software tasks. For WFC3, this directory is referred to as ‘iref’. The raw image headers already contain the appropriate keywords that list the reference file names that were assigned during STScI pipeline processing. The user must simply define the location of the ‘iref’ directory in the Unix environment:
 
setenv iref /mydisk/myiref/
If done from the command line, this setup must be done in the same window in which PyRAF will be started. Setting ‘iref’ from within PyRAF will not work, even though subsequently typing ‘show iref’ would suggest it might. For convenience, this setup command can be added to your .setenv file, so that the iref environment variable will always be defined. The ‘iref’ environment variable is required whether you are reprocessing through python, PyRAF or from your OS command line.
Selecting Calibration Switches
The MAST uses the latest available calibration reference files by default. In order to use non-default reference files, manual recalibration is required. The calibration reference file keywords will need to be updated manually in the raw data files with the desired file names before running calwf3. In addition, the user can choose to change which calibration steps are performed by calwf3 by resetting the values of the calibration switch keywords. These keywords are listed in Table 3.8 along with their default values as used in the STScI pipeline. To change the values of any of the keyword switches, use a FITS keyword editor, such as the IRAF hedit task or the python package pytfits:
python>pyfits.setval(myfile_raw.fits,extname=sci,extver=1,keyword=darkcorr,value=omit)
Table 3.8: Calibration switch and default settings.**
Set Up to Run the PHOTCORR Step
calwf3 does not alter the units of the pixels in the image when calculating photometric information. Instead it calculates and writes the inverse sensitivity conversion factors (PHOTFLAM and PHOTFNU) and the ST magnitude scale zero point (PHOTZPT) into header keywords in the calibrated data files. Refer to subsections on PHOTCORR in Section 3.4.2 (UVIS) and Section 3.4.3 (IR) for more information.
Speed of Pipeline Processing
Reprocessing WFC3 UVIS and IR datasets can stress some computing platforms because of the potentially large data volume and CPU-intensive calculations. Great care has been taken to minimize the memory requirements of the pipeline software. Line-by-line I/O used during UVIS processing is particularly useful when more than one image is operated on at a time, such as during flat-field application or combining images. Unfortunately, this places an extra burden on the I/O capabilities of the computer. calwf3 requires up to 130MB of memory to process a full-frame UVIS image and up to 250MB for an IR exposure containing a full set of 16 non-destructive reads. AstroDrizzle requires up to 400MB.
Timing tests for processing WFC3 datasets using calwf3 are given in Table 3.9 Geometric correction or dither-combining using AstroDrizzle will take extra time, because these are performed separately. The CPU usage column reports the amount of time the CPU was active and reflects the amount of time waiting for disk I/O. WFC3 observers should keep these requirements in mind when securing computing resources for data processing.
Table 3.9: Timing tests for running calwf3 alone (no drizzling) using Dell Optiplex GX280, P4 3.2GHz CPU, 1GB RAM, SATA (SCSI) HD
UVIS Assoc. CR-SPLIT=2 (full-frame, unbinned)
Single IR (NSAMP=16)
3.7.2 calwf3 Examples
This section presents several examples of calwf3 reprocessing. The boxes show commands and output to the screen. The following examples indicate commands which are typed into PyRAF (pyraf>) and pure python (python>). Pure python commands may also be used in a PyRAF session.
Example 1: Reprocessing a Single Exposure
The following example uses hypothetical UVIS observations of a stellar cluster, observed with the F814W filter. The exposures are CR-SPLIT into two exposures of 20 seconds each. The association table for this observation is i8bt07020_asn.fits. Typing ‘pyraf> tprint i8bt07020_asn.fits’ reveals the rootnames of the individual exposures: i8bt07oyq and i8bt07ozq.
For the purposes of this first example, assume that the observer desires to reprocess only one of these exposures. This example illustrates the steps required to reprocess a single exposure after changing the bias reference file from the default value to a file specified by the user.
1.
We assume here that the user has followed the instructions in the section See “Setting up ‘iref’”. We also assume that calibration reference files were obtained as part of the HDA request, and are stored in the local ‘iref’ directory (i.e., ‘/mydisk/myiref/’). In this example, the directory ‘/mydisk/’ contains the pipeline data.
2.
3.
pyraf> i8bt07oyq_raw.fits[0] updated
4.
5.
Finally, load the package and run calwf3 on the single raw exposure.
pyraf and python> calwf3.calwf3(i8bt07oyq_raw.fits)
The product will be a single calibrated image with the ‘_flt’ file name suffix.
Example 2: Reprocessing Multiple Exposures within an Association
This example uses the same data from Example 1 and illustrates the steps required to reprocess a WFC3 association after changing the bias reference file from the default value to a file specified by the user. The steps required are similar to the previous example, with a few modifications. IRAF output comments that are similar to Example 1 have been omitted.
1.
2.
3.
Note: If this command is executed in the same directory in which you have run the previous example, then one of the flt files will already exist and calwf3 will not overwrite existing images. Either delete the existing flt file, move it to a separate directory, or rename it.
The products will be two separate calibrated flt images (i8bt07oyq_flt.fits, i8bt07oyq_flt.fits) and a single CR-combined crj image (i8bt07021_crj.fits).
Example 3: Reprocessing Images taken as part of a Dither Pattern
The following example uses IR images that are part of a 2-point line dither pattern. This example illustrates the steps required to reprocess images that are part of a dither pattern using a non-default dark reference file. The steps are similar to Example 2, but the format of the association and the data products are unique.
 
1.
2.
3.
python> pyfits.setval(i8e654c0q_raw.fits,darkfile,iref$mydark.fits)
4.
5.
Finally, run calwf3 on the image association.
pyraf and python> calwf3.calwf3(i8e654010_asn.fits)
The output products will be two separate calibrated datasets, consisting of ima and flt files for each of the input images. In subsequent processing AstroDrizzle can be used to combine the two flt files into a single drz image (i8e654010_drz.fits).
Example 4: Reprocessing IR data to reject data from entire readouts
The following example is for a hypothetical IR exposure that has some number of individual readouts affected by an anomaly, such as scattered Earth light. In this example we reprocess the raw data using calwf3 after flagging all the pixels in the last 3 readouts of the exposure, so that the data from those readouts is not used in the ramp fitting process (CRCORR step). A convenient data quality flag value to use is 256, which causes the ramp fitting step to ignore any flagged reads as if the data were saturated.
Raw WFC3 FITS files contain null DQ arrays, so it is not possible to directly edit the pixel values in the DQ extensions of the raw files. Instead, the DQ extension keyword ‘pixvalue’ is modified, which will cause all pixels within the DQ extensions to take on that value when the raw files are read into calwf3. The modified raw image file is then processed normally with calwf3.
 
1.
python> for value in [1,2,3]\: pyfits.setval(ia2k19k6q_raw.fits,exver=value,extname=DQ,keyword=pixvalue,value=256)
2.
pyraf and python> calwf3.calwf3(ia2k19k6q_raw.fits)

WFC3 Data Handbook > Chapter 3: WFC3 Data Calibration > 3.7 Manual Recalibration of WFC3 Data

Table of Contents Previous Next PDF