Presented here is information on the reference files supplied by Jon Holtzman, WFPC2 IDT, in November 1994. For convenience, these files can be obtained via the links on the CDBS WFPC2 Reference Files page.
In some cases, these files are the same as the products in the STScI pipeline, but generally they're not.
For some of these products, proper usage depends on the usage of other of these products, so it is important to read completely the information in this README file to determine the proper sequence for calibration using these files, and to understand the dependencies of some files on others.
The WFPC2 Calibration uses the following steps:
1. Correction of ADC Errors in both Raw Data and Engineering (Overscan) Frame. 2. Determination of Bias Levels from Overscan Data and Subtraction of these from Data Frames. 3. Subtraction of Superbias Frames. 4. Subtraction of Superdark Frames. 5. Subtraction of Deltadark Frames. 6. Application of Shutter Shading Correction. 7. Application of Flat Fields. 8. Pixel Area Correction.
Additional data is provided in this directory on geometric distortion which is not used in the standard reduction. As described further below, the distort/subdirectory gives distortion and inverse distortion coefficients as well as relative pixel area maps for each chip.
NOTE: These procedures are to be applied to raw data; the engineering data is used for overscan data. The raw data are the DOH/D0D frames; the engineering data are the X0H/X0D frames. NOTE: We recommend as you apply the following procedures that you add cards in the image header describing what you have done and what products you used, so you can later track these down if you need to.
The details for the reduction steps are as follows:
1. ADC Correction:
The directory adc contains tables to use for correcting the small analog-to-digital conversion errors. There are separate tables for each bay (bay 3=gain 14, bay 4=gain 7). These tables consist of 4 header lines, then a line for each DN value. The second column gives the most probable input DN value to produce the given DN value.
To apply these files, for every pixel in the raw and overscan data, find the corresponding entry in the table, and replace the raw DN value with the corrected value from the second column.
Same as files currently in pipeline...
2. Bias Subtraction:
Determine a bias level from the overscan data. It is important to ignore the first few overscan columns as they are affected by the level of signal in the raw data frame. Also, we have found occasional evidence for a difference in bias value between the odd and even columns, so we recommend a separate subtraction for the odd and even columns. If the frames are referenced to origin (1,1), then use the average of columns (9,11,13) in the engineering frame for the EVEN columns in the data frame; use the average of columns (10,12,14) in the engineering frame for the ODD columns (note the mismatch between odd and even between the engineering frames and data frames). Properly, the means should be determined with some sort of outlying pixel rejection.
Note that the odd-even difference is at the fraction of a DN level at most, and sometimes is nonexistent, so if you don't care about things at this level, you can use a single bias value determined from columns 9-14 in the engineering frame. DON'T use overscan data from any columns before column 9 in the engineering frame.
Same as section used by current calwp2 to determine blevcorr...
3. Superbias Subtraction:
Subtract a superbias frame which contains bias structure and a little bit of dark current. There is a separate superbias for each chip and for each gain (gain 7=bay4, gain14=bay3). The biases have changed in time: they were different at -76° C than at -88° C, then in 6/94, they seem to have changed character (gotten smoother) for an unknown reason. Consequently, we currently have 3 epochs of superbiases:
apr94 : launch until 4/24/94 jun94 : 4/24/94 until 6/13/94 sep94 : 6/14/94 until 10/94 useafter mode gain filename comment 19931201 full 15 ec21021pu IDT superbias 19931201 full 7 ec21021tu IDT superbias 19940424 full 15 ec21259fu IDT superbias 19940424 full 7 ec21259iu IDT superbias 19940613 full 15 ec21306au IDT superbias 19940613 full 7 ec21306fu IDT superbias
4. Superdark Subtraction:
Subtract a superdark frame which has high S/N dark rate information for those pixels which have a constant dark rate (i.e., those that haven't gone hot or cold or varied). There is a separate superdark for each chip. The darks are the same for both gains and were determined from gain=7 data; for gain=14 data, you need to divide the superdarks by a factor of two.
The superdarks have units DN(gain=7)/s. Before subtracting the superdark, it must be scaled to the exposure time of your data. To do this, multiply the appropriate superdark by a factor t_sd, given by:
t_sd = s + 60*int[(t+16.4)/60]
t is the requested open shutter time which you can get from the header card UEXPODUR (which may not be the same as the final open shutter EXPTIME if the exposure was interrupted) s is 60 for exposures taken with serial clocks off (default for long exposures) and 0 for exposures with serial clocks on int represents integer truncation
Remember to divide by a factor of two if applying to gain=14 (bay=3) data.
Since the superdarks have themselves been superbias subtracted, they are tied to the superbias frames. So there is a separate epoch superdark for each of the epochs of superbiases.
useafter mode clks gain filename comment 19931201 full off 15 ec60846ju IDT superdark 19931201 full off 7 ec60846nu IDT superdark 19931201 full on 15 ec60846lu IDT superdark 19931201 full on 7 ec60846pu IDT superdark 19940424 full off 15 ec608478u IDT superdark 19940424 full off 7 ec60847cu IDT superdark 19940424 full on 15 ec60847au IDT superdark 19940424 full on 7 ec60847eu IDT superdark 19940613 full off 15 ec60848hu IDT superdark 19940613 full off 7 ec60848lu IDT superdark 19940613 full on 15 ec60848ju IDT superdark 19940613 full on 7 ec60848nu IDT superdark
5. Delta Dark Subtraction:Subtract a delta dark frame taken near in time to your exposure to remove pixels which have a dark rate significantly different than was measured for the superdark. As with superdarks, there is a separate delta dark for each chip. The delta darks are the same for both gains and were determined from gain=7 data; for gain=14 data, you need to divide the delta darks by a factor of two.
The delta darks are each composed of only ~5 dark frames so they do not have high S/N. Consequently, they should only be used to remove dark rate from pixels which have rates significantly different than those measured in the superdarks. Low values in the delta darks should NOT be used (see step b below). IF YOU USE THE DELTA DARKS WITHOUT CLIPPING OUT LOW VALUES YOU DESTROY THE BENEFITS OF USING SUPERDARKS.
The delta darks have units DN(gain=7)/s.
The procedure to use delta darks is as follows:
A. find the dark nearest in date to your observation;
B. in the delta dark, select only pixels with ABSOLUTE value greater than 0.002 DN/s (corresponds to dark rates of 3.6 DN/1800s, or something like 5 sigma - you can experiment with reducing this value). Set all other pixels to zero - IMPORTANT, or else you'll introduce a noisy dark subtraction.
C. scale the darks for your exposure time, using:
t_dd = 60 + (n-1)*14.5 + s + 60*int[(t+16.4)/60]
t is the requested open shutter time (maybe not the same as the final open shutter EXPTIME if the exposure was interrupted) s is 60 for exposures taken with serial clocks off (default for long exposures) and 0 for exposures with serial clocks on int represents integer truncation n is the chip number (1-4)
NOTE: this is different from the time used for the superdark.
D. if applying to gain=14 data, divide the delta dark by two;
E. subtract the scaled delta dark from your data.
Since the delta darks have themselves been superbias and superdark subtracted, they are tied to the superbias and superdark frames used for their construction. So each delta dark needs to be used with the correct corresponding superdark.
useafter mode clks gain filename 19940101 full off 7 ecg1136au IDT deltadark 19940102 full off 7 ecg1136du IDT deltadark 19940117 full off 7 ecg1136fu IDT deltadark 19940201 full off 7 ecg1136lu IDT deltadark 19940201 full off 7 ecg1136ou IDT deltadark 19940218 full off 7 ecg1136ru IDT deltadark 19940219 full off 7 ecg1136tu IDT deltadark 19940221 full off 7 ecg11372u IDT deltadark 19940223 full off 7 ecg11374u IDT deltadark 19940303 full off 7 ecg11377u IDT deltadark 19940308 full off 7 ecg11379u IDT deltadark 19940314 full off 7 ecg1137du IDT deltadark 19940321 full off 7 ecg1137fu IDT deltadark 19940328 full off 7 ecg1137iu IDT deltadark 19940404 full off 7 ecg1137lu IDT deltadark 19940415 full off 7 ecg1137nu IDT deltadark 19940418 full off 7 ecg1137qu IDT deltadark 19940424 full off 7 eck1344tu IDT deltadark 19940502 full off 7 eck13452u IDT deltadark 19940509 full off 7 eck13457u IDT deltadark 19940516 full off 7 eck1345au IDT deltadark 19940523 full off 7 eck1345cu IDT deltadark 19940530 full off 7 eck1345fu IDT deltadark 19940606 full off 7 eck1345hu IDT deltadark 19940613 full off 7 ecl0919cu IDT deltadark 19940620 full off 7 ecl0919eu IDT deltadark 19940627 full off 7 ecl0919gu IDT deltadark 19940704 full off 7 ecl0919ju IDT deltadark 19940718 full off 7 ecl0919lu IDT deltadark 19940725 full off 7 ecl0919nu IDT deltadark 19940802 full off 7 ecm0904du IDT deltadark 19940809 full off 7 ecm0904fu IDT deltadark 19940816 full off 7 ecm0904hu IDT deltadark 19940823 full off 7 ecm0904ju IDT deltadark 19940830 full off 7 ecm0904mu IDT deltadark 19940906 full off 7 ecm0904ou IDT deltadark 19940915 full off 7 ecm0904qu IDT deltadark 19940920 full off 7 ecm0945eu IDT deltadark 19940930 full off 7 ecm0945hu IDT deltadark 19941003 full off 7 ecm0945ku IDT deltadark 19941011 full off 7 ecm0945mu IDT deltadark 19941017 full off 7 ecm0945pu IDT deltadark 19941025 full off 7 ecm0945ru IDT deltadark 19941101 full off 7 ecm09460u IDT deltadark
6. Shutter Shading Correction:
The WFPC2 shutter travel times causes slightly different exposure times across the field of view. The effect is very small: it gives ~ 5% errors across the field for a 0.11s exposure and thus is essentially negligible for exposures longer than a few seconds. Still if you want to correct for it, we have constructed shutter shading frames from TV data.
To apply the shutter shading frames, you need to determine which shutter was used for your exposure (there are two). Determine this from the UBLDASNR and UBLDBSNR cards in the header.
if UBLDASNR=0 and UBLDBSNR=1, we call this shutter A; if UBLDASNR=1 and UBLDBSNR=0, we call this shutter B;
There is a separate shutter shading correction frame for each chip.
Find the appropriate frame, divide it by the exposure time (EXPTIME), and add one. Then take the data frame and divide it by this frame.
Same as files currently in pipeline...
7. Flat Fielding:
Finally, correct for pixel-to-pixel response and variations in the illumination by application of a flat field. There's a separate flat for each chip and for each filter. The flats have been normalized and are stored in the database as inverse flats, so you need simply to multiply the data by the inverse flat to apply the correction.
The flats are normalized in WF3, and the flats for the rest of the chips should be appropriately normalized for proper chip-to-chip normalization.
Same as files currently in pipeline...
8. Pixel Area Correction:
Finally, when performing integrated photometry, users should be aware that WFPC2 has significant distortion which causes the pixel area to vary across the field. The application of flat-fields "corrects" for this and normalizes all pixels to have the same area. However, this renders integrated photometry incorrect, because all of the light falls on the distorted pixels and correcting for the pixel area changes the amount of total light.
Consequently if you are measuring integrated brightnesses (as opposed to surface brightnesses), you will want to put back the true relative pixel areas. Maps of these can be found in the distort flats directory. Multiply by these maps to put the relative areas back into the image. The maps are normalized to unity at (400,400) in each chip.
This file is available from the HST Archive...
filename: f1k1552bu, extension is r9h/r9d
when requesting from Starview, go into "Searches", select "DataFiles", then "Dataset Name" screen and enter your desired file name (without extension) in the "dataset name" field.
Or available via anonymous ftp in:
Also included in the distort/ subdirectory are data files which give the cubic distortion coefficients for each chip. The distortion solution is described in detail in Holtzman et al, PASP. Briefly, a solution was obtained which takes pixel locations from any chip and returns pixel locations in a master corrected rectilinear frame. For the master frame, a system with the orientation and scale of PC1 was chosen. Consequently, the distortion coefficients for the WF chips not only correct for distortion, but they also include rotations and scalings, as well as relative pointings, to bring the coordinates into the orientation of the master frame. In the master corrected frame, data from the entire four chips will roughly fall in the region (-1750:1750, -1750:1750); the origin was chosen to roughly be located at the apex of the pyramid. The cubic distortion solution is of the form:
x = x_obs - 400 y = y_obs - 400 x' = C_1i + C_2i*x + C_3i*y + C_4i*x2 + C_5i*x*y + C_6i*y2 + C_7i*x3 + C_8i*x2*y + C_9i*x*y2 + C_10i*y3 y' = D_1i + D_2i*x + D_3i*y + D_4i*x2 + D_5i*x*y + D_6i*y2 + D_7i*x3 + D_8i*x2*y + D_9i*x*y2 + D_10i*y3
where the C's and D's are the coefficients, and i runs from 1 to 4 for the chip number.
Note that since relative pointings are included, the C_1 and D_1 coefficients are sensitive to the exact position of the articulated fold mirrors, which can change the chip-to-chip pointing. These were moved to improve alignment on 4 March 1994 and consequently there are different values before and after this date. In addition, it is known that the positions are sensitive to temperature, so it is possible that the chip-to-chip pointings can vary slightly (up to a few tenths of a pixel max) from exposure to exposure.
We have also computed inverse distortion solutions to go from pixel locations in the master frame back to locations in the raw frame. These are given by:
x_obs = c_1i + c_2i*x' + c_3i*y' + c_4i*x'2 + c_5i*x'*y' + c_6i*y'2 + c_7i*x'3 + c_8i*x'2*y' + c_9i*x'*y'2 + c_10i*y'3 y_obs = d_1i + d_2i*x' + d_3i*y' + d_4i*x'2 + d_5i*x'*y' + d_6i*y'2 + d_7i*x'3 + d_8i*x'2*y' + d_9i*x'*y'2 + d_10i*y'3
As with the forward solution, there are differences before and after the dates of the AFM moves.
Files (available in /instrument_news/WFPC2/Wfpc2_memos) names are:
wfpc2dist_jan94.dat Distortion coefs before 94/3/4 wfpc2dist_mar94.dat Distortion coefs after 94/3/4 wfpc2invdist_jan94.dat Inv. Distortion coefs before 94/3/4 wfpc2invdist_mar94.dat Inv. Distortion coefs after 94/3/4