STScI Logo

Hubble Space Telescope
WFPC2 IDT Reference Files

Presented here is information on the reference files supplied by Jon Holtzman, WFPC2 IDT, in November 1994. For convenience, these files can be obtained via the links on the CDBS WFPC2 Reference Files page.

In some cases, these files are the same as the products in the STScI pipeline, but generally they're not.

For some of these products, proper usage depends on the usage of other of these products, so it is important to read completely the information in this README file to determine the proper sequence for calibration using these files, and to understand the dependencies of some files on others.

The WFPC2 Calibration uses the following steps:

1. Correction of ADC Errors in both Raw Data and 
   Engineering (Overscan) Frame.

2. Determination of Bias Levels from Overscan Data 
   and Subtraction of these from Data Frames.

3. Subtraction of Superbias Frames.
  
4. Subtraction of Superdark Frames.
    
5. Subtraction of Deltadark Frames.
 
6. Application of Shutter Shading Correction.

7. Application of Flat Fields.

8. Pixel Area Correction.

Additional data is provided in this directory on geometric distortion which is not used in the standard reduction. As described further below, the distort/subdirectory gives distortion and inverse distortion coefficients as well as relative pixel area maps for each chip.



NOTE: These procedures are to be applied to raw data; 
      the engineering data is used for overscan data. 
      The raw data are the DOH/D0D frames; the engineering 
      data are the X0H/X0D frames.

NOTE: We recommend as you apply the following procedures 
      that you add cards in the image header describing 
      what you have done and what products you used, so 
      you can later track these down if you need to.

The details for the reduction steps are as follows:

1. ADC Correction:

The directory adc contains tables to use for correcting the small analog-to-digital conversion errors. There are separate tables for each bay (bay 3=gain 14, bay 4=gain 7). These tables consist of 4 header lines, then a line for each DN value. The second column gives the most probable input DN value to produce the given DN value.

To apply these files, for every pixel in the raw and overscan data, find the corresponding entry in the table, and replace the raw DN value with the corrected value from the second column.

Same as files currently in pipeline...


2. Bias Subtraction:

Determine a bias level from the overscan data. It is important to ignore the first few overscan columns as they are affected by the level of signal in the raw data frame. Also, we have found occasional evidence for a difference in bias value between the odd and even columns, so we recommend a separate subtraction for the odd and even columns. If the frames are referenced to origin (1,1), then use the average of columns (9,11,13) in the engineering frame for the EVEN columns in the data frame; use the average of columns (10,12,14) in the engineering frame for the ODD columns (note the mismatch between odd and even between the engineering frames and data frames). Properly, the means should be determined with some sort of outlying pixel rejection.

Note that the odd-even difference is at the fraction of a DN level at most, and sometimes is nonexistent, so if you don't care about things at this level, you can use a single bias value determined from columns 9-14 in the engineering frame. DON'T use overscan data from any columns before column 9 in the engineering frame.

Same as section used by current calwp2 to determine blevcorr...


3. Superbias Subtraction:

Subtract a superbias frame which contains bias structure and a little bit of dark current. There is a separate superbias for each chip and for each gain (gain 7=bay4, gain14=bay3). The biases have changed in time: they were different at -76° C than at -88° C, then in 6/94, they seem to have changed character (gotten smoother) for an unknown reason. Consequently, we currently have 3 epochs of superbiases:

                 apr94  :   launch until 4/24/94
               jun94  :   4/24/94 until 6/13/94
               sep94  :   6/14/94 until 10/94

      useafter  mode  gain filename   comment

      19931201  full   15  ec21021pu  IDT superbias
      19931201  full    7  ec21021tu  IDT superbias
      19940424  full   15  ec21259fu  IDT superbias
      19940424  full    7  ec21259iu  IDT superbias
      19940613  full   15  ec21306au  IDT superbias
      19940613  full    7  ec21306fu  IDT superbias

4. Superdark Subtraction:

Subtract a superdark frame which has high S/N dark rate information for those pixels which have a constant dark rate (i.e., those that haven't gone hot or cold or varied). There is a separate superdark for each chip. The darks are the same for both gains and were determined from gain=7 data; for gain=14 data, you need to divide the superdarks by a factor of two.

The superdarks have units DN(gain=7)/s. Before subtracting the superdark, it must be scaled to the exposure time of your data. To do this, multiply the appropriate superdark by a factor t_sd, given by:

              t_sd = s + 60*int[(t+16.4)/60]

where:

t is the requested open shutter time which you can get from
the header card UEXPODUR (which may not be the same as
the final open shutter EXPTIME if the exposure was 
interrupted)
  
s is 60 for exposures taken with serial clocks off (default for
long exposures) and 0 for exposures with serial clocks on
int[] represents integer truncation

Remember to divide by a factor of two if applying to gain=14 (bay=3) data.

Since the superdarks have themselves been superbias subtracted, they are tied to the superbias frames. So there is a separate epoch superdark for each of the epochs of superbiases.

      useafter  mode clks gain filename   comment

      19931201  full off   15  ec60846ju  IDT superdark
      19931201  full off    7  ec60846nu  IDT superdark
      19931201  full  on   15  ec60846lu  IDT superdark
      19931201  full  on    7  ec60846pu  IDT superdark
      19940424  full off   15  ec608478u  IDT superdark
      19940424  full off    7  ec60847cu  IDT superdark
      19940424  full  on   15  ec60847au  IDT superdark
      19940424  full  on    7  ec60847eu  IDT superdark
      19940613  full off   15  ec60848hu  IDT superdark
      19940613  full off    7  ec60848lu  IDT superdark
      19940613  full  on   15  ec60848ju  IDT superdark
      19940613  full  on    7  ec60848nu  IDT superdark

5. Delta Dark Subtraction:

Subtract a delta dark frame taken near in time to your exposure to remove pixels which have a dark rate significantly different than was measured for the superdark. As with superdarks, there is a separate delta dark for each chip. The delta darks are the same for both gains and were determined from gain=7 data; for gain=14 data, you need to divide the delta darks by a factor of two.

IMPORTANT:

The delta darks are each composed of only ~5 dark frames so they do not have high S/N. Consequently, they should only be used to remove dark rate from pixels which have rates significantly different than those measured in the superdarks. Low values in the delta darks should NOT be used (see step b below). IF YOU USE THE DELTA DARKS WITHOUT CLIPPING OUT LOW VALUES YOU DESTROY THE BENEFITS OF USING SUPERDARKS.

The delta darks have units DN(gain=7)/s.

The procedure to use delta darks is as follows:

A. find the dark nearest in date to your observation;

B. in the delta dark, select only pixels with ABSOLUTE value greater than 0.002 DN/s (corresponds to dark rates of 3.6 DN/1800s, or something like 5 sigma - you can experiment with reducing this value). Set all other pixels to zero - IMPORTANT, or else you'll introduce a noisy dark subtraction.

C. scale the darks for your exposure time, using:

t_dd = 60 + (n-1)*14.5 + s + 60*int[(t+16.4)/60]

where:

t is the requested open shutter time (maybe not the same as the final open shutter EXPTIME if the exposure was interrupted) s is 60 for exposures taken with serial clocks off (default for long exposures) and 0 for exposures with serial clocks on int[] represents integer truncation n is the chip number (1-4)

NOTE: this is different from the time used for the superdark.

D. if applying to gain=14 data, divide the delta dark by two;

E. subtract the scaled delta dark from your data.

Since the delta darks have themselves been superbias and superdark subtracted, they are tied to the superbias and superdark frames used for their construction. So each delta dark needs to be used with the correct corresponding superdark.

      useafter  mode clks gain filename 

      19940101  full off    7  ecg1136au    IDT deltadark
      19940102  full off    7  ecg1136du    IDT deltadark
      19940117  full off    7  ecg1136fu    IDT deltadark
      19940201  full off    7  ecg1136lu    IDT deltadark
      19940201  full off    7  ecg1136ou    IDT deltadark
      19940218  full off    7  ecg1136ru    IDT deltadark
      19940219  full off    7  ecg1136tu    IDT deltadark
      19940221  full off    7  ecg11372u    IDT deltadark
      19940223  full off    7  ecg11374u    IDT deltadark
      19940303  full off    7  ecg11377u    IDT deltadark
      19940308  full off    7  ecg11379u    IDT deltadark
      19940314  full off    7  ecg1137du    IDT deltadark
      19940321  full off    7  ecg1137fu    IDT deltadark
      19940328  full off    7  ecg1137iu    IDT deltadark
      19940404  full off    7  ecg1137lu    IDT deltadark
      19940415  full off    7  ecg1137nu    IDT deltadark
      19940418  full off    7  ecg1137qu    IDT deltadark
      19940424  full off    7  eck1344tu    IDT deltadark
      19940502  full off    7  eck13452u    IDT deltadark
      19940509  full off    7  eck13457u    IDT deltadark
      19940516  full off    7  eck1345au    IDT deltadark
      19940523  full off    7  eck1345cu    IDT deltadark
      19940530  full off    7  eck1345fu    IDT deltadark
      19940606  full off    7  eck1345hu    IDT deltadark
      19940613  full off    7  ecl0919cu    IDT deltadark
      19940620  full off    7  ecl0919eu    IDT deltadark
      19940627  full off    7  ecl0919gu    IDT deltadark
      19940704  full off    7  ecl0919ju    IDT deltadark
      19940718  full off    7  ecl0919lu    IDT deltadark
      19940725  full off    7  ecl0919nu    IDT deltadark
      19940802  full off    7  ecm0904du    IDT deltadark
      19940809  full off    7  ecm0904fu    IDT deltadark
      19940816  full off    7  ecm0904hu    IDT deltadark
      19940823  full off    7  ecm0904ju    IDT deltadark
      19940830  full off    7  ecm0904mu    IDT deltadark
      19940906  full off    7  ecm0904ou    IDT deltadark
      19940915  full off    7  ecm0904qu    IDT deltadark
      19940920  full off    7  ecm0945eu    IDT deltadark
      19940930  full off    7  ecm0945hu    IDT deltadark
      19941003  full off    7  ecm0945ku    IDT deltadark
      19941011  full off    7  ecm0945mu    IDT deltadark
      19941017  full off    7  ecm0945pu    IDT deltadark
      19941025  full off    7  ecm0945ru    IDT deltadark
      19941101  full off    7  ecm09460u    IDT deltadark

6. Shutter Shading Correction:

The WFPC2 shutter travel times causes slightly different exposure times across the field of view. The effect is very small: it gives ~ 5% errors across the field for a 0.11s exposure and thus is essentially negligible for exposures longer than a few seconds. Still if you want to correct for it, we have constructed shutter shading frames from TV data.

To apply the shutter shading frames, you need to determine which shutter was used for your exposure (there are two). Determine this from the UBLDASNR and UBLDBSNR cards in the header.

 
   if UBLDASNR=0 and UBLDBSNR=1, we call this shutter A; 
      
   if UBLDASNR=1 and UBLDBSNR=0, we call this shutter B; 

There is a separate shutter shading correction frame for each chip.

Find the appropriate frame, divide it by the exposure time (EXPTIME), and add one. Then take the data frame and divide it by this frame.

Same as files currently in pipeline...


7. Flat Fielding:

Finally, correct for pixel-to-pixel response and variations in the illumination by application of a flat field. There's a separate flat for each chip and for each filter. The flats have been normalized and are stored in the database as inverse flats, so you need simply to multiply the data by the inverse flat to apply the correction.

The flats are normalized in WF3, and the flats for the rest of the chips should be appropriately normalized for proper chip-to-chip normalization.

Same as files currently in pipeline...


8. Pixel Area Correction:

Finally, when performing integrated photometry, users should be aware that WFPC2 has significant distortion which causes the pixel area to vary across the field. The application of flat-fields "corrects" for this and normalizes all pixels to have the same area. However, this renders integrated photometry incorrect, because all of the light falls on the distorted pixels and correcting for the pixel area changes the amount of total light.

Consequently if you are measuring integrated brightnesses (as opposed to surface brightnesses), you will want to put back the true relative pixel areas. Maps of these can be found in the distort flats directory. Multiply by these maps to put the relative areas back into the image. The maps are normalized to unity at (400,400) in each chip.

This file is available from the HST Archive...

filename: f1k1552bu, extension is r9h/r9d

when requesting from Starview, go into "Searches", select "DataFiles", then "Dataset Name" screen and enter your desired file name (without extension) in the "dataset name" field.

Or available via anonymous ftp in:

ftp://ftp.stsci.edu/cdbs/uref/

Distortion Data:

Also included in the distort/ subdirectory are data files which give the cubic distortion coefficients for each chip. The distortion solution is described in detail in Holtzman et al, PASP. Briefly, a solution was obtained which takes pixel locations from any chip and returns pixel locations in a master corrected rectilinear frame. For the master frame, a system with the orientation and scale of PC1 was chosen. Consequently, the distortion coefficients for the WF chips not only correct for distortion, but they also include rotations and scalings, as well as relative pointings, to bring the coordinates into the orientation of the master frame. In the master corrected frame, data from the entire four chips will roughly fall in the region (-1750:1750, -1750:1750); the origin was chosen to roughly be located at the apex of the pyramid. The cubic distortion solution is of the form:

x = x_obs - 400
y = y_obs - 400
x' = C_1i + C_2i*x + C_3i*y + C_4i*x2 + C_5i*x*y + 
     C_6i*y2 + C_7i*x3 + C_8i*x2*y + C_9i*x*y2

     + C_10i*y3
y' = D_1i + D_2i*x + D_3i*y + D_4i*x2 + D_5i*x*y + 
     D_6i*y2 + D_7i*x3 + D_8i*x2*y + D_9i*x*y2 
     + D_10i*y3

where the C's and D's are the coefficients, and i runs from 1 to 4 for the chip number.

Note that since relative pointings are included, the C_1 and D_1 coefficients are sensitive to the exact position of the articulated fold mirrors, which can change the chip-to-chip pointing. These were moved to improve alignment on 4 March 1994 and consequently there are different values before and after this date. In addition, it is known that the positions are sensitive to temperature, so it is possible that the chip-to-chip pointings can vary slightly (up to a few tenths of a pixel max) from exposure to exposure.

We have also computed inverse distortion solutions to go from pixel locations in the master frame back to locations in the raw frame. These are given by:

x_obs = c_1i + c_2i*x' + c_3i*y' + c_4i*x'2 + c_5i*x'*y' 
        + c_6i*y'2 + c_7i*x'3 + c_8i*x'2*y' + 
        c_9i*x'*y'2 + c_10i*y'3

y_obs = d_1i + d_2i*x' + d_3i*y' + d_4i*x'2 + d_5i*x'*y' 
        + d_6i*y'2 + d_7i*x'3 + d_8i*x'2*y' + 
        d_9i*x'*y'2 + d_10i*y'3

As with the forward solution, there are differences before and after the dates of the AFM moves.

Files (available in /instrument_news/WFPC2/Wfpc2_memos) names are:

wfpc2dist_jan94.dat     Distortion coefs before 94/3/4
wfpc2dist_mar94.dat     Distortion coefs after 94/3/4
wfpc2invdist_jan94.dat  Inv. Distortion coefs before 94/3/4
wfpc2invdist_mar94.dat  Inv. Distortion coefs after 94/3/4