STScI logoThe Hubble Deep Field South

WFPC2 Data Reduction / Technical Issues

Check the page with warnings and advisories before making any use of the data.

Information is available about the following topics:


WFPC2 imaging


Description of data products

1. General

Follow this link for a legend of the file names for the final reprocessed data.

The version 1 images represent our best shot at a first combination of the WFPC2 data taken in the main portion of the Hubble Deep Field South. Images were taken in the same four filters as for the original Hubble Deep Field: F300W, F450W, F606W, and F814W. For each filter, we have produced a combined data image, named fxxx_comb.fits (xxx stands for 300, 450, 606 or 814), and a weight image, named fxxx_weight.fits. Note that both data and weight images are presented in a single mosaic, with all four WFPC2 detectors combined onto the same image plane.

2. Combined Image

The combined image is the result of combining all individual exposures, optimally weighted for the background signal, and resampled to a pixel scale of 0.0398" using the "Drizzle" package (Fruchter & Hook 1998). The images are expressed in counts per second at a gain of 7. They have been rotated to have North approximately up (about 0.5 degrees from vertical). The coordinates of the images are set in the header WCS parameters, and can be retrieved for example via the IRAF task xy2rd. The absolute astrometry of the field has been derived from a match with four astrometrically measured stars kindly provided by the Naval Observatory, and is likely to be accurate to about 40 mas.

The depth and coverage for each filter varies across the field of view, due to the variety of pointings that were combined together. The image depth wanes when approaching the edges of the area covered, as well as in a near-vertical seam between detectors which received much lower coverage than the central region of each chip. This variable depth is reflected in the image weights, described below.

As a consequence of the decreased image quality, the outer regions of each image are less reliable, especially in terms of cosmic ray rejection. Any oddly-shaped objects appearing in only one filter near the edge of the images presented here has a good probability of being a piece of an unrejected cosmic ray.

Images have been trimmed where the (median) weight per pixel is less than a filter-dependent cutoff value: 7.e7 for F300W, 5.e7 for F450W, 1.e7 for F606W, and 2.e7 for F814W. Images also take the value of 0 where no valid pixels are available, such as at the center of the brightest stars - which saturate in all F606W and F814W images.

3. Exposure time and weight

The total effective exposure time for these images is:

F300W   140,185 s
F450W   100,950 s
F606W    81,275 s 
F814W   100,300 s
Several images need special treatment and have been rejected in this combination; their inclusion in a future version is likely to improve the noise by about 5 to 10%.

The weight images we provide represent the expected (theoretical) inverse variance due to the background sky in each image. However, due to the correlated nature of noise in resampled images, the variance measured on a single-pixel scale will in general differ from the inverse of the weight. The weight image can be used to estimate the expected noise on large scales, where the noise correlation abates. Specifically, the variance of the *total* signal in an area including N pixels scales with sqrt(N / < W >) for N sufficiently large (> 100 or so), where W is the weight reported for each pixel.

4. Photometry

WFPC2 detectors differ slightly in their sensitivity. The individual input images have been scaled to the response of the WF3 detector, thus the zero points determined for WF3 (at gain 7) apply. For reference, the most recent determination of the zero point for infinite aperture in the VEGAMAG system is:

F300W     19.43
F450W     22.02
F606W     22.90
F814W     21.66


Data reduction

A short description of the steps taken to produce the WFPC2 Version 1 images.

1. Preparation

The WFPC2 group produced a new superbias, superdark, and flat field for each of the filters. The superdark showed a general increase in dark current with respect to the previous version, and a larger number (about 4000 per detector) of permanent hot pixels. Although permanent hot pixels seem to subtract reasonably well, all pixels with dark current greater than 0.02 DN/s (0.14 e/s) were flagged as bad and not used in further combinations.

In addition, daily hot pixel masks were produced to track the newly produced hot pixels. Daily masks were produced by median combination of all F300W images taken in a given day. The median combination excludes objects, which fall on different places in the chip due to the dither and roll changes in the telescope, and brings up hot pixels, which are then flagged. This procedure failed for October 4, 9 and 10, when not enough F300W images were taken; we used the October 5 mask for October 4, and the October 8 mask for October 9 and 10. These masks were used in step 2 and in generating the individual weight files for step 5.

2. Quality verification and pipeline

First, each image was indidually inspected by eye, and any problems - moving objects, charge trails, cosmic ray showers, bias jumps, the characteristic cross due to Earth-scattered light - noted in the master log file. For charge trails, the affected areas were masked out by editging the image quality file. Moving objects and cosmic ray showers generally are rejected automatically in further processing, but the quality of the rejection was checked explicitly later on.

Individual images were processed by the standard CALWP2 pipeline, using the new superbias, superdark and flat field.

Finally, each image had the sky subtracted and the value recorded in the header.

3. Image registration and first combination

A preliminary image registration was obtained with two independent methods. The first was to adopt the position recorded in the jitter file, derived from the average measured position of the guide stars throughout the observation. The second was to cross-correlate the image with an image pair chosen as reference. [The second method could not be used for F300W because of the low signal levels.] Images for which the two methods yielded comparable results were used in the first combination, while images for which they disagreed were excluded from the first combination and included again after their position was redetermined in step 3.

The first combination was done as follows:

4. Refinement of the pointing measurement

The median image was then used to refine the position measurement for each of the input images. For F450W, F606W and F814W, this was done by cross-correlating each individual mosaic with the median image. In a few cases the quality of the cross-correlation indicated a possible rotation; these cases were excluded from the Version 1 combination.

Again, F300W required a different procedure, since the low signal level causes the full-image cross-correlation to be dominated by cosmic rays. Instead, we cut out two 100-pixel regions around the two brighest stars, and cross-correlated those against the reference image. The shifts were generally in good agreement and their average was adopted; in a few cases, one of the stars was affected by a cosmic ray and the other was used. For one image, both stars were affected by a cosmic ray and unusable; that image, 2704, was excluded from the final combination.

The typical uncertainties in the alignment of individual images are between 5 and 10 mas for F450W, F606W, and F814W, and between 10 and 20 mas for F300W.

5. Cosmic rays and weight image

The identification of cosmic rays, a difficult problem in the absence of cosmic-ray splits, was done by comparing each image to an "expected" image based from the median image. The expected image was produced by BLOT using the updated positions, and the combination of DERIV and DRIZ_CR was used to identify cosmic rays. This process was in general very successful except close to the edges of the median image, where there was not enough information to identify cosmic rays properly. Pixels affected by cosmic rays were identified in the cosmic ray mask.

6. Inter-chip alignment verification

The relative positions of the four detectors in the sky were verified using the following procedure:

7. Problem cases

Images affected by a bias jump or by the cross pattern due to scattered Earth light were corrected by applying a median filter (width 21 pixels) to the image obtained after subtracting the "expected" image produced by BLOT in step 3. The median filter preserves the edges and produces an adequate subtraction for a sharply varying background, such as that produced by either a bias jump or the scattered Earth light. Direct inspection confirmed the subtraction to be satisfactory.

Cosmic rays were identified properly in the vast majority of cases. A handful of exceptions - usually due to a very large cosmic ray - were identified at this stage and excluded from further processing; such images will be recovered as part of the Version 2 combination.

Similarly, moving targets usually were flagged properly, but in some cases left visible residuals; these were excluded from further Version 1 processing.

Altogether, approximately 7% of the data were excluded from the final combination because of uncertain pointing or uncorrected blemishes; these are identified in the master log file.

8. Final combination

First, the (small) difference in sensitivity between the detectors was corrected by scaling to a common zero point; WF3 was chosen as reference.

Second, the pixel weights of each input file were determined as the inverse variance of the background within each pixel, scaled to the output pixel size.

DRIZZLE was then run on all acceptable images for each filter with the following parameters:

resulting in a final pixel size of 0.0398" (verified by coalignment with astrometric stars, see below).

The size of the output image is 4096 x 4600 pixels, or 163" x 183" - enough to include all the area with valid data. The final image was set to 0 for low-weight pixels, which suffer from uncorrected cosmic rays and other blemishes and have low scientific value.

As a final step, the image was astrometrically calibrated by using four stars with absolute astrometry determined by the Naval Observatory in the Hipparcos reference frame. The WCS parameters in the header were updated to reflect the best astrometric solution, with an expected error of 40 mas dominated by systematics in the astrometric position of the reference stars.


Appendices

A. The weight images forDRIZZLE

The DRIZZLE program allows each input image to be assigned a pixel-by-pixel weight. We used the inverse variance of the background in each input pixel, scaled by the fourth power of the scale ratio; this produces final weigths that are representative of the inverse variance per pixel on large areas. (The variance measured on small areas is smaller because of the pixel-to-pixel correlation.)

Specifically, we defined the weight image as follows:

VARIANCE = [(DARK+BACKGROUND)*FLATFIELD/GAIN + READNOISE^2] / 
   (FLATFIELD^2 * EXPTIME^2)

WEIGHT = 1 / (SCALE^4 * VARIANCE)
where the dark current DARK and background BACKGROUND were estimated from the image average, rather than on a pixel-by-pixel basis. DARK, BACKGROUND and READNOISE are in DN/pixel; the GAIN and READNOISE were taken from the WFPC2 Instrument Handbook, and are different for each chip; and the FLATFIELD is the inverse of the value in the flat field reference file, which is the INVERSE flat field.

The SCALE parameter is the ratio of output to input pixel size. Since the output size is 0.4 WF pixels, SCALE = 0.400 for WF2, WF3 and Wf4, and SCALE = 0.875 for the PC.

The WEIGHT was set to zero if any of the following conditions applies:

dark current > 0.02 DN/s       (hot pixel)
flagged as cosmic ray 
inverse flat field > 1.7       (too close to the pyramid edge, vignetted)
x, y < 60 (PC) or < 40 (WF)    (too close to the pyramid edge)
x, y > 795                     (too close to the outer edge)

B. Photometric Information

Filter                      F300W       F450W      F606W      F814W
Multiplicative factor:
PC                          1.025       1.027      1.008      1.019
WF2                         0.984       1.008      0.979      0.994
WF3                         1.000       1.000      1.000      1.000
WF4                         1.019       1.030      1.015      1.017
Final zero point
Vegamag                     19.43       22.01      22.90      21.66
ABMAG                       20.77       21.93      23.02      22.09
STMAG                       19.45       21.53      23.21      22.91
Photflam                    5.985e-17   8.797e-18  1.888e-18  2.449e-18
These zero points differ very slightly from those used for the original HDF (-0.02 in F300W, -0.01 in F606W). The camera throughput has remained constant through the years, and the slight differences are due to new measurements of the zero points over the last three years.

C. New Superbias reference file

Name I9817383U.R2H

Created by: Shireen Gonzaga, August 25, 1998

The reference bias file ("superbias") was produced from an average of 120 on-orbit bias frames taken between December 4, 1997 and August 13, 1998 (listed in the file header).

The raw images were retrieved from the archive and recalibrated (mask, atod, and bias level corrections) using CALWP2 Version 1.3.5.2, which removes separate bias levels for the even and odd data columns based on values in columns 9-14 of the extracted engineering datafile (x0h).

The STSDAS wfpc "mkdark" task (identical to a version of "crrej" previously used to make reference bias files) was used to combine the calibrated frames and remove cosmic rays. The task was run with four iterations, using sigmas set to 6,5,5,and 4; Pixels are rejected if they are N sigma above or below the initial guess image. The first initial guess image was taken to be the median of the stack; subsequent iterations use the computed average image as the initial guess. In addition, pixels within 1 pixel of a rejected pixel (in a '+' pattern) were discarded if they deviated by more than 2,2,1.5, and 1 sigmas, respectively for each iteration. Pixels using less than 100 input images were marked in the DQF.

D. New Superdark reference file

Name I9T1701QU.R3H

Created by: Michael S. Wiggs and Stefano Casertano, September 25, 1998.

The reference dark file ("superdark") was created from an average of 120 input darks. The input darks were from the date range of 11/05/1998 thru 21/08/1998, and are named in the image header. All datasets were calibrated using CALWP2.1.3.5.2, utilizing the most up-to-date reference files, including the new superbias: i9817383u. The datasets were then run thru the STSDAS task CRREJECT in 8 groups of 15. This was done assuming a total readout, a-to-d conversion, etc. noise of 6 electrons. The cosmic rays were removed iteratively, with 4 sigma rejection, and then additional iterations were made with 3 sigma and 2 sigma rejection levels.

The resulting 8 "crrejected" datasets were averaged together to create the final 120 image superdark. The superdark image was then normalized to a darktime of 1.0 second for each CCD.

The associated DQF file was computed in the following manner:

E. New Inverse Flat Field reference file

F300W: I9T1701IU.R4H
F450W: I9T1701KU.R4H
F606W: I9T1701MU.R4H
F814W: I9T1701OU.R4H
Created by: John Biretta and Michael S. Wiggs, September 28, 1998.

The flat fields will be described in a forthcoming document.



Send us your feedback or questions.
Visit the Hubble Deep Field South main page.
Visit the Space Telescope Science Institute home page.

This page was last updated on November 24, 1998.
Copyright Notice.