• 1.0 General Information

• 2.0 What is being released
  • 2.1 Description of the files
  • 2.2 File nomenclature
  • 2.3 The Science Images
  • 2.4 The Weight Map Images
  • 2.5 File Size and Data Set Size

• 3.0 Data Reduction and Calibration
  • 3.1 Superbias
  • 3.2 Superdarks
  • 3.3 Flat fields

• 4.0 Astrometry
  • 4.1 Geometrical Distortion Coefficients
  • 4.2 Velocity Aberration Correction
  • 4.3 The GOODS Astrometrical Solution
    • 4.3.1 Fundamental solution for the F850LP image
    • 4.3.2 Derived astrometric solution for F435W, F606W, and F775W

• 5.0 Drizzling
  • 5.1 The Multidrizzle Algorithm
  • 5.2 Cosmic Ray Rejection

• 6.0 Known Issues and Problems
  • 6.1 Missing Data
  • 6.2 Increased background in some of the Epoch 1 B-band images
  • 6.3 Astrometry and Rejection of Cosmic Ray and of Other Blemishes
  • 6.4 Photometry

• 7.0 Updates and Future Releases
  • 7.1 Updates to version v1.0
  • 7.2 Version v2.0

README

1.0 General Information

On Friday, August 29 2003, at 5pm EST, the GOODS Teams has released the version v1.0 of the reduced, calibrated, stacked and mosaiced images acquired with HST and ACS as part of the GOODS ACS Treasury program.

Version v1.0 is a significant improvement upon the previous v0.5 release of GOODS reduced HST/ACS images. Not only are the released images comprised of the mosaiced stack of all the data acquired during the five epochs of GOODS observation. They have also been re-processed using the best calibration and reference files available to the team at this time. Improved geometrical distortion coefficients have been used together with corrections for the velocity aberration distortion, and improved rejection of cosmic rays and other blemishes has been performed.

2.0 What is being released

The version v1.0 release consists of the full, multi-epoch stacked mosaics of the GOODS ACS data in each passband and in both fields of the survey.

No source catalogs or other higher-level data products are being released at this time.

• 2.1 Description of the files

  The data set is organized as a series of images in each of the four GOODS passbands (F435W, F606W, F775W, and F850LP) and in each of the GOODS fields (HDF-N and CDF-S). We refer to these images as "sections".

  Each field has been divided into sections for the following reasons. To optimize PSF sampling, the original, pipeline-processed 4Kx4K ACS images, which have a scale of 0.05 arcsec per pixel, have been drizzled onto a very large virtual grid with a scale of 0.03 arcsec/pixel. Such grid would be 40,000 x 40,000 pixels in size in the HDF-N, and 32,000 x 40,000 pixels in the CDF-S. It would be too large to exist as an individual file (it would exceed the 2 GB limit for file size of 32-bit computers). It would also be highly impractical to work with and to manage, in particular during transfer operations over the Internet. The grid, therefore has been divided into sections.

  Each section is an image 8,192 x 8,192 pixels in size. A total of 17 sections cover the HDF-N, and a total of 18 sections cover the CDF-S.

  Sections are contiguous, namely if all adjacent sections were stitched together with no gaps and in the proper order, with the last line or row of a given section abutting the first line or row of an adjacent section, then it would be possible to reconstruct the original grid (if such a large file could be created).

  Sections are labeled according to their position in the field, identified by a two digit number (see Figures h_cdfs_v1.0sects_plt.jpg and h_hdfn_v1.0sects_plt.jpg below). The first digit of this number represents the "X" position of the section in the field, the second digit the "Y" position, with X and Y increasing towards to the right and upward, respectively. Thus, the section located in the lower left corner is called "section11". The one in the upper right corner is called "section45" in the CDF-S, and "section55" in the HDF-N.

  CDF-S Section Layout

  

  HDF-N Section Layout

  

  Clarifying the relationship between the version v0.5 GOODS data release and the current v1.0 release, there is no one-to-one correspondence between v1.0 sections and v0.5 tiles. A section covers about 44% more area than a tile, and it contains a number of different tiles. Sections include a varying number of tiles, and one tile can contribute to a number of contiguous sections. For more information on v0.5 tile structure see: V0.5 README.

• 2.2 File nomenclature

  The name of the files relative to the sections contain information on the field (whether the CDF-S or the HDF-N), the passband, the data release version (v1.0 in this case), the section, and the image type (whether it is a science image or a weight map). For example, the FITS file containing the F850LP science data for section11 in the CDF-S field is called

  h_sz_sect11_v1.0_drz_img.fits,

  where the prefix "h" and subfix "img" are adopted to satisfy the general nomenclature rules of the STScI data archive, "sz" specifies that the section is relative to the GOODS southern field (s) and is an F850LP image (z, or z band), "sect11" specifies the location of the section in the field, "v1.0" specifies the data release, and "drz" specifies that the file contains an image (drz stands for drizzled). The file containing the corresponding weight map is called

  h_sz_sect11_v1.0_wht_img.fits,

  where "wht" specifies that it is a weight map image.

• 2.3 The Science Images

  The pixel values of the science images report the flux count rate calibrated in electron/second.

  The zero points to convert the count rate into an AB magnitude for the four GOODS passbands are the following:

  • Z0_F435W = 25.65288
  • Z0_F606W = 26.49341
  • Z0_F775W = 25.64053
  • Z0_F850LP = 24.84315

  For the details of the photometric calibration of the ACS camera we refer to the Instrument Handbook.
  Also useful is the Instrument Data Handbook.

• 2.4 The Weight Map Images

  The weight maps are images which are produced as part of the data reduction process, and which give a measure of the background + instrumental nominal noise per unit area (pixel) in the science data. Because the GOODS images are mosaics consisting of different numbers of overlapping images at each place on the sky, the total exposure time varies as a function of position. In addition, pixels have been masked for a variety of reasons, including rejection of cosmic rays, masking of satellite trails and reflection ghosts, etc. Finally, the sky background varied somewhat from exposure to exposure. All of these effects contribute to variable "depth" across the image mosaics.

  During the data reduction, a noise model was used to calculate the expected noise per pixel at the background level from the combination of sky background (modulated by the flat field), readout noise, and dark current. The effects of Poisson shot noise due to signal from objects in the image were not included in this noise model. This noise model was then used to construct weight maps equal to the expected inverse variance per pixel. These weight maps were combined with masks that excluded (i.e., set to zero weight) pixels for various reasons outlined above, and were then used to weight the combination of images in the drizzling process.

  The resulting output weight map should be equal to the expected inverse variance (i.e., 1/RMS^2) per pixel. The interpolations introduced by drizzling the images (shifting, rotating, correcting distortion, and subsampling pixels onto a finer grid) result in correlations between pixels in the drizzled science images. Therefore, the apparent RMS background noise that one measures in the image is smaller than that given by the (inverse) weight maps, because the apparent RMS is suppressed by the effects of these correlations. The weight maps are normalized to show the expected noise per pixel that the images would have in the absence of these correlations. Or, put another way, the sum of the variances (inverse weight values) over some aperture larger than the correlation scale (a few pixels) should accurately reflect the measurement uncertainty due to the background + instrument noise. (We note again that no attempt is made to include Poisson uncertainty due to signal from objects.) For a more detailed discussion of weight map conventions and noise correlation in drizzling, please see Casertano et al. 2000, AJ, 120, 2747, especially Section 3.5 and Appendix A.

  The scaling of the weight maps has been validated by comparing their values to the measured image noise, after a correction for the measured autocorrelation of background pixels. This test indicates that the weight maps correctly predict the image noise to an accuracy of about 5% (perhaps systematically underestimating the noise by about that much, although this needs to be checked more carefully with additional tests). Any updated information will be posted on the GOODS web pages as necessary.

• 2.5 File Size and Data Set Size

  The files for each section, both the science and the weight map images, are ~268.4 MB in size each (8192 x 8192 pixels, each with a 32 depth), and there are images in each of the four GOODS passbands. Thus, the CDF-S data set requires 2x268.4x18x4 = 38.655 GB, the HDF-N requires 2x268.4x17x4 = 36.507 GB. The full data set requires 38.655 + 36.507 = 75.162 GB.

3.0 Data Reduction and Calibration

Raw data read from the ACS CCD array are processed by the ACS pipeline (CALACS), which provides the basic reduction steps of dark and bias subtractions and flat-fielding. It also provides data quality files that flag known hot pixels, bad column and other cosmetic defects. These basically reduced images consist of the individual exposures, or "dithers", taken in each band for each ACS pointings (tiles).

For the v1.0 data release, all raw data have been recalibrated using the best reference files available at this time. This has resulted in new .flt files produced for use in the post-pipeline reduction (multidrizzling). We will replace the previously reduced .flt files with the new one immediately after the release of the v1.0 data.

• 3.1 The Bias Reference Frame

  The bias reference frame is the combination of seven frames acquired with daily cadence. The bias pedestal is removed from the science frames using the 7-day bias frame closest in time to individual observations.

• 3.2 The Dark Reference Frame

  The dark reference frame is relative to a period of two weeks, and is a hybrid composed of a base dark frame that accounts for the removal of the dark pedestal pixel by pixel, and of a component of hot pixels added daily. The base dark frame is the combination of 56 frame, acquired with a cadence of 4 frames per day (thus, it include 2 weeks worth of dark frames). Unfortunately, hot pixels in the ACS CCDs vary with time, as a result of radiation damage and annealing operations. Therefore, the base dark frame is then "sprinkled" at a daily cadence with hot pixel updates from the corresponding set of 4 dark frames. The dark pedestal is removed from the science frames using the bi-weekly frame closest in time to observations with an up-to-date content of hot pixels.

• 3.3 Flat fields The flat field frames in each of the F435W, F606W, F775, and F850LP passbands contain about 100,000 counts per pixel (p-flats). These frames also include large-scale flat-fields (l-flats), obtained through dithered observations of stellar fields. The flat field frames have been tested to be accurate to about 1%.

• 4.1 Geometrical Distortion Coefficients

  The geometric distortion applied to the individual input images uses the most recent (July 2003) values of the IDCTAB distortion coefficients released by the ACS group at STScI. The geometric distortion of ACS was determined on the basis of multiple observations of a star field in 47 Tucanae, including observations at multiple roll angles to determine the presence of skew terms in the solution. Unlike for Version 0.5, the residual skew is found to be very small.

  Currently a single geometric solution is available, independent of wavelength; wavelength-dependent terms are believed to be <~ 0.1 pixels over the full extent of the detector.

• 4.2 Velocity Aberration Correction

  As of July 2003, the geometric solution for ACS includes a small correction for the effect of differential velocity aberration. Each image header includes a parameter called VAFACTOR which represents the correction to the pixel scale in that exposure due to the heliocentric velocity of HST projected onto the line of sight of the telescope. The VAFACTOR term is taken into account when drizzling individual images onto the mosaic, and should require no additional action. As verification, we attempted an astrometric solution in which the pixel scale was allowed to vary from image to image, and we found no residual correlation (at the 1.e-5 level) between the derived scale and the value of VAFACTOR. We did find a small scale variation, which we interpret as related to the limitation of the reference image; see Section 4.3.1 for more details.

• 4.3 The GOODS Astrometrical Solution

  • 4.3.1 Fundamental solution for the F850LP image

    The astrometric solution obtained for the GOODS V1.0 images is based on the F850LP data and was obtained through several steps. The preliminary phase consisted of cross-identifying sources among ACS images, as follows. First, a reference ground-based image was selected for each hemisphere: for the North it was a Subaru R-band image, and for the South a composite image based on deep WFI data. Second, the reference image was astrometrically matched to stellar positions in the GSC2 catalog, allowing for a second order distortion. Third, the F850LP ACS images were combined into "tiles", with one tile for each epoch and each pointing; a tile consists of four individual exposures and is rather clean, containing only a small number of residual cosmic rays and other imperfections. Fourth, each tile was registered to the reference ground-based image in order to cross-identify sources; the match between ground-based and ACS images had a typical noise of 0.1".

    With the initial cross-identification complete, the global astrometric solution was obtained by a multi-parameter chi-squared minimization for each hemisphere. Each tile was allowed three free parameters, namely the RA and DEC of the reference point and the position angle. In addition, the scale and two skew terms were allowed to vary, but were forced to be the same for all tiles in each hemisphere. For each cross-identified source, the quantity to be minimized included the difference between the ground based and the ACS position, with nominal error 0.1", and the difference between each ACS measurement and their average, with nominal error 0.005". Therefore the optimal solution matches ACS-to-ACS measurements with a very stiff penalty, and matches ACS-to-ground positions with a softer penalty. This approach allowed us to take advantage of the relative stiffness of ACS to constrain the overall distortion of the solution, while retaining the global scale and orientation constraints from the ground-based image. Allowing the scale to vary from image to image would introduce a small position-dependent gradient in the pixel scale, which we interpret as a residual effect of the imperfection of the ground-based solution. The final solution had a clipped RMS deviation of 0.006-0.007" in each coordinate in the ACS-to-ACS difference, and of 0.12" in each coordinate in the ACS-to-ground difference.

    We are aware that slight improvements are possible in the procedure we used, in particular in the quality of the reference ground-based image; such improvements will be applied for V2.0.

    Once the astrometric solution was obtained for each tile, the information was back-propagated to the constituent images and each image was drizzled onto the final grid as described in Section 5.

  • 4.3.2 Derived astrometric solution for F435W, F606W, and F775W

    The images in the other bands were aligned to the F850LP solution by matching source positions to the average position of the corresponding source in F850LP, as determined from the fundamental astrometric solution. The typical clipped rms residual (F850LP average to other filters) was 0.007" to 0.010" in each coordinate; we noticed a significant non-Gaussian tail especially in F435W, presumably due to morphological differences between images in different bands.

5.0 Drizzling

This processing is done in two independent phases. In the first phase, images of the same tile taken in the same filters are identified, sky-subtracted and drizzled onto a common pixel grid with the same scale as the input images (0.05 arcsec/pixel). Cosmic rays and deviant pixels are identified during this process and flagged in mask files specifically created for this purpose. Information included in post-pipeline masks (which have also been drizzled onto the same grid) is included in the new masks at this time.

During the second phase, the images and the mask files are blotted back to the original positions, drizzled again onto a common astrometric grid with scale 0.03 arcsec/pixel, and stacked together. During this process corrections for the ACS geometrical distortion are applied, cosmic rays flagged during the previous processing block are masked out from the stack, and additional, low-level cosmic rays and defects are identified and masked, too.

• 5.1 The Multidrizzle Algorithm

  MultiDrizzle (Koekemoer et al. 2002, HST Calibration Workshop) is a single Pyraf script which automatically performs the following steps on a list of input flatfielded ACS images (flt.fits files):

  • - bad pixel identification
  • - sky subtraction
  • - drizzle each input image onto separate, registered output images
  • - create a median from these images
  • - transform ("blot") the median back to the frame of each input image
  • - create a derivative of the median image, and compare with the original image to obtain a cosmic ray mask
  • - refine the shifts by cross-comparing the images
  • - run drizzle a second time, to combine the images with the cosmic ray masks into a final, combined image.

  This technique is sufficiently flexible to enable adjustment by means of a range of parameters, but is also sufficiently automated that it is capable of being run in a pipeline such as that which was used by GOODS.

• 5.2 Cosmic Ray Rejection

  Initially a set of shallow, single-exposure-depth mosaics are created which are then combined to create a clean "median" mosaic, using signal-to-noise thresholds to reject cosmic rays and any remaining bad pixels. This technique is extremely robust at producing a clean median mosaic, with the primary requirement being accurate astrometric alignment between epochs. The relevant pieces of this median mosaic are then transformed back to the frame of each of the original input flt.fits files, using "wblot" which is essentially a WCS-based version of "blot".

  Next, the standard dither package tasks of "deriv" and "driz_cr" are used to compare this blotted image and its derivative image with the original input flt.fits file, and generate a cosmic ray mask based on the comparison. Finally, all the flt.fits files, together with their newly created cosmic ray masks, are drizzled using "wdrizzle" onto a single output mosaic, which has units of countrate in electrons/second in each pixel.

• 6.1 Missing Data

  A small number of individual exposures are missing from the final multi-epoch HDF-N mosaics. The areas where exposures are missing can be easily identified by displaying and inspecting the weight maps. In one case, one tile (i.e. an area with the footprint of the ACS WFC field of view) in the V (2 exposures), i (2 exposures and z (4 exposures) bands was completely lost during epoch 5 due to a failed guide star acquisition. The GOODS Team plans to request 2 orbits to re-observe the missed tile and produce a new stacked mosaic. We will post a note in the GOODS Web page and release an update to the data when this is done.

  In another instance a few individual exposures have not been included in the final z-band stack in the HDF-N, because using them in the multidrizzle pipeline caused a software problem. It is not understood why this happens as no obvious anomalies were observed. It is likely that there is a discrepancy in some of the header parameters that has not been identified. Once this problem is corrected, we will post a note in the GOODS Web page and release an update to the data.

• 6.2 Increased background in some of the Epoch-1 B-band images

  Some B-band tiles acquired during Epoch 1 in the HDF-N are affected by a relatively large background scattered light due to an error in the scheduling of the observations. This allowed the limb angle to become smaller than about 25 degree, which resulted in increased Earth scattered light. Such tiles have been re-observed at later epochs. The old and new B-band images have been co-added in the mosaic with optimal weighting to maximize signal-to-noise ratio. The affected tiles (and therefore, the affected sections or part of sections) can be identified by inspection of the weight maps. Upon visual inspection, this scattered light appears rather uniform. No special attention or actions are required when working with these images.

• 6.3 Astrometry and Rejection of Cosmic Ray and of Other Blemishes

  Inspection of the weight maps sometimes reveals that residual mis-alignments due to scatter in the astrometric solutions cause rejections of the brightest pixels in bright, unresolved sources by the cosmic-ray rejection algorithm. Only a handful of bright sources are affected. The Team will refine the astrometric solution in the following weeks. We also plan to release a list of stars where we do find evidence of this problem. In no case the rejection problem has been observed in diffuse sources.

• 6.4 Photometry

  One known problem is that the photometry of a very small number of bright sources in the mosaics is systematically biased toward low fluxes because of small astrometrical errors, as mentioned above.

  Another known problem is that the photometry of bright stars in the multi-epoch stacked mosaic is systematically off typically by ~1 percent and in some case by up to ~2 percent, the sense being that the magnitude appears fainter. No such effect is observed in single-epoch stacked images. We discovered that this effect is due to the rejection of the "rotated" diffraction spikes by the cosmic-ray rejection algorithm. This problem seems to affect only bright (z<~20) unresolved sources, namely those for which the energy fraction contained in the diffraction spikes is the highest. It effect becomes progressively less noticeable at fainter flux, because fluctuations in the background become comparable to the spikes. It also becomes less significant if small photometric apertures are used (e.g. less than ~1% for 0.5 arcsec diameter apertures). This is a limitation, albeit a minor one, of the "multi-epoch" approach to the observations.

• 6.5 PSF Issues

  During a small number of exposures fine guidance was lost, and the observations were completed in gyro mode. Some of the images affected show a small trailing effect (about 1 pixel). For some applications (e.g. weak lensing), this might be a concern. The GOODS Team will post a list of the affected areas as soon as the list is complete.

7.0 Updates and Future Releases

• 7.1 Updates to version v1.0

  We will announce updates to the v1.0 release data set on the GOODS website together with the instructions on how to download them.

• 7.2 Version v2.0

  A version v2.0 is planned for some time in 2004. Details of this release will be made available when they are finalized.

Back to ftp site.

Back to top of page.