3.2 Pipeline Overview
Pipeline processing is carried out by 2 separate packages: CALACS, which corrects for instrumental effects and produces calibrated products, and PyDrizzle, which corrects for distortion and combines associated dithered data.
3.2.1 CALACS: Image Calibration
The CALACS software consists of a series of individual tasks that:
- Orchestrate the flow of data through the pipeline.
- Perform the initial tasks of basic two-dimensional image reduction (e.g., overscan subtraction, bias subtraction) for CCD data.
- Reject cosmic rays from CR-SPLIT CCD data.
- Perform the remaining tasks of basic two-dimensional image reduction (e.g., dark subtraction, flat fielding) for CCD and MAMA data.
- Sum any REPEAT-OBS exposures.
The calibrated products from the pipeline may still contain artifacts such as hot pixels and cosmic-rays. If hot pixels are to be detected and flagged by CALACS, dithered exposures are required. Similarly, cosmic ray rejection by the pipeline requires the specification of CR-SPLIT exposures in the observing program. In the future, improved pipeline software will allow all associated data (including REPEAT-OBS exposures and dithered exposures) to be automatically corrected for cosmic-rays.
While intermediate steps in CALACS make use of sky subtraction, for example for identifying cosmic-rays, all data products created by the pipeline will not be sky subtracted. Fully calibrated data products (with suffixes FLT, CRJ, SFL) will be in units of electrons.
CALACS and Single exposures
A single raw ACS exposure is processed by CALACS following the steps outlined in Section 3.3. These steps include the standard detector calibrations: bias subtraction, dark subtraction, flat fielding, etc. When CALACS processes a single raw exposure, the calibrated product is given the FLT suffix. The data in the calibrated SCI and ERR extensions are in units of electrons.
CALACS and Multiple exposures
The same processing steps performed on single images are performed for multiple images which are part of an association. The result is a single calibrated product in units of electrons.
CALACS will recognize and correctly process CR-SPLIT or REPEAT-OBS exposures by interpreting the association table and determining which exposures should be combined during calibration. REPEAT-OBS exposures are individually run through all the calibration steps to create calibrated, flat-fielded (FLT) product for each input file. The FLT images are then summed to create the summed flat-fielded (SFL) product. No cosmic-ray rejection is performed for REPEAT-OBS images.
Observations taken as part of a CR-SPLIT association, on the other hand, are combined during cosmic-ray rejection. First, the bias and dark subtraction is performed. Then, the images are combined while at the same time rejecting cosmic rays. The product is then flat fielded to create a single calibrated, cosmic-ray rejected, flat-fielded (CRJ) image.
CALACS and Dithered exposures
Observations which use the dither patterns provided in the proposal instructions will be automatically associated for combining in the pipeline. Any pattern designed with POS-TARGs will not be associated. When processing observations which are part of a dither pattern, CALACS will produce a calibrated FLT, CRJ, or SFL file at each dither position. It will not, however, combine images from multiple positions. Further pipeline processing by PyDrizzle will correct for geometric distortion and combine dithered images.
Table 3.1: The input and output image suffixes from CALACS and PyDrizzle are given for various observing modes.
| Image Type |
CALACS Input |
CALACS Output |
PyDrizzle Input |
PyDrizzle Output |
Cosmic Ray Rejected? |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3.2.2 PyDrizzle: Distortion Correction and Dither Combination
All ACS data will be automatically corrected for distortion during pipeline processing using a task called PyDrizzle. This task relies on the IDCTAB reference file for a description of the distortion model. PyDrizzle understands the ACS association tables which allow the pipeline to combine dithered observations. Only observations which use the dither patterns provided in the proposal instructions will be automatically associated for combining in the pipeline. Programs which rely on explicit POS-TARG commands will NOT be associated, resulting in separately calibrated images for each position. PyDrizzle will automatically produce images which are both astrometrically and photometrically accurate regardless of whether they were taken as a single exposure or a set of dithered exposures.
The flat-fielded, calibrated products from CALACS are listed in Table 3.1. The calibrated product of a single ACS exposure is an FLT file. The product of a CR-SPLIT or REPEAT-OBS association is a CRJ or SFL file, respectively. The product of a dither pattern is an FLT, CRJ or SFL file at each dither position. Using these files as input, PyDrizzle performs the geometric correction on all data (dithered or not) and combines multiple dithered images into a single output image with the DRZ suffix. For WFC observations, both chips are combined into a single extension. PyDrizzle then converts the data to units of count rate (electrons/sec).
It is important to note that PyDrizzle, when combining dithered images, does not remove cosmic-rays. This can be done offline with a new task called MultiDrizzle using the individual FLT files as input. Subsequent reprocessing with PyDrizzle or MultiDrizzle offline may be required to obtain the desired scientific result. For information on obtaining this software, we refer the reader to Section 3.5.1.
3.2.3 When is OTFR not Appropriate?
The goal of the ACS pipeline is to provide data calibrated to a level suitable for initial evaluation and analysis for all users. Observers frequently require a detailed understanding of the calibrations applied to their data and the ability to repeat, often with improved products, the calibration process at their home institution. There are several occasions when OTFR is not ideal and when off-line interactive processing by the user is required:
- when running CALACS with personal versions of reference files,
- when running CALACS with non-default calibration switch values,
- when images must be cleaned of artifacts such as new hot pixels or cosmic-rays.
Many ACS datasets do not contain multiple exposures at the same pointing and hence will not be cleaned of cosmic rays by OTFR processing. In addition, the increasing prevalence of hot pixels, particularly in the WFC mode, makes dithered observations highly desirable. In these cases the pipeline will provide products which are geometrically corrected but which still contain cosmic rays.
To address this problem, the MultiDrizzle script has been developed to be run interactively outside the pipeline. MultiDrizzle provides a single-step interface to the complex suite of tasks in the STSDAS dither package. It makes use of the PyDrizzle software and can only be executed in the PyRAF environment. For more information, we refer the reader to Chapter 4.
