As mentioned earlier, the calacs
pipeline consists of four individual calibration tasks: acsccd
, acsrej, acs2d
, and acssum
. These tasks are listed in Table 3.2
and diagrammed in Figure 3.1
is responsible for controlling the processing rather than the actual calibration of the data. The individual tasks apply the appropriate calibration steps to the data and create the output products.
Raw ACS images are in units of Data Numbers (DN)
. Calibrated images are in units of electrons
, and drizzled images are in units of electrons/second
This task contains the initial processing functions for all ACS CCD data. These
functions are listed, in operational order, in Table 3.3
. The bulk of this routine was based on the code for calstis1
processes each image in the input list, one at a time, using the header keywords to determine which calibration steps to perform. Upon completion of acsccd
, the overscan regions will be trimmed from the image and an output image with the file suffix “blv_tmp.fits
” is created.
Table 3.3: acsccd
- Error Array Initialization
images delivered by Generic Conversion will contain a null
error (ERR) array, defined by the header keywords NPIX11
, where PIXVALUE=0
The first processing step is to initialize the image error array: if this ERR array has
already been expanded in prior processing and contains values other than zero, doNoise
does nothing. Otherwise, calacs
initializes the array and the doNoise
function assigns a simple noise model to the ERR array.
The noise model reads the science array and, for each pixel, calculates the error
value σ (in units of Data Numbers [DN] for the CCD and in units of counts for the MAMA):
For CCD observations: prior to SM4, there was a negligible gradient in the bias,
therefore a constant value was used for the bias
variable in the noise equation. However, for post-SM4 processing, the bias
variable now also incorporates the bias gradient as given in the corresponding bias calibration image.
Because MAMA data is not processed by acsccd
, the doNoise
function is later called, just for processing SBC data, as part of acs2d
The “CCD Characteristics Table” reference file, recorded in the image header
, is used to determine the bias, gain, and read noise values for an observation, and for use in calculating error values (σCCD
) for the ERR array. The table columns are
reference table uses these commanded values to determine an observation’s physical readout characteristics that are written to the flt.fits
calibration file. For instance, readnoise values for each amplifier are populated in header keywords READNSEA
, and READNSED
; A-to-D gain in ATODGNA
, and ATODGND
; bias voltage offset values in CCDOFSTA
, and CCDOFSTD
- Bad Pixel Determination
The function doDQI
initializes the Data Quality (DQ) array by combining it with a table of known permanent bad pixels for the detector, stored in the bad pixel reference table (named by the image header keyword BPIXTAB
). The type of bad pixel flags are listed in Table 3.4
The DQ array may have already been partially populated to reflect pixels which
were affected by telemetry problems or compression buffer overflow. Other DQ flags will be added in further processing steps (such as cosmic ray rejection). For CCD data, the values in the SCI extension are also checked for saturation by comparing image pixel values with the value of the SATURATE column in the CCD parameters table (named in header keyword CCDTAB
For each observation, the doDQI
function combines the DQ flags from prior calibration processing (obtained from the BPIXTAB
reference table) and from saturation tests, into a single DQ result. These values are combined using a “bitwise/logical OR operator” for each pixel. Thus, if a single pixel is affected by two DQ flags, the sum of those flag values is assigned to the corresponding pixel in the final DQ array. This array then serves as a mask so that calacs
will ignore bad pixels during processing.
- A-to-D Correction (skipped)
function converts pixel values in the raw input image from units of DN (16-bit integer) to counts (32-bit float). An analog-to-digital correction would be applied if the CCD electronic circuitry, which performs the analog-to-digital conversion, were biased toward the assignment of certain DN (Data Number) values. Ground test results show that this correction is not currently needed, so the ATODCORR
switch is always set to OMIT
- Bias level Correction
function fits the bias level from the physical overscan section and subtracts it from the image data. The 2
overscan area region is defined in the OSCNTAB
A subset of the pixels along each row of the vertical overscan is used to obtain a
linear fit of the bias level as a function of image row. If the measured overscan level for a given row is more than 3σ from the mean of all rows, that row will be excluded from the linear fit. The fitted value for each overscan row is then subtracted from the image pixels for that row. If the overscan level cannot be determined, or if the overscan region is not present in the input image, a default value from the CCD parameters table (CCDBIAS
) will be subtracted and an error message will be written to the image trailer file.
, the overscan regions were removed from the image just after the bias level subtraction, since no subsequent processing was required. calacs
, on the other hand, retains the output image in memory until the completion of all the processing steps in acsccd
. The overscan regions will not be trimmed until the image is written to disk at the completion of acsccd
. The positional image header keywords CRPIX1
are updated to reflect the offset due to removal of the overscan. In addition, the mean value of all overscan levels is computed and written to the SCI extension header keyword MEANBLEV
. The individual bias level measured for each amplifier is written to the SCI header keywords BIASLEVA
, and BIASLEVD
The “Overscan Region Table” reference file, named in the header keyword OSCNTAB
, describes the overscan regions to be used for measuring the bias level of the observation. Each row corresponds to a specific configuration, given by the CCD amplifier and chip. The columns TRIMX* give the number of columns to trim off the beginning and end of each line (the physical overscan region), while the TRIMY* columns give the number of rows to trim off the top and bottom of each column (the virtual overscan region). The result of trimming (TRIMX1 + TRIMX2) columns and (TRIMY1 + TRIMY2) rows gives the final calibrated image sizes, 4096 x 2048 for a full WFC image and 1024 x 1024 for a full HRC image.
columns BIASSECTA* and BIASSECTB* give the range of image columns to be used for determining the bias level in the leading and trailing regions, respectively, of the physical overscan region.
The virtual overscan region is also defined in the OSCNTAB
, where (VX1, VY1) indicates the beginning of the upper or lower overscan region, extending to the position (VX2, VY2).
To determine which overscan regions were actually used for determining the bias
level, users are encouraged to check the OSCNTAB
reference file. If desired for manual calibration, users may modify the bias section and virtual overscan region definitions in the reference table, but the TRIMX*, TRIMY* columns must not be changed.
- Bias Image Subtraction
Subtraction of the bias image is performed prior to cosmic ray rejection using the
. The appropriate reference image, named in the header keyword BIASFILE
, is selected using the DETECTOR
, and CCDGAIN
image header keyword values. The dimensions of the science image are used to distinguish observations which use the entire chip from sub-array images.
Because the bias image has already been scaled by the gain and had the overscan
values subtracted, the bias image may have a mean pixel value which is less than one.
Due to the way the bias reference image is created, part of the dark subtraction is
also included in this step. Dark counts accumulate for an additional time beyond the exposure time, primarily the time required to read out the detector, and this portion of the dark current is subtracted along with the bias. This is described further in the section on “doDark - Dark Image Subtraction”
The bias reference files are updated frequently (about weekly). The BIASFILE
has the same dimensions as a full-size science image, 1062 x 1044 for HRC and 4144 x 2068 for WFC, allowing for simple image arithmetic on arrays of equal size. Only after the completion of acsccd
are the images trimmed to 1024x1024 (HRC) and 4096x2048 (WFC). calacs
assumes that CCD data will not be read out in binned mode, so any image which is not the full size is assumed to be a sub-array.
Prior to the ACS failure in January 2007, for sub-array images, calacs
uses the LTV[1,2]
keywords to extract the appropriate region from the reference file and apply it to the sub-array input image. But in post-SM4 data, the ACS WFC bias structure in sub-arrays is qualitatively different from full-frame readouts. Therefore, bias calibration images are available for supported sub-array configurations. For users who take their own bias files, the BIASFILE
keyword will need to be updated manually before recalibrating. This is described in detail in Section 3.5.2
ACS has a post-flash capability to provide the means of mitigating the effects of
charge transfer efficiency degradation. The proposer controls the use of this capability via the “FLASHEXP” optional parameter in the Phase II proposal.
The reference file, named in the header keyword FLSHFILE
, has the same dimensions as a full-size science image, 1062
1044 for HRC and 4144
2068 for WFC, allowing for simple image arithmetic on arrays of equal size. The appropriate FLSHFILE
reference image is selected using the following keywords from the image header: DETECTOR
, and SHUTRPOS
image header switch has a default value of OMIT
for pipeline processing even if a post-flash was performed, thus producing an image with excess sky background in the calibrated products. To remove the post-flash effect, the user would have to recalibrate the data with the header keyword switch FLSHCORR =
.” The function doFlash
will subtract the post-flash in the following way:
reference image is normalized to one second for the appropriate post-flash current level (LOW
, given by the FLASHCUR
image is then multiplied by the flash duration in seconds (given in the header keyword FLASHDUR
) and is subtracted from the science image.
Upon completion of acsccd
, the overscan regions will be trimmed from the image when it is written out, but only if doBlev is performed successfully
. Otherwise, the full image array will be written out.
, the cosmic ray rejection task in calacs
, combines “CR-SPLIT” exposures and repeated sub-exposures in an exposure, into a single image, free of cosmic rays. The task uses the same statistical detection algorithm developed for STIS data (ocrrej
) and WFPC2 data (crrej
), providing a well-tested and robust procedure. The parameters used by acsrej
are obtained from the “Cosmic Ray Rejection Table”
(named in header keyword CRREJTAB
), and depend on the type of chip, number of “CR-SPLIT” or repeated sub-exposure images, and exposure time of each image.
will compute the sky background using the mode of each image. Sky subtraction is performed before any statistical checks are made for cosmic rays. Next, acsrej
constructs an initial comparison image from each input exposure. The comparison image can either be a median- or minimum-value sky-subtracted image constructed from all the input images, and represents the “first guess” of a cosmic ray-free image. This comparison image serves as the basis for determining the statistical deviation of each pixel from the input image.
A detection threshold is then calculated for each pixel based on the comparison
image. This threshold is equal to a constant times sigma squared, given in the equation below:
is the read noise squared (in DN) and gain
is the CCD gain (in e-
/DN) for the amplifier used to read the pixel,
is the scale factor for the noise model,
is the pixel value (in DN) from the median- or minimum- combined comparison image.
is the sky background (in DN) of image n
If , the pixel is flagged as a cosmic ray in the input image’s DQ array and is
ignored when images are summed together. Surrounding pixels within a given expansion radius are marked as “spill” pixels and are given less stringent detection thresholds.
When all input images have been processed, the values of the acceptable pixels are
summed over all input images. Each pixel in the summed output array is then scaled by the total exposure time:
is the total exposure time (regardless of whether all input images were used for that particular pixel). This corresponds to the header keyword values TEXPTIME
The following crj.fits
image keywords are derived from the variables in this computation:
The remaining keywords EXPSTART
are updated based on the input image headers.
In summary, the cosmic ray rejection task sums all accepted pixel values, computes
the true exposure time for that pixel, and scales the sum to correspond to the total exposure time. The final scaled, cleaned pixel is written to the comparison image to be used for the next iteration. This process is then repeated with successively less stringent detection thresholds, as specified by the crj.fits
image header keyword CRSIGMAS
. Further processing by calacs
will scale the “pixout(x,y)” array by the gain, resulting in the summed, cosmic ray eliminated, but not sky-subtracted product (crj.fits
) in units of electrons
uses the “Cosmic Ray Rejection Parameter Table” (header keyword CRREJTAB
) to determine the number of iterations for cosmic ray rejection, the sigma levels to use for each iteration, and the spill radius to use during detection. This allows the rejection process to be tuned to each detector, with suitable defaults being applied during pipeline processing. Observers may fine-tune the cosmic ray rejection parameters when manually reprocessing data with acsrej
by editing the CRREJTAB
reference file contains the basic parameters necessary for performing cosmic ray rejection. The column names and default values for the CRREJTAB
are given in Table 3.5
. The appropriate row is selected based on the chip being processed (CCDCHIP
), the number of images into which the exposure was split (“CR-SPLIT” Phase II optional parameter), and the exposure time of each “CR-SPLIT” image (MEANEXP
). The sky fitting algorithm is controlled by the parameter acsrej.skysub
which can have values of “mode
” or “none
”. The “first guess” CR-combined image is then created using the median or minimum value of the input exposures, as specified by the table column INITGUES.
Cosmic ray detection requires the specification of a threshold above which a pixel
value is considered a cosmic ray. This threshold, defined earlier as
, uses the sigma rejection thresholds
that correspond to the column CRSIGMAS in the CRREJTAB
reference file. The table column SCALENSE is a multiplicative term (in percent) for the noise model and is given as scale
in the threshold equation. This term can be useful when the pointing of the telescope has changed by a small fraction of a pixel between images. Under such circumstances, the undersampling of the image by the CCD will cause stars to be rejected as cosmic rays if a scale noise term is not included. This is a crude but effective step taken to satisfy the maxim of “first do no harm.” However, for cases in which there have been no frame-to-frame offsets or if the image is locally well-sampled, this will unduly bias against
rejecting cosmic rays.
Pixels within a given radius of a cosmic ray, specified in the table column
CRRADIUS, will also be treated as cosmic rays. A less stringent rejection threshold, CRTHRESH, can be used for detecting pixels adjacent to a cosmic ray. As in CRSIGMAS, CRTHRESH is also given as a sigma value. If CRTHRESH is exceeded, pixels within a defined radius of the cosmic ray will also be flagged. All pixels determined to be affected by a cosmic ray will have their DQ values set to 8192, as described in Table 3.4
The pipeline adopts a conservative value for SCALENSE to avoid doing harm. In
recalibrating several frames for a new cosmic ray elimination, it would be advisable to determine the full range of relative x,y offsets. An appropriate value of SCALENSE is 100*(maximum offset in pixels), thus if the full offset range was 0.1 pixels an appropriate SCALENSE is 10.0. To alter this value, one may use tedit
on the CRREJTAB
table (calibration file with the extension crr.fits
). The number of exposures obtained in a repeated observation set can be larger than the maximum “CR-SPLIT=8” allowed in Phase II proposals.
For instance, in program 9662, 14 individual 1.0 sec. HRC exposures were
obtained at the same pointing. A check of relative offsets showed that a shift of about 0.2 pixels occurred between the first and last exposures. To obtain a crj.fits
combined image of all 14 inputs at 1.0s and with better sensitivity to cosmic ray elimination than provided by the conservative default of SCALENSE = 30.0, the following should be done:
|Create a new asn.fits
table with one EXP-CRJ entry for each of the 14 raw 1.0s images, and an appropriately named PROD-CRJ file line.
on the crr.fits
table adding a line with a “CR-SPLIT” value of 14, and a SCALENSE value of 20.0.
|Then rerun calacs
on the new association table with 14 entries to obtain a crj.fits
extension image based on the full stack of 1.0s images.
Every observation, whether taken with the MAMA or CCD detectors, will be
processed by acs2d
. The primary functions of the task, listed in Table 3.6,
include dark current subtraction, flat fielding, and photometric keyword calculations. acs2d
contains the same data quality and error array initialization functions used in acsccd
, but acs2d
will check to ensure that the array initialization is not performed twice on CCD data. Calibration switches in the image header control the performance of the remaining calibration functions, with MAMA-specific functions being initiated only when the relevant calibration switches are set.
- Error Array Initialization
checks the image error array (ERR) to determine if there are non-zero values that were created from previous processing steps. If all pixel values are zero, indicating no prior calibration steps were performed, acs2d
runs the same initializations described in “doNoise - Error Array Initialization”
. However, if the input image’s ERR array has non-zero values, indicating processing in earlier calibration steps, the doNoise
function in acs2d
does not change the ERR array.
- Bad Pixel Determination
If the DQICORR
header keyword switch is set to COMPLETE
(e.g., for CCD data), this step is skipped. Otherwise, the same initialization is performed as described in “doDQI - Bad Pixel Determination”
- Linearity Correction for MAMA Data
This routine flags global and local nonlinearity in ACS MAMA observations; the
term “global” refers to the entire ACS MAMA detector, while “local” refers to an individual detector pixel. The MAMA Linearity Table, MLINTAB
, provides the basic parameters for determining linearity.
The global limit (GLOBAL_LIMIT) column in this table refers to the total count
rate at which the data are affected by greater than 10% non-linearity across the detector.
attempts to correct for non-linearity up to the global limit using the non-linearity time constant in the column TAU. The global linearity correction is computed for every pixel below the global linearity limit specified by iteratively solving the equation
to get the true count rate N.
The LOCAL_LIMIT can actually be much higher than the global limit and is
difficult to correct using a simple algorithm. Each pixel found to exceed this limit will simply be marked as non-linear in the DQ file. This DQ flag will be extended by a fixed radius from the original pixel, given in the EXPAND column and is currently set to 2 pixels.
If the LFLGCORR
switch is set to PERFORM
excessive global and local nonlinearity in the DQ array. If GLINCORR
is set to PERFORM
, it will correct
excessive global nonlinearity in the SCI array, if it is not too large. If the global linearity limit is exceeded, the keyword GLOBLIM
in the SCI extension header will be set to EXCEEDED
. Otherwise, it will have the value NOT-EXCEEDED
- Dark Image Subtraction
For the SBC, the header switch DARKCORR
is set to OMIT
because the SBC dark countrate is so negligible that corrections are not needed, even for long exposures.
For CCD data, the dark reference file image (in electrons/sec
) is multiplied by the image’s exposure time, then divided by the gain. This scaled dark reference image is subtracted from the input image in memory. The mean dark value is then computed from the scaled dark image and used to update the MEANDARK
keyword in the SCI image header. The “dark time” is simply the exposure time; it does not
include the idle time since the last flushing of the chip or the readout time. Any dark accumulation during readout time or post-flash idle is automatically included in the bias image reference file (BIASFILE
Each daily dark reference image is created by incorporating the hot pixel
information from a daydark with its corresponding basedark. Only hot pixels above 0.08 e−
/sec. are identified with the flag 16 in the data quality (DQ) array of WFC and HRC reference darks; this flag value is propagated to the DQ array of the calibrated science data. The flagged hot pixels are typically excluded in subsequent processing steps, as they are likely too noisy to be correctable. But the many “warm” pixels below this threshold are assumed to be adequately corrected by the dark calibration. This produces a high signal-to-noise reference file which accurately reflects (and corrects/flags) the hot pixels present for a given observation date. The “best” dark file is typically not available in the pipeline until 2 to 3 weeks after the date of observation, because it takes a few weeks to collect enough frames to make a basedark (see ACS ISR 2004-07
for more information).
The reference file for dark subtraction, given in the header keyword DARKFILE
, is selected based on the values of the keywords DETECTOR
, and CCDGAIN
in the image header, and of course, the date of observation. The dark correction is applied after the overscan regions are trimmed from the input science image. As in the bias image correction (BIASFILE
assumes that the images have not been binned, so any input image smaller than the full detector size will be interpreted as a sub-array image.
- Flat Field Image(s) Correction
As in STIS, the ACS doFlat
routine corrects for pixel-to-pixel and large-scale sensitivity gradients across the detector by dividing the data with the flat field image. (This is in contrast to WFPC2 and NICMOS data that is flat-fielded by multiplying the image with an “inverse” flat field reference file.)
When performing the flat field correction, calacs
also multiplies by the gain so that the calibrated data will be in units of electrons.
Because of geometric distortion effects, the area of the sky seen by a given pixel is
not constant, and therefore, observations of a constant surface brightness object will have count rates per pixel that vary over the detector, even if every pixel has the same sensitivity. In order to produce images that appear uniform for uniform illumination, the flat fields make an implicit correction for the geometric distortion across the field that is equivalent to dividing each pixel by the optical distortion which is normalized to unity at the center of the field. A consequence of this procedure is that two stars of equal brightness do not have the same total counts after the flat fielding step. Thus, point source photometry extracted from a flat fielded
image must be multiplied by the effective pixel area map
, as shown in Figure 5.1
Geometric distortion corrections are also implemented in the pipeline by multidrizzle
which uses the geometric distortion solution to correct all pixels to equal areas. Thus, in drizzled images, photometry is correct for both point and extended sources.
The flat field image used to correct the data is created using up to four flat field
reference files: the pixel-to-pixel file (PFLTFILE
), the low-order flat (LFLTFILE
), the delta flat (DFLTFILE
), and for HRC, the coronagraphic spot flat (CFLTFILE
is a pixel-to-pixel flat field correction file containing the small scale flat field variations.
is a low-order flat which will correct for any large-scale flat field variations across each detector. This file is stored as a binned image which is expanded when it’s applied by calacs.
is a delta-flat containing any needed changes to the small-scale PFLTFILE
is a spot mask which contains the vignetting patterns of the HRC occulting spots and is applied to coronagraphic observations only.
If the LFLTFILE
, or CFLTFILE
reference images are not specified in the SCI image header, only the PFLTFILE
is used for the flat field correction. If all four reference files are specified, they are multiplied together to form a complete flat field correction image which is then applied to the science data.
Currently, the LFLTFILE
flats are not used for ACS data. The PFLTFILE
reference flat in the pipeline is actually a combination of the pixel-to-pixel flats taken during the ground calibration and the low-order flat correction derived in-flight. The CFLTFILE
is applied only when the OBSTYPE
image header keyword is equal to CORONAGRAPHIC
All flat field reference images will be chosen based on the detector, amplifier, and
filters used for the observation. Any sub-array science image will use the same reference file as a full-size image. calacs
will extract the appropriate region from the reference file and apply it to the sub-array input image.
- Shutter Shading File Correction (skipped)
routine would apply the shutter shading correction image (named in image header keyword SHADFILE
) to the science data if the SHADCORR
header keyword switch were set to PERFORM
. However, the SHADCORR
calibration switch is currently set to OMIT
and is unlikely to be used in future versions of calacs
. Calibration data show that a shading correction is not needed for ACS data. For more information, refer to ACS ISR 2003-03
and to the report
“WFC and HRC Shutter Shading and Accuracy” on the ACS Science and Engineering Team Web site at:
- Photometry Keyword Calculation
Before photometry can be performed on ACS observations, a transformation from
electrons to absolute flux units must be performed. For a given instrument configuration, as described in the PHOTMODE
header keyword, calacs
uses the GRAPHTAB
reference look-up tables to determine which individual throughput component tables are needed to construct the total throughput of an observing mode.
SYNPHOT throughput component tables contain throughputs as a function of
wavelength for various components that determine the absolute sensitivity for an observing mode: HST and ACS optics transmissivity; detector QE tables; aperture and filter throughputs. ACS-related throughput tables are updated as needed if there are improved calibrations or for time-dependent sensitivity changes. The individual components tables for a given observing mode is used to create a total throughput table which is used to determine the PHOTFLAM3
header keyword values. The synphot
can be used to see the individual component throughput tables used to construct the total throughput for an observing mode. For example:
Users who wish to convert calibrated images (in units of electrons) to flux units
may simply divide the image by the exposure time and then multiply by the PHOTFLAM
keyword value. Drizzled images are already in units of electrons per second and may simply be multiplied by the PHOTFLAM
value in the drizzled image header to create an image in flux units.
- Image Statistics Determination
This routine computes the number of pixels which are flagged as “good” in the data
quality array. The minimum, mean, and maximum pixel values are then calculated for data flagged as “good” in both the science and error arrays. Similarly, the minimum, mean, and maximum signal-to-noise of “good” pixels is derived for the science array. These quantities are updated in the image header.
Multiple exposures of SBC MAMA data obtained using the Phase II repeated
specification are flagged in the association table for summing by acssum.
A straight pixel-to-pixel addition of the science values is applied, and the error calculated as the square root of the sum of the squares of the errors in the individual exposures. The calibration switch RPTCORR
is set to COMPLETE
upon successful completion of the summation. In addition, the keywords NCOMBINE
, and EXPEND
are adjusted to reflect the total of the summed images.