As mentioned earlier, the calacs
pipeline consists of five individual calibration tasks: acsccd
, and acssum
. These tasks are listed in Table 3.2
and diagrammed in Figure 3.1
is responsible for controlling the processing rather than the actual calibration of the data. The individual tasks within calacs
apply the appropriate calibration steps to the data and create the output products.
processes each image in the input list, one at a time, using the header keywords to determine which calibration steps to perform. Upon completion of acsccd
, the overscan regions will be trimmed from the image and an output image with the file suffix "blv_tmp.fits
" is created.
Table 3.3: acsccd
- Bad Pixel Determination
The function doDQI
initializes the Data Quality (DQ) array by combining it with a table of known permanent bad pixels for the detector, stored in the bad pixel reference table (named by the image header keyword BPIXTAB
). The type of bad pixel flags are listed in Table 3.4
For each observation, the doDQI
function combines the DQ flags from prior calibration processing (obtained from the BPIXTAB
reference table) and from saturation tests, into a single DQ result. These values are combined using a "bitwise OR operator" for each pixel. Thus, if a single pixel is affected by two DQ flags, the sum of those flag values is assigned to the corresponding pixel in the final DQ array. This array then serves as a mask so that calacs
will ignore bad pixels during processing.
- Bias Image Subtraction
removes the low order, quasi-static structure of the bias, including the bias gradient in post-SM4 images.
Subtraction of the bias image is performed prior to cosmic ray rejection using the function doBias
. The default reference image, named in the header keyword BIASFILE
, is assigned based on the DETECTOR
(for post-SM4 WFC images), CCDAMP
, and CCDGAIN
image header keyword values. The dimensions of the science image are used to distinguish observations which use the entire chip from subarray images.
The bias reference files are updated regularly (for more information, see the ACS website
). For full frame images, the BIASFILE
has the same dimensions as a full-size science image, 1062 x
1044 for HRC and 4144 x
2068 for WFC, allowing for simple image arithmetic on arrays of equal size. For WFC subarray images in the post-SM4 Cycles 17–23, aperture-specific BIASFILEs are used to account for different bias gradients, but the concept of simple image arithmetic remains.
Only after the completion of acsccd
are the images trimmed to 1024 x
1024 (HRC) and 4096 x
2048 (WFC). ACS CCD data are not binned, so any image which is not the full size is assumed to be a subarray.
For WFC subarray images in HST
Cycles other than 17–23, calacs
uses the LTV[1,2]
keywords to extract the appropriate region from the reference file and apply it to the subarray input image. For HST
Cycles 17–23, the low order bias structure in WFC subarray images differs significantly from that in full frame WFC images. Therefore, bias calibration images are available for supported subarray configurations.
For users who make their own bias files, the BIASFILE
keyword will need to be updated manually before recalibrating. This is described in detail in Section 3.5.2
Data is multiplied by gain for respective amplifiers. BUNIT
is changed from DN to electrons in SCI and ERR extensions.
– Bias level Correction
There are two branches in the doBlev
function, one for pre-SM4 data and another for post-SM4 data.
fits the bias level from the physical pre-scan1
section (defined in the OSCNTAB
reference table), and subtracts it from the image data.
A subset of the pixels along each row of the vertical pre-scan is used to obtain a linear fit of the bias level as a function of image row. If the measured pre-scan level for a given row is more than 3
σ from the mean of all rows, that row will be excluded from the linear fit. The fitted value for each pre-scan row is then subtracted from the image pixels for that row. If the pre-scan level cannot be determined, or if the pre-scan region is not present in the input image, a default value gets derived from the CCD reference table (given by the keyword CCDTAB
), with the value coming from the "CCDBIAS" column. That value will be subtracted and a warning message will be written to the image trailer file.
The positional image header keywords CRPIX1
are updated to reflect the offset due to removal of the overscan. In addition, the mean value of all overscan levels is computed and written to the SCI extension header keyword MEANBLEV
. The individual bias level measured for each amplifier is written to the SCI header keywords BIASLEVA
, and BIASLEVD
The "Overscan Region Table" reference file, named in the header keyword OSCNTAB
, describes the overscan regions to be used for measuring the bias level of the observation. Each row corresponds to a specific configuration, given by the CCD amplifier, chip, binning ratio, and dimensions. The columns TRIMX*2
give the number of columns to trim off the beginning and end of each line (the physical overscan region), while the TRIMY* columns give the number of rows to trim off the top and bottom of each column (the virtual overscan region). The result of trimming (TRIMX1 + TRIMX2) columns and (TRIMY1 + TRIMY2) rows gives the final calibrated image sizes, 4096 x
2048 for a full WFC image and 1024 x
1024 for a full HRC image.
and BIASSECTB* give the range of image columns to be used for determining the bias level in the leading and trailing regions, respectively, of the physical overscan region.
The VX* and VY* columns in the OSCNTAB
define the virtual overscan region, but are not used in calibration.
To determine which overscan regions were actually used for determining the bias level, users are encouraged to check the OSCNTAB
reference file. If desired for manual calibration, users may modify the bias section and virtual overscan region definitions in the reference table, but the TRIMX*, TRIMY* columns must not be changed.
First, a bias shift correction
is done to remove an artifact associated with the CCDs' external pre-amplifiers and the dual-slope integrators within the CCD Electronics Box Replacement. This effect is signal-dependent, causing the pixel-to-pixel bias level to be offset by 0.02–0.3%, with an offset that decays slowly in the serial (horizontal) clocking direction. Detailed information about the bias shift can be found in ACS ISR 12-02
The effects of cross-talk
, negative ghost images that are mirror-symmetric to bright sources in adjacent quadrants, are also removed. To learn more about cross-talk, please refer to ACS ISR 10-02
Corrections are simultaneously made for two other artifacts: amplifier offsets
, slight bias level offsets in each amplifier, and bias stripes
, low amplitude, horizontal striping caused by electronic 1/f
noise by the replacement electronics (see Grogin et al., ACS ISR 11-05
For post-SM4 full frame WFC images, the bias level subtraction is more complicated, as this step also removes the 1/f
bias striping noise. To overcome read noise fluctuations, all of the pre-scan pixels are used, not just those in columns 20–24 where the bias level has stabilized.
With all the pre-scan pixels normalized in this way, the next step is estimation and removal of the bias striping noise. This noise is highly consistent along rows, and is also highly consistent among the four amplifiers. For each of the 2048 rows, the 24 pixels from each of the four amplifiers' pre-scans are combined for a total of 96 estimates of the systematic 1/f
noise residual. The robust average of these 96 pixels is then subtracted from the corresponding row of all four amplifier quadrants. The striping correction has a read noise-driven uncertainty of 0.4 e–
, compared with the highly stable 0.9 e–
standard deviation of the striping noise. A more accurate removal of the striping noise may be possible outside calacs
by using the entire image region rather than only the pre-scan—please refer to the ACS Web page about de-striping
for additional information, as well as the documentation for acstools
, and see Example 5 in Section 3.5.2
- Error Array Initialization
images delivered by Generic Conversion will contain a null
error (ERR) array, defined by the header keywords NPIX1
where PIXVALUE = 0
If this ERR array has already been expanded in prior processing and contains values other than zero, doNoise
does nothing. Otherwise, calacs
initializes the array and the doNoise
function assigns a simple noise model to the ERR array.
For CCD observations: prior to SM4, there was a negligible gradient in the bias, therefore a constant value was used for the bias
variable in the noise equation. However, for post-SM4 processing, the bias
variable now also incorporates the bias gradient as given in the corresponding bias calibration image.
Because MAMA data is not processed by acsccd
, the doNoise
function is later called, just for processing SBC data, as part of acs2d
The "CCD Characteristics Table" reference file, recorded in the image header keyword CCDTAB
, is used to determine the bias (for certain subarrays only), gain, and read noise values for an observation, and for use in calculating error values (σCCD
) for the ERR array. The table columns are
reference table uses these commanded values to determine an observation's physical readout characteristics that are written to the flt.fits
calibration file. For instance, read noise values for each amplifier are populated in header keywords READNSEA
, and READNSED
; A-to-D gain in ATODGNA
, and ATODGND
; bias voltage offset values in CCDOFSTA
, and CCDOFSTD
flags sink pixels and the adjacent affected pixels with the value 1024 in the DQ array of WFC images. See Section 4.3.3
for a description of sink pixels. One sink pixel reference image is created for each anneal cycle after January 2015. This reference image encodes the locations of the sink pixels, trailing low pixels, and excessively high downstream pixels, as well as information relevant to flagging the correct number of trailing pixels based on the background level in a science image (see ACS ISR 2017-01
for more details). The reference image in the header keyword SNKCFILE
is assigned based on the DETECTOR keyword and the observation date of the science image. All WFC images taken after January 2015 will have doSink
Upon completion of acsccd
, the overscan regions will be trimmed from the image when it is written out, but only if doBlev is performed successfully
. Otherwise, the full image array will be written out.
3.4.2 acscte –
Pixel-Based CTE Corrections
|Header keyword added or updated: CTE_VER, CTE_NAME, CTEDATE0, CTEDATE1, PCTENFOR, PCTENPAR, PCTETLEN, FIXROCR, PCTEFRAC, PCTERNOI, PCTETRSH, PCTENSMD
For all WFC exposures, except bias images, the pixel-based CTE correction is applied between the acsccd
steps, after BLEVCORR
. Parameters characterizing the CTE correction are stored in a reference table, PCTETAB
, and also in SCI image headers. As long as PCTECORR
is set to PERFORM
and the correct reference file or parameters are provided, CTE loss correction is also available for a WFC 2K subarray of the old format or any WFC subarray of the new format. However, due to uncorrected striping in all WFC subarrays, PCTECORR
is set to OMIT
by default in the pipeline for all subarrays. To perform CTE loss correction for an eligible WFC subarray, please set PCTECORR
in the image header and use the acs_destripe_plus
task in acstools
to de-stripe the data. Then CTE correction will be performed. Upon completion, a CTE-corrected version of "blv_tmp.fits
" (created in acsccd
), named "blc_tmp.fits
," is created. Both files are passed to acs2d
for further processing. (Please see Example 5 in Section 3.5.2
for a worked example.)
, the cosmic ray rejection task in calacs
, combines "CR-SPLIT" exposures and repeated sub-exposures in an exposure, into a single image, free of cosmic rays. The task uses the same statistical detection algorithm developed for STIS data (ocrrej
) and WFPC2 data (crrej
), providing a well-tested and robust procedure. The parameters used by acsrej
are obtained from the "Cosmic Ray Rejection Table"
(named in header keyword CRREJTAB
), and depend on the type of chip, number of "CR-SPLIT" or repeated sub-exposure images, and exposure time of each image.
will compute the sky background using the mode of each image. Sky subtraction is performed before any statistical checks are made for cosmic rays. Next, acsrej
constructs an initial comparison image from each input exposure. The comparison image can either be a median- or minimum-value sky-subtracted image constructed from all the input images, and represents the "first guess" of a cosmic ray-free image. This comparison image serves as the basis for determining the statistical deviation of each pixel from the input image.
is the read noise squared (in DN) and gain
is the CCD gain (in
/DN) for the amplifier used to read the pixel,
is the scale factor for the noise model,
is the pixel value (in DN) from the median- or minimum- combined comparison image.
is the sky background (in DN) of image n
is the total exposure time (regardless of whether all input images were used for that particular pixel). This corresponds to the header keyword values TEXPTIME
The following crj.fits
image keywords are derived from the variables in this computation:
The remaining keywords EXPSTART
are updated based on the input image headers.
uses the "Cosmic Ray Rejection Parameter Table" (header keyword CRREJTAB
) to determine the number of iterations for cosmic ray rejection, the sigma levels to use for each iteration, and the spill radius to use during detection. This allows the rejection process to be tuned to each detector, with suitable defaults being applied during pipeline processing. Observers may fine-tune the cosmic ray rejection parameters when manually reprocessing data with acsrej
by editing the CRREJTAB
reference file contains the basic parameters necessary for performing cosmic ray rejection. The column names and default values for the CRREJTAB
are given in Table 3.5
. The appropriate row is selected based on the chip being processed (CCDCHIP
), the number of images into which the exposure was split ("CR-SPLIT" Phase II optional parameter), and the exposure time of each "CR-SPLIT" image (MEANEXP
). The sky fitting algorithm is controlled by the parameter SKYSUB
which can have values of "mode
" or "none
". The "first guess" CR-combined image is then created using the median or minimum value of the input exposures, as specified by the table column INITGUES.
Cosmic ray detection requires the specification of a threshold above which a pixel value is considered a cosmic ray. This threshold, defined earlier as , uses the sigma rejection thresholds that correspond to the column CRSIGMAS in the CRREJTAB
reference file. The table column SCALENSE is a multiplicative term (in percent) for the noise model and is given as scale
in the threshold equation. This term can be useful when the pointing of the telescope has changed by a small fraction of a pixel between images. Under such circumstances, the undersampling of the image by the CCD will cause stars to be rejected as cosmic rays if a scale noise term is not included. This is a crude but effective step taken to satisfy the maxim of "first do no harm." However, for cases in which there have been no frame-to-frame offsets or if the image is locally well-sampled, this will unduly bias against
rejecting cosmic rays.
The pipeline adopts a conservative value for SCALENSE to avoid doing harm. In recalibrating several frames for a new cosmic ray elimination, it would be advisable to determine the full range of relative x,y
offsets. An appropriate value of SCALENSE is 100*(maximum offset in pixels), thus if the full offset range was 0.1 pixels an appropriate SCALENSE is 10.0. To alter this value, one may edit the CRREJTAB
table (calibration file with the extension crr.fits
). The number of exposures obtained in a repeated observation set can be larger than the maximum "CR-SPLIT=8" allowed in Phase II proposals.
|Create a new asn.fits
table with one EXP-CRJ entry for each of the 14 raw 1.0s images, and an appropriately named PROD-CRJ file line.
|Edit the crr.fits
table adding a line with a "CR-SPLIT" value of 14, and a SCALENSE value of 20.0.
|Then rerun calacs
on the new association table with 14 entries to obtain a crj.fits
extension image based on the full stack of 1.0s images.
Every observation, whether taken with the MAMA or CCD detectors, will be processed by acs2d
. The primary functions of the task, listed in Table 3.6,
include but not limited to dark current subtraction, flat-fielding, and photometric keyword calculations. acs2d
contains the same data quality and error array initialization functions used in acsccd
, but acs2d
will check to ensure that the array initialization is not performed twice on CCD data. Calibration switches in the image header control the performance of the remaining calibration functions, with MAMA-specific functions being initiated only when the relevant calibration switches are set.
– Bad Pixel Determination
If the DQICORR
header keyword switch is set to COMPLETE
(e.g., for CCD data), this step is skipped. Otherwise, the same initialization is performed as described in "doDQI – Bad Pixel Determination"
– Linearity Correction for MAMA Data
attempts to correct for non-linearity up to the global limit using the non-linearity time constant in the column TAU. The global linearity correction is computed for every pixel below the global linearity limit specified by iteratively solving the equation
to get the true count rate N.
If the LFLGCORR
switch is set to PERFORM
excessive global and local nonlinearity in the DQ array. If GLINCORR
is set to PERFORM
, it will correct
excessive global nonlinearity in the SCI array, if it is not too large. If the global linearity limit is exceeded, the keyword GLOBLIM
in the SCI extension header will be set to EXCEEDED
. Otherwise, it will have the value NOT-EXCEEDED
- Error Array Initialization
checks the image error array (ERR) to determine if there are non-zero values that were created from previous processing steps. If all pixel values are zero, indicating no prior calibration steps were performed, acs2d
runs the same initializations described in "doNoise – Error Array Initialization"
. However, if the input image's ERR array has non-zero values, indicating processing in earlier calibration steps, the doNoise
function in acs2d
does not change the ERR array.
– Dark Image Subtraction
For the SBC, the header switch DARKCORR
is set to OMIT
because the SBC dark count rate is so negligible that corrections are not needed, even for long exposures.
For CCD data, the dark reference file image (in electrons/sec
) is multiplied by the total of the image's exposure time, flash time (if post-flashed), and a 3-second idle time (post-SM4 non-BIAS WFC images only). This scaled dark reference image is subtracted from the input image in memory. The mean dark value is then computed from the scaled dark image and used to update the MEANDARK
keyword in the SCI image header. Any dark accumulation during readout time is automatically included in the bias image reference file (BIASFILE
), and already removed during the doBias
step in acsccd
A dark reference file is regularly created4
from the combination of 1000 sec. dark frames taken four times per day during several days per week. These individual dark frames are bias-corrected and combined with cosmic ray rejection. Since 2015, dark frames have been flashed to better remove CTE loss effects on warm and hot pixels, thus these darks are also flash corrected using a flash reference file.
Only hot pixels above a certain threshold (this number changes with time and is currently set at 0.08 e–
/sec) are identified with the flag 16 in the data quality (DQ) array of WFC and HRC reference darks; this flag value is propagated to the DQ array of the calibrated science data. The flag 32 now marks pixels that are unstable. We now only exclude the pixels that are unstable in the subsequent processing steps since stable hot pixels, while undesirable, can be calibrated. (See ACS ISR 2017-05
by Borncamp et al. for more on unstable pixels, the reprocessing of older darks to reflect the new flag 32 scheme, and modifications to the MDRIZTAB
associated with these changes for not only darks but also science files.)
The many "warm" pixels below this threshold are assumed to be adequately corrected by the dark calibration. This produces a high signal-to-noise reference file which accurately reflects (and corrects/flags) the hot pixels present for a given observation date. The "best" dark reference file is typically not available in the pipeline for several weeks after the date of observation, because it takes a few weeks to collect enough frames to make a basedark (see ACS ISR 2004-07
for more information).
The default reference files for dark subtraction, given in the header keywords DARKFILE
, are assigned based on the values of the keywords DETECTOR
, and EXPSTART
in the image header. The dark correction is applied after the overscan regions are trimmed from the input science image. As in the bias image correction (BIASFILE
assumes that the images have not been binned, so any input image smaller than the full detector size will be interpreted as a subarray image. The reference file named by the DRKCFILE
header keyword is the CTE-corrected version of the dark reference file named by the DARKFILE
keyword, and it's only used in WFC image calibrations when pixel-based CTE correction is performed.
The reference file, named in the header keyword FLSHFILE
, has the same dimensions as a full size WFC science image, 4144 x
2068, allowing for simple image arithmetic on arrays of equal size. The appropriate FLSHFILE
reference image is selected using the following keywords from the image header: DETECTOR
, and SHUTRPOS
The function doFlash
will subtract the post-flash in the following way:
reference image, which is normalized to one second for the appropriate post-flash current level (LOW
, given by the FLASHCUR
keyword), is multiplied by the flash duration in seconds (given in the header keyword FLASHDUR
) and subtracted from the science image.
– Flat-Field Image(s) Correction
The ACS doFlat
routine corrects for pixel-to-pixel and large-scale sensitivity gradients across the detector by dividing the data with the flat-field image. (Conversion to electrons no longer depends on doFlat
, as it did in the past.)
Geometric distortion corrections are also implemented in the pipeline by AstroDrizzle
which uses the geometric distortion solution to correct all pixels to equal areas. Thus, in drizzled images, photometry is correct for both point and extended sources.
is a pixel-to-pixel flat-field correction file containing the small scale flat-field variations.
is a low-order flat which will correct for any large-scale flat-field variations across each detector. This file is stored as a binned image which is expanded when it's applied by calacs.
is a delta-flat containing any needed changes to the small-scale PFLTFILE
is a spot mask which contains the vignetting patterns of the HRC occulting spots and is applied to coronagraphic observations only.
If the LFLTFILE
, or CFLTFILE
reference images are not specified in the SCI image header, only the PFLTFILE
is used for the flat-field correction. If all four reference files are specified, they are multiplied together to form a complete flat-field correction image which is then applied to the science data.
Currently, the LFLTFILE
flats are not used for ACS data. The PFLTFILE
reference flat in the pipeline is actually a combination of the pixel-to-pixel flats taken during the ground calibration and the low-order flat correction derived in-flight. The CFLTFILE
is applied only when the OBSTYPE
image header keyword is equal to CORONAGRAPHIC
– Shutter Shading File Correction (skipped)
routine would apply the shutter shading correction image (named in image header keyword SHADFILE
) to the science data if the SHADCORR
header keyword switch were set to PERFORM
. However, the SHADCORR
calibration switch is currently set to OMIT
and is unlikely to be used in future versions of calacs
. Calibration data show that a shading correction is not needed for ACS data. For more information, refer to ACS ISR 2003-03
Photometry Keyword Calculation
Before photometry can be performed on ACS observations, a transformation from electrons to absolute flux units must be performed. For a given instrument configuration, as described in the PHOTMODE
header keyword, calacs
uses the IMPHTTAB5
reference look-up table to determine the total throughput of an observing mode (linear interpolation is done using pre-computed values when necessary).
Users who wish to convert calibrated images (in units of electrons) to flux units may simply divide the image by the exposure time and then multiply by the PHOTFLAM
keyword value. Drizzled images are already in units of electrons per second and may simply be multiplied by the PHOTFLAM
value in the drizzled image header to create an image in flux units.
– Image Statistics Determination
Multiple exposures of SBC MAMA data obtained using the Phase II repeated sub-exposures specification are flagged in the association table for summing by acssum.
A straight pixel-to-pixel addition of the science values is applied, and the error calculated as the square root of the sum of the squares of the errors in the individual exposures. The calibration switch RPTCORR
is set to COMPLETE
upon successful completion of the summation. In addition, the keywords NCOMBINE
, and EXPEND
are adjusted to reflect the total of the summed images.