Introduction
Updates made to the ACS calibration pipeline, CALACS, are documented here. Three times a year, updates to the software packages in the HST calibration pipelines (HSTCAL) are released in an HST data processing (HSTDP) build (e.g. HSTDP 2018.1). After each build is released, it is incorporated into the AstroConda channel and made available to the science community.
Latest Updates
Current Version Information
ACSTOOLS version: 3.2.0
HSTCAL version: 2.3.1
CALACS version: 10.2.1
Build Information
-
CALACS updates to Version 10.1.0 due to updates in the ACSCCD and ACS2D algorithms.
-
The dark correction, DARKCORR step of the ACS2D algorithm, has been modified to work directly on the full 2D data rather than on a line-by-line basis.
-
The bias shift correction, BLEVCORR step of the ACSCCD algorithm, has been upgraded to accommodate the new supported subarrays which have both physical and virtual overscan. This is the same algorithm which applies to full frame data.
-
Forward model functionality was implemented which exploits the existing CTE code and usage (Generation 2 only). Instead of correcting the CTE trails, the forward model functionality simulates (adds) the trails. This functionality should be accessed via the new ACSTOOLS function, acscteforwardmodel.py
-
Updated ACS library routine, getacskeys, updated to call the new HSTIO function which robustly determines the number of IMSETS in a FITS file.
-
CALACS updates to Version 10.0.0 due to the modifications imposed by the upgraded ACSREJ algorithm.
-
An updated ACSREJ algorithm was implemented where all calculations are now done in electrons versus DN. The threshold computation uses the ERR arrays to account for post-flash noise. The C interface to the ACSREJ standalone component of CALACS was modified for clarity. Specifically, the "newbias" option was renamed "readnoise_only". The functionality associated with this option remains the same. For more information see ACS ISR 2018-05.
-
CALACS now uses the amp-dependent read noise as the threshold to determine if a pixel has enough signal-to-noise ratio such that a reasonable CTE correction can be applied.
-
A bug fix has been implemented which is only applicable for RAWs with PCTECORR=PERFORM for ASNs used for cosmic ray rejection. In this case the CR rejected output (CRJ) previously used the wrong dark; it used the CTE-corrected dark. This has been addressed, and the CRJ product is now correct utilizing the non-CTE corrected dark.
-
All BIAS files are now processed by CALACS to use only read noise (vs Poisson) for the computation of the ERR array as the date discriminant (pre- and post-SM4) has been removed.
-
CALACS updates to version 9.2.0 with this release.
-
New BLEVCORR algorithm that uses new OSCNTAB, which now correctly calculates bias level for AMP B and D in subarrays.
-
New SINKCORR step to apply new SNKCFILE in order to flag sink pixels. For information about the SINKCORR step see ACS ISR 2017-01.
-
New pixel based PCTECORR algorithm implemented (
CTE_NAME= 'PixelCTE 2017').- The code defaults to this new algorithm, however, the old one can still be executed using the command line option
--ctegen <1|2>. Note: The correct accompanying PCTETAB file must also be used. - The user can now override the PCTETAB file used (normally specified in the image header) with the command line option
--pctetab <file path>. - The number of threads used for computing the PCTECORR can now be specified with the command line option
--nthreads <N>. Note: OPENMP is required for N > 1.
- The code defaults to this new algorithm, however, the old one can still be executed using the command line option
-
Fixed uninitialized variable in
prinfo.c. -
Corrected
--versionreporting and added-rto report version string that was previously reported by--version.
Overview of CALACS
CALACS, is divided up into five main tasks that are run sequentially. In order, these five tasks are: acsccd, acscte, acsrej, acs2d, acssum. The exact path an image takes through the pipeline is controlled by various switches stored in the image's primary header. In the boxes below, we describe each of the processing steps applied in the five main tasks and the relevant header switches that control them. For a complete description of each of these tasks, we refer users to the ACS Data Handbook.
This task contains the initial processing functions for all ACS CCD data.
| Function Name | Header Switch | Action Applied |
|---|---|---|
| doDQI |
DQICORR |
Initializes the data quality (DQ) array by combining it with a table of known permenant bad pixels marked in the bad pixel table reference file. |
|
doBias |
BIASCORR |
Subtracts the superbias reference file to remove the low order, quasi-static structure of the bias, including the bias gradient in post-SM4 images |
| toElectrons | NONE, applied to all CCD data | Mulitplies the image data by the gain to convert from DN to electrons. |
| doBlev |
BLEVCORR |
For observations taken before Servicing Mission 4 (SM4), computes and subtracts the bias level from the image data. For observations taken after SM4, corrects for the signal-dependent bias shift, cross-talk, and bias-striping. |
| doNoise | NONE, applied to all CCD data | Initializes the error array. |
| doSink |
SINKCORR |
Flags sink pixels and the adjacent affected pixels with the value 1024 in the DQ array of ACS/WFC images. |
This task applies to pixel-based CTE correction to the ACS/WFC data.
| Function Name | Header Switch | Action Applied |
|---|---|---|
| N/A |
PCTECORR |
Applies the pixel-based CTE correction software to generate CTE-corrected products (e.g. FLC, CRC, or DRC) |
For CR-SPLIT observations, this task will combine the individual observations and perform cosmic ray rejection.
| Function Name | Header Switch | Action Applied |
|---|---|---|
| N/A |
CRCORR |
Runs the cosmic ray rejection routine to generate a single CRJ or CRC from a list of FLTs or FLCs. |
This task is run on all ACS observations. A number of the header switches are detector-dependent.
| Function Name | Header Switch | Action Applied |
|---|---|---|
| doDQI |
DQICORR |
If the DQ array has not already been initialized by acsccd, initializes the DQ array using information from the bad pixel table reference file. |
| doNonLin |
LFLGCORR, GLINCORR |
Corrects and flags non-linear data in observations taken with the SBC MAMA |
| toElectrons | None | If the units of CCD data are not already electrons, multiplies by the gain to convert them. |
| doNoise | None | Checks the error extension of the image to determine if there are non-zero values generated in previous steps. If the values are all zero, it computes the error using the applicable noise model. |
| doDark |
DARKCORR |
Performs the dark subtraction. |
| doFlash |
FLSHCORR |
Performs the flash subtraction. |
| doFlat |
FLATCORR |
Performs the flat-field division. |
| doShad |
SHADCORR |
Performs the CCD shutter shading correction. Currently skipped as calibration data show that a shading correction is not needed for ACS. |
| doPhot |
PHOTCORR |
Computes the photometric keywords values and populates them in the header. |
| doStat | None | Computes basic image statistics. |
This task will combine repeated sub-exposures taken with the SBC MAMA
| Function Name | Header Switch | Action Applied |
|---|---|---|
| N/A | RPTCORR | Combines repeated sub-exposures using a straight pixel-to-pixel addition of the science values. Propagates uncertainties by summing in quadrature. |
ACSTOOLS
To facilitate scripting of calibration workflows, the ACS Team developed the python package, ACSTOOLS. This package is a wrapper around the compiled C executables used by the CALACS pipeline. It provides users with an easy-to-use python interface, as well as, access to other utility functions developed by the ACS Team (e.g. the CTE forward modeler). The interface simplifies the process of building up calibration workflows, making it much easier for users to reprocess their data if they so choose. For more information on ACSTOOLS, please see the website.
Example Usage
Suppose you wanted to reprocess CR-SPLIT observations while experimenting with the parameters in CRREJTAB. Accomplishing this using the executables is a drawn out process that involves iteratively modifying the parameters stored in the CRREJTAB reference file. A simpler way to do this is by leveraging the python interface provided with ACSTOOLS. In the section above on CALACS, we see that only two tasks are executed before acsrej. We write a simple python script to process our observations through acsccd and acscte to generate intermediate data products with this minimum level of required calibration completed.
from acstools import acsccd, acscte
# Run the acsccd task (*raw.fits -> *blv_tmp.fits)
acsccd.acsccd('*raw.fits', time_stamps=True, verbose=True)
# Run CTE correction (*blv_tmp.fits -> *blc_tmp.fits)
acscte.acscte('*blv_tmp.fits')
Now that we have reached the level of calibration expected before entering the acsrej task, we can experiment with the parameters of CRREJTAB much more easily. The values stored in the CRREJTAB reference file (see Section 2 in ACS ISR 2018-05) can be accessed through keyword arguments in the ACSTOOLS implementation. Instead of updating the reference file, change the values passed in the function call:
from acstools import acsrej
acsrej.acsrej('*blc_tmp.fits',
output='combined_product.fits', # output filename
crsigmas='6,5,4', # specify the rejection threshold
crradius=0.75, # specify the propagation radius
initgues='med') # specify the method for computing the initial guess