|Space Telescope Science Institute|
|ACS Data Handbook 2011|
- raw (raw.fits) files from Generic Conversion.
• A single fully-calibrated MAMA image is given the suffix flt.fits. (MAMA images do not have an overscan region, and they are not affected by cosmic rays.)
- after the bias level and bias image are subtracted, and the overscan regions are trimmed, a temporary file, with the suffix blv_tmp.fits, is created.
- Upon completion of additional calibrations (flat fielding, dark subtraction, etc.), the temporary file is renamed with the suffix flt.fits. The flt.fits files will later serve as input for multidrizzle.
- raw images from a “CR-SPLIT” observation undergo bias level and bias image subtraction. They’re then combined with cosmic ray rejection to create a temporary image with suffix crj_tmp.fits.
- Other basic calibrations are performed on the temporary combined image; upon completion, it’s renamed with the crj.fits suffix.
- Individual calibrated images (flt.fits) are also created for each exposure in the “CR-SPLIT” observation.
- Follows the same procedure described for “CR-SPLIT” CCD observations.
- calibrated flt.fits images are created for each sub-exposure.
- A summed flat-fielded image is created, with suffix sfl.fits. (MAMA images are not affected by cosmic rays.)Table 2.1: ACS File Suffixes
Trailer file containing calacs processing comments. Overscan-trimmed individual exposure (renamed to flt.fits after all basic calibrations are completed). CR-rejected combined image created using blv_tmp.fits images (renamed to crj.fits after all basic calibrations are completed). Calibrated and combined image, with CR rejection (using images from sub-exposures in a “CR-SPLIT” observation) Calibrated and summed MAMA image (no CR rejection needed) created from sub-exposures in an observation. Calibrated, geometrically-corrected, dither-combined image (created by multidrizzle, which is not a part of calacs)
Intermediate calibrated products created by calacs—sfl.fits, blv_tmp.fits, crj_tmp.fits, and crj.fits—are, by default, not delivered by the Archive. A standard calibrated files delivery contains these extensions: asn.fits, spt.fits, trl.fits, flt.fits, and drz.fits. If you want intermediate calacs products, there is a field in the Archive data request form that allows you to enter specific extensions for retrieval (e.g., crj, crj_tmp).
2.1.2 Association tables describe and track the relationship or “associations” between data products for an observation(s). Such relationships include repeated exposures in observations, “CR-SPLIT” observations, and dithered observations. ACS association tables can be used to instruct calacs to create different levels of calibration products. These tables are particularly useful for keeping track of complex observations, for example, an observation at a specific dither position that may be additionally split into multiple exposures. Edited association tables can also be used with calacs to create non-default calibration products (see Example 3 in Section 3.5).
• a single image from an exposure or sub-exposure is the “atomic unit” of HST data.
• A dataset is a collection of files having a common rootname (first nine characters of the image name).
• A sub-product is created by combining a subset of the exposures in an association.
• A product is created by combining sub-products, or in some cases, individual exposures (before they’re incorporated into a sub-product), of an association.An ACS association table has three primary columns: MEMNAME, MEMTYPE, and MEMPRSNT.
• The column MEMNAME lists the name of each exposure in the association, and names of calacs output products.
• The MEMPRSNT column indicates the calibration status of each product.
An image from a “CR-SPLIT” observation. (“EXP-CRJ” is used when there is only one “CR-SPLIT” observation in an association) Same as “EXP-CRJ,” but used when there are multiple “CR-SPLIT” observations in an association. n is the numerical ID of a “CR-SPLIT” set (e.g., “EXP-CR1,” “EXP-CR2”). A calibrated and CR-rejected combined image created from a “CR-SPLIT” observation. (“PROD-CRJ” is used when there is only one “CR-SPLIT” observation in an association.) Same as “PROD-CRJ” but used when there are multiple “CR-SPLIT” observations in an association. n is the numerical ID of a “CR-SPLIT” set (e.g., “PROD-CR1,” “PROD-CR2”). An image from an observation containing a series of repeated sub-exposures. (“EXP-RPT” is used when there is only one such observation in the association.) Same as “EXP-RPT,” but used when there are multiple observations, each containing a series of repeated sub-exposures, in an association. n is the numerical ID of each such set (e.g., “EXP-RP1,” “EXP-RP2”).. A calibrated summed MAMA image, created from an observation containing a series of repeated sub-exposures,. (“PROD-RPT” is used when there is only one such observation in the association.). Same as “PROD-RPT,” but used when there are multiple observations, each containing a series of repeated exposures, in an association. n is the numerical ID of each such observation (e.g., “PROD-RPT1,” “PROD-RPT-2”). An example of an association table is shown in Table 2.3. But first, to trace back its origins, we start by looking at the Phase 2 proposal commands that created the data. This example came from proposal 10605, visit 1, exposure logsheet number 1. The observations were in the form of a 2-point dither with a “CR-SPLIT=2” at each dither point. The proposal’s dither pattern specification looked like this:
-------------------------------------------------------------------------------------------------------------Exp | Target | Instr | Oper. | Aper | Spectral | Central | Optional |Num | Time | SpecialNum | Name | Config | Mode | or FOV | Element | Waveln. | Parameters |Exp | | Requirements-------------------------------------------------------------------------------------------------------------1 MESSIER-081 ACS/WFC ACCUM WFC F555W CR-SPLIT=2 1 2900 S PATTERN 1 1-2-------------------------------------------------------------------------------------------------------------A standard calibrated data retrieval from the Archive for images taken by Visit 1, exposure 1, are listed below. (Intermediate calacs products have to be specifically requested, therefore, the crj.fits files are not included in the delivery.)
• The combined image created by multidrizzle from four flt.fits images:Table 2.3: Contents of Association Table, j9cm01010_asn.fits
• Rows 1 and 2 show values associated with the first “CR-SPLIT” set in the association. (This “CR-SPLIT” set was obtained at the first dither point.)
- the “MEMNAME” column shows the image rootnames for sub-exposures in the first “CR-SPLIT set: J9CM01JVQ and J9CM01JWQ.
- “MEMTYPE” value “EXP-CR1” indicates that the images correspond to sub-exposures from the first “CR-SPLIT” set in the association.
- “MEMPRSNT” set to “yes” indicates that the images underwent standard calibrations (excluding geometric distortion corrections).
• Rows 3 and 4 show the same parameters as described in (1) for the second “CR-SPLIT” set in the association. (This “CR-SPLIT” set was taken at dither point 2.)
• Row 5 has the “MEMTYPE” value of “PROD-DTH.” This is a drizzle-combined image product of the four flt.fits images, which has the rootname J9CM01010. (If “MEMPRSNT” was set to “no” in line 5, the pipeline would not have created the multidrizzled image.) The multidrizzled image is corrected for geometric distortion.
• In Row 6, the “MEMTYPE” value “PROD-CR1” indicates that the image j9cm01011_crj.fits was created by combining, with cosmic ray rejection, the two images from the first “CR-SPLIT” pair in the association. Note that crj.fits images are not corrected for geometric distortion.
• Similarly, in Row 7, the image j9cm01012_crj.fits was created using the images that came out of the second “CR-SPLIT” set in the association.
2.1.3 Each task in the calacs package creates messages which describe the progress of the calibration; these messages are directed to STDOUT (STanDard OUTput), which simply means that processing messages appear on the screen during the calacs run.In pipeline processing for first and second generation HST instruments, where data files were calibrated one at a time, trailer files were created by simply redirecting the STDOUT contents to a file. However, the ACS pipeline was designed to calibrate several images at a time, such as those from a single dither pattern (as described in Section 2.1.2), and create different types of output files. Therefore, each task within the calacs package must decide how to populate the trailer files associated with each product.calacs will always overwrite information in existing trailer files from previous runs of calacs while preserving any comments generated by Generic Conversion. This ensures that the trailer files accurately reflect the most recent processing performed by calacs. After the generic conversion entries, the string “CALACSBEG” marks the first comment added to a trailer file by calacs. If the trailer file already exists, calacs searches for this string, then begins to write new processing comments from that point onwards, over-writing previous calacs comments. If “CALACSBEG” is not found in an existing trailer file, calacs will write that string at the end of the trailer file, then continue populating the trailer file with calibration processing comments.As each image is reprocessed, an accompanying trailer text file with the suffix “tra” (without the .fits extension) is created. (The trl.fits file from the Archive has the same content; it can be read using the tdump command, e.g., “tdump jb5u34010_trl.fits”.)Following the processing hierarchy specified in the association table, information in trailer files belonging to images used for creating a higher level product will be included in the trailer file of that higher level product. In other words, the trailer file for any product processed by the pipeline will contain processing comments from trailers belonging to each input file.Linking trailer files together can result in multiple occurrences of the CALACSBEG string. Only the first, however, determines where calacs will begin overwriting comments if an observation is reprocessed.
An “observation” set is a group of exposures under the umbrella of one unique ID. Each set has the same target, instrument configuration, operating mode, aperture, and spectral elements. An observation set created by the pipeline usually contains a set of dithered, repeated, and/or “CR-SPLIT” exposures taken within a visit. The data products in each calibrated observation set is described in its association table.