|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
 |
 |
|
|
| Part II: ACS Data Handbook | |||||
| ||||||
4.5 Post-Pipeline Reprocessing
The goal of the ACS pipeline is to provide data calibrated to a level suitable for initial evaluation and analysis for all users. Observers frequently require a detailed understanding of the calibrations applied to their data and the ability to repeat, often with improved products, the calibration process at their home institution. There are several occasions when off-line interactive processing with MultiDrizzle may be desired:
- Observations which have been manually reprocessed with CALACS will still require a correction for geometric distortion.
- Users may wish to modify the default MultiDrizzle parameters, specifying an alternate output image orientation, pixel "shrinking factor" pixfrac, or pixel scale.
- When images have been taken across multiple visits, the observations are not automatically associated. Creating custom association tables will allow data from multiple visits to be combined. See Section 4.7 for details.
- For optimal image registration and refining the CR rejection, the user may wish to fine-tune the WCS information by supplying additional shifts and/or rotations.
- During pipeline processing, MultiDrizzle makes assumptions about which DQ flags should be considered bad. These pixels will be excluded when creating the drizzled product. Users may wish to select which flagged pixels should actually be considered good and included in the drizzled product.
- If cosmic ray rejection parameters need to be fine-tuned for specific datasets, for which the pipeline defaults may not have produced optimal results.
The MultiDrizzle software uses the Python language and can only be executed in the PyRAF environment. PyRAF, developed at STScI, allows users to run tasks using the standard IRAF parameter-based interface or by using separate Python programs. The PyRAF interface and the most recent version of MultiDrizzle can be downloaded from:
http://www.stsci.edu/resources/software_hardware/stsdas/pydrizzle.MultiDrizzle creates many large files and does a considerable amount of processing. It is hence recommended that a modern fast machine with several GBytes of free space be used if possible. In the example to follow, each of the input FLT files is ~168MB and the final combined output product is 310MB. Many large intermediate files are also created but may be cleaned up if required.
The main calibration reference files required by MultiDrizzle are the IDCTAB and DGEOFILE files discussed in Section 4.1.4. The IDCTAB file describes the ACS distortion in terms of polynomial coefficients. The appropriate reference file is recorded in the image header keyword IDCTAB. This file should be obtained from the STScI archive (via Starview) and placed in some local directory. Before starting MultiDrizzle the environment variable "jref" must be set appropriately to point to this directory (see Section 3.5 for more information). Once the FLT files have been collected and the 'jref' environment established, PyRAF should be started and the stsdas, analysis, and dither packages loaded.
The calibrated products from CALACS and the corresponding input to MultiDrizzle are described for various observing modes in Table 3.1. For example, a single image will produce an FLT file, a RPT-OBS exposure will produce an SFL file, a CR-SPLIT exposure will produce a CRJ file, and a DITH-PATTERN will produce a series of FLT, CRJ, or SFL files at each dither position. MultiDrizzle may be executed using any of the calibrated data products or the association table itself, and creates a single drizzled image with the DRZ suffix. In the case of dithered observations which are part of a pattern, the individual FLT files will be combined to create a single DRZ image.
An important variable to set up prior to running MultiDrizzle is the "jref" path, since this specifies the directory where MultiDrizzle searches for the IDCTAB and other distortion-related files. This is described in Section 3.5 and should be set up before starting PyRAF, by typing:
setenv jref /mydisk/jref/After defining the "jref" environment and starting PyRAF, the stsdas, analysis, and dither packages should be loaded.
To avoid potential problems, it is also recommended that the parameters for both MultiDrizzle and PyDrizzle be reset to their default values until users are sufficiently familiar with them:
The examples in this section describe the steps required for reprocessing after retrieving pipeline products, and serve as a continuation of the CALACS examples described in Section 3.5.2. We advise the reader to work through the CALACS examples first to develop a solid understanding of the calibrated data products and their potential limitations.
4.5.1 Example 1: Running PyDrizzle on a Single Exposure
The following is a continuation of Example 1 from Section 3.5.2 where manual recalibration of a single raw exposure with CALACS produced a single image with the FLT suffix. Here, we run PyDrizzle on the calibrated FLT image (j8bt07oyq_flt.fits) to illustrate the geometric distortion. By default, the drizzled output product will be named j8bt07oyq_drz.fits. To specify an output file which is different than the default, the user may set the 'outfile' parameter to 'myfile_drz.fits', for example. Section 4.4.3 defined the 'bits' parameter; here we adopt the recommended value of 96. The reader may also wish to add 8 to this value to see the effect of not dropping pixels behind the occulting finger. Prior to running, we recommended clearing any preset values using the 'unlearn' command.
The distortion corrected HRC science image is shown in Figure 4.7. The image on the left is the flat fielded FLT product from CALACS and still contains distortion. The image on the right is the calibrated, distortion-corrected DRZ product.
Figure 4.7: The effect of applying the distortion correction to a single HRC image. The image on the left is the flat-fielded FLT product from CALACS. The image on the right is the distortion-corrected DRZ product.
4.5.2 Example 2: Running PyDrizzle on an Association file
The following is a continuation of Example 5 from Section 3.5.2 where manual recalibration of a 2-point line dither pattern produced two FLT images (j8e654c0q_flt.fits, j8e654c4q_flt.fits).
Using tprint, we can view the contents of the image association, including the rootnames of the individual dithered images and the default rootname of the drizzled product.
Next we run PyDrizzle on the image association. The two FLT images are combined to create a single DRZ image called 'j8e654011_drz.fits'.
Note that PyDrizzle is not designed to remove cosmic rays from these images.
4.5.3 Example 3: Running MultiDrizzle with Different Output Image Options
The simple examples in the preceding Section 4.5.1 and Section 4.5.2 demonstrate the use of PyDrizzle as an easy interface to the Drizzle program. However, in general we recommend the use of MultiDrizzle which includes more functionality, for example cosmic ray rejection, while at the same time combining an easy user interface together with substantial flexibility through access to a wide range of parameters.
The default MultiDrizzle parameters within the pipeline produce output images with a scale equal to unity, i.e. with output pixels equal to the pixel scale of whichever camera was used to obtain the images (e.g., ACS/WFC, HRC or SBC). In addition, the pixfrac parameter is set to 1.0 for associations with less than 6 images, and 0.8 for 6 or more images. Finally, the images are generally oriented with North to the top, except for polarizer images which are left unrotated for easier ingest by polarization analysis software. An estimate of the sky background will be determined by the software and written to the keyword MDRIZSKY.
While these parameters are satisfactory for a wide range of scientific applications, there may be occasions where the user wishes to re-run MultiDrizzle offline to repeat the final drizzle combination step using a different output image size or scale, pixfrac or orientation. This firstly involves turning off the intermediate steps for MultiDrizzle (except for sky subtraction), which can be done either within the epar interface:
or on the PyRAF command line:
pyraf> iraf.multidrizzle.static = no
pyraf> iraf.multidrizzle.skysub = yes
pyraf> iraf.multidrizzle.driz_separate = no
pyraf> iraf.multidrizzle.median = no
pyraf> iraf.multidrizzle.blot = no
In this particular example, most of the steps are turned off since the cosmic ray rejection and other steps were already performed in the pipeline and are not being re-done, only the final drizzle step.
Note that we recommend sky subtraction to be turned on for broad-band data or other data with significant amounts of sky. This is because the sky background is retained in the science data in the FLT files; the pipeline MultiDrizzle calculates a sky background value and subtracts it on-the-fly before creating the pipeline drizzled product, but does not modify the science pixel values in the original FLT files. Therefore, turning on this step when running MultiDrizzle offline will ensure that the sky background keywords are read from the header and are applied before drizzling the data.
Next, the final output parameters can be specified. In this example, by way of illustration we wish to re-drizzle with a smaller scale (0.015 arcsec/pixel, instead of the HRC default of 0.025 arcsec/pixel), an orientation placing the y-axis at a position angle of 20 degrees, a different pixfrac and a specific central RA, Dec position, as well as a specific output image size. Since the FLT files already have cosmic ray flag values set in their DQ extensions, these will be applied by MultiDrizzle to produce a clean image.
Note that the original Drizzle program expresses the output scale from 0 to 1, as a fraction of the default scale, while MultiDrizzle expresses the output scale in arcsec/pixel.
The output settings can be specified as separate parameters, before running MultiDrizzle:
or on the command line when running MultiDrizzle:
or in the epar interface, prior to running MultiDrizzle:
4.5.4 Example 4: Re-Running MultiDrizzle to Improve Sky Subtraction
The default behavior of MultiDrizzle in the pipeline is to estimate the sky by means of the statistical distribution of pixels in the image, and subtract that from copies of the FLT files that are used to create the final drizzled image. The FLT files that are delivered to the user from the archive are not modified, ie they are not sky subtracted, but the sky values calculated by MultiDrizzle are contained in the header keyword MDRIZSKY.
For cameras with multiple detectors (such as ACS/WFC, or WFPC2), the sky values in each exposure are first measured separately for the different detectors. Then these different values are compared, and the lowest measured sky value is used as the estimate for all the detectors for that exposure. This is based on the premise that for targets consisting of large extended or bright sources, the pixel intensity distribution in one or more of the detectors may be significantly skewed toward the bright end by the target itself, thereby overestimating the sky on that detector. If the other detector is less affected by such a target, then its sky value will be lower, and can therefore also be substituted as the sky value for the detector with the bright source.
The default behavior for MultiDrizzle in the pipeline is to perform iterative sigma-clipping, starting with the full range of pixel intensity values and calculating the standard deviation, then rejecting pixels with values deviating from the mean by more than 4 sigma either way, repeating this for a total of 5 iterations, and then using the median value of the final distribution as the sky value. This gives good results for a wide range of datasets, but occasionally the sky is still slightly overestimated and can be improved by post-pipeline processing.
The example in the preceding section (taken from Section 3.5.2) is of a bright globular cluster where the counts in the calibrated image extend beyond 100,000 electrons per pixel. The sky values calculated by MultiDrizzle in the pipeline are about 97 electrons, and inspection of the pipeline DRZ image product reveals that this is apparently an overestimate, since the background in the DRZ image clearly has a net negative offset
Therefore, MultiDrizzle can be run off-line, in this case to limit the range of initial pixel intensity values used in computing the sky, and also to tighten up the sigma-clipping iterations. First, the 'imhist' command is used to examine the distribution of pixel intensity values on the FLT file (see Figure 4.8), and an upper limit of 500 electrons is chosen - high enough to be well above the sky value, but much lower than the counts in the stars. Next, MultiDrizzle is re-run:
pyraf> multidrizzle j8bt07*flt.fits output=example4a skystat=mode skylower=-100 skyupper=500 skyclip=5 skylsigma=3 skyusigma=3 static- skysub+ driz_separate- median- blot- driz_cr- driz_combine+
This produces sky values of the order of 60-70 counts, thus the background in the final drizzled image is less negative than before, but it is still negative indicating that the sky is still being somewhat overestimated. At this point, the best solution is for the user to measure the sky independently, and provide this as a value to MultiDrizzle. Displaying the image (Figure 4.9) shows that scattered light from the bright stars is very strong across the entire image.
Figure 4.8: Histogram of pixel intensity values in the calibrated science data of j8bt07oyq_flt.fits, which is dominated by scattered light from bright stars, introducing a significant bright skew to the distribution of pixel intensities near the background, thereby strongly affecting any automated estimate of the background.
Figure 4.9: Demonstration of the impact of scattered light from bright sources in a crowded field on the accuracy of background determination. In this example, all pixels in the image are affected to some measurable degree by scattered light, so the least affected regions need to be selected manually after visual inspection.
Therefore, we use imstat to measure the background values in a few manually selected regions that are less affected:
In this particular case, we find average values of 31.11 for j8bt07oyq_flt.fits and 31.77 for j8bt07ozq_flt.fits, based on these manually selected regions. Therefore, we add these to the headers as new keywords (arbitrary called 'NEWSKY'), and then re-run MultiDrizzle. Note that the new keywords for the sky background must go into the image primary header [0], not the individual science exposure headers.
Indeed, this finally yields a drizzled image where the background pixels no longer appear to have a net negative offset. Of course, it should be kept in mind that if the background of a field is this strongly affected by the bright sources within it, then the purpose of the background measurement is primarily to ensure that there are no more varying offset levels in all the exposures prior to combination, due to sky variations from one exposure to the next; the true background may be difficult or impossible to determine in such cases.
This is also why background subtraction is turned off by default when running MultiDrizzle in the pipeline on narrow-band data and UV observations that are "dark" (i.e., have no geocoronal emission in the filter bandpass), since: 1) the sky background through such filters is much lower than through an optical broad-band filter, and (2) such observations are often of extended diffuse emission-line targets whose flux is much higher than the background, thus any automated attempt to measure the background may introduce errors that are larger than the background itself. However, if the user is able to determine an accurate background level in such cases, then the above mechanism may be used to propagate those values directly to MultiDrizzle.
Sky subtraction is generally recommended for optimal flagging and removal of CR's, when the sky background is more than a few electrons. However, some science applications may require the sky to not be removed. Thus, the final drizzle step can be performed with no sky subtraction.
|
Space Telescope Science Institute http://www.stsci.edu Voice: (410) 338-1082 help@stsci.edu |