The MultiDrizzle Handbook
Cosmic ray rejection using the separate tasks in IRAF's dither package required the interactive use of many separate tasks; a complicated and often error-prone procedure. In addition, the geometric distortion must be removed from each input exposure in order to calculate precise shifts for alignment of the images well enough to recognize the presence of cosmic rays in each exposure. MultiDrizzle automates this entire process into a single operation based on the basic drizzle algorithm for removing the distortion and aligning the images. It then provides control over the individual steps through a separate set of parameters for each step in the process while taking into account many of the unique characteristics of the different HST cameras. As a result, a single set of dithered exposures can be combined into a single mosaic while removing the distortion and cosmic rays simply by running this one task.
5.4.2 Software Requirements
MultiDrizzle has been designed as a Python package with its own interface and can be run from the Python environment in conjunction with STSDAS 3.2 or later. However, it can also be loaded into STSDAS under PyRAF to enable use of the graphical EPAR interface that comes with the PyRAF command interface. These packages have been designed to work together in an almost seamless manner without requiring any manual setting of the environment by the user. MultiDrizzle requires at least Python 2.4, Numpy v1.0, and F2PY to be installed for operation with the Python interfaces. In addition, STSDAS 3.8 and PyRAF 1.1 are required for use of the GUI EPAR interface.
5.4.3 Data and Header Specifics
This is a detailed example for ACS data, but will be generally applicable for any instrument. For a specific, more detailed example of the instrument you are working with please see Real-World Examples Chapter 6.
All ACS data calibrated using the standard HST calibration pipeline has been processed by MultiDrizzle to remove geometric distortion and cosmic rays when associated images are combined. Data taken using a standard dither pattern, using CR-SPLIT on a long exposure or using REPEAT-OBS, as specified in the observing proposal will produce associated datasets that will be automatically combined into a dither product by MultiDrizzle. There will be times, though, when the results are not sufficient for scientific analysis and the images require reprocessing off-line using MultiDrizzle. This example illustrates how to take observations associated by the HST calibration pipeline and reprocess them with MultiDrizzle.
Table 5.6: Association table for dithered observations. This example will be based on observations taken using a 2-point Dither-Line pattern with CR-SPLIT=2 at each pointing using the WFC detector of the ACS.
MEMNAME MEMTYPE MEMPRSNT
It is assumed that this association has already been processed using calacs. The products created by calacs include both cosmic ray cleaned, calibrated products with `crj.fits' suffixes and fully calibrated input images with `flt.fits' suffixes. This will result in having at least the following files available in the current directory:
Table 5.7: Files That Need to be Available to run MultiDrizzle
18.104.22.168 Reference Files
The default files which were used to process data retrieved from the HST archive are referenced in the header using an environment variable which points to the directory where they are stored. For ACS this variable is called "jref". If you retrieved the best reference files along with your data, make sure that this points to the directory in which they were saved. If you have used specialized reference files make sure that the full path to their location is also referenced correctly in the data headers. A quick way to check the current value of the variable (in IRAF or PyRAF) is to use the "show" command:>show jref
If the variable is incorrect, it can be set using the standard IRAF command "set", where the directory shown in this example would be replaced by the correct one for your local system. It might, for example, be located at:>set jref = /grp/hst/cdbs/jref
You should also set jref in your appropriate operating system environment setup file (such as .setenv on Mac X11, Unix and Linux)>setenv jref /grp/hst/cdbs/jref
Two important file are the distortion coefficients reference file, IDCTAB (Section 4.2.4), and MultiDrizzle parameter table, mdriztab (Section 5.4.7), which are referenced in the primary headers of each dataset. The MultiDrizzle parameter table provides all the default settings used by the pipeline to run MultiDrizzle on any given single image or association. For these images, using the IRAF syntax, we find that idctab and mdriztab are:>hedit j8cw03a1q_flt.fits idctab,mdriztab . >j8cw03a1q_flt.fits,IDCTAB = jref$wfc_smov_idc.fits >j8cw03a1q_flt.fits,MDRIZTAB = jref$wfc_smov_mdz.fits
5.4.4 Running Under PyRAF
MultiDrizzle has been incorporated into STSDAS not only as a Python task in its own right, but also as an IRAF task with the usual parameter driven interface. The Python syntax can also be used from PyRAF, since PyRAF provides full Python functionality as well as access to IRAF tasks.
PyRAF can usually be started by simply running the command:>your_computer> pyraf
Once PyRAF has started up, load the STSDAS, then dither packages:>stsdas >dither
At this point, the PyDrizzle code has been loaded into PyRAF and is available for use. The IRAF EPAR interface can be used to review and set the available parameters which guide MultiDrizzle's processing routines.
5.4.5 Running under Python
MultiDrizzle can also be used directly from Python without loading IRAF, STSDAS or PyRAF. MultiDrizzle has been written as a Python package and can be treated just like any other Python package. Therefore, to use MultiDrizzle directly from Python, go to the directory with the data and start up Python:your_computer> python
In order to use MultiDrizzle with the Python interface the code must be installed in a directory accessible by Python. If that directory is not already included in the PYTHONPATH or default Python system path, you can point to it inside Python using:>>> sys.path.insert(1,'/directory/with/Multidrizzle/code')
MultiDrizzle can then be loaded using the usual import mechanism in Python:>>> import multidrizzle
At this point, MultiDrizzle can be run on the calibrated images in the local directory. Interactive help can be obtained for MultiDrizzle using the built-in help mechanism. This will provide basic information on the methods available for use and their syntax as well as the most basic set of commands outlined in this example for running MultiDrizzle.>>> multidrizzle.help()
5.4.6 Running MultiDrizzle
The Python syntax for running MultiDrizzle does not vary that much from the PyRAF syntax. Unlike the PyRAF syntax, it provides the user with the ability to examine the inputs and computed parameters before spending the majority of time actually doing the image combination. MultiDrizzle creates an instance of a MultiDrizzle object which contains all the methods necessary for editing the input parameters, computing the image combination parameters, then performing the actual cosmic ray detection and image combination. All of these functions get wrapped into one call through the PyRAF interface, making for simpler operation at the cost of the ability to inspect the object and its contents.
The processing starts with the creation of a MultiDrizzle object, which for the sake of this example will simply be called `md':>>>md = multidrizzle.Multidrizzle(input='j8cw030x0_asn.fits', mdriztab=yes, editpars=False, **input_dict)
The parameter `input_dict' serves as the means for the user to specify any of the remaining parameters from the parameter table which should have non-default values. Setting `editpars' to True here will cause the Python GUI parameter editor to start automatically. You may also choose to use the separate `editpars()' method.
The specification of the mdriztab parameter in the initialization demonstrates how any of the input parameters whose values need to be changed from the default value can be provided at the start. If many parameters need to be edited, the Python GUI interface can be started, simply closing this GUI saves the values in the MultiDrizzle object for use.>>>md.editpars()
This editor allows the user to examine all the inputs for MultiDrizzle and set them as necessary to work best with the input data. See the parameter table (Section 5.4.7) for the full available set of MultiDrizzle parameters. At this point the MultiDrizzle object contains all the input parameter values necessary for the processing. However, the computation of the parameters necessary for doing the image combination and cosmic ray detection still need to be performed using the build method:>>>md.build()
The MultiDrizzle object (md) now contains everything necessary for the processing. It can still be inspected using simple python commands, but no further method operations are necessary to complete the setup process. Alternatively, a help method can provide a reminder of the available methods for any MultiDrizzle object already created using:>>>md.help()
The full MultiDrizzle process can now be started:>>>md.run(static=None, skysub=None, driz_separate=None, median=None, blot=None, driz_cr=None, driz_combine=None)
This method allows the user one last chance to specify what processing steps should be performed. If no parameters are specified, all processing steps turned on during the instantiation of the MultiDrizzle object will be performed. Generally, though, the user simply wants to run this method without any specified parameters, such as:>>>md.run()
This represents the last step in processing images with MultiDrizzle, with the final product being a registered, cosmic ray cleaned, distortion-free, photometrically uniform image.
22.214.171.124 Session Summary
Very few commands are actually needed in order to process images or sets of images using MultiDrizzle. The total session described here simply boils down to:>your_computer> cd /directory/with/data/ >your_computer> pyraf >stsdas >dither >multidrizzle j8cw030x0_asn.fits mdriztab=yes mode=h
The Python syntax does not require many more commands:>import multidrizzle >md = multidrizzle.Multidrizzle('j8cw030x0_asn.fits',mdriztab=yes) >md.editpars() >md.build() >md.run()
Both ways of running MultiDrizzle will produce the exact same output products if the same input files are supplied.
126.96.36.199 Intermediate Products
There are many intermediate steps during the full course of "drizzling" your input images into a mosaic. Each of these steps has particular output which informs the next stage of the proceedure. The intermediate products are all those files which get created during the process but are not necessarily considered final science products. These may include files with the extensions:
- _single_sci.fits: The individually drizzled, distortion corrected versions of each input image
- _single_mask1.fits: These are the first version of the bad pixel files that are generated
- _final_mask1.fits: These are the final masks to be used for each science image
- _bl.fits: The blotted images which were created by undistorting and shifting the part of the median image which corresponds to the FOV of the original science exposure
- _cr.fits: cosmic ray masks which are created as part of the cosmic ray detection step, accomplished in the driz_cr subroutine
- _drw.fits: pixel weighting images which are output whenever the drizzle step is run
- _med.fits: most likely these are median combined singly drizzled images
- _coeffs2.dat: this is a text file which contains information extracted from the IDC table about the distortion in the instrument. This will be replaced by the SIP header information which will also be added to the headers.
MultiDrizzle allows an option (clean) to turn on and off the saving of files which are created during intermediate steps. While it is good to keep intermediate files so that you can assess the full performance of the tasks, you must keep in mind the size of your input images and the disk capacity of the machine on which you're running, and set this parameter accordingly.
188.8.131.52 Final Products
The output from PyDrizzle is a single multi-extension FITS file with the suffix "_drz.fits". The first extension contains the science (SCI) image which is corrected for distortion and dither-combined (or mosaiced), if applicable. The drizzled SCI image is typically in units of electrons per second, which is the default for ACS images. The user can choose to have the output in either electrons, electrons per second, data numbers (DN) or DN per second, some of these options retain the natural units of the instrument. All pixels have equal area on the sky and equal photometric normalization across the field of view, giving an image which is both photometrically and astrometricaly accurate for point and extended sources. The dimensions of the output image will be computed on-the-fly by PyDrizzle and the default output plate scale will be read from the idctab. These parameters, however, may be chosen by the user during reprocessing to best suit the actual data and the users needs.
The second extension of the output image contains the weight (WHT) image. It gives the relative weight of the output pixels and can be considered an effective exposure time map. The data quality map created by the instrument calibration pipelines is used by PyDrizzle in creating the weight image.
The third extension of the PyDrizzle output image contains the context (CTX) image which encodes information about which input image contributes to a specific output pixel. This is done using a bitmask for each output pixel, where `bit set' means that the image, in the order it was combined, contributed with non-zero weight to that output pixel. The context image starts as a single 32-bit integer image but is extended as a cube with additional 32-bit deep planes as required to handle all the input images.
NNICMOS users should note that this may seem further complicated by the nature of NICMOS data. Since the final science level NICMOS images that come out of calnica are really mathematical fits to the ramp of observations, the true exposure time varies across the calibrated science image. The default output from calnica is to have unitcorr set to perform, which divides the pixel counts by the full exposure time for that pixel, producing an output image in units of data numbers/second. NICMOS _cal.fits images contain a TIME extension which details the true exposure level of the image, pixel-by-pixel, after the individual reads have been combined into the ramp image and after cosmic ray rejection, which includes read rejection, has been done on the image. Since the calibrated image represents the actual countrate the pixel should have reported in the absence of defects in any of the individual reads, multiplying this final image by the full exposure time of the dataset (using the exptime keyword value) is an acceptable practice to convert the science image back into relative counts. The [TIME] extension in the output _cal file should only be used as a processing reference by the user.
The WFC3/IR pipeline is based on the NICMOS pipeline, and produces a calibrated _flt.fits file using the same up-the-ramp fitting procedure which rejects cosmic rays. The units of the WFC3/IR _flt.fits file are electrons per second, rather than data numbers per second, to be more consistent with the calibrated WFC3/UVIS _flt.fits files, which are in units of electrons.
ACS and WFC3 users should note that the calibrated FLT files contain additional bits in the DQ mask that identify pixels flagged as cosmic rays by MultiDrizzle. The default value for these bits is 4096, and if MultiDrizzle is re-run on the FLT files offline it will be able to use these flags and proceed directly to the final drizzle step without the need to re-do cosmic ray masking, unless that is required. Alternatively, if the cosmic ray rejection needs to be improved, then the pipeline CRs can be ignored/reset by adding 4096 to the "bits" parameter value when running MultiDrizzle.
5.4.7 MultiDrizzle Parameters
MultiDrizzle processing starts with a list of input images, then applies a number of operations on them to produce the final cosmic ray cleaned image; specifically:
- Initialization (Section 5.4.8)
- Bad pixel (Section 5.4.9) (static) mask creation
- Sky Subtraction (Section 5.5.4)
- Drizzle onto separate (Section 5.4.10), registered output images
- Combine the separate drizzled images (Section 5.4.11) into a median
- "blot" (Section 5.4.12), or transform back the median to each input image
- Compare the median with original images to make cosmic ray masks (Section 5.4.13)
- Drizzle all the images onto a final image (Section 5.4.14), using the cosmic ray masks
MultiDrizzle relies on a large number of parameters for specifying what steps get performed and for controlling the algorithm used by each step. The complete list of parameters and their default values are given in the parameter table (Section 5.4.7). It is generally recommended to try running the script first with the default parameters, which should allow the task to process nearly any set of images for an initial review. The script can then be re-run, or restarted at a particular step, with modified parameters if this is necessary. MultiDrizzle is not designed to be restarted in the middle of processing, so when computing improved cosmic ray masks, the user must turn on all steps prior to 'driz_cr'. If this is not done, the header keyword mdrizsky will be set to zero, and the software will flag any reasonably bright objects in the image. The result is a cosmic ray mask that looks like a map of the image, where all pixels containing sources have been flagged. When MultiDrizzle is run by the archive instrument pipelines it references the mdriztab file in the header of the dataset. mdriztab is a fits table file which contains carefully chosen input parameters for MultiDrizzle which are optimized for the widest range of science cases. These are the default parameters which are used in the instrument pipelines for data retrieved from the HST archive, and may be different for each instrument. Users are encouraged to examine whether these parameters are appropriate for their particular datasets and recombine their science data as necessary.
Table 5.8: Parameters for MultiDrizzle
Parameter Default Value Description Format Initialization Static Mask Creation Sky Subtraction Create Separate Drizzled Images (Section 5.4.10) Create Median Image Blot Median Image Remove Cosmic Rays (Section 5.4.13) Create Final Cleaned, Combined Product Override Instrument Specific Parameters
184.108.40.206 General Explanation
MultiDrizzle starts by determining what files are being specified as inputs. These files then get converted as necessary to a usable input format with appropriate units. This conversion includes making copies of each input file, if `workinplace' is set to `False', so that the original input SCI arrays are not altered and can be used for successive runs. A set of drizzle parameter values needed to combine the images into a final output image then get computed using PyDrizzle.
220.127.116.11 Parameter Details
Input: This controls all inputs for the entire MultiDrizzle task. In addition, it requires access to:
- The name or names of the input files to be processed which can be provided in any of several forms; namely,
- filename of a single image
- filename of an association (ASN) table
- wild-card specification for files in directory
- comma-separated list of filenames
- `@file' filelist containing list of desired input filenames. The file list needs to be provided as an ASCII text file containing a list of filenames for all input images with one filename on each line of the file. If inverse variance maps (IVM maps) have also been created by the user and are to be used (by specifying `IVM' to the parameter final_wht_type (Section 18.104.22.168), then these are simply provided as a second column in the filelist, with each IVM filename listed on the same line as a second entry, after its corresponding exposure filename. If the user specifies `IVM' for the `final_wht_type' but does not provide the names of IVM files, MultiDrizzle will automatically generate the IVM files itself for each input exposure.
- the mdriztab (Section 5.4.7) reference table which specifies default parameters, contained in the input image headers
- the idctab (Section 4.2.4) reference table specified in the input image headers
- any specified reference image as named in the `refimage' parameter
Output: The rootname for the output drizzled products. This step can result in the creation of several files, including:
- copies of each input image as a FITS image, if workinplace=yes and/or input images are in GEIS format
- mask files and coeffs files created by PyDrizzle for use by drizzle
- If an association file has been given as input, this name will be used instead of the product name specified in the ASN file. Similarly, if a single exposure is provided, this rootname will be used for the output product instead of relying on input rootname. If no value is provided when a filelist or wild-card specification is given as input, then a rootname of `final' will be used.
mdriztab: Specifies whether or not to use an mdriztab reference table to specify the remaining MultiDrizzle parameter settings. If `True', the values in the table will override the settings for the remainder of the parameters.
refimage: Optional "reference image" that can be provided, in which case MultiDrizzle will create a final product with the same WCS. This reference image should be a simple FITS file (single-group, no multiple extensions), and should have been already drizzled so that all of its distortion has been removed, and its WCS is completely rectified.
runfile: This log file will contain the IRAF CL commands necessary for performing the final combination manually using the drizzle task directly.
workinplace: This parameter specifies whether to perform all processing on the original, including sky subtraction and update of the DQ array. If set to `True', then no copy of the input will be created for processing, and the original input will be modified directly by MultiDrizzle.
updatewcs: This parameter specifies whether the WCS keywords are to be updated by running makewcs on the input data, or left alone. The update performed by makewcs not only recomputes the WCS based on the currently used idctab, but also populates the header with the SIP coefficients. Also, for ACS/WFC images, the time-dependence correction will also be applied to the WCS and SIP keywords. This parameter should usually be set to `yes', unless the WCS keywords have been carefully set by some other method, and need to be passed through to drizzle `as is'.
coeffs: The source of the distortion coefficients gets specified using:
- header: use the `header' for determining what distortion coefficients to use.
- cubic: use `cubic' solutions originally provided with `drizzle'
- trauger: use `trauger' solutions originally provided with `drizzle'
- none: Do not apply any distortion correction to the images
Alternatively, an arbitrary distortion coefficients file can be specified (including optionally the full pathname). This distortion file is used in computing the WCS of the header. NOTE: When the `header' option has been selected, if the idctab file is not found on disk, then the task will exit for all instruments except for WFPC2 data, which will default to the older "Trauger" model.
context: This parameter specifies whether or not to create a context image during the final drizzle combination. The context image contains the information on what image(s) contributed to each pixel encoded as a bit-mask. More information on context images can be obtained from the
ACS Data Handbook.
clean: The temporary files created by MultiDrizzle can be automatically removed by setting this parameter to `True'. The affected files would include the coefficients files and static mask files created by PyDrizzle, along with other intermediate files created by MultiDrizzle. It is often useful to retain the intermediate files and examine them when first learning how to run MultiDrizzle. But when running it routinely, or on a small disk drive, these files can be removed to save space.
group: A single FITS extension or group can be drizzled by setting this parameter. If an extension is provided, then only that chip will be drizzled onto the output frame. Either a FITS extension number or GEIS group number (such as `1'), or a FITS extension name (such as `sci,1') may be specified.
ra: Right ascension (in decimal degrees) of the center of the output image. If this is not specified, the code will calculate the center automatically based on the distribution of image dither positions.
dec: Declination (in decimal degrees) of the center of the output image. If this is not specified, the code will calculate the center automatically based on the distribution of image dither positions.
build: MultiDrizzle will combine the separate `drizzle' output files into a single multi-extension format FITS file when this parameter gets set to `True'. This combined output file will contain a SCI (science), a WHT (weight), and a CTX (context) extension. If set to `False', each extension would remain as a separate simple FITS file on its own.
shiftfile: Name of optional input file containing the shifts to be applied to the input images to improve their registration.These shifts will be added to those calculated automatically from the image headers. The file should be organized as follows:# units: pixels # frame: input # form: delta rootname xshift yshift rotation staticfile: Name of the optical input static bad-pixel mask. The input data are multiplied by the contents of this static mask file.
gain: Value used to override instrument specific default gain values. The value is assumed to be in units of electrons/count. This parameter should not be populated if the gain keyword parameter is in use.
gainkeyword: Keyword used to specify a value to be used to override instrument specific default gain values. The value is assumed to be in units of electrons/count. This parameter should not be populated if the gain parameter is in use.
rdnoise: Value used to override instrument specific default readnoise values. The value is assumed to be in units of electrons. This parameter should not be populated if the rnkeyword parameter is in use.
rnkeyword: Keyword used to specify a value to be used to override instrument specific default readnoise values. The value is assumed to be in units of electrons. This parameter should not be populated if the rdnoise parameter is in use.
exptime: Value used to override default exposure time image header values. The value is assumed to be in units of seconds. This parameter should not be populated if the expkeyword parameter is in use.
expkeyword: Keyword used to specify a value to be used to override default exposure time image header values. The value is assumed to be in units of seconds. This parameter should not be populated if the exptime parameter is in use.
crbit: Integer used to override instrument specific cosmic ray bit values. This value is used by MultiDrizzle to update data quality arrays when cosmic rays or other image defects are identified as "bad" in the DRIZ_CR step. To prevent the image's data quality array from being updated set the crbit value to 0.
22.214.171.124 Basic Example
An example of an input image that you would like to drizzle is in Figure 5.2.Figure 5.2: Single Extension from an ACS FLT File
A single extension from a calibrated ACS FLT file, j8cw54orq_flt.fits, which still contains numerous cosmic rays, hot pixels, and other artifacts.
5.4.9 Static Mask Creation
126.96.36.199 General Explanation
If the static flag is set to "yes", a static mask will be created for all input images. The mask contains pixels which fall more than `static_sig' *RMS below the mode for a given chip or extension. Those severely negative or low pixels might result from over subtraction of bad pixels in the dark image, or high sky levels, during calibration. For example, each ACS WFC image contains a separate image for each of 2 CCDs and separate mask will be generated for each chip.
The final static mask for each chip contains all the bad pixels that meet this criteria from all the input images along with any bad pixels which satisfied the final_bits value specified by the user and found in the images DQ mask. This static mask could also be combined with a user supplied static mask specified in the `staticfile' parameter.
Users should consider the details of their science image and decide whether creating this mask is appropriate for the resulting science. For example, if your field is very crowded, contains mostly nebulous or extended objects, the statistics could be heavily skewed and the mask could end up containing sources.
Input: Aside from the input parameters, this step requires opening each input image to access the science (SCI) extensions for generating the static masks.
Output: The generated static masks exist strictly in memory as numarray objects that get applied during the single drizzle step. They also get used to update the input mask files for the final image combination.
188.8.131.52 Parameter Details
static: This parameter specifies whether to create a static bad-pixel mask from the data or not. This mask flags all pixels that deviate by more than `static_sig' sigma below the image median.
static_sig: Number of sigma below the RMS to use as the clipping limit for creating the static mask, defaulted to 4.0
184.108.40.206 Basic Examples
An example of what a bad pixel mask might look like. (Figure 5.3) is a badpixel mask from a NICMOS camera 3 image where the differing gray levels signify different bits which have been set in the Data Quality (DQ) extension of the science image with which it is associated.Figure 5.3: An Example Bad Pixel Mask Taken from a NICMOS Image
Bad pixel mask from NICMOS camera 3. There are multiple shades of grey because the DQ image contains the logical AND of all pixel processing flags generated during calnica.
5.4.10 Create Separate Drizzled Images
220.127.116.11 General Explanation
Each input image gets drizzled onto a separate copy of the output frame. When stacked, these copies would correspond to the final combined product. As separate images, they allow for treatment of each input image separately in the undistorted, final WCS system. These images provide the information necessary for refining the image registration for each of the input images. They also provide the images which will later be combined into a median image used for the later blot and cosmic ray detection steps.
Input: Aside from the input parameters, this step requires:
- valid input images with SCI extensions
- valid distortion coefficients tables
- any optional secondary distortion correction images
- numarray object (in memory) for static mask
Output: This step produces:
- singly drizzled science image (simple FITS format)
- singly drizzled weight images (simple FITS format)
These images all have the same WCS based on the original input parameters and those provided for this step; specifically, output shape, pixel size, and orientation, if any have been specified at all.
18.104.22.168 Parameter Details
driz_separate: This parameter specifies whether or not to drizzle each input image onto separate output images. The separate output images will all have the same WCS as the final combined output frame. These images are used to create the median image, needed for the cosmic ray rejection step further on.
driz_sep_outnx: Size of the X axis of the output images, in pixels, on which each input will be drizzled onto. If no value is specified, it will use the smallest size that can accommodate the full dithered field.
driz_sep_outny: Size of the Y axis of the output images, in pixels, on which each input will be drizzled onto. If no value is specified, it will use the smallest size that can accommodate the full dithered field.
driz_sep_kernel: For the initial separate drizzling operation only, this specifies the form of the kernel function used to distribute flux onto the separate output images. The options are currently:
- square: original classic drizzling kernel
- point: this kernel is a point so each input pixel can only contribute to the single pixel which is closest to the output position. It is equivalent to the limit pixfrac -> 0 and is very fast.
- gaussian: this kernel is a circular gaussian with FWHM equal to the value of pixfrac, measured in input pixels.
- turbo: this is similar to kernel="square" but the box is always the same shape and size on the output grid and always aligned with the X and Y axes. This may result in a significant speed increase.
- tophat: the kernel is a circular "top hat" shape of width pixfrac. It effects only output pixels within pixfrac/2 of the output position are affected.
- lanczos3: a Lanczos style kernel extending 3 pixels from the center. The Lanczos kernel is a damped, bounded form of the "sinc" interpolator and is very effective for resampling single images when scale=pixfrac=1. It leads to less resolution loss than the other kernels, and also less correlated noise in outputs. It is however much slower. It should never be used for pixfrac not equal to 1.0 and is not recommended for scale not equal to 1.0.
The default for this step is "turbo" since it is much faster than "square", and it is quite satisfactory for the purposes of generating the median image. More information about the different kernels can be found in the help for the task `drizzle'.
driz_sep_wt_scl: Weighting factor for input image. If driz_sep_wt_scl=exptime then the scaling value will be set equal to the exposure time found in the image header. This is the default recommended behavior. It is also possible to give wt_scl=expsq for weighting by the square of exposure time, which is optimal for read-noise dominated images.
driz_sep_scale: Linear size of the output pixels in arcseconds/pixel for each separate drizzled image (to be used in creating the median for cosmic ray rejection). The default value of INDEF specifies that the undistorted pixel scale for the first input image, as computed by PyDrizzle, will be used as the pixel scale for all the output images.
driz_sep_pixfrac: Fraction by which input pixels are "shrunk" before being drizzled onto the output image grid, given as a real number between 0 and 1. This specifies the size of the footprint, "dropsize", of a pixel in units of the input pixel size. If pixfrac is set to less than 0.001, the kernel will be reset to "point" for more efficient processing. For the step of drizzling each input image onto a separate output image, the default value of 1 is best in order to ensure that each output drizzled image is fully populated with pixels from the input image. For more information, see the help file for the drizzle task.
driz_sep_rot: Position Angle of output image's Y-axis relative to North. A value of 0.0 would orient the final output image with North up. The default of INDEF specifies that the images will not be rotated, but will instead be drizzled in the default orientation for the camera, with the x and y axes of the drizzled image corresponding approximately to the detector axes. This conserves disk space, since these single drizzled images are only used in the intermediate step of creating a median image.
driz_sep_fillval: Value to be assigned to output pixels that have zero weight or did not receive flux from any input pixels during drizzling. This parameter corresponds to the `fillval' parameter of the drizzle. If the default of `INDEF' is used and if the weight in both the input and output images for a given pixel are zero, then the output pixel will be set to the value it would have had if the input had a non-zero weight. Otherwise, if a numerical value is provided (e.g. 0), then these pixels will be set to that value.
driz_sep_bits: Integer sum of all the DQ bit values from the input image's DQ array that should be considered `good' when building the weighting mask. This can also be used to reset pixels to good if they had been flagged as cosmic rays during a previous run of MultiDrizzle, by adding the value 4096 for ACS and WFPC2 data. Please see the section on Selecting the `Bits' Parameter (Section 5.5.7) for a more detailed discussion.
22.214.171.124 Basic Example
Figure 5.4 and Figure 5.5 contain two basic examples.Figure 5.4: A Singly Drizzled ACS Image
The singly drizzled FLT image `j8cw54ovq_flt_single_sci.fits'. The two chips have been combined and the geometric distortion has been removed.
Figure 5.5: Weight Map Corresponding to the Singly Drizzled ACS Image
The weight image `j8cw54ovq_flt_single_wht.fits' corresponding to the singly drizzled image in Figure 5.5 where white indicates zero weight.
5.4.11 Create Median Image
126.96.36.199 General Explanation
The singly drizzled science images are combined to create a single median image. This median combination gets performed section-by-section from the input single drizzle images. Each section corresponds to a contiguous set of lines from each image taking up no more than 1Mb in memory, so that combining 10 input images would only require 10Mb for these sections. The goal of this step is to created an estimate of what the fully cleaned image should look like in order to enable better bad pixel detection as well as a chance to improve the alignment of the image stack. Creating a median image from the aligned and undistorted input images allows for a statistical rejection of bad pixels.
Input: Aside from the input parameters, this step requires access to the single drizzled images on disk.
Output: The final median image serves as the only output from this step.
188.8.131.52 Parameter Details
median: The user can specify whether or not to create a median image with this parameter. This median image will be used as the comparison `truth' image: in the cosmic ray rejection step.
median_newmasks: The user can specify whether or not to create new mask files when creating the median image. These masks are generated from the weight files produced previously by the "driz_separate" step, and would contain all the bad pixel information. These pixels will be excluded when calculating the median. Generally this step should be set to "yes", unless it is desired to include bad pixels in generating the median.
combine_type: This parameter allows the user to choose which method is used to create the median image. Valid options are:
The `average', `median', and `sum' options set the mode of operation for using numcombine, a numarray method for median-combining arrays, to create the median image. The "minmed" option will produce an image that is generally the same as the median, except in cases where the median is significantly higher than the minimum good pixel value, in which case it will choose the minimum. The sigma thresholds for this decision are provided by the "combine_nsigma" parameter.
combine_nsigma: Sigmas used for accepting minimum values instead of median values when using the `minmed' combination method. If two values are specified, then the first value will be used in the initial choice between median and minimum, while the second value will be used in the "growing" step to reject additional pixels around those identified in the first step. If only one value is specified, then it is used in both steps.
combine_nlow: When using a `minmed' rejection method, this parameter sets the number of low value pixels to reject.
combine_nhigh: When using a `minmed' rejection method, this parameter sets the number of high value pixels to reject.
combine_lthresh: Sets the lower threshold for clipping input pixel values during image combination. This value gets passed directly to `imcombine' for use in creating the median image. If None, no thresholds are used at all.
combine_hthresh: Sets the upper threshold for clipping input pixel values during image combination. This value gets passed directly to `imcombine' for use in creating the median image. If None, no thresholds are used at all.
combine_grow: Width in pixels for additional pixels to be rejected in an image with a rejected pixel from one of the rejection algorithms. This parameter is used to set the `grow' parameter in imcombine for use in creating the median image.
184.108.40.206 Basic Example
This is a cleaned median image created using six separately drizzled images and their associated bad pixel masks.Figure 5.6: ACS Example of a Cleaned Median Image
The cleaned median image created using the 6 separately drizzled Tadpole images and their bad pixel masks.
5.4.12 Blot Median Image
220.127.116.11 General Explanation
The median image is the combination of the WCS aligned input images which have already had the distortion model applied. Taking the median of the aligned images allows for a statistical rejection of bad pixels from the image stack. The resulting median image can then be input to the blot task with the goal of creating `cleaned' versions of the input images at each of their respective dither locations. These "blotted" images can then be directly compared to the original distorted input images for detection of bad-pixels, hot pixels, and cosmic rays whose locations will be saved to the output badpixel masks.
Input: Aside from the input parameters, this step only requires opening the single median image created from all the input images.
Output: A distorted version of the median image corresponding to each input `chip' (extension) gets written out as output from this step as separate simple FITS images.
18.104.22.168 Parameter Details
blot: Perform the blot operation on the median image. The output will be median smoothed images which match each input chips location, these are used in the cosmic ray rejection step.
blot_interp: Type of interpolation to use when blotting drizzled images back to their original WCS. Valid options are:
- nearest: Nearest neighbor
- linear: Bilinear interpolation in x and y
- poly3: Third order interior polynomial in x and y
- poly5: Fifth order interior polynomial in x and y
- sinc: Sinc interpolation; accurate but slow
The poly5 interpolation method has been chosen as the default because it is relatively fast and accurate. If `sinc' interpolation has been selected, then it will use the value of the parameter `blot_sinscl' to specify the size of the sinc interpolation kernel.
blot_sinscl: Size of the sinc interpolation kernel in pixels.
22.214.171.124 Basic ExampleFigure 5.7: Original ACS FLT Image Fed into the Drizzle Step
The original FLT image that was fed into the single drizzle step.
Figure 5.8: The Returned Corresponding Blotted Imager
The blotted image, this is a cutout of the median image at the location of the original image. Note the absence of cosmic rays because they were statistically rejected.
5.4.13 Remove Cosmic Rays
126.96.36.199 General Information
The blotted median images which are now transformed back into the original reference frame, get compared to the original input images to detect any spurious pixels (which may include pixels impacted by cosmic rays) in each input. Those spurious pixels then get flagged as `bad' in the output mask files, which get used as input for the final combination, so that they do not show up in the final product.
Input: Aside from the input parameters, this step requires:
Output: The identified bad pixels get flagged by updating the input mask files. Optionally, copies of the original images with the bad pixels removed can be created through the use of the `driz_cr_corr' parameter.
188.8.131.52 Parameter Details
driz_cr: Perform cosmic ray detection? If set to "yes", it will detect cosmic rays and create cosmic ray masks using the algorithms from deriv and driz_cr.
driz_cr_corr: Create a cosmic ray cleaned input image? The cosmic ray cleaned _cor image will be generated directly from the input image, and a corresponding _crmask file will be written to document the pixels detected as affected by cosmic rays.
driz_cr_snr: These values specify the signal-to-noise ratios for the driz_cr task to use in detecting cosmic rays. This parameter value gets passed directly to driz_cr;
driz_cr_scale: Scaling factor applied to the derivative in driz_cr when detecting cosmic rays. This parameter gets passed directly to driz_cr.
driz_cr_grow: Radius (in pixels) around each detected cosmic ray to use more stringent detection criteria for additional cosmic rays.
driz_cr_cte_grow: Length (in pixels) of CTE tail to mask in drizzled output.
184.108.40.206 Basic ExampleFigure 5.9: Single chip ACS Cosmic Ray Mask
A single cosmic ray mask `j8cw54orq_flt_sci2_crderiv.fits'. This mask should be blinked with the original image `j8cw54orq_flt' input file to assure that optimal parameters were chosen in the driz_cr task.
5.4.14 Create Final Cleaned, Combined Product
220.127.116.11 General Explanation
This step performs the final image combination of the original input images (or their copies) using the updated mask files to remove any cosmic rays. The output frame, just like the single drizzle step, can be redefined here using some parameters for this step, otherwise it will use the defaults to (ideally) include all the input pixels from all the input images after registering them according to their header's WCS information.
Input: Aside from the input parameters, this step requires:
Output: The final product of MultiDrizzle is a registered, cosmic ray cleaned, distortion-free, photometrically flat science image with associated weight and context images. By default, these will be written out as a single multi-extension FITS file, but the user could simply have them written out as separate simple FITS images.
18.104.22.168 Parameter Details
driz_combine: This parameter specifies whether or not to perform the final drizzle image combination. This applies the bad pixel masks to the input images and creates a final, cleaned, distortion-corrected product.
final_wht_type: Specify the type of weighting image to combine with the bad pixel mask for the final drizzle step. The options are:
The default of `EXP' indicates that the images will be weighted according to their exposure time, which is the standard behavior for drizzle. This weighting is a good approximation in the regime where the sky noise dominates. This option is provided as the default since it produces reliable weighting for all types of data, including older instruments (e.g., WFPC2) where more sophisticated options may not be available.
Specifying `ERR' is an alternative for ACS and STIS data, in which case the final drizzled images will be weighted according to the inverse variance of each pixel in the input exposure files, calculated from the error array data extension that is in each calibrated input exposure file. This array encapsulates all the noise sources in each exposure, including read-noise, dark current and sky background, as well as Poisson noise from the sources themselves, and this also includes a dependence upon exposure time. For WFPC2, the ERR array is not produced during calibration, therefore this option is not available. But for ACS and STIS datasets this option is generally recommended to be the most accurate type of weighting for producing the final drizzled image.
In this case the user can either supply their own inverse-variance weighting map or let MultiDrizzle generate one on-the-fly automatically during the final drizzle step. This may be necessary for specific purposes, for example to create a drizzled weight file for software such as SExtractor, which expects a weight image that contains all the background noise sources (sky level, read-noise, dark current, etc.) but not the Poisson noise from the objects themselves. The user can create the inverse variance images and then specify their names using the `input' parameter for MultiDrizzle to specify an `@file'. This would be a single ASCII file containing the list of input calibrated exposure filenames (one per line), with a second column containing the name of the IVM file corresponding to each calibrated exposure. Each IVM file must have the same file format as the input file, and if given as multi-extension FITS files (for example, ACS or STIS data) then the IVM extension must have the extname of `IVM'. If no IVM files are specified on input, then MultiDrizzle will rely on the flat-field reference file and computed dark value from the image header to automatically generate an IVM file specific to each exposure.
final_outnx: Size of the X axis of the final drizzled image (in pixels). If no value is specified, it will use the smallest size that can accommodate the full dithered field.
final_outny: Size of the Y axis of the final drizzled image (in pixels). If no value is specified, it will use the smallest size that can accommodate the full dithered field.
final_kernel: Shape of the kernel used by `drizzle' in the final image combination. The supported choices are:
- square: original classic drizzling kernel
- point: this kernel is a point so each input pixel can only contribute to the single pixel which is closest to the output position. It is equivalent to the limit pixfrac -> 0. It is very fast.
- gaussian: this kernel is a circular gaussian with FWHM equal to the value of pixfrac, measured in input pixels.
- turbo: this is similar to kernel="square" but the box is always the same shape and size on the output grid and always aligned with the X and Y axes. This results in a significant speed increase in some cases.
- tophat: the kernel is a circular "top hat" shape of width pixfrac. In effect only output pixels within pixfrac/2 of the output position are affected.
- lanczos3: a Lanczos style kernel extending 3 pixels from the center. The Lanczos kernel is a damped, bounded form of the sinc interpolator and is very effective for resampling single images when scale=pixfrac=1. It leads to less resolution loss than the other kernels, and also less correlated noise in outputs. It is however much slower. It should never be used for pixfrac not equal to 1.0 and is not recommended for scale not equal to 1.0.
- The default for this step is "square"
final_wt_scl: Weighting factor for input image. If final_wt_scl = exptime then the scaling value will be set equal to the exposure time found in the image header. This is the default recommended behavior. It is also possible to give wt_scl=expsq for weighting by the square of exposure time which is optimal for read-noise dominated images.
final_scale: Linear size of the output pixels in arcseconds/pixel for the final combined product. The default value of INDEF specifies that the undistorted pixel scale for the first input image, as computed by PyDrizzle, will be used as the pixel scale for the final output image.
final_pixfrac: Fraction by which input pixels are "shrunk" before being drizzled onto the output image grid, given as a real number between 0 and 1. This specifies the size of the footprint, "dropsize", of a pixel in units of the input pixel size. If pixfrac is set to less than 0.001, the kernel is reset to `point' for more efficient processing. If more than a few images are being combined, values smaller than 1 (e.g. 0.7 or 0.8) can be specified, which result in a slightly sharper output image. For more information, read the help for the task `drizzle'.
final_rot: Position Angle of output image's Y-axis relative to North. The default of 0.0 would orient the final output image with North up. A value of INDEF would specify that the images will not be rotated, but will instead be drizzled in the default orientation for the camera, with the x and y axes of the drizzled image corresponding approximately to the detector axes.
final_fillval: Value to be assigned to output pixels that have zero weight or did not receive flux from any input pixels during drizzling. This parameter corresponds to the `fillval' parameter of the drizzle task. If the default of INDEF is used and if the weight in both the input and output images for a given pixel are zero, then the output pixel will be set to the value it would have had if the input had a non-zero weight. Otherwise, if a numerical value is provided (e.g. 0), then these pixels will be set to that value.
final_bits: Integer sum of all the DQ bit values from the input image's DQ array that should be considered `good' when building the weighting mask. This can also be used to reset pixels to good if they had been flagged as cosmic rays during a previous run of MultiDrizzle, by adding the value 4096 for ACS and WFPC2 data. The section on selecting the bits parameter (Section 5.5.7) provides more details on which bits should be included.
22.214.171.124 Basic Example
Figure 5.10 shows the SCI extension, Figure 5.11 shows the WHT extension, and Figure 5.12 shows the CTX extension of the drizzled product.Figure 5.10: Science Extension of an ACS Drizzled Output Product
The science (SCI) extension of the drizzled product. This image has been corrected for distortion and drizzled onto a single mosaic using the six images in the dither pattern.
Figure 5.11: Corresponding Weight Extension of the ACS Drizzled Output
The corresponding weight (WHT) extension of the drizzled product.
Figure 5.12: Corresponding Context Extension of the ACS Drizzled Output
The corresponding context (CTX) extension of the drizzled product.
5.4.15 Override Instrument Specific Parameters
126.96.36.199 General Information
The user may choose to override the information contained in the headers of the images by setting these parameters directly.
188.8.131.52 Parameter Details
gain: Detector gain
gnkeyword: Detector gain keyword in header
rdnoise: Detector read noise
rnkeyword: Detector read noise keyword in header
exptime: Exposure time
expkeyword: Exposure time keyword in header
crbit: Bit value for CR identification in DQ array
Space Telescope Science Institute
Voice: (410) 338-1082