Release Notes for stsci_python version 2.1 ------------------------------------------ stsci_python is a library of Python routines and C extensions that has been developed to provide a general astronomical data analysis infrastructure. It includes the following: * PyRAF, an environment for running IRAF tasks without using the IRAF CL * Numarray, a general array manipulation package * MultiDrizzle, a task for combining dithered STScI images * PyDrizzle, used by MultiDrizzle to provide a convenient but powerful interface to the Drizzle task of Fruchter and Hook * PyFITS, used to read FITS images and tables into numarray objects and to manipulate FITS headers * Numdisplay, used to visualize numarray array objects using image display tools like ds9 and ximtool PyRAF changes: ------------- PyRAF v 1.1.2 is released with stsci_python 2.1. The file chooser option in epar was changed to allow operation under both Python 2.2 and 2.3; Tkinter changed the way Tcl variables were passed around from strings to objects. The 'help()' method in 'epar.py' was modified to include the package name with the task name when extracting the help information from IRAF. This avoids picking up the help file for the wrong task when there is a name conflict between tasks from different IRAF packages, such as 'dither.tran' and 'noao.translit'. Python 2.4 changed the signature of the 'checkcache' function in the 'linecache' module and this function was added to traceback calls in Python. The PyRAF overloaded version needed to be modified to be consistent with that new signature in order to avoid crashing out of the command-line interpreter when Python hits an Exception. A Python 2.3 boolean bug was fixed in the IRAF bool class Support was added for extended log axes in plots (type=ELOG). IRAF plots often switch to to this type if there are 0 or negative values in a log plot. This type uses log for values greater than 10, -log10(-x) for negative values less than -10, and x/10 for values between 10 and -10. A bug was fixed wherein PyRAF would crash when exiting from a session initiated without graphics support (-n). numarray changes: ---------------- stsci_python 2.1 includes numarray 1.2.3, replacing numarray 1.1 which was included in stsci_python 2.0. The release notes to the numarray versions between 1.1 and 1.2.3 can be found at http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367 (click on each release number to see the release notes). Notable enhancements: - Support for user defined universal functions with arbitrary numbers of parameters. - Additional modifications to support numarray operation with scipy_base. - Import time roughly halved. - Improved sorting functions. - Simplified array() and fromfile() functions. - Port of Numeric's dotblas extension for faster matrix multiplication when linked against ATLAS, LAPACK, BLAS, etc. - Improvements to array indexing speed MultiDrizzle changes: -------------------- The Multidrizzle software automatically creates cosmic-ray cleaned, geometric distortion corrected, mosaics from multiple input images. This patch to Multidrizzle corrects bugs that have been uncovered since the release of Multidrizzle 2.3.6 with STSDAS 3.3 and stsci_python 2.0. In addition, this release provides new functionality and broader HST instrument support. Bug fixes in this release: ------------------------- The following problems have been identified and corrected in Multidrizzle 2.5.5: - Multidrizzle will no longer delete output files when the parameters 'build=no' and 'clean=yes' are set. - The proper EXPTIME value will be reported with the D00*WTSC keyword. - Shift information provided for WFPC2 data files in the GEIS file format will no longer be discarded when the data files are converted to the multi-extension FITS format during MultiDrizzle processing. - A bug that prevented the use of a user supplied static mask file has been corrected. This fix required changing the format requirements of the user supplied static files to generalize them for use with WFPC2, STIS, and NICMOS instead of just ACS. A description of the static file format requirements can be found in the Multidrizzle help file. - Multidrizzle can now accept NICMOS data in counts or counts per second. Also, the TIME array is no longer used to scale the data on input, only the EXPTIME keyword is used. - A bug that resulted in a context image always being created for the final drizzle step was corrected. Enhancements in this release: ---------------------------- A number of features have been added to MultiDrizzle 2.5.5 to enhance its support for STIS and NICMOS data, add additional data reduction capabilities, and improve the user experience. These new features are enumerated below: - Each input image's MDRIZSKY value is now recorded in the primary header of the final output product in the HISTORY cards. - MultiDrizzle now allows for the use of multi-imset STIS _flt files as input. These files will be automatically split into separate files prior to processing by MultiDrizzle. These files will be named "rootname_flt_extnX.fits" where X represents the original extension number of the input file. - Better error trapping and expanded error messages have been included with this version of MultiDrizzle. We have tried to test for the most common user errors and give messages that explain the problems and, where appropriate, how to correct the errors. - Multidrizzle will not include input images with EXPTIME values of 0.0 in its processing. These images will be removed from the list of input images and a warning message displayed explaining why the images were removed and instructing the user how to correct the problem. If all images are rejected by Multidrizzle for this reason then multidrizzle will print an ERROR message and exit while creating an empty DRZ product. - Multidrizzle now allows NICMOS '_asc.fits' files as input. - Multidrizzle now accepts the use of different BITS values for the single and final drizzle steps. - Multidrizzle will now ignore association table entries with a MEMPRSNT keyword value of 'F'. - A user supplied static mask file will now be applied even if the STATIC MASK step is turned off. In earlier versions of MultiDrizzle a user supplied static mask would not be applied if STATIC MASK was not run. - Multidrizzle will now automatically search for SPT files when a PA_V3 value has not been found in the primary header of input files. The process Multidrizzle now follows is: 1) Check the primary header for a PA_V3 value. If found, run MAKEWCS 2) If the PA_V3 value is not found in the primary header of the input file, guess at what the name of the corresponding SPT file would be. If the SPT file can be found, extract the PA_V3 value from the SPT file and add it to the primary header of the input file. Then run MAKEWCS. 3) If the PA_V3 value cannot be found anywhere, issue an error message explaining why MultiDrizzle cannot run and exit. - An improved method of creating mask files for use in the CREATE MEDIAN step has been implemented. The new method uses a clipped mean instead of a straight mean calculation. Interface changes in this release: --------------------------------- A number of changes have been made in the Multidrizzle interface that will require users to unlearn any existing Multidrizzle parameter files and modify any existing scripts. The interface changes are as follows: - Multidrizzle no longer allows for the implicit use of wild-cards in an input string. This means that a user will need to specify "*flt.fits" instead of "flt.fits" for the input parameter. - We have added separate parameters for the BITS value for the single and final drizzle processing steps. The new parameters are called DRIZ_SEP_BITS and FINAL_BITS. The existing BITS parameter was removed from the interface. This version of MultiDrizzle retains backward compatibility with existing MDRIZTAB files. - The 'wt_scl' parameter has been exposed in both the single and final drizzle steps. The default for this parameter is "EXPTIME". This was done to support the drizzling of NICMOS data. - The default for the runfile parameter is now the empty string ''. If no value is provided for runfile, the default behavior for MultiDrizzle will be to create a runfile name based upon the rootname for the output DRZ product. Comments about this version: --------------------------- This implementation of the Multidrizzle software fully supports all ACS imaging modes. At this time only the most preliminary support exists for WFPC2 observing modes. Support for STIS imaging modes is limited at this time. Full support for STIS imaging data is currently under development. NICMOS is fully supported, but users must copy the PA_V3 keyword from the *_spt.fits file headers into the multidrizzle input file headers manually, and the keyword IDCTAB containg the name of a geometric distortion correction table must be added. IDCTAB files are availble from the NICMOS group at STScI and will be installed in the archive in the near future. Both the IDCTAB and PA_V3 keywords will be added to the headers automatically beginning with the next OPUS release scheuled for early summer 2005. Although every effort was made to test the different modes of MultiDrizzle prior to release, for a software project of this complexity we anticipate that some problems do exist. We strongly encourage users to submit bug reports to firstname.lastname@example.org. PyDrizzle changes: ----------------- PyDrizzle has undergone some modifications to better support MultiDrizzle and related tasks. Some key changes include: - correcting use of 'blot' so that it works properly when the original drizzle products used a non-unity scale, such as WFPC2 data. - expanding the WCSObject class to archive original WCS keyword values and to support writing out a new WCS FITS object. - breaking up the 'bits' parameter into separate parameters for single drizzling and final drizzling; namely, 'bits_single' and 'bits_final', respectively. This allows DQ information to be used for both steps as desired, unlike before, where input DQ arrays were only used for final drizzling. - allowing Python lists of filenames as input, along with a shiftfile. - adding support for NICMOS IDCTAB usage. - properly supporting turning off context image generation for single drizzling, reducing memory usage for that step for large numbers of input images. As a result, PyDrizzle now correctly supports a wider range of datatypes more accurately than ever, especially WFPC2 and NICMOS images. It also provides the user with better control over what masking to use for both single and final drizzling, and over the WCS information being used. Dependency changes in this release: ---------------------------------- - Removed dependency on IRAF environment variable 'stsdas'. It will now default to the current working directory for default coefficients directory if 'stsdas$' is not defined. - Removed all dependencies on DQPars module and classes. - The function 'readShiftFile' was moved from 'buildasn' to fileutil so that other tasks can use this function without also needing to import 'buildasn'. Interface changes in this release: --------------------------------- - Replaced 'bits' parameter with 'bits_final' and 'bits_single'. - EPAR interface change: removed 'use_mask', 'bits' and added 'bits_final', 'bits_single', and 'prodonly'. - Added 'shiftfile' parameter as input parameter. - Added support for Python list of filenames as input. - The behavior of 'idckey' was changed such that a value of ''(blank) will cause PyDrizzle to default to instrument-specific default cases and a value of None will result in no distortion coefficients being applied. - The 'build' attribute was added to the PyDrizzle class so that it can be set when 'run()' method gets called, and used in the 'clean()' method. This allows PyDrizzle to know that it should not delete simple FITS output from drizzle if not combined into a multi-extension output. - The 'getMembers' and 'getMemberNames' methods were added to the Observation classes to more readily support coordinate transformations using PyDrizzle, especially for prototype tweakshifts. - The 'wtraxy' method was modified to work with list input as well as single position input to better support coordinate transformations with PyDrizzle. - Added support for NICMOS IDCTAB usage, including support for quadratic coefficients in IDCTAB. - Modified reading of ASN tables to maintain order of filenames and correctly identify when only 1 exposure gets read from an ASN table. - Correctly support use of 'blot' for images which were drizzled using non-unity scale values; specifically, WFPC2 data. This results in WFPC2 data being properly blotted back to correctly align with the original input data. Parlist changes in this version: ------------------------------- - Added 'single_driz_mask' to the parlist to support separate masks for single and final drizzle processing. - Added 'nimages' to parlist to record the number of files/images combined for this product. Used only to update 'NCOMBINE' if it exists in the SCI header. Functionality changes in this version: ------------------------------------- WCS Module 'wcsutil': - WCS keywords are now archived within the image header allowing them to be updated without losing the original values. Multiple updates of the header will NOT overwrite archived keyword values. Methods were added to support use of the archived values. - Added 'createReferenceWCS' method to WCSObject class to support output of WCS for use as a reference WCS with shiftfiles. Required for 'tweak.py' (tweakshifts) - A bug in the 'rd2xy' method was fixed. - Additional help information has been added to the class level docstring. Mask file usage: - Separate masks are now used for single drizzle and final drizzle processing. The 'single_driz_mask' entry was added to the parlist to support separate masks for single and final drizzle processing. - Problems with building mask files when no DQ array is present in the input image were fixed, so that ACQ or RAW files can be successfully processed if necessary. - Names for the output mask images are built and remembered by PyDrizzle now, regardless of whether they get used. This allows external programs, such as MultiDrizzle, to have a consistent filename to use when it builds its own mask for the input. Operational changes: - Logic for turning off context images during drizzling was modified to work with final drizzling as well as single drizzling. This now supports turning off context image building altogether resulting in the same memory usage for processing 33 or more chips as it would take to drizzle 2 chips. - A bug in computing the RA and Dec values when applying shifts was fixed. This bug caused a problem with the final rotation values when a shiftfile was used. Changes to output: - Filename for the final output product will be derived DIRECTLY from the rootname provided in the ASN table, including retaining the case of the rootname. Previously, all output products used all lower-case filenames. - The Exposure class was modified to not expand DGEOFILE filename when reporting the XGIM and YGIM drizzle parameters. - Primary header output was modified to report NEXTEND == 0 when not building a multi-extension output file. - Output to the 'D0*WTSC' keywords was changed to report value of EXPTIME for wt_scl instead of the string 'exptime' or 'expsq' to be consistent with IRAF 'drizzle' task output. - The NCOMBINE keyword now gets updated with the correct number of chips drizzled into the output image. PyFITS changes: -------------- - Variable length array tables now supported. - Writing ASCII table extensions now supported. - Reading and writing random groups now supported. - Numbers with leading zeros in an ASCII table extension now supported. numdisplay changes: ------------------ Version 0.2.0: The need to explicitly call 'open()' when first starting numdisplay was removed. It will now automatically call 'open()' when this module gets imported, so that numdisplay.display() can be called immediately. The only time the 'open()' method needs to called explicitly now is after the display object was explicitly closed and the frame buffer has been resized. Documentation on the Python API for Numdisplay has been added to the package so that it will be installed alongside the executable code for easier reference. Makewcs (as part of the pytools package) ------- Makewcs was modified to fix a number of bugs and add support for more instruments. Specifically, - Added support for NICMOS images - Revised to properly compute WCS keywords for subarrays - Revised to always start with archived WCS keywords Dependency changes - Requires version of 'wcsutil.py' in PyDrizzle Version 5.4.5 (or greater). Functionality changes - The module was revised to properly compute WCS keywords for subarray images taking into account offsets relative to the full chip and relative to the distortion model reference position. - WCS keyword value input was made more efficient by reading the input header in once into a PyFITS header object and using that throughout the module. - The module was modified to insure that archived keywords are always read in when present or created if not and used in 'read-only' mode. Makewcs also needed to reset WCS values read in to always use the archived values as the starting values for the computations.