STScI Logo

Release Notes for stsci_python
stsci_python 2.1 Release Notes

  Release Notes for stsci_python version 2.1
  ------------------------------------------

  stsci_python is a library of Python routines and C extensions
that has been developed to provide a general astronomical data
analysis infrastructure. It includes the following:

    * PyRAF, an environment for running IRAF tasks without using
      the IRAF CL
    * Numarray, a general array manipulation package
    * MultiDrizzle, a task for combining dithered STScI images
    * PyDrizzle, used by MultiDrizzle to provide a convenient but
      powerful interface to the Drizzle task of Fruchter and Hook
    * PyFITS, used to read FITS images and tables into numarray
      objects and to manipulate FITS headers
    * Numdisplay, used to visualize numarray array objects using
      image display tools like ds9 and ximtool

PyRAF changes:
-------------

  PyRAF v 1.1.2 is released with stsci_python 2.1.
  The file chooser option in epar was changed to allow operation under
both Python 2.2 and 2.3; Tkinter changed the way Tcl variables were
passed around from strings to objects. 

The 'help()' method in 'epar.py' was modified to include the package
name with the task name when extracting the help information from
IRAF.  This avoids picking up the help file for the wrong task when
there is a name conflict between tasks from different IRAF packages,
such as 'dither.tran' and 'noao.translit'.

Python 2.4 changed the signature of the 'checkcache' function in
the 'linecache' module and this function was added to traceback calls in
Python.  The PyRAF overloaded version needed to be modified to be consistent
with that new signature in order to avoid crashing out of the command-line
interpreter when Python hits an Exception.

A Python 2.3 boolean bug was fixed in the IRAF bool class

Support was added for extended log axes in plots (type=ELOG).
IRAF plots often switch to to this type if there are 0 or negative
values in a log plot. This type uses log for values greater than
10, -log10(-x) for negative values less than -10, and x/10 for
values between 10 and -10.

A bug was fixed wherein PyRAF would crash when exiting from a
session initiated without graphics support (-n).

numarray changes:
----------------

stsci_python 2.1 includes numarray 1.2.3, replacing numarray 1.1 which
was included in stsci_python 2.0.  The release notes to the numarray
versions between 1.1 and 1.2.3 can be found at

http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367

(click on each release number to see the release notes).

Notable enhancements:

 - Support for user defined universal functions with arbitrary numbers
   of parameters.

 - Additional modifications to support numarray operation with scipy_base.

 - Import time roughly halved.

 - Improved sorting functions.

 - Simplified array() and fromfile() functions.

 - Port of Numeric's dotblas extension for faster matrix
   multiplication when linked against ATLAS, LAPACK, BLAS, etc.

 - Improvements to array indexing speed
   
MultiDrizzle changes:
--------------------
 
  The Multidrizzle software automatically creates cosmic-ray cleaned, 
  geometric distortion corrected, mosaics from multiple input images.
  This patch to Multidrizzle corrects bugs that have been uncovered
  since the release of Multidrizzle 2.3.6 with STSDAS 3.3 and
  stsci_python 2.0. In addition, this release provides new
  functionality and broader HST instrument support.

  Bug fixes in this release:
  -------------------------

  The following problems have been identified and corrected in
  Multidrizzle 2.5.5:

  - Multidrizzle will no longer delete output files when the
    parameters 'build=no' and 'clean=yes' are set.

  - The proper EXPTIME value will be reported with the D00*WTSC
    keyword.

  - Shift information provided for WFPC2 data files in the GEIS
    file format will no longer be discarded when the data files
    are converted to the multi-extension FITS format during
    MultiDrizzle processing.

  - A bug that prevented the use of a user supplied static mask
    file has been corrected.  This fix required changing the
    format requirements of the user supplied static files to
    generalize them for use with WFPC2, STIS, and NICMOS instead
    of just ACS.  A description of the static file format
    requirements can be found in the Multidrizzle help file.

  - Multidrizzle can now accept NICMOS data in counts or counts
    per second.  Also, the TIME array is no longer used to scale
    the data on input, only the EXPTIME keyword is used.

  - A bug that resulted in a context image always being created
    for the final drizzle step was corrected.

  Enhancements in this release:
  ----------------------------

  A number of features have been added to MultiDrizzle 2.5.5 to
  enhance its support for STIS and NICMOS data, add additional
  data reduction capabilities, and improve the user experience.
  These new features are enumerated below:

  - Each input image's MDRIZSKY value is now recorded in the
    primary header of the final output product in the HISTORY cards.

  - MultiDrizzle now allows for the use of multi-imset STIS _flt
    files as input.  These files will be automatically split into
    separate files prior to processing by MultiDrizzle.  These files
    will be named "rootname_flt_extnX.fits" where X represents the
    original extension number of the input file.

  - Better error trapping and expanded error messages have been
    included with this version of MultiDrizzle.  We have tried
    to test for the most common user errors and give messages that
    explain the problems and, where appropriate, how to correct
    the errors.

  - Multidrizzle will not include input images with EXPTIME
    values of 0.0 in its processing. These images will be removed
    from the list of input images and a warning message displayed
    explaining why the images were removed and instructing the
    user how to correct the problem.  If all images are rejected
    by Multidrizzle for this reason then multidrizzle will print
    an ERROR message and exit while creating an empty DRZ product.

  - Multidrizzle now allows NICMOS '_asc.fits' files as input.
 
  - Multidrizzle now accepts the use of different BITS values
    for the single and final drizzle steps.
 
  - Multidrizzle will now ignore association table entries with a
    MEMPRSNT keyword value of 'F'.

  - A user supplied static mask file will now be applied even
    if the STATIC MASK step is turned off.  In earlier versions
    of MultiDrizzle a user supplied static mask would not
    be applied if STATIC MASK was not run.

  - Multidrizzle will now automatically search for SPT files
    when a PA_V3 value has not been found in the primary header
    of input files.  The process Multidrizzle now follows is:
    1) Check the primary header for a PA_V3 value.  If found,
       run MAKEWCS
    2) If the PA_V3 value is not found in the primary header of
       the input file, guess at what the name of the corresponding
       SPT file would be.  If the SPT file can be found, extract
       the PA_V3 value from the SPT file and add it to the primary
       header of the input file.  Then run MAKEWCS.
    3) If the PA_V3 value cannot be found anywhere, issue an
       error message explaining why MultiDrizzle cannot run and exit.

  - An improved method of creating mask files for use in the
    CREATE MEDIAN step has been implemented.  The new method uses
    a clipped mean instead of a straight mean calculation.

  Interface changes in this release:
  ---------------------------------

  A number of changes have been made in the Multidrizzle interface
  that will require users to unlearn any existing Multidrizzle
  parameter files and modify any existing scripts. The interface
  changes are as follows:
  
  - Multidrizzle no longer allows for the implicit use of
    wild-cards in an input string.  This means that a user will
    need to specify "*flt.fits" instead of "flt.fits" for the
    input parameter.  
    
  - We have added separate parameters for the BITS value for
    the single and final drizzle processing steps.  The new
    parameters are called DRIZ_SEP_BITS and FINAL_BITS.  The
    existing BITS parameter was removed from the interface.
    This version of MultiDrizzle retains backward compatibility
    with existing MDRIZTAB files.  

  - The 'wt_scl' parameter has been exposed in both the single
    and final drizzle steps. The default for this parameter is
    "EXPTIME".  This was done to support the drizzling of
    NICMOS data.

  - The default for the runfile parameter is now the empty
    string ''.  If no value is provided for runfile, the default
    behavior for MultiDrizzle will be to create a runfile name 
    based upon the rootname for the output DRZ product.

  Comments about this version:
  ---------------------------

  This implementation of the Multidrizzle software fully supports
  all ACS imaging modes.  At this time only the most preliminary
  support exists for WFPC2 observing modes.  Support for STIS
  imaging modes is limited at this time.  Full support for STIS
  imaging data is currently under development.  NICMOS is fully
  supported, but users must copy the PA_V3 keyword from the
  *_spt.fits file headers into the multidrizzle input file
  headers manually, and the keyword IDCTAB containg the name of
  a geometric distortion correction table must be added. IDCTAB
  files are availble from the NICMOS group at STScI and will be
  installed in the archive in the near future.  Both the IDCTAB
  and PA_V3 keywords will  be added to the headers automatically
  beginning with the next OPUS release scheuled for early summer
  2005.

  Although every effort was made to test the different modes of
  MultiDrizzle prior to release, for a software project of this
  complexity we anticipate that some problems do exist.  We
  strongly encourage users to submit bug reports to help@stsci.edu.

PyDrizzle changes:
-----------------

  PyDrizzle has undergone some modifications to better support
MultiDrizzle and related tasks.  Some key changes include:

    - correcting use of 'blot' so that it works properly when the
      original drizzle products used a non-unity scale, such as
      WFPC2 data.
            
    - expanding the WCSObject class to archive original WCS
      keyword values and to support writing out a new WCS FITS
      object. 
        
    - breaking up the 'bits' parameter into separate parameters
      for single drizzling and final drizzling; namely,
      'bits_single' and 'bits_final', respectively. This allows
      DQ information to be used for both steps as desired, unlike
      before, where input DQ arrays were only used for final
      drizzling.
        
    - allowing Python lists of filenames as input, along with a
      shiftfile.
        
    - adding support for NICMOS IDCTAB usage.
        
    - properly supporting turning off context image generation
      for single drizzling, reducing memory usage for that step
      for large numbers of input images.
        
  As a result, PyDrizzle now correctly supports a wider range of
datatypes more accurately than ever, especially WFPC2 and NICMOS
images. It also provides the user with better control over what
masking to use for both single and final drizzling, and over the
WCS information being used.  
        
  Dependency changes in this release:
  ----------------------------------

    - Removed dependency on IRAF environment variable 'stsdas'.
      It will now default to the current working directory for
      default coefficients directory if 'stsdas$' is not defined.

    - Removed all dependencies on DQPars module and classes.
      
    - The function 'readShiftFile' was moved from 'buildasn' to
      fileutil so that other tasks can use this function without
      also needing to import 'buildasn'.
        
  Interface changes in this release:
  ---------------------------------

    - Replaced 'bits' parameter with 'bits_final' and 'bits_single'. 
    - EPAR interface change: removed 'use_mask', 'bits' and added
      'bits_final', 'bits_single', and 'prodonly'. 
    - Added 'shiftfile' parameter as input parameter.
    - Added support for Python list of filenames as input.
    - The behavior of 'idckey' was changed such that a value of
      ''(blank) will cause PyDrizzle to default to instrument-specific
      default cases and a value of None will result in no distortion
      coefficients being applied.
    - The 'build' attribute was added to the PyDrizzle class so
      that it can be set when 'run()' method gets called, and used
      in the 'clean()' method. This allows PyDrizzle to know that
      it should not delete simple FITS output from drizzle if not
      combined into a multi-extension output.
    - The 'getMembers' and 'getMemberNames' methods were added
      to the Observation classes to more readily support coordinate
      transformations using PyDrizzle, especially for prototype
      tweakshifts.
    - The 'wtraxy' method was modified to work with list input as
      well as single position input to better support coordinate
      transformations with PyDrizzle.

    - Added support for NICMOS IDCTAB usage, including support for
      quadratic coefficients in IDCTAB.
    - Modified reading of ASN tables to maintain order of filenames
      and correctly identify when only 1 exposure gets read from an
      ASN table.  
    - Correctly support use of 'blot' for images which were drizzled 
      using non-unity scale values; specifically, WFPC2 data. This
      results in WFPC2 data being properly blotted back to correctly
      align with the original input data.

  Parlist changes in this version:
  -------------------------------

    - Added 'single_driz_mask' to the parlist to support separate
      masks for single and final drizzle processing.
    - Added 'nimages' to parlist to record the number of
      files/images combined for this product. Used only to update
      'NCOMBINE' if it exists in the SCI header.
            
  Functionality changes in this version:
  -------------------------------------

    WCS Module 'wcsutil':

    - WCS keywords are now archived within the image header allowing
      them to be updated without losing the original values. Multiple
      updates of the header will NOT overwrite archived keyword
      values. Methods were added to support use of the archived values.
 
    - Added 'createReferenceWCS' method to WCSObject class to support
      output of WCS for use as a reference WCS with shiftfiles.
      Required for 'tweak.py' (tweakshifts)

    - A bug in the 'rd2xy' method was fixed.

    - Additional help information has been added to the class level
      docstring.
        
    Mask file usage:

    - Separate masks are now used for single drizzle and final drizzle
      processing. The 'single_driz_mask' entry was added to the
      parlist to support separate masks for single and final drizzle
      processing.

    - Problems with building mask files when no DQ array is present
      in the input image were fixed, so that ACQ or RAW files can be 
      successfully processed if necessary.

    - Names for the output mask images are built and remembered by 
      PyDrizzle now, regardless of whether they get used.  This allows
      external programs, such as MultiDrizzle, to have a consistent 
      filename to use when it builds its own mask for the input.

    Operational changes:
        
    - Logic for turning off context images during drizzling was
      modified to work with final drizzling as well as single drizzling.
      This now supports turning off context image building altogether
      resulting in the same memory usage for processing 33 or more chips 
      as it would take to drizzle 2 chips. 

    - A bug in computing the RA and Dec values when applying shifts was
      fixed.  This bug caused a problem with the final rotation values
      when a shiftfile was used.
         
    Changes to output:
                
    - Filename for the final output product will be derived DIRECTLY
      from the rootname provided in the ASN table, including retaining
      the case of the rootname. Previously, all output products used
      all lower-case filenames.

    - The Exposure class was modified to not expand DGEOFILE filename
      when reporting the XGIM and YGIM drizzle parameters. 

    - Primary header output was modified to report NEXTEND == 0 
      when not building a multi-extension output file. 

    - Output to the 'D0*WTSC' keywords was changed to report value
      of EXPTIME for wt_scl instead of the string 'exptime' or 'expsq'
      to be consistent with IRAF 'drizzle' task output.

    - The NCOMBINE keyword now gets updated with the correct number of 
      chips drizzled into the output image.

PyFITS changes:
--------------

 - Variable length array tables now supported.
 - Writing ASCII table extensions now supported.
 - Reading and writing random groups now supported.
 - Numbers with leading zeros in an ASCII table extension now supported.

numdisplay changes:
------------------

Version 0.2.0:

    The need to explicitly call 'open()' when first starting numdisplay
    was removed. It will now automatically call 'open()' when this module
    gets imported, so that numdisplay.display() can be called immediately.
    
    The only time the 'open()' method needs to called explicitly now is
    after the display object was explicitly closed and the frame buffer
    has been resized.
    
    Documentation on the Python API for Numdisplay has been added to the
    package so that it will be installed alongside the executable code for
    easier reference.

Makewcs (as part of the pytools package)
-------

  Makewcs was modified to fix a number of bugs and add support for 
more instruments. Specifically,

    - Added support for NICMOS images
    - Revised to properly compute WCS keywords for subarrays
    - Revised to always start with archived WCS keywords
    

  Dependency changes

    - Requires version of 'wcsutil.py' in PyDrizzle Version 5.4.5
      (or greater).

  Functionality changes

    - The module was revised to properly compute WCS keywords for subarray
      images taking into account offsets relative to the full chip and
      relative to the distortion model reference position.

    - WCS keyword value input was made more efficient by reading the 
      input header in once into a PyFITS header object and using that
      throughout the module.

    - The module was modified to insure that archived keywords are
      always read in when present or created if not and used in 'read-only'
      mode.  Makewcs also needed to reset WCS values read in to always
      use the archived values as the starting values for the computations.