Release Notes for stsci_python version 2.1
------------------------------------------
stsci_python is a library of Python routines and C extensions
that has been developed to provide a general astronomical data
analysis infrastructure. It includes the following:
* PyRAF, an environment for running IRAF tasks without using
the IRAF CL
* Numarray, a general array manipulation package
* MultiDrizzle, a task for combining dithered STScI images
* PyDrizzle, used by MultiDrizzle to provide a convenient but
powerful interface to the Drizzle task of Fruchter and Hook
* PyFITS, used to read FITS images and tables into numarray
objects and to manipulate FITS headers
* Numdisplay, used to visualize numarray array objects using
image display tools like ds9 and ximtool
PyRAF changes:
-------------
PyRAF v 1.1.2 is released with stsci_python 2.1.
The file chooser option in epar was changed to allow operation under
both Python 2.2 and 2.3; Tkinter changed the way Tcl variables were
passed around from strings to objects.
The 'help()' method in 'epar.py' was modified to include the package
name with the task name when extracting the help information from
IRAF. This avoids picking up the help file for the wrong task when
there is a name conflict between tasks from different IRAF packages,
such as 'dither.tran' and 'noao.translit'.
Python 2.4 changed the signature of the 'checkcache' function in
the 'linecache' module and this function was added to traceback calls in
Python. The PyRAF overloaded version needed to be modified to be consistent
with that new signature in order to avoid crashing out of the command-line
interpreter when Python hits an Exception.
A Python 2.3 boolean bug was fixed in the IRAF bool class
Support was added for extended log axes in plots (type=ELOG).
IRAF plots often switch to to this type if there are 0 or negative
values in a log plot. This type uses log for values greater than
10, -log10(-x) for negative values less than -10, and x/10 for
values between 10 and -10.
A bug was fixed wherein PyRAF would crash when exiting from a
session initiated without graphics support (-n).
numarray changes:
----------------
stsci_python 2.1 includes numarray 1.2.3, replacing numarray 1.1 which
was included in stsci_python 2.0. The release notes to the numarray
versions between 1.1 and 1.2.3 can be found at
http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=32367
(click on each release number to see the release notes).
Notable enhancements:
- Support for user defined universal functions with arbitrary numbers
of parameters.
- Additional modifications to support numarray operation with scipy_base.
- Import time roughly halved.
- Improved sorting functions.
- Simplified array() and fromfile() functions.
- Port of Numeric's dotblas extension for faster matrix
multiplication when linked against ATLAS, LAPACK, BLAS, etc.
- Improvements to array indexing speed
MultiDrizzle changes:
--------------------
The Multidrizzle software automatically creates cosmic-ray cleaned,
geometric distortion corrected, mosaics from multiple input images.
This patch to Multidrizzle corrects bugs that have been uncovered
since the release of Multidrizzle 2.3.6 with STSDAS 3.3 and
stsci_python 2.0. In addition, this release provides new
functionality and broader HST instrument support.
Bug fixes in this release:
-------------------------
The following problems have been identified and corrected in
Multidrizzle 2.5.5:
- Multidrizzle will no longer delete output files when the
parameters 'build=no' and 'clean=yes' are set.
- The proper EXPTIME value will be reported with the D00*WTSC
keyword.
- Shift information provided for WFPC2 data files in the GEIS
file format will no longer be discarded when the data files
are converted to the multi-extension FITS format during
MultiDrizzle processing.
- A bug that prevented the use of a user supplied static mask
file has been corrected. This fix required changing the
format requirements of the user supplied static files to
generalize them for use with WFPC2, STIS, and NICMOS instead
of just ACS. A description of the static file format
requirements can be found in the Multidrizzle help file.
- Multidrizzle can now accept NICMOS data in counts or counts
per second. Also, the TIME array is no longer used to scale
the data on input, only the EXPTIME keyword is used.
- A bug that resulted in a context image always being created
for the final drizzle step was corrected.
Enhancements in this release:
----------------------------
A number of features have been added to MultiDrizzle 2.5.5 to
enhance its support for STIS and NICMOS data, add additional
data reduction capabilities, and improve the user experience.
These new features are enumerated below:
- Each input image's MDRIZSKY value is now recorded in the
primary header of the final output product in the HISTORY cards.
- MultiDrizzle now allows for the use of multi-imset STIS _flt
files as input. These files will be automatically split into
separate files prior to processing by MultiDrizzle. These files
will be named "rootname_flt_extnX.fits" where X represents the
original extension number of the input file.
- Better error trapping and expanded error messages have been
included with this version of MultiDrizzle. We have tried
to test for the most common user errors and give messages that
explain the problems and, where appropriate, how to correct
the errors.
- Multidrizzle will not include input images with EXPTIME
values of 0.0 in its processing. These images will be removed
from the list of input images and a warning message displayed
explaining why the images were removed and instructing the
user how to correct the problem. If all images are rejected
by Multidrizzle for this reason then multidrizzle will print
an ERROR message and exit while creating an empty DRZ product.
- Multidrizzle now allows NICMOS '_asc.fits' files as input.
- Multidrizzle now accepts the use of different BITS values
for the single and final drizzle steps.
- Multidrizzle will now ignore association table entries with a
MEMPRSNT keyword value of 'F'.
- A user supplied static mask file will now be applied even
if the STATIC MASK step is turned off. In earlier versions
of MultiDrizzle a user supplied static mask would not
be applied if STATIC MASK was not run.
- Multidrizzle will now automatically search for SPT files
when a PA_V3 value has not been found in the primary header
of input files. The process Multidrizzle now follows is:
1) Check the primary header for a PA_V3 value. If found,
run MAKEWCS
2) If the PA_V3 value is not found in the primary header of
the input file, guess at what the name of the corresponding
SPT file would be. If the SPT file can be found, extract
the PA_V3 value from the SPT file and add it to the primary
header of the input file. Then run MAKEWCS.
3) If the PA_V3 value cannot be found anywhere, issue an
error message explaining why MultiDrizzle cannot run and exit.
- An improved method of creating mask files for use in the
CREATE MEDIAN step has been implemented. The new method uses
a clipped mean instead of a straight mean calculation.
Interface changes in this release:
---------------------------------
A number of changes have been made in the Multidrizzle interface
that will require users to unlearn any existing Multidrizzle
parameter files and modify any existing scripts. The interface
changes are as follows:
- Multidrizzle no longer allows for the implicit use of
wild-cards in an input string. This means that a user will
need to specify "*flt.fits" instead of "flt.fits" for the
input parameter.
- We have added separate parameters for the BITS value for
the single and final drizzle processing steps. The new
parameters are called DRIZ_SEP_BITS and FINAL_BITS. The
existing BITS parameter was removed from the interface.
This version of MultiDrizzle retains backward compatibility
with existing MDRIZTAB files.
- The 'wt_scl' parameter has been exposed in both the single
and final drizzle steps. The default for this parameter is
"EXPTIME". This was done to support the drizzling of
NICMOS data.
- The default for the runfile parameter is now the empty
string ''. If no value is provided for runfile, the default
behavior for MultiDrizzle will be to create a runfile name
based upon the rootname for the output DRZ product.
Comments about this version:
---------------------------
This implementation of the Multidrizzle software fully supports
all ACS imaging modes. At this time only the most preliminary
support exists for WFPC2 observing modes. Support for STIS
imaging modes is limited at this time. Full support for STIS
imaging data is currently under development. NICMOS is fully
supported, but users must copy the PA_V3 keyword from the
*_spt.fits file headers into the multidrizzle input file
headers manually, and the keyword IDCTAB containg the name of
a geometric distortion correction table must be added. IDCTAB
files are availble from the NICMOS group at STScI and will be
installed in the archive in the near future. Both the IDCTAB
and PA_V3 keywords will be added to the headers automatically
beginning with the next OPUS release scheuled for early summer
2005.
Although every effort was made to test the different modes of
MultiDrizzle prior to release, for a software project of this
complexity we anticipate that some problems do exist. We
strongly encourage users to submit bug reports to help@stsci.edu.
PyDrizzle changes:
-----------------
PyDrizzle has undergone some modifications to better support
MultiDrizzle and related tasks. Some key changes include:
- correcting use of 'blot' so that it works properly when the
original drizzle products used a non-unity scale, such as
WFPC2 data.
- expanding the WCSObject class to archive original WCS
keyword values and to support writing out a new WCS FITS
object.
- breaking up the 'bits' parameter into separate parameters
for single drizzling and final drizzling; namely,
'bits_single' and 'bits_final', respectively. This allows
DQ information to be used for both steps as desired, unlike
before, where input DQ arrays were only used for final
drizzling.
- allowing Python lists of filenames as input, along with a
shiftfile.
- adding support for NICMOS IDCTAB usage.
- properly supporting turning off context image generation
for single drizzling, reducing memory usage for that step
for large numbers of input images.
As a result, PyDrizzle now correctly supports a wider range of
datatypes more accurately than ever, especially WFPC2 and NICMOS
images. It also provides the user with better control over what
masking to use for both single and final drizzling, and over the
WCS information being used.
Dependency changes in this release:
----------------------------------
- Removed dependency on IRAF environment variable 'stsdas'.
It will now default to the current working directory for
default coefficients directory if 'stsdas$' is not defined.
- Removed all dependencies on DQPars module and classes.
- The function 'readShiftFile' was moved from 'buildasn' to
fileutil so that other tasks can use this function without
also needing to import 'buildasn'.
Interface changes in this release:
---------------------------------
- Replaced 'bits' parameter with 'bits_final' and 'bits_single'.
- EPAR interface change: removed 'use_mask', 'bits' and added
'bits_final', 'bits_single', and 'prodonly'.
- Added 'shiftfile' parameter as input parameter.
- Added support for Python list of filenames as input.
- The behavior of 'idckey' was changed such that a value of
''(blank) will cause PyDrizzle to default to instrument-specific
default cases and a value of None will result in no distortion
coefficients being applied.
- The 'build' attribute was added to the PyDrizzle class so
that it can be set when 'run()' method gets called, and used
in the 'clean()' method. This allows PyDrizzle to know that
it should not delete simple FITS output from drizzle if not
combined into a multi-extension output.
- The 'getMembers' and 'getMemberNames' methods were added
to the Observation classes to more readily support coordinate
transformations using PyDrizzle, especially for prototype
tweakshifts.
- The 'wtraxy' method was modified to work with list input as
well as single position input to better support coordinate
transformations with PyDrizzle.
- Added support for NICMOS IDCTAB usage, including support for
quadratic coefficients in IDCTAB.
- Modified reading of ASN tables to maintain order of filenames
and correctly identify when only 1 exposure gets read from an
ASN table.
- Correctly support use of 'blot' for images which were drizzled
using non-unity scale values; specifically, WFPC2 data. This
results in WFPC2 data being properly blotted back to correctly
align with the original input data.
Parlist changes in this version:
-------------------------------
- Added 'single_driz_mask' to the parlist to support separate
masks for single and final drizzle processing.
- Added 'nimages' to parlist to record the number of
files/images combined for this product. Used only to update
'NCOMBINE' if it exists in the SCI header.
Functionality changes in this version:
-------------------------------------
WCS Module 'wcsutil':
- WCS keywords are now archived within the image header allowing
them to be updated without losing the original values. Multiple
updates of the header will NOT overwrite archived keyword
values. Methods were added to support use of the archived values.
- Added 'createReferenceWCS' method to WCSObject class to support
output of WCS for use as a reference WCS with shiftfiles.
Required for 'tweak.py' (tweakshifts)
- A bug in the 'rd2xy' method was fixed.
- Additional help information has been added to the class level
docstring.
Mask file usage:
- Separate masks are now used for single drizzle and final drizzle
processing. The 'single_driz_mask' entry was added to the
parlist to support separate masks for single and final drizzle
processing.
- Problems with building mask files when no DQ array is present
in the input image were fixed, so that ACQ or RAW files can be
successfully processed if necessary.
- Names for the output mask images are built and remembered by
PyDrizzle now, regardless of whether they get used. This allows
external programs, such as MultiDrizzle, to have a consistent
filename to use when it builds its own mask for the input.
Operational changes:
- Logic for turning off context images during drizzling was
modified to work with final drizzling as well as single drizzling.
This now supports turning off context image building altogether
resulting in the same memory usage for processing 33 or more chips
as it would take to drizzle 2 chips.
- A bug in computing the RA and Dec values when applying shifts was
fixed. This bug caused a problem with the final rotation values
when a shiftfile was used.
Changes to output:
- Filename for the final output product will be derived DIRECTLY
from the rootname provided in the ASN table, including retaining
the case of the rootname. Previously, all output products used
all lower-case filenames.
- The Exposure class was modified to not expand DGEOFILE filename
when reporting the XGIM and YGIM drizzle parameters.
- Primary header output was modified to report NEXTEND == 0
when not building a multi-extension output file.
- Output to the 'D0*WTSC' keywords was changed to report value
of EXPTIME for wt_scl instead of the string 'exptime' or 'expsq'
to be consistent with IRAF 'drizzle' task output.
- The NCOMBINE keyword now gets updated with the correct number of
chips drizzled into the output image.
PyFITS changes:
--------------
- Variable length array tables now supported.
- Writing ASCII table extensions now supported.
- Reading and writing random groups now supported.
- Numbers with leading zeros in an ASCII table extension now supported.
numdisplay changes:
------------------
Version 0.2.0:
The need to explicitly call 'open()' when first starting numdisplay
was removed. It will now automatically call 'open()' when this module
gets imported, so that numdisplay.display() can be called immediately.
The only time the 'open()' method needs to called explicitly now is
after the display object was explicitly closed and the frame buffer
has been resized.
Documentation on the Python API for Numdisplay has been added to the
package so that it will be installed alongside the executable code for
easier reference.
Makewcs (as part of the pytools package)
-------
Makewcs was modified to fix a number of bugs and add support for
more instruments. Specifically,
- Added support for NICMOS images
- Revised to properly compute WCS keywords for subarrays
- Revised to always start with archived WCS keywords
Dependency changes
- Requires version of 'wcsutil.py' in PyDrizzle Version 5.4.5
(or greater).
Functionality changes
- The module was revised to properly compute WCS keywords for subarray
images taking into account offsets relative to the full chip and
relative to the distortion model reference position.
- WCS keyword value input was made more efficient by reading the
input header in once into a PyFITS header object and using that
throughout the module.
- The module was modified to insure that archived keywords are
always read in when present or created if not and used in 'read-only'
mode. Makewcs also needed to reset WCS values read in to always
use the archived values as the starting values for the computations.
