STScI Logo

Release Notes for stsci_python
stsci_python 2.9 Release Notes

STScI_Python Version 2.9 Release Notes

November 2009


This release includes new versions of PyRAF, PyFITS, pysynphot, pytools,
Multidrizzle, and related tasks.  This release also includes more
instrument-specific packages to support WFPC2, NICMOS, STIS and COS.

The convolve, image, and ndimage packages have been included with
this release to insure compatibility with the rest of our code.

Platform Support 
================
Normally, platform support issues do not apply to Python tasks,
as most Python code will run on all platforms on which Python has
been installed. This distribution was tested to correctly support
installation on Linux, Mac OS X 10.5 (Leopard), and Solaris, while
also being provided for installation under Windows. The single
exception is that the text-based epar functionality now available
in PyRAF (in addition to the already existing GUI-based epar) is
not available under Solaris, and likely will never be. In addition,
PyRAF currently requires the installation of IRAF, yet the only
IRAF distribution for Windows runs under Cygwin and no testing of
PyRAF has been conducted under Cygwin.
 
Documentation
=============
Documentation for these tasks has been consolidated into a
single location complete with an index viewable as a local web
page. The documentation includes any available user guides and
API documentation generated using 'EPYDOC' for the modules or
packages. This index can be accessed using:
 
 --> import stscidocs
 --> stscidocs.viewdocs()
 --> stscidocs.help()
 
This will automatically bring up the default web browser application
to display the index page. All the available documentation for
software packaged with this distribution can be accessed through this
interface, as this documentation resides locally upon installation.


Python Environment
================== 
This release has been tested using Python 2.5.4 and requires
numpy 1.3.  This release may work on versions of Python as old as
Python 2.4 depending on the module being used. 

Future Division support
-----------------------
    - Future division support was added to numerous routines in preparation
    for Python 2.6 to 3.0 transition. Packages updated to use the
    new division include:
    + CALCOS
    + NICTOOLS modules CalTempFromBias, finesky, nic_rem_persist
        (and supporting utility modules)
    + PyRAF
    + pysynphot
    + imagestats
    + WFPC2TOOLS module wfpc2destreak


PyRAF Version 1.8.1
-------------------
The PyRAF v1.8 series supports Python 2.6.  Since the last full
release of PyRAF (v1.8), the following enhancements have been made:

    - Process caching was made more flexible with the addition of
    the prcacheOff() and prcacheOn() functions.  The former sets
    the process cache size to 0 and flushes it, while the latter
    re-enables caching. (ticket #95)

    - On OSX the user might have a Python build which is linked with
    X11, and/or the user might have a Python build which is linked
    with native graphics (Cocoa/Aqua).  For the former case, PyRAF
    needs to load an X11 shared object.  For the latter case, PyRAF
    needs to load an Aqua module.  PyRAF will now attempt to determine
    upon startup which is the case, and load the appropriate item.
    This affects EPAR and all graphics windows. (r1063, see ticket #86)

    - To the Python API (iraf.task()) was added the ability to easily
    send multi-line CL strings to be executed.  Currently, this is
    most useful in regression testing PyRAF. (ticket #104)

    - In an attempt to start moving the codebase toward Python 3
    capability, future division is now in use in all PyRAF modules
    as well as those relied upon in PyTools. (ticket #102)

    - For code maintainability a large set of functions in
    iraffunctions.py had repeated code (related to I/O redirection)
    removed/consolidated via the use of decorators. (ticket #106)

And the following bugs have been fixed:

    - PyRAF started with the '-s' option is now correctly silent
    as advertised.  See 'pyraf -h' for a list of command-line
    options. (ticket #97)

    - Mixed-casing in task names was causing problems for the
    command-line help system. (r1059)

    - A warning-generating incompatibility with matplotlib 0.99.1
    was fixed. (r1077)
    

PyFITS Version 2.2.2
--------------------
Updates described in this release are only supported in the NUMPY
version of pyfits and include all revisions and bug fixes implemented
since the last public release of STScI_Python which included PyFITS
Version 2.1.1 (22-April-2009).

The following enhancements were made:

    - Provide support for the FITS Checksum Keyword Convention.

      * Adding the checksum=True keyword argument to the open
          convenience function will cause checksums to be verified
          on file open.

            hdul=pyfits.open('in.fits',checksum=True)

      * On output, CHECKSUM and DATASUM cards may be output to
          all HDU's in a fits file by using the keyword argument
          checksum=True in calls to the writeto convenience function,
          the HDUList.writeto method, the writeto methods of all of
          the HDU classes, and the append convenience function.

            hdul.writeto('out.fits',checksum=True)

    - Implemented a new insert method to the HDUList class that
        allows for the insertion of a HDU into a HDUList at a given
        index.

         hdul.insert(2,hdu)

    - Provided the capability to handle unicode input for file names.

    - Provided support for integer division required by Python 3.0. 

The following bugs were fixed:

    - Corrected a bug that caused an exception to be raised when
    creating a CompImageHDU using an initial header that does not
    match the image data in terms of the number of axes.

    - Corrected a bug that prevented the opening of a fits file
    where a header contained a CHECKSUM card but no DATASUM card.

    - Corrected a bug that caused NULLs to be written instead of
    blanks when an ASCII table was created using a numpy chararray in
    which the original data contained trailing blanks. 

    - Corrected a bug that caused an index out of bounds exception
    to be raised when iterating over the rows of a binary table
    HDU using the syntax " for row in tbhdu.data: ". 

    - Corrected a bug that prevented the use of the writeto
    convenience function for writing table data to a
    file. 

    - Modified the code to raise an IOError exception with the
    comment "Header missing END card." when pyfits can't find a
    valid END card for a header when opening a file.

        * This change addressed a problem with a non-standard fits
        file that contained several new-line characters at the end
        of each header and at the end of the file. However, since
        some people want to be able to open these non-standard
        files anyway, an option was added to the open convenience
        function to allow these files to be opened without exception.

          pyfits.open('infile.fits',ignore_missing_end=True)

    - Corrected a bug that prevented the use of StringIO objects
    as fits files when reading and writing table data. Previously,
    only image data was supported. 

    - Corrected a bug that caused a bus error to be generated
    when compressing image data using GZIP_1 under the Solaris
    operating system.

    - Corrected bugs that prevented pyfits from properly reading
    Random Groups HDU's using numpy. 

    - Corrected a bug that can occur when writing a fits
    file. 

        * If no default SIGINT signal handler has been assigned,
        before the write, a TypeError exception is raised in the
        _File.flush() method when attempting to return the signal
        handler to its previous state. Notably this occurred when
        using mod_python. The code was changed to use SIG_DFL when
        no old handler was defined.

    - Corrected a bug in CompImageHDU that prevented rescaling the
    image data using hdu.scale(option='old').


pysynphot Version 0.7
---------------------

This release includes version 0.7 of pysynphot, which is presently being 
developed as a successor to STSDAS.SYNPHOT. It is not a simple
port into Python, but provides improved algorithms that address known
shortcomings in SYNPHOT. It does not require IRAF, and is suitable
for either library or interactive use.

Version 0.7 implements much of the core SYNPHOT functionality, and has 
been validated by an extensive commissioning process described in 
Technical Software Report 2009-01. User documentation is still being
developed, but preliminary versions are available, including a guide to 
pysynphot for SYNPHOT users. 

Once the core SYNPHOT functionality is supported (presently planned for 
Spring 2010), SYNPHOT will be formally deprecated.

Preliminary documentation for pysynphot can be obtained at:
  
     http://stsdas.stsci.edu/pysynphot

============
Applications
============
    
PyDrizzle Version 6.3.2
-----------------------
    - 'section' parameter now supports multiple groups to be specified

    - Updates were made to the Fortran 'drizzle' code to correctly
        scale the photometry when using SIP coefficients.
    
MultiDrizzle v3.3.5
-------------------
    - changes to support multiple chip specification for the
        'group' paramater.

    - supports WFC3 data processing

    - Fixed problems with updating PHOTFLAM, BUNIT keywords in
        image headers

    - updated to handle proc_unit settings more robustly; namely,
        converts all inputs to electrons, including some parameters,
        then converts output to user specified units only in the
        final drizzle step.

    - logic problems dealing with image iterator were resolved by
        removing duplicate logic from multidrizzle and computing
        buffer sizes correctly in the first place.

Pytools
-------
    - NMPFIT: fixed a problem with constrained parameters

    - FILEUTIL: parseExtn() now returns the Primary extension if no 
        extension is specified on input

    - MAKEWCS Version 1.1.4:
        + supports binned data correctly 
        + corrected a problem with how to define the 
            reference chip for WFC3/UVIS data

    - The set of modules in Pytools which are used by PyRAF have been
    converted over to Python 3 "future" division. (r8085/87/88/90,
    r8114)

    - Logic was added to the parameter interface tool (TEAL) so that
    it more accurately determines on the fly whether or not the Python
    executable in used is linked with X11.  (r8295)

    - TEAL was not correctly handling the case of tasks located inside
    of larger primary tasks.  Also, exception handling in this area
    of the code was enhanced for better error messaging.  (ticket #434)

    - TEAL was changed so that the INDEF value is handled elegantly
    during validation.  (ticket #432)
    
Numdisplay v.1.5.5
------------------
    - A new parameter, quiet, was added to the display() method to
    turn off messages to STDOUT.

CALCOS v2.11b
-------------
    - This release includes many updates, bug fixes and algorithm
        changes from the last released version of CALCOS (V2.8a). The
        primary changes include:

    - Significant changes were made to wavecal processing.
        For NUV data, the shift of each stripe will be found
        separately, instead of adding all the stripes together
        and finding the global shift.  The shift in the dispersion
        direction is found by minimizing chi square, rather than
        by cross correlation.  Different information is printed to
        the trailer file.

    - The input may be a corrtag file instead of a raw file.
        In this case, the input and output directories must not be
        the same.  The association table may contain a full file
        name rather than just a rootname, so that a corrtag file
        may be specified in an association table.

    - Imaging wavecals can be processed (usually just calibrated
        as images).

    - New command-line arguments have been added:
            --compress      compress the calcos sum (csum) image
            --binx, --biny  bin the csum image in X or Y 
            --shift         specify a file to override shift1 values

    - Data for stripe C with opt_elem = G230L and cenwave = 3360
        will be calibrated (previously stripe C was skipped).

    - For TIME-TAG data, a new GTI table will be appended if time
        intervals were flagged as bad due to FUV bursts or due to
        the BADTTAB.

    - The values of some output keywords are rounded off, usually
        to four decimal places.

    - Photometry keywords (e.g. inverse sensitivity) are populated
        for imaging data, currently with pre-launch values, however.

    - ACQ/IMAGE data can be calibrated.

    - A bug in setting the data quality flags has been fixed.
        In some cases for FUV TIME-TAG data, the flagged regions
        are extended by large amounts in the dispersion direction,
        due to an offset in the flags indicating which events are
        within the active area.

    - For NUV spectroscopic data, the background smoothing was
        extending into the buffer zones on the left and right edges,
        which was not appropriate because the values in the buffer
        zones are all zero.  This bug has been fixed.

    - There was a bug in extracting the spectrum for auto and
        GO wavecals. The SHIFT2A (and B, C) keywords were not
        being used, so the spectrum was being extracted from the
        wrong location in the cross-dispersion direction.  In some
        cases, the extraction region was not large enough to include
        the spectrum.

    - Some changes were made to keywords.  For tagflash data,
        LAMPUSED can be NONE in the raw header, even though the
        lamp was on at intervals during the exposure, so LAMPUSED
        may now be reset to the value of the new keyword LAMPPLAN.
        Some keyword names were changed.  SEGMENT is set to "BOTH"
        in an x1d header; WAVECALS and SEGMENT are deleted from
        x1dsum headers.  A WAVECALS keyword was added, giving the
        names of the wavecal files.  The number of events and time
        lost due to bursts or flagged in the BADTTAB are recorded
        in keywords.

    - The heliocentric radial velocity V_HELIO is calculated more
        accurately.

    - For ACCUM data, the rawaccum headers are used as a template
        when writing the extensions in the flt and counts files,
        because the headers for SCI, ERR and DQ are different from
        each other.