STScI Logo

Release Notes for stsci_python
stsci_python 2.10 Release Notes

STScI_Python Version 2.10 Release Notes

June 2010

This release includes new versions of PyRAF, PyFITS, pysynphot, pytools,
Multidrizzle, and related tasks.  This release also includes
instrument-specific packages to support WFPC2, NICMOS, STIS and COS.

The convolve, image, and ndimage packages have been included with
this release to insure compatibility with the rest of our code.

Platform Support 
Normally, platform support issues do not apply to Python tasks,
as most Python code will run on all platforms on which Python has
been installed. This distribution was tested to correctly support
installation on Linux, Mac OS X 10.5 (Leopard), and Solaris, while
also being provided for installation under Windows. The single
exception is that the text-based epar functionality now available
in PyRAF (in addition to the already existing GUI-based epar) is
not available under Solaris, and likely will never be. As of the
2010 release, PyRAF no longer requires the installation of IRAF.
Note that the only IRAF distribution for Windows runs under Cygwin,
but no testing of PyRAF has been conducted under Cygwin.
Documentation for these tasks has been consolidated into a
single location complete with an index viewable as a local web
page. The documentation includes any available user guides and
API documentation generated using 'EPYDOC' for the modules or
packages. This index can be accessed using:
 --> import stscidocs
 --> stscidocs.viewdocs()
This will automatically bring up the default web browser application
to display the index page. All the available documentation for
software packaged with this distribution can be accessed through this
interface, as this documentation resides locally upon installation.

Python Environment
This release has been tested using Python 2.5.4 and requires
numpy 1.3. This release has also been tested under Python 2.6, but only 
under Linux. 

Future Division support
    - Future division support has been added to all our code in preparation
    for Python 2.6 to 3.0 transition.

PyRAF Version 1.9
PyRAF v1.9 supports Python 2.5, 2.6 and 2.7-beta.  Since the 1.8.1 bundled release, the following enhancements have been made:

   - Matplotlib graphics kernel speed improvement: PyRAF v1.6 brought
   us the optional matplotlib graphics kernel (ticket #80). It provided
   cleanly resizable fonts and much smoother looking graphics than
   the default Tk kernel, but it was slower. This new version makes
   some assumptions about the GKI command stream from the IRAF task
   which allows it to severely limit the amount of drawing to the
   matplotlib canvas, resulting in improved performance. In some cases,
   graphics render as much as 5 to 10 times faster. (ticket #122)

   - PyRAF without IRAF: PyRAF can now run in "No-IRAF" mode, where
   the PyRAF Python shell will run, and - although no IRAF tasks will
   be found (imheader, implot, etc.) - the basic capabilities of the
   PyRAF command-line are still supported.  Some examples of working
   features are:

       - parameter handling (e.g. epar/tpar/lpar/dpar)
       - CL script conversion to Python (except of course any IRAF tasks called)
       - both Python and CL syntax on the command-line
       - minimum matching
       - tab-completion
       - aliases
       - command history (up-arrow), which is persistent after exit
       - access to shell variables and commands
       - spawning native executables
       - behind-the-scenes conversion to parenthesis, "sleep 3" -> "sleep(3)"
       - the full span of Python's capabilities and Tkinter graphics

      Please see PyRAF FAQ #1.3 for a full description. (ticket #107)

   - Command history persistence was added to PyRAF. Upon starting a
   PyRAF session, the up-arrow can be used to recall commands from
   the previous session(s), similar to ipython or your favorite
   shell. (ticket #115)

   - X11 graphics initialization has been changed to help Linux users
   (see FAQ #5.7). By setting an environment variable within the
   Python process, PyRAF now preempts the majority of issues with
   the X11 composite extension. (ticket #123)

   - Mouse-wheels are now supported in EPAR (and TEAL) for those
   Python installations with Tk 8.5 or better. (ticket #120)

   - Commonly used invocation options may now be specified in the
   environment variable named 'PYRAF_ARGS'. (ticket #114)

   - For PyRAF-only users (as opposed to those downloading all of
   stsci_python), installation instructions have been created on the
   web site. A README file was also created to direct such users to
   that page. (ticket #121)

The following bugs have been fixed:

   - Testing under Python 2.7-beta revealed an import error which
   was due to a change in Python's own UserDict type. (ticket #126)

   - PyRAF on OSX 64-bit Python had an issue where mouse-warping
   during graphics tasks left the mouse cursor in the top left corner
   of the screen. (ticket #110)

   - PyRAF initialization code was refactored so that PyRAF will
   perform no graphics setup when run in non-graphics mode (e.g. setenv
   PYRAF_NO_DISPLAY 1). (r1092-1096)

   - A bug in "tpar" was causing control characters to be shown on the
   screen (making navigation difficult) upon the second call within
   the same PyRAF session (OSX). This has been fixed. (ticket #117)

   - Also on the Mac, the "help" command was emitting BadWindow
   X errors when called from PyRAF running inside the Terminal
   application. This was related to X11 focus-switching which was
   not actually necessary in that specific situation. (ticket #124)

   - sleep: version 1.8.1 included the cleaning up of some command-line
   functions' signatures. The sleep() function had been changed to
   require an argument (seconds), since a call with no arguments
   was equivalent to "sleep 0" which does not seem useful. However,
   there are existing IRAF CL scripts with such "sleep" calls, so
   PyRAF was changed in 1.8.2 to re-allow this. (r1120)

PyFITS Version 2.3.1
Updates described in this release include all revisions and bug fixes
implemented since the last public release of STScI_Python which included
PyFITS Version 2.2.2 (12-October-2009).  PyFITS now only supports the
NUMP array package.  Support for NUMARRAY has been eliminated from PyFITS.

The following enhancements were made:

    - Rework documentation to use Sphinx. 

    - Added method HDUList.filename() to return the name of a file associated
      with an HDU.

    - Support the Python 2.5 'with' statement when opening files as in the

      >>> with"input.fits") as hdul:
      ...    #process hdul

    - Added support for writing FITS data to file-like objects that do not
      support the random access methods seek() and tell().  Most PyFITS 
      functions or methods will treat these file-like objects as an empty file
      that cannot be read, only written.  It is also expected that the
      file-like object is in a writable condition (ie. opened) when passed into
      a PyFITS function or method.  The following methods and functions will
      allow writing to a non-random access file-like object: HDUList.writeto(),
      HDUList.flush(), pyfits.writeto(), and pyfits.append().  The
      convenience function may be used to create an HDUList object that is
      associated with the provided file-like object.

      An illustration of the new capabilities follows.  In this example FITS
      data is written to standard output which is associated with a file opened
      in write-only mode:

      >>> import pyfits
      >>> import numpy as np
      >>> import sys
      >>> hdu = pyfits.PrimaryHDU(np.arange(100,dtype=np.int32))
      >>> hdul = pyfits.HDUList()
      >>> hdul.append(hdu)
      >>> tmpfile = open('','w')
      >>> sys.stdout = tmpfile
      >>> hdul.writeto(sys.stdout, clobber=True)
      >>> sys.stdout = sys.__stdout__
      >>> tmpfile.close()
      No.    Name         Type      Cards   Dimensions   Format
      0    PRIMARY     PrimaryHDU       5  (100,)        int32

    - Support for reading unsigned integer 16 values from an ImageHDU extended
      to include unsigned integer 32 and unsigned integer 64 values.
      ImageHDU data is considered to be unsigned integer 16 when the data type
      is signed integer 16 and BZERO is equal to 2**15 (32784) and BSCALE is
      equal to 1.  ImageHDU data is considered to be unsigned integer 32 when
      the data type is signed integer 32 and BZERO is equal to 2**31 and BSCALE
      is equal to 1.  ImageHDU data is considered to be unsigned integer 64
      when the data type is signed integer 64 and BZERO is equal to 2**63 and
      BSCALE is equal to 1.  An optional keyword argument (uint) was added to
      the open convenience function for this purpose.  Supplying a value of
      True for this argument will cause data of any of these types to be read
      in and scaled into the appropriate unsigned integer array (uint16,
      uint32, or uint64) instead of into the normal float 32 or float 64 array.
      If an HDU associated with a file that was opened with the 'int' option
      and containing unsigned integer 16, 32, or 64 data is written to a file,
      the data will be reverse scaled into a signed integer 16, 32, or 64 array
      and written out to the file along with the appropriate BSCALE/BZERO
      header cards.  Note that for backward compatability, the 'uint16'keyword
      argument will still be accepted in the open function when handling
      unsigned integer 16 conversion.

    - Added the capability to access the data for a column of a FITS table by
      indexing the table using the column name.  This is consistent with Record
      Arrays in NUMPY (array with fields).  The following example will
      illustrate this:

      >>> import pyfits
      >>> hdul ='input.fits')
      >>> table = hdul[1].data
      >>> table.names
      >>> print table.field('c2') # this is the data for column 2
      ['abc' 'xy']
      >>> print table['c2'] # this is also the data for column 2
      array(['abc', 'xy '],
      >>> print table[1] # this is the data for row 1
      (2, 'xy', 6.6999997138977054, True)

    - Provided support for slicing a FITS_record object.  The FITS_record
      object represents the data from a row of a table.  PyFITS now supports
      the slice syntax to retrieve values from the row.  The following
      illustrates this new syntax:

      >>> hdul ='table.fits')
      >>> row = hdul[1].data[0]
      >>> row
      ('clear', 'nicmos', 1, 30, 'clear', 'idno= 100')
      >>> a, b, c, d, e = row[0:5]
      >>> a
      >>> b
      >>> c
      >>> d
      >>> e
    - Added capability to create a BinaryTableHDU directly from a numpy Record
      Array (array with fields). The new capability includes table creation,
      writing a numpy Record Array directly to a FITS file using the
      pyfits.writeto and pyfits.append convenience functions.  Reading the data
      for a BinaryTableHDU from a FITS file directly into a numpy Record Array
      using the pyfits.getdata convenience function.  The following illustrates
      these new capabilities:

      >>> import pyfits
      >>> import numpy

      >>> t=numpy.zeros(5,dtype=[('x','f4'),('y','2i4')]) \
      ... # Create a numpy Record Array with fields

      >>> hdu = pyfits.BinTableHDU(t) \
      ... # Create a Binary Table HDU directly from the Record Array
      >>> print
      [(0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))]

      >>> hdu.writeto('test1.fits',clobber=True) \
      ... # Write the HDU to a file
      Filename: test1.fits
      No.    Name         Type      Cards   Dimensions   Format
      0    PRIMARY     PrimaryHDU       4  ()            uint8
      1                BinTableHDU     12  5R x 2C       [E, 2J]

      >>> pyfits.writeto('test.fits', t, clobber=True) \
      ... # Write the Record Array directly to a file

      >>> pyfits.append('test.fits', t) \
      ... # Append another Record Array to the file
      Filename: test.fits
      No.    Name         Type      Cards   Dimensions   Format
      0    PRIMARY     PrimaryHDU       4  ()            uint8
      1                BinTableHDU     12  5R x 2C       [E, 2J]
      2                BinTableHDU     12  5R x 2C       [E, 2J]

      >>> d=pyfits.getdata('test.fits',ext=1) \
      ... # Get the first extension from the file as a FITS_rec
      >>> print type(d)
      >>> print d
      [(0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))
       (0.0, array([0, 0], dtype=int32))]

      >>> d=pyfits.getdata('test.fits',ext=1,view=numpy.ndarray) \
      ... # Get the first extension from the file as a numpy Record Array
      >>> print type(d)
      >>> print d
      [(0.0, [0, 0]) (0.0, [0, 0]) (0.0, [0, 0]) (0.0, [0, 0])
       (0.0, [0, 0])]
      >>> print d.dtype
      [('x', '>f4'), ('y', '>i4', 2)]

      >>> d=pyfits.getdata('test.fits',ext=1,upper=True, view=pyfits.FITS_rec)
      ... # Force the Record Array field names to be in upper case
            regardless of how they are stored in the file
      >>> print d.dtype
      [('X', '>f4'), ('Y', '>i4', 2)]

    - Allow the assignment of a row value for a PyFITS table using a tuple or a
      list as input.  The following example illustrates this new feature:

      >>> c1=pyfits.Column(name='target',format='10A')
      >>> c2=pyfits.Column(name='counts',format='J',unit='DN')
      >>> c3=pyfits.Column(name='notes',format='A10')
      >>> c4=pyfits.Column(name='spectrum',format='5E')
      >>> c5=pyfits.Column(name='flag',format='L')
      >>> coldefs=pyfits.ColDefs([c1,c2,c3,c4,c5])
      >>> tbhdu=pyfits.new_table(coldefs, nrows = 5)
      >>> # Assigning data to a table's row using a tuple
      >>>[2] = ('NGC1',312,'A Note',
      ... num.array([1.1,2.2,3.3,4.4,5.5],dtype=num.float32), True)
      >>> # Assigning data to a tables row using a list
      >>>[3] = ['JIM1','33','A Note',
      ... num.array([1.,2.,3.,4.,5.],dtype=num.float32),True]

    - Allow the creation of a Variable Length Format (P format) column from a
      list of data.  The following example illustrates this new feature:

      >>> a = [num.array([7.2e-20,7.3e-20]),num.array([0.0]), num.array([0.0])]
      >>> acol = pyfits.Column(name='testa',format='PD()',array=a)
      >>> acol.array
      _VLF([[  7.20000000e-20   7.30000000e-20], [ 0.], [ 0.]],

    - Allow the assignment of multiple rows in a table using the slice syntax.
      The following example illustrates this new feature:

      >>> counts = num.array([312,334,308,317])
      >>> names = num.array(['NGC1','NGC2','NGC3','NCG4'])
      >>> c1=pyfits.Column(name='target',format='10A',array=names)
      >>> c2=pyfits.Column(name='counts',format='J',unit='DN',
      ... array=counts)
      >>> c3=pyfits.Column(name='notes',format='A10')
      >>> c4=pyfits.Column(name='spectrum',format='5E')
      >>> c5=pyfits.Column(name='flag',format='L',array=[1,0,1,1])
      >>> coldefs=pyfits.ColDefs([c1,c2,c3,c4,c5])
      >>> tbhdu1=pyfits.new_table(coldefs)
      >>> counts = num.array([112,134,108,117])
      >>> names = num.array(['NGC5','NGC6','NGC7','NCG8'])
      >>> c1=pyfits.Column(name='target',format='10A',array=names)
      >>> c2=pyfits.Column(name='counts',format='J',unit='DN',
      ... array=counts)
      >>> c3=pyfits.Column(name='notes',format='A10')
      >>> c4=pyfits.Column(name='spectrum',format='5E')
      >>> c5=pyfits.Column(name='flag',format='L',array=[0,1,0,0])
      >>> coldefs=pyfits.ColDefs([c1,c2,c3,c4,c5])
      >>> tbhdu=pyfits.new_table(coldefs)
      >>>[0][3] = num.array([1.,2.,3.,4.,5.],
      ... dtype=num.float32)
      >>> tbhdu2=pyfits.new_table(, nrows=9)
      >>> # Assign the 4 rows from the second table to rows 5 thru
      ...   8 of the new table.  Note that the last row of the new
      ...   table will still be initialized to the default values.
      >>>[4:] =
      >>> print
      [ ('NGC1', 312, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), True)
        ('NGC2', 334, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), False)
        ('NGC3', 308, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), True)
        ('NCG4', 317, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), True)
        ('NGC5', 112, '0.0', array([ 1.,  2.,  3.,  4.,  5.],
      dtype=float32), False)
        ('NGC6', 134, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), True)
        ('NGC7', 108, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), False)
        ('NCG8', 117, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), False)
        ('0.0', 0, '0.0', array([ 0.,  0.,  0.,  0.,  0.],
      dtype=float32), False)]

The following bugs were fixed:

    - Corrected bugs in HDUList.append and HDUList.insert to correctly handle
      the situation where you want to insert or append a Primary HDU as
      something other than the first HDU in an HDUList and the situation where
      you want to insert or append an Extension HDU as the first HDU in an

    - Corrected a bug involving scaled images (both compressed and not
      compressed) that include a BLANK, or ZBLANK card in the header.  When the
      image values match the BLANK or ZBLANK value, the value should be
      replaced with NaN after scaling.  Instead, PyFITS was scaling the BLANK
      or ZBLANK value and returning it.

    - Corrected a bug involving reading and writing compressed image data.
      When written, the header keyword card ZTENSION will always have the value
     'IMAGE' and when read, if the ZTENSION value is not 'IMAGE' the user will
     receive a warning, but the data will still be treated as image data.

    - Corrected a byteswapping bug that occurs when writing certain column data.

    - Corrected a bug that occurs when creating a column from a chararray when
      one or more elements are shorter than the specified format length.  The
      bug wrote nulls instead of spaces to the file.

    - Corrected a bug in the HDU verification software to ensure that the
      header contains no NAXISn cards where n > NAXIS.

    - Corrected a bug that restricted the ability to create a custom HDU class
      and use it with PyFITS.  The bug fix will allow something like this:

      >>> import pyfits
      >>> class MyPrimaryHDU(pyfits.PrimaryHDU):
      ...     def __init__(self, data=None, header=None):
      ...         pyfits.PrimaryHDU.__init__(self, data, header)

      ...     def _summary(self):
      ...         """
      ...         Reimplement a method of the class.
      ...         """
      ...         s = pyfits.PrimaryHDU._summary(self)
      ...         # change the behavior to suit me.
      ...         s1 = 'MyPRIMARY ' + s[11:]
      ...         return s1
      ... classExtensions={pyfits.PrimaryHDU: MyPrimaryHDU})

      Filename: pix.fits
      No.    Name         Type      Cards   Dimensions   Format
      0    MyPRIMARY  MyPrimaryHDU     59  (512, 512)    int16

    - Modified ColDefs.add_col so that instead of returning a new ColDefs
      object with the column added to the end, it simply appends the new column
      to the current ColDefs object in place.

    - Corrected a bug in ColDefs.del_col which raised a KeyError exception when
      deleting a column from a ColDefs object.

    - Modified the open convenience function so that when a file is opened in
      readonly mode and the file contains no HDU's an IOError is raised.

    - Modified _TableBaseHDU to ensure that all locations where data is
      referenced in the object actually reference the same ndarray, instead of
      copies of the array.

    - Corrected a bug in the Column class that failed to initialize data when
      the data is a boolean array.

    - Corrected a bug that caused an exception to be raised when creating a
      variable length format column from character data (PA format).

    - Modified installation code so that when installing on Windows, when a C++
      compiler compatable with the Python binary is not found, the installation
      completes with a warning that all optional extension modules failed to
      build.  Previously, an Error was issued and the installation stopped.
    - Replaced code in the Compressed Image HDU extension which was covered
      under a GNU General Public License with code that is covered under a BSD
      License.  This change allows the distribution of PyFITS under a BSD

pysynphot Version 0.8
This version has the following dependencies:
  - numpy v1.0 or higher
  - PyFITS v1.1 or higher
  - matplotlib v0.9 or higher

Changes from version 0.7 include:

  - support added for subracting spectra
  - partial support for new keywords to countrate:
      - range to specify integration range
      - force=True to force the integration when the specified range 
        only partially overlaps the observed spectrum
  - .thermback() method on the ObsBandpass class to provide user access to 
     the basic SYNPHOT.thermback functionality
  - bugfix to so changes to setref(area) properly take effect
  - bugfix to observation.sample() 
  - improved doctrings for extinction and analytic spectra

More details can be seen in the changelog at


Imagestats Version 1.3

  The computation of the median was completely replaced with the
  results returned by numpy, after applying any clipping to the data
  as specified by the user.  The previous computation of the median
  has been renamed as the 'midpt' result to match exactly what IRAF's
  imstatistics task returns (both in terms of the result name and
  algorithm used for the computation).  The new median result no
  longer depends on binning like IRAF's computation of the 'midpt'
  as it relies on the implementation within numpy to return a true
  median every time.


wfpc2destreak 2.2 (2010 March 12):
  - Support was added for GEIS, waivered FITS, and multi-extension FITS 
  formatted input.

  This is the first release of an HST instrument specific library
  which provides detector to world coordinate transformations
  including all available WCS corrections. It is based on the more
  general PyWCS and WCSLIB libraries.  Its primary use is to support
  the new redesigned multidrizzle (betadrizzle), although it can be
  used interactively. STWCS consists of two subpackages - UPDATEWCS
  and WCSUTIL.
  - Updates the WCS of individual HST files based
  on information in reference files and header keywords. In
  general it will be run in the pipeline in order to provide
  science files with the best available WCS to archive users.
  - Defines an HST instrument specific WCS object with
  access to all instrument specific distortions and corrections. It
  provides methods for transformation from detector to world
  coordinates applying all available corrections.  This can be
  done either directly in one-step or in a multi-step interactive
  fashion.  Examples on how to do this can be found in this
  draft document:
  - A new function 'interpretDQvalue()' was added to split a DQ value
  into its bit values.
  - A new function, checkPhotKeywords, was added to pytools.check_files
  in order to insure that all copies contained the photometry keywords
  in the science extensions. This function will copy the keywords
  from the PRIMARY header if necessary.

Multidrizzle v3.3.7
This version of MultiDrizzle relies on PyDrizzle Version 6.3.5, whose
updates are included here as they directly support changes in MultiDrizzle.

  - The code will now look for the 'PFLTFILE' keyword for the
  flat-field reference file for WFC3 IR data, instead of looking for
  'FLATFILE' as used by NICMOS.

  - The use of the flat-field with WFC3 IR data was updated to reflect
  the convention in place for WFC3 data, where it, like ACS, uses
  a flat that is divided into the science data, as opposed to the
  inverse-flat convention used by NICMOS.

  - Velocity aberration correction will now be applied to all HST
  data that specifies the VAFACTOR keyword, instead of applying the
  correction to ACS data only. This insures that WFC3 data will be
  properly corrected for velocity aberration.

  - The PHOTFLAM keyword now gets scaled by the gain depending on
  whether the final output units were requested to be ELECTRONS
  or COUNTS.

  - The 'group' parameter now gets used to determine which FITS header
  should be used as the basis for the output product's header.

  - Updates to PyDrizzle to support use of PRIMARY header keywords
  from IDCTAB for computing the TDD coefficients for ACS/WFC data. If
  no keywords are present, the same hard-coded values will be used
  as before for ACS/WFC data only.

CALCOS v2.12
  - The algorithm for wavecal processing was significantly improved.  More
  information is printed regarding the wavecal shifts, including the FPOFFSET
  for the exposure and an error estimate for the shift in the dispersion
  direction.  Emission lines in the wavecal spectrum that are truncated by
  the edge of the NUV detector can interfere with finding the shift, so a test
  has been added for this case.  For G140L data, the wavecal shift information
  from segment A is now copied and used to correct the data for segment B.
  Improvements were made to the code to find the location of the wavecal
  spectrum in the cross-dispersion direction.
  - A WAVELENGTH column was added to the corrtag table, and a GCOUNTS column
  (gross counts, as opposed to count rate) was added to the x1d and x1dsum
  - Keywords pertaining to the deadtime correction are now written to the
  extension header instead of the primary header.
  - A new GTI table (good time intervals) will only be added to the corrtag
  file if the GTI list was actually modified (e.g. due to FUV bursts).
  There were cases where the flagged regions in the data quality extension
  were much wider than they should have been, resulting in lost data in the
  x1dsum tables.  This has been fixed.