STScI Logo

Release Notes for stsci_python
stsci_python 2.12 Release Notes

STScI_Python Version 2.12 Release Notes

August 2011

This release includes new versions of PyRAF, PyFITS, STWCS, pytools,
Multidrizzle, and related tasks.  This release also includes
instrument-specific packages to support ACS, NICMOS, and COS.

The convolve, image, and ndimage packages have been included with
this release to insure compatibility with the rest of our code.

Platform Support
================
Normally, platform support issues do not apply to Python tasks,
as most Python code will run on all platforms on which Python has
been installed. This distribution was tested to correctly support
installation on Linux and Mac OS X 10.6 (Snow Leopard), while an
installation for Windows will be provided at a later date.  As of the
June 2010 release, PyRAF no longer requires the installation of IRAF.
Note that the only IRAF distribution for Windows runs under Cygwin,
but no testing of PyRAF has been conducted under Cygwin.

No Solaris Support
------------------
This release has NOT been tested on the Solaris platform.  We no longer
build Solaris binaries for STSDAS, although the source code can always
be downloaded and compiled locally as needed.
 
Documentation
=============
Documentation for these tasks has been consolidated into a
single location complete with an index viewable as a local web
page. The documentation includes any available user guides and
API documentation generated using 'EPYDOC' for the modules or
packages. This index can be accessed using:
 
 --> import stscidocs
 --> stscidocs.viewdocs()
 --> stscidocs.help()
 
This will automatically bring up the default web browser application
to display the index page. All the available documentation for
software packaged with this distribution can be accessed through this
interface, as this documentation resides locally upon installation.


Python Environment
==================
This release has been tested using Python Versions 2.5.4 and 2.7. This
release also requires at least numpy 1.3.  All packages within this
release have been revised to support easy_install of the code, while
still allowing the code to be installed using the disutils command
'python setup.py'.


Namespace Changes
==================
The organization of the stsci_python package has been significantly
changed to move many packages into the 'stsci' namespace and remove
any package name confusion (such as 'pytools' which is used as a name
by other Python software). The namespace changes require dependent
packages to update to point to the new names of the packages based
on this translation table:

old name   --->         new name
----------              -------------
convolve               stsci.convolve
image                  stsci.image
imagemanip             stsci.imagemanip
imagestats             stsci.imagestats
numdisplay             stsci.numdisplay
pytools                stsci.tools
stimage                stsci.stimage
ndimage                stsci.ndimage   

The 'ndimage', 'image', and 'convolve' packages included in our release
are forked from SciPy in order to support our software needs without
requiring all of SciPy to be installed separately.

PyFITS Version 3.0.0
====================
This release now includes and relies on PyFITS 3.0.0, a version which
contains changes to interfaces that are incompatible with previous
versions of PyFITS. These changes were implemented to allow PyFITS to
be translated to work with Python 3.0 while refactoring the code to make
it easier to maintain. 

The PyFITS Release Notes (and file 'CHANGES.txt' included in the release)
contain detailed descriptions of all the changes made to PyFITS 3.0.0.
The code in this release has been modified to be compatible with both
PyFITS 3.0.0 and earlier versions of PyFITS, however, this compatibility
will be removed at some point in the future.

This release addresses an extensive list of bug fixes and improvements,
including:

- Calls to deprecated functions will display a Deprecation warning.  However,
  in Python 2.7 and up Deprecation warnings are ignored by default, so run
  Python with the `-Wd` option to see if you're using any deprecated
  functions.  If we get close to actually removing any functions, we might
  make the Deprecation warnings display by default.

- Added basic Python 3 support

- Added support for multi-dimensional columns in tables as specified by the
  TDIMn keywords (#47)

- Fixed a major memory leak that occurred when creating new tables
  with the `new_table()` function (where strings are padded with
  zero-bytes) vs ASCII tables (where strings are padded with spaces)

- Fixed a bug in which the case of Random Access Group parameters names
  was not preserved when writing

- Added support for binary table fields with zero width

- Added support for wider integer types in ASCII tables; although this
  is non- standard, some GEIS images require it

- Fixed a bug that caused the index_of() method of HDULists to crash
  when the HDUList object is created from scratch

- Fixed the behavior of string padding in binary tables (where strings
  should be padded with nulls instead of spaces)

- Fixed a rare issue that caused excessive memory usage when computing
  checksums using a non-standard block size

- Add support for forced uint data in image sections

- Fixed tuple and list-based indexing of FITS_rec objects

- Fixed an issue where BZERO and BSCALE keywords were appended to
  headers in the wrong location

- `FITS_record` objects (table rows) have full slicing support, including
  stepping, etc.

- Fixed a bug where updating multiple files simultaneously (such as when
  running parallel processes) could lead to a race condition with mktemp()

- Fixed a bug where compressed image headers were not in the order
  expected by the funpack utility

stsci.tools (formerly: pytools)
============
This package provides utilities which support multiple packages due
to the more general nature.  Updates include:

    - New modules were added to work with GEIS (IRAF) images:
      specifically, the 'swapgeis' module to byteswap a GEIS image
      and the 'convertgeis' to replace the 'stwfits' task in IRAF to
      write out a waivered-FITS image from a GEIS image.

    - Removed the use of 'numerixenv' so that code will now only
      support numpy.

    - Updated the 'buildRootname()' function in 'tools.fileutil'
      to retain and properly handle any paths provided for the input
      filename

    - There were many improvements to the TEAL parameter interface:

        - TEAL now handles .cfgspc changes/upgrades with a notice
          stating which parameters are "problems" in the changed
          version (#728, r11322, PyRAF ticket #127).

        - Changes were made to the TEAL code to prepare for the port
          to Python 3 (#584), although Python 2.x will be supported
          for quite some time.  TEAL does now run under Python 3
          (via 2to3), but this has not yet been heavily tested
          (expected next release).

        - TEAL is now smarter about showing task help in HTML where
          possible, and it also has hooks to jump to parameter-specific
          help topics (#614, r12000, r12863)

        - Some GUI items were fixed: font colors, Save-As dialog,
          a FileDialog which changes directory when it shouldn't
          (#727, r12344, r12462, r12995).

        - Parameter mismatch errors are now more easy to
          read/understand.

        - TEAL supports two new trigger keywords: "set_yes_if" and
          "set_no_if". Set the env. variable TEAL_DEBUG to have TEAL
          print out trigger logic (#725).

calcos
============
Version 2.14.4 was previously released as a patch in April. This
release includes CALCOS Version 2.15.4 with updates from 2.14.4
that include:

    - A 'walk' correction was added to CALCOS to correct the pixel
      coordinates of events in an FUV TIME-TAG exposure for shifts
      which are dependent on the pulse height amplitude (PHA).

    - Use exposure appropriate for each FUV data segment in the
      x1d correction.

    - A bug in background smoothing during spectral extraction has
      been fixed.  This mostly affects values near the endpoints of
      the spectrum.

In this release, there is an unexplained problem on Windows; CALCOS
cannot be imported.

costools
============
A new package containing COS-specific analysis tasks has been added to
this release.  Currently, the only task included in this new package
is the 'timefilter' task. This program reads a COS corrtag file and
sets flags in the DQ column of the EVENTS extension depending on the
filter that was specified.


imagestats (now: stsci.imagestats)
============
The imagestats package was updated to take into account any
user-specified limits when clipping has been requested so that it
behaves more like IRAF's imagestats task.


numdisplay (now: stsci.numdisplay)
============
Numdisplay version 1.6.1 corrects a problem with compatibility with
numpy versions > 2.0. This version has been tested to work under
Python 2.7 using all the latest versions of numpy.


pydrizzle
============
Only minor changes were made to allow this version to work with PyFITS 3.0.0.


PyRAF v1.11
============
This release supports Python 2.5, 2.6, and 2.7.  Since the 1.10 bundled release, the following enhancements have been made:

 - PyRAF is being prepared for the port to Python 3. Many code changes
   were made, though there should be no behavior changes noticeable
   to the user (#137, #154).

 - PyRAF was modified to maintain compatibility with IRAF 2.15.*,
   both 32 and 64-bit (mostly due to graphics WCS changes). Also,
   better error messaging was implemented within PyRAF's core in
   order to debug related problems, often seen during startup. See
   PyRAF FAQ #1.5 (#156 and r1324).

 - PyRAF is now more robust in the face of outdated or changing CL
   scripts, e.g. login.cl. The ErrorTracker class, used internally, is
   now keeping better track of offensive CL script code for reporting
   to the user.

 - The 'unlearn' command is now available for TEAL-enabled tasks
   (r1412,3). TEAL also now automatically switches focus like EPAR
   (#159).

 - PyRAF on Windows now has a desktop icon for launching the
   command-line interface (#130).

and the following bugs have been fixed:

 - EPAR for TEAL-enabled tasks (.cfg file) now more elegantly handles
   changes to that task's .cfgspc definition: no traceback, simpler
   error message or, in some cases, even a dialog (#127). Validation
   errors are also handled more gracefully now (#158).

 - It was noticed that text-based help in the terminal would not run
   without graphics capabilities, even though it should not require
   them (#147).

 - On OSX, Python itself will no longer crash in a remote session if
   aqutil fails to load (#149).

 - A bug in the Page menu (graphics windows) was fixed for the
   matplotlib graphics kernel (#150). Also fixed in this graphics
   kernel was redraws not occurring with 't' during implot (r1387).

 - CL script parsing is now catching cases where a "#" is found in
   the right hand side of a set/reset statement (#155).

 - Integer division was not behaving correctly in parsed CL code. See
   PyRAF FAQ #6.6 for a full discussion of the expected behavior
   (#157).

 - PyRAF was throwing a traceback during a warning about certain CL
   system/shell commands (#162).

 - PyRAF on Windows no longer requires the use of the HOME environment
   variable (#130).

Python 3 Note: It is anticipated that the next full release of PyRAF
(late 2011/early 2012) will have basic capabilities working under
Python 3. Python 2.x however will be supported for quite a while. Many
users will be cautious about converting their scripts and processing.


pysynphot
============

The following changes have been made since the 0.8.2 release:


Bugfixes
---------
- Calculation of the pivot wavelength is now correctly implemented
  as a method (.pivot) on Bandpass objects.

- The GaussianSource, Powerlaw, FlatSpectrum, and BlackBody classes
  have been fixed so that they are consistent after unit changes
  using the convert() method.

- Fixed multiple bugs in the InterpolatedSpectralElement class so that
  it correctly handles tables with different column designs and
  provides correctly interpolated results for all table input
  types. (Previously, interpolation was implemented correctly only
  for ramp filters.)

- Handling of MergedWavesets was corrected to strictly eliminate
  duplicate entries, which previously sometimes occurred due to
  numerical rounding during merge operations.


Performance
-----------
- Calculating binned flux for Observation objects has been moved to
an optional C extension to improve speed. If the C extension is not
available at run time,
  the calculations are performed in Python.

- Objects created when opening and reading tables are cached to
improve speed
  in situations where a table previously would have been opened and
  closed many times within a program.


Other changes 
------------- 
- Custom pysynphot exceptions have been implemented for some conditions
  that previously raised ValueErrors. Additional custom exceptions
  will be implemented in the next release.

- The CompTable and GraphTable classes have been moved to a new
  pysynphot module called tables.

- The etc module has been deprecated. The etc.parse_spec function
  has been relocated to spparser.parse_spec.


stwcs
============
The package has been updated with these changes: 
    - added logging to STWCS

    - added support for a WCSCORR table which summarizes all updates
      made to the WCS information of each SCI extension

    - The HSTWCS method 'wcs2header' was modified to correctly append
      the SIP and IDC distortion model keywords to the header

    - The logic for using the 'readModel()' method of HSTWCS was
      also improved to not try and read the IDCModel when no 'idctab'
      had been specified in the header.

    - eliminated the 1 pixel offset introduced when creating an output
      WCS with an input WCS which has no distortion model.

    - makes recognition of non-HST data (slightly) more general so
      that it will work on data created with IRAF's artdata package

    - added 'all_sky2pix()' method to HSTWCS objects which implements
      an iterative solution using the full distortion model to convert
      sky positions into pixel positions

    - Added a tool to create a virtual mosaic based on the WCS of the
      input observations, with the option to provide an output tangent
      plane for the virtual mosaic

    - Added a method to update the scale of a WCS object:
      updatePscale()