STScI_Python Version 2.9 Release Notes
November 2009
This release includes new versions of PyRAF, PyFITS, pysynphot, pytools,
Multidrizzle, and related tasks. This release also includes more
instrument-specific packages to support WFPC2, NICMOS, STIS and COS.
The convolve, image, and ndimage packages have been included with
this release to insure compatibility with the rest of our code.
Platform Support
================
Normally, platform support issues do not apply to Python tasks,
as most Python code will run on all platforms on which Python has
been installed. This distribution was tested to correctly support
installation on Linux, Mac OS X 10.5 (Leopard), and Solaris, while
also being provided for installation under Windows. The single
exception is that the text-based epar functionality now available
in PyRAF (in addition to the already existing GUI-based epar) is
not available under Solaris, and likely will never be. In addition,
PyRAF currently requires the installation of IRAF, yet the only
IRAF distribution for Windows runs under Cygwin and no testing of
PyRAF has been conducted under Cygwin.
Documentation
=============
Documentation for these tasks has been consolidated into a
single location complete with an index viewable as a local web
page. The documentation includes any available user guides and
API documentation generated using 'EPYDOC' for the modules or
packages. This index can be accessed using:
--> import stscidocs
--> stscidocs.viewdocs()
--> stscidocs.help()
This will automatically bring up the default web browser application
to display the index page. All the available documentation for
software packaged with this distribution can be accessed through this
interface, as this documentation resides locally upon installation.
Python Environment
==================
This release has been tested using Python 2.5.4 and requires
numpy 1.3. This release may work on versions of Python as old as
Python 2.4 depending on the module being used.
Future Division support
-----------------------
- Future division support was added to numerous routines in preparation
for Python 2.6 to 3.0 transition. Packages updated to use the
new division include:
+ CALCOS
+ NICTOOLS modules CalTempFromBias, finesky, nic_rem_persist
(and supporting utility modules)
+ PyRAF
+ pysynphot
+ imagestats
+ WFPC2TOOLS module wfpc2destreak
PyRAF Version 1.8.1
-------------------
The PyRAF v1.8 series supports Python 2.6. Since the last full
release of PyRAF (v1.8), the following enhancements have been made:
- Process caching was made more flexible with the addition of
the prcacheOff() and prcacheOn() functions. The former sets
the process cache size to 0 and flushes it, while the latter
re-enables caching. (ticket #95)
- On OSX the user might have a Python build which is linked with
X11, and/or the user might have a Python build which is linked
with native graphics (Cocoa/Aqua). For the former case, PyRAF
needs to load an X11 shared object. For the latter case, PyRAF
needs to load an Aqua module. PyRAF will now attempt to determine
upon startup which is the case, and load the appropriate item.
This affects EPAR and all graphics windows. (r1063, see ticket #86)
- To the Python API (iraf.task()) was added the ability to easily
send multi-line CL strings to be executed. Currently, this is
most useful in regression testing PyRAF. (ticket #104)
- In an attempt to start moving the codebase toward Python 3
capability, future division is now in use in all PyRAF modules
as well as those relied upon in PyTools. (ticket #102)
- For code maintainability a large set of functions in
iraffunctions.py had repeated code (related to I/O redirection)
removed/consolidated via the use of decorators. (ticket #106)
And the following bugs have been fixed:
- PyRAF started with the '-s' option is now correctly silent
as advertised. See 'pyraf -h' for a list of command-line
options. (ticket #97)
- Mixed-casing in task names was causing problems for the
command-line help system. (r1059)
- A warning-generating incompatibility with matplotlib 0.99.1
was fixed. (r1077)
PyFITS Version 2.2.2
--------------------
Updates described in this release are only supported in the NUMPY
version of pyfits and include all revisions and bug fixes implemented
since the last public release of STScI_Python which included PyFITS
Version 2.1.1 (22-April-2009).
The following enhancements were made:
- Provide support for the FITS Checksum Keyword Convention.
* Adding the checksum=True keyword argument to the open
convenience function will cause checksums to be verified
on file open.
hdul=pyfits.open('in.fits',checksum=True)
* On output, CHECKSUM and DATASUM cards may be output to
all HDU's in a fits file by using the keyword argument
checksum=True in calls to the writeto convenience function,
the HDUList.writeto method, the writeto methods of all of
the HDU classes, and the append convenience function.
hdul.writeto('out.fits',checksum=True)
- Implemented a new insert method to the HDUList class that
allows for the insertion of a HDU into a HDUList at a given
index.
hdul.insert(2,hdu)
- Provided the capability to handle unicode input for file names.
- Provided support for integer division required by Python 3.0.
The following bugs were fixed:
- Corrected a bug that caused an exception to be raised when
creating a CompImageHDU using an initial header that does not
match the image data in terms of the number of axes.
- Corrected a bug that prevented the opening of a fits file
where a header contained a CHECKSUM card but no DATASUM card.
- Corrected a bug that caused NULLs to be written instead of
blanks when an ASCII table was created using a numpy chararray in
which the original data contained trailing blanks.
- Corrected a bug that caused an index out of bounds exception
to be raised when iterating over the rows of a binary table
HDU using the syntax " for row in tbhdu.data: ".
- Corrected a bug that prevented the use of the writeto
convenience function for writing table data to a
file.
- Modified the code to raise an IOError exception with the
comment "Header missing END card." when pyfits can't find a
valid END card for a header when opening a file.
* This change addressed a problem with a non-standard fits
file that contained several new-line characters at the end
of each header and at the end of the file. However, since
some people want to be able to open these non-standard
files anyway, an option was added to the open convenience
function to allow these files to be opened without exception.
pyfits.open('infile.fits',ignore_missing_end=True)
- Corrected a bug that prevented the use of StringIO objects
as fits files when reading and writing table data. Previously,
only image data was supported.
- Corrected a bug that caused a bus error to be generated
when compressing image data using GZIP_1 under the Solaris
operating system.
- Corrected bugs that prevented pyfits from properly reading
Random Groups HDU's using numpy.
- Corrected a bug that can occur when writing a fits
file.
* If no default SIGINT signal handler has been assigned,
before the write, a TypeError exception is raised in the
_File.flush() method when attempting to return the signal
handler to its previous state. Notably this occurred when
using mod_python. The code was changed to use SIG_DFL when
no old handler was defined.
- Corrected a bug in CompImageHDU that prevented rescaling the
image data using hdu.scale(option='old').
pysynphot Version 0.7
---------------------
This release includes version 0.7 of pysynphot, which is presently being
developed as a successor to STSDAS.SYNPHOT. It is not a simple
port into Python, but provides improved algorithms that address known
shortcomings in SYNPHOT. It does not require IRAF, and is suitable
for either library or interactive use.
Version 0.7 implements much of the core SYNPHOT functionality, and has
been validated by an extensive commissioning process described in
Technical Software Report 2009-01. User documentation is still being
developed, but preliminary versions are available, including a guide to
pysynphot for SYNPHOT users.
Once the core SYNPHOT functionality is supported (presently planned for
Spring 2010), SYNPHOT will be formally deprecated.
Preliminary documentation for pysynphot can be obtained at:
http://stsdas.stsci.edu/pysynphot
============
Applications
============
PyDrizzle Version 6.3.2
-----------------------
- 'section' parameter now supports multiple groups to be specified
- Updates were made to the Fortran 'drizzle' code to correctly
scale the photometry when using SIP coefficients.
MultiDrizzle v3.3.5
-------------------
- changes to support multiple chip specification for the
'group' paramater.
- supports WFC3 data processing
- Fixed problems with updating PHOTFLAM, BUNIT keywords in
image headers
- updated to handle proc_unit settings more robustly; namely,
converts all inputs to electrons, including some parameters,
then converts output to user specified units only in the
final drizzle step.
- logic problems dealing with image iterator were resolved by
removing duplicate logic from multidrizzle and computing
buffer sizes correctly in the first place.
Pytools
-------
- NMPFIT: fixed a problem with constrained parameters
- FILEUTIL: parseExtn() now returns the Primary extension if no
extension is specified on input
- MAKEWCS Version 1.1.4:
+ supports binned data correctly
+ corrected a problem with how to define the
reference chip for WFC3/UVIS data
- The set of modules in Pytools which are used by PyRAF have been
converted over to Python 3 "future" division. (r8085/87/88/90,
r8114)
- Logic was added to the parameter interface tool (TEAL) so that
it more accurately determines on the fly whether or not the Python
executable in used is linked with X11. (r8295)
- TEAL was not correctly handling the case of tasks located inside
of larger primary tasks. Also, exception handling in this area
of the code was enhanced for better error messaging. (ticket #434)
- TEAL was changed so that the INDEF value is handled elegantly
during validation. (ticket #432)
Numdisplay v.1.5.5
------------------
- A new parameter, quiet, was added to the display() method to
turn off messages to STDOUT.
CALCOS v2.11b
-------------
- This release includes many updates, bug fixes and algorithm
changes from the last released version of CALCOS (V2.8a). The
primary changes include:
- Significant changes were made to wavecal processing.
For NUV data, the shift of each stripe will be found
separately, instead of adding all the stripes together
and finding the global shift. The shift in the dispersion
direction is found by minimizing chi square, rather than
by cross correlation. Different information is printed to
the trailer file.
- The input may be a corrtag file instead of a raw file.
In this case, the input and output directories must not be
the same. The association table may contain a full file
name rather than just a rootname, so that a corrtag file
may be specified in an association table.
- Imaging wavecals can be processed (usually just calibrated
as images).
- New command-line arguments have been added:
--compress compress the calcos sum (csum) image
--binx, --biny bin the csum image in X or Y
--shift specify a file to override shift1 values
- Data for stripe C with opt_elem = G230L and cenwave = 3360
will be calibrated (previously stripe C was skipped).
- For TIME-TAG data, a new GTI table will be appended if time
intervals were flagged as bad due to FUV bursts or due to
the BADTTAB.
- The values of some output keywords are rounded off, usually
to four decimal places.
- Photometry keywords (e.g. inverse sensitivity) are populated
for imaging data, currently with pre-launch values, however.
- ACQ/IMAGE data can be calibrated.
- A bug in setting the data quality flags has been fixed.
In some cases for FUV TIME-TAG data, the flagged regions
are extended by large amounts in the dispersion direction,
due to an offset in the flags indicating which events are
within the active area.
- For NUV spectroscopic data, the background smoothing was
extending into the buffer zones on the left and right edges,
which was not appropriate because the values in the buffer
zones are all zero. This bug has been fixed.
- There was a bug in extracting the spectrum for auto and
GO wavecals. The SHIFT2A (and B, C) keywords were not
being used, so the spectrum was being extracted from the
wrong location in the cross-dispersion direction. In some
cases, the extraction region was not large enough to include
the spectrum.
- Some changes were made to keywords. For tagflash data,
LAMPUSED can be NONE in the raw header, even though the
lamp was on at intervals during the exposure, so LAMPUSED
may now be reset to the value of the new keyword LAMPPLAN.
Some keyword names were changed. SEGMENT is set to "BOTH"
in an x1d header; WAVECALS and SEGMENT are deleted from
x1dsum headers. A WAVECALS keyword was added, giving the
names of the wavecal files. The number of events and time
lost due to bursts or flagged in the BADTTAB are recorded
in keywords.
- The heliocentric radial velocity V_HELIO is calculated more
accurately.
- For ACCUM data, the rawaccum headers are used as a template
when writing the extensions in the flt and counts files,
because the headers for SCI, ERR and DQ are different from
each other.
