STScI_Python Version 2.9 Release Notes November 2009 This release includes new versions of PyRAF, PyFITS, pysynphot, pytools, Multidrizzle, and related tasks. This release also includes more instrument-specific packages to support WFPC2, NICMOS, STIS and COS. The convolve, image, and ndimage packages have been included with this release to insure compatibility with the rest of our code. Platform Support ================ Normally, platform support issues do not apply to Python tasks, as most Python code will run on all platforms on which Python has been installed. This distribution was tested to correctly support installation on Linux, Mac OS X 10.5 (Leopard), and Solaris, while also being provided for installation under Windows. The single exception is that the text-based epar functionality now available in PyRAF (in addition to the already existing GUI-based epar) is not available under Solaris, and likely will never be. In addition, PyRAF currently requires the installation of IRAF, yet the only IRAF distribution for Windows runs under Cygwin and no testing of PyRAF has been conducted under Cygwin. Documentation ============= Documentation for these tasks has been consolidated into a single location complete with an index viewable as a local web page. The documentation includes any available user guides and API documentation generated using 'EPYDOC' for the modules or packages. This index can be accessed using: --> import stscidocs --> stscidocs.viewdocs() --> stscidocs.help() This will automatically bring up the default web browser application to display the index page. All the available documentation for software packaged with this distribution can be accessed through this interface, as this documentation resides locally upon installation. Python Environment ================== This release has been tested using Python 2.5.4 and requires numpy 1.3. This release may work on versions of Python as old as Python 2.4 depending on the module being used. Future Division support ----------------------- - Future division support was added to numerous routines in preparation for Python 2.6 to 3.0 transition. Packages updated to use the new division include: + CALCOS + NICTOOLS modules CalTempFromBias, finesky, nic_rem_persist (and supporting utility modules) + PyRAF + pysynphot + imagestats + WFPC2TOOLS module wfpc2destreak PyRAF Version 1.8.1 ------------------- The PyRAF v1.8 series supports Python 2.6. Since the last full release of PyRAF (v1.8), the following enhancements have been made: - Process caching was made more flexible with the addition of the prcacheOff() and prcacheOn() functions. The former sets the process cache size to 0 and flushes it, while the latter re-enables caching. (ticket #95) - On OSX the user might have a Python build which is linked with X11, and/or the user might have a Python build which is linked with native graphics (Cocoa/Aqua). For the former case, PyRAF needs to load an X11 shared object. For the latter case, PyRAF needs to load an Aqua module. PyRAF will now attempt to determine upon startup which is the case, and load the appropriate item. This affects EPAR and all graphics windows. (r1063, see ticket #86) - To the Python API (iraf.task()) was added the ability to easily send multi-line CL strings to be executed. Currently, this is most useful in regression testing PyRAF. (ticket #104) - In an attempt to start moving the codebase toward Python 3 capability, future division is now in use in all PyRAF modules as well as those relied upon in PyTools. (ticket #102) - For code maintainability a large set of functions in iraffunctions.py had repeated code (related to I/O redirection) removed/consolidated via the use of decorators. (ticket #106) And the following bugs have been fixed: - PyRAF started with the '-s' option is now correctly silent as advertised. See 'pyraf -h' for a list of command-line options. (ticket #97) - Mixed-casing in task names was causing problems for the command-line help system. (r1059) - A warning-generating incompatibility with matplotlib 0.99.1 was fixed. (r1077) PyFITS Version 2.2.2 -------------------- Updates described in this release are only supported in the NUMPY version of pyfits and include all revisions and bug fixes implemented since the last public release of STScI_Python which included PyFITS Version 2.1.1 (22-April-2009). The following enhancements were made: - Provide support for the FITS Checksum Keyword Convention. * Adding the checksum=True keyword argument to the open convenience function will cause checksums to be verified on file open. hdul=pyfits.open('in.fits',checksum=True) * On output, CHECKSUM and DATASUM cards may be output to all HDU's in a fits file by using the keyword argument checksum=True in calls to the writeto convenience function, the HDUList.writeto method, the writeto methods of all of the HDU classes, and the append convenience function. hdul.writeto('out.fits',checksum=True) - Implemented a new insert method to the HDUList class that allows for the insertion of a HDU into a HDUList at a given index. hdul.insert(2,hdu) - Provided the capability to handle unicode input for file names. - Provided support for integer division required by Python 3.0. The following bugs were fixed: - Corrected a bug that caused an exception to be raised when creating a CompImageHDU using an initial header that does not match the image data in terms of the number of axes. - Corrected a bug that prevented the opening of a fits file where a header contained a CHECKSUM card but no DATASUM card. - Corrected a bug that caused NULLs to be written instead of blanks when an ASCII table was created using a numpy chararray in which the original data contained trailing blanks. - Corrected a bug that caused an index out of bounds exception to be raised when iterating over the rows of a binary table HDU using the syntax " for row in tbhdu.data: ". - Corrected a bug that prevented the use of the writeto convenience function for writing table data to a file. - Modified the code to raise an IOError exception with the comment "Header missing END card." when pyfits can't find a valid END card for a header when opening a file. * This change addressed a problem with a non-standard fits file that contained several new-line characters at the end of each header and at the end of the file. However, since some people want to be able to open these non-standard files anyway, an option was added to the open convenience function to allow these files to be opened without exception. pyfits.open('infile.fits',ignore_missing_end=True) - Corrected a bug that prevented the use of StringIO objects as fits files when reading and writing table data. Previously, only image data was supported. - Corrected a bug that caused a bus error to be generated when compressing image data using GZIP_1 under the Solaris operating system. - Corrected bugs that prevented pyfits from properly reading Random Groups HDU's using numpy. - Corrected a bug that can occur when writing a fits file. * If no default SIGINT signal handler has been assigned, before the write, a TypeError exception is raised in the _File.flush() method when attempting to return the signal handler to its previous state. Notably this occurred when using mod_python. The code was changed to use SIG_DFL when no old handler was defined. - Corrected a bug in CompImageHDU that prevented rescaling the image data using hdu.scale(option='old'). pysynphot Version 0.7 --------------------- This release includes version 0.7 of pysynphot, which is presently being developed as a successor to STSDAS.SYNPHOT. It is not a simple port into Python, but provides improved algorithms that address known shortcomings in SYNPHOT. It does not require IRAF, and is suitable for either library or interactive use. Version 0.7 implements much of the core SYNPHOT functionality, and has been validated by an extensive commissioning process described in Technical Software Report 2009-01. User documentation is still being developed, but preliminary versions are available, including a guide to pysynphot for SYNPHOT users. Once the core SYNPHOT functionality is supported (presently planned for Spring 2010), SYNPHOT will be formally deprecated. Preliminary documentation for pysynphot can be obtained at: http://stsdas.stsci.edu/pysynphot ============ Applications ============ PyDrizzle Version 6.3.2 ----------------------- - 'section' parameter now supports multiple groups to be specified - Updates were made to the Fortran 'drizzle' code to correctly scale the photometry when using SIP coefficients. MultiDrizzle v3.3.5 ------------------- - changes to support multiple chip specification for the 'group' paramater. - supports WFC3 data processing - Fixed problems with updating PHOTFLAM, BUNIT keywords in image headers - updated to handle proc_unit settings more robustly; namely, converts all inputs to electrons, including some parameters, then converts output to user specified units only in the final drizzle step. - logic problems dealing with image iterator were resolved by removing duplicate logic from multidrizzle and computing buffer sizes correctly in the first place. Pytools ------- - NMPFIT: fixed a problem with constrained parameters - FILEUTIL: parseExtn() now returns the Primary extension if no extension is specified on input - MAKEWCS Version 1.1.4: + supports binned data correctly + corrected a problem with how to define the reference chip for WFC3/UVIS data - The set of modules in Pytools which are used by PyRAF have been converted over to Python 3 "future" division. (r8085/87/88/90, r8114) - Logic was added to the parameter interface tool (TEAL) so that it more accurately determines on the fly whether or not the Python executable in used is linked with X11. (r8295) - TEAL was not correctly handling the case of tasks located inside of larger primary tasks. Also, exception handling in this area of the code was enhanced for better error messaging. (ticket #434) - TEAL was changed so that the INDEF value is handled elegantly during validation. (ticket #432) Numdisplay v.1.5.5 ------------------ - A new parameter, quiet, was added to the display() method to turn off messages to STDOUT. CALCOS v2.11b ------------- - This release includes many updates, bug fixes and algorithm changes from the last released version of CALCOS (V2.8a). The primary changes include: - Significant changes were made to wavecal processing. For NUV data, the shift of each stripe will be found separately, instead of adding all the stripes together and finding the global shift. The shift in the dispersion direction is found by minimizing chi square, rather than by cross correlation. Different information is printed to the trailer file. - The input may be a corrtag file instead of a raw file. In this case, the input and output directories must not be the same. The association table may contain a full file name rather than just a rootname, so that a corrtag file may be specified in an association table. - Imaging wavecals can be processed (usually just calibrated as images). - New command-line arguments have been added: --compress compress the calcos sum (csum) image --binx, --biny bin the csum image in X or Y --shift specify a file to override shift1 values - Data for stripe C with opt_elem = G230L and cenwave = 3360 will be calibrated (previously stripe C was skipped). - For TIME-TAG data, a new GTI table will be appended if time intervals were flagged as bad due to FUV bursts or due to the BADTTAB. - The values of some output keywords are rounded off, usually to four decimal places. - Photometry keywords (e.g. inverse sensitivity) are populated for imaging data, currently with pre-launch values, however. - ACQ/IMAGE data can be calibrated. - A bug in setting the data quality flags has been fixed. In some cases for FUV TIME-TAG data, the flagged regions are extended by large amounts in the dispersion direction, due to an offset in the flags indicating which events are within the active area. - For NUV spectroscopic data, the background smoothing was extending into the buffer zones on the left and right edges, which was not appropriate because the values in the buffer zones are all zero. This bug has been fixed. - There was a bug in extracting the spectrum for auto and GO wavecals. The SHIFT2A (and B, C) keywords were not being used, so the spectrum was being extracted from the wrong location in the cross-dispersion direction. In some cases, the extraction region was not large enough to include the spectrum. - Some changes were made to keywords. For tagflash data, LAMPUSED can be NONE in the raw header, even though the lamp was on at intervals during the exposure, so LAMPUSED may now be reset to the value of the new keyword LAMPPLAN. Some keyword names were changed. SEGMENT is set to "BOTH" in an x1d header; WAVECALS and SEGMENT are deleted from x1dsum headers. A WAVECALS keyword was added, giving the names of the wavecal files. The number of events and time lost due to bursts or flagged in the BADTTAB are recorded in keywords. - The heliocentric radial velocity V_HELIO is calculated more accurately. - For ACCUM data, the rawaccum headers are used as a template when writing the extensions in the flt and counts files, because the headers for SCI, ERR and DQ are different from each other.