STScI Logo
STScI Logo
The Institute
HST
JWST
Community Missions
Data Archives
News and Outreach
Resources
Banner
Software & Hardware
PyFITS
User's Manual
Release notes
Download
Code Repository
Project Page
Credits
Citation
License
Data Analysis In Python Tutorial
STSDAS
TABLES
PyRAF
numarray
MultiDrizzle
specview

PyFITS release notes

PyFITS is being used within the STScI operations pipelines and is a supported product. Currently PyFITS takes a fairly strict interpretation of FITS files. There are likely to be problems with FITS data that do not strictly conform to the standard. Undoubtedly, use with a wider variety of files and uses will uncover problems; please contact help@stsci.edu when such problems are encountered.

We intend to accomodate such variances, particularly if they involve widely used or available data (so please let us know when such problems occur).

PyFITS requires numpy (or numarray) to be installed.

Version 1.3; February 22 2008

  • Updates described in this release are only supported in the NUMPY version of pyfits.
  • The following enhancements were made:
    • Provided support for a new extension to pyfits called stpyfits.
      • The stpyfits module is a wrapper around pyfits. It provides all of the features and functions of pyfits along with some STScI specific features. Currently, the only new feature supported by stpyfits is the ability to read and write fits files that contain image data quality extensions with constant data value arrays. See stpyfits for more details on stpyfits.
    • Added a new feature to allow trailing HDUs to be deleted from a fits file without actually reading the data from the file.
      • This supports a JWST requirement to delete a trailing HDU from a file whose primary Image HDU is too large to be read on a 32 bit machine.
    • Updated pyfits to use the warnings module to issue warnings. All warnings will still be issued to stdout, exactly as they were before, however, you may now suppress warnings with the -Wignore command line option. For example, to run a script that will ignore warnings use the following command line syntax: 

      python -Wignore yourscript.py

    • Updated the open convenience function to allow the input of an already opened file object in place of a file name when opening a fits file.
    • Updated the writeto convenience function to allow it to accept the output_verify option.
      • In this way, the user can use the argument output_verify='fix' to allow pyfits to correct any errors it encounters in the provided header before writing the data to the file.
    • Updated the verification code to provide additional detail with a VerifyError exception.
    • Added the capability to create a binary table HDU directly from a numpy.ndarray. This may be done using either the new_table convenience function or the BinTableHDU constructor.
  • The following performance improvements were made:
    • Modified the import logic to dramatically decrease the time it takes to import pyfits.
    • Modified the code to provide performance improvements when copying and examining header cards.
  • The following bugs were fixed:
    • Corrected a bug that occurs when reading the data from a fits file that includes BZERO/BSCALE scaling. When the data is read in from the file, pyfits automatically scales the data using the BZERO/BSCALE values in the header. In the previous release, pyfits created a 32 bit floating point array to hold the scaled data. This could cause a problem when the value of BZERO is so large that the scaled value will not fit into the float 32. For this release, when the input data is 32 bit integer, a 64 bit floating point array is used for the scaled data.
    • Corrected a bug that caused an exception to be raised when attempting to scale image data using the ImageHDU.scale method.
    • Corrected a bug in the new_table convenience function that occurred when a binary table was created using a ColDefs object as input and supplying an nrows argument for a number of rows that is greater than the number of rows present in the input ColDefs object. The previous version of pyfits failed to allocate the necessary memory for the additional rows.
    • Corrected a bug in the new_table convenience function that caused an exception to be thrown when creating an ASCII table.
    • Corrected a bug in the new_table convenience function that will allow the input of a ColDefs object that was read from a file as a binary table with a data value equal to None.
    • Corrected a bug in the construction of ASCII tables from Column objects that are created with noncontinuous start columns.
    • Corrected bugs in a number of areas that would sometimes cause a failure to improperly raise an exception when an error occurred.
    • Corrected a bug where attempting to open a non-existent fits file on a windows platform using a drive letter in the file specification caused a misleading IOError exception to be raised.

Version 1.1; June 15 2007

  • Modified to use either NUMPY or NUMARRAY.
  • New file writing modes have been provided to allow streaming data to extensions without requiring the whole output extension image in memory. See documentation on StreamingHDU.
  • Improvements to minimize byteswapping and memory usage by byteswapping in place.
  • Now supports ':' characters in filenames.
  • Handles keyboard interrupts during long operations.
  • Preserves the byte order of the input image arrays.

Version 1.0.1; March 24 2006

The changes to PyFITS were primarily to improve the docstrings and to reclassify some public functions and variables as private. Readgeis and fitsdiff which were distributed with PyFITS in previous releases were moved to pytools. This release of PyFITS is v1.0.1. The next release of PyFITS will support both numarray and numpy (and will be available separately from stsci_python, as are all the python packages contained within stsci_python). An alpha release for PyFITS numpy support will be made around the time of this stsci_python release.

  • Updated docstrings for public functions.
  • Made some previously public functions private.

Version 1.0; November 1 2005

Major Changes since v0.9.6:

  • Added support for the HEIRARCH convention
  • Added support for iteration and slicing for HDU lists
  • PyFITS now uses the standard setup.py installation script
  • Add utility functions at the module level, they include:
    • getheader
    • getdata
    • getval
    • writeto
    • append
    • update
    • info

Minor changes since v0.9.6:

  • Fix a bug to make single-column ASCII table work.
  • Fix a bug so a new table constructed from an existing table with X-formatted columns will work.
  • Fix a problem in verifying HDUList right after the open statement.
  • Verify that elements in an HDUList, besides the first one, are ExtensionHDU.
  • Add output verification in methods flush() and close().
  • Modify the the design of the open() function to remove the output_verify argument.
  • Remove the groups argument in GroupsHDU's contructor.
  • Redesign the column definition class to make its column components more accessible. Also to make it conducive for higher level functionalities, e.g. combining two column definitions.
  • Replace the Boolean class with the Python Boolean type. The old TRUE/FALSE will still work.
  • Convert classes to the new style.
  • Better format when printing card or card list.
  • Add the optional argument clobber to all writeto() functions and methods.
  • If adding a blank card, will not use existing blank card's space.

PyFITS Version 1.0 REQUIRES Python 2.3 or later.

Version 0.9.6; 11 November 2004

Major changes since v0.9.3:

  • Support for variable length array tables.
  • Support for writing ASCII table extensions.
  • Support for random groups, both reading and writing.

Some minor changes:

  • Support for numbers with leading zeros in an ASCII table extension.
  • Changed scaled columns' data type from Float32 to Float64 to preserve precision.
  • Made Column constructor more flexible in accepting format specification.

Version 0.9.3; 2 July 2004

Changes since v0.9.0:

  • Lazy instanciation of full Headers/Cards for all HDU's when the file is opened. At the open, only extracts vital info (e.g. NAXIS's) from the header parts. This change will speed up the performance if the user only needs to access one extension in a multi-extension FITS file.
  • Support the X format (bit flags) columns, both reading and writing, in a binary table. At the user interface, they are converted to Boolean arrays for easy manipulation. For example, if the column's TFORM is "11X", internally the data is stored in 2 bytes, but the user will see, at each row of this column, a Boolean array of 11 elements.
  • Fix a bug such that when a table extension has no data, it will not try to scale the data when updating/writing the HDU list.

Version 0.9; 27 April 2004

Changes since v0.8.0:

  • Rewriting of the Card class to separate the parsing and verification of header cards
  • Restructure the keyword indexing scheme which speed up certain applications (update large number of new keywords and reading a header with larger numbers of cards) by a factor of 30 or more
  • Change the default to be lenient FITS standard checking on input and strict FITS standard checking on output
  • Support CONTINUE cards, both reading and writing
  • Verification can now be performed at any of the HDUList, HDU, and Card levels
  • Support (contiguous) subsection (attribute .section) of images to reduce memory usage for large images

Version 0.8.0; August 19, 2003

NOTE: This version will only work with numarray Version 0.6. In addition, earlier versions of PyFITS will not work with numarray 0.6. Therefore, both must be updated simultaneously.

Changes since 0.7.6:

  • Compatible with numarray 0.6/records 2.0
  • For binary tables, now it is possible to update the original array if a scaled field is updated.
  • Support of complex columns
  • Modify the __getitem__ method in FITS_rec. In order to make sure the scaled quantities are also viewing ths same data as the original FITS_rec, all fields need to be "touched" when __getitem__ is called.
  • Add a new attribute mmobject for HDUList, and close the memmap object when close HDUList object. Earlier version does not close memmap object and can cause memory lockup.
  • Enable 'update' as a legitimate memmap mode.
  • Do not print message when closing an HDUList object which is not created from reading a FITS file. Such message is confusing.
  • remove the internal attribute "closed" and related method (__getattr__ in HDUList). It is redundant.

Version 0.7.6; November 22, 2002

NOTE: This version will only work with numarray Version 0.4.

Changes since 0.7.5:

  • Change x*=n to numarray.multiply(x, n, x) where n is a floating number, in order to make pyfits to work under Python 2.2. (2 occurrences)

  • Modify the "update" method in the Header class to use the "fixed-format" card even if the card already exists. This is to avoid the mis-alignment as shown below:

    After running drizzle on ACS images it creates a CD matrix whose elements have very many digits, e.g.:

    CD1_1   =  1.1187596304411E-05 / partial of first axis coordinate w.r.t. x
CD1_2   = -8.502767249350019E-06 / partial of first axis coordinate w.r.t. y
    with pyfits, an "update" on these header items and write in new values which has fewer digits, e.g.:

    CD1_1   =        1.0963011E-05 / partial of first axis coordinate w.r.t. x
CD1_2   =          -8.527229E-06 / partial of first axis coordinate w.r.t. y
  • Change some internal variables to make their appearance more consistent: 

            old name                new name
        
        __octalRegex            _octalRegex
        __readblock()           _readblock()
        __formatter()           _formatter().
        __value_RE              _value_RE
        __numr                  _numr 
        __comment_RE            _comment_RE 
        __keywd_RE              _keywd_RE 
        __number_RE             _number_RE.
        tmpName()               _tmpName()
        dimShape                _dimShape
        ErrList                 _ErrList
  • Move up the module description. Move the copywright statement to the bottom and assign to the variable __credits__.

  • change the following line:

    self.__dict__ = input.__dict__
    to 

    self.__setstate__(input.__getstate__())
    in order for pyfits to run under numarray 0.4.

  • edit _readblock to add the (optional) firstblock argument and raise IOError if the the first 8 characters in the first block is not 'SIMPLE ' or 'XTENSION'. Edit the function open to check for IOError to skip the last null filled block(s). Edit readHDU to add the firstblock argument.

Version 0.7.5; 16 August 2002

Changes since v0.7.3:

  1. Memory mapping now works for readonly mode, both for images and binary tables.
    Usage: pyfits.open('filename', memmap=1)
  2. Edit the field method in FITS_rec class to make the column scaling for numbers use less temporary memory. (does not work under 2.2, due to Python "bug" of array *=)
  3. Delete bscale/bzero in the ImageBaseHDU constructor.
  4. Update bitpix in BaseImageHDU.__getattr__ after deleting bscale/bzero. (bug fix)
  5. In BaseImageHDU.__getattr__ point self.data to raw_data if float and if not memmap. (bug fix).
  6. Change the function get_tbdata() to private: _get_tbdata().

Version 0.7.3; 12 July 2002

Changes since v0.7.2:
  1. It will scale all integer image data to Float32, if BSCALE/BZERO != 1/0. It will also expunge the BSCALE/BZERO keywords.
  2. Add the scale() method for ImageBaseHDU, so data can be scaled just before being written to the file. It has the following arguments:
    type:
    destination data type (string), e.g. Int32, Float32, UInt8, etc.
    option:
    scaling scheme. if 'old', use the old BSCALE/BZERO values. if 'minmax', use the data range to fit into the full range of specified integer type. Float destination data type will not be scaled for this option.
    bscale/bzero:
    user specifiable BSCALE/BZERO values. They overwrite the "option".
  3. Deal with data area resizing in 'update' mode.
  4. Make the data scaling (both input and output) faster and use less memory.
  5. Bug fix to make column name change takes effect for field.
  6. Bug fix to avoid exception if the key is not present in the header already. This affects (fixes) add_history(), add_comment(), and add_blank().
  7. Bug fix in __getattr__() in Card class. The change made in 0.7.2 to rstrip the comment must be string type to avoid exception.

    8. Version 0.7.2.1; June 25, 2002

    A couple of bugs were addressed in this version.

    • Fix a bug in _add_commentary(). Due to a change in index_of() during version 0.6.5.5, _add_commentary needs to be modified to avoid exception if the key is not present in the header already. This affects (fixes) add_history(), add_comment(), and add_blank().
    • Fix a bug in __getattr__() in Card class. The change made in 0.7.2 to rstrip the comment must be string type to avoid exception.

    Version 0.7.2; June 19, 2002

    The two major improvements from Version 0.6.2 are:

    • support reading tables with "scaled" columns (e.g. tscal/tzero, Boolean, and ASCII tables)
    • a prototype output verification.
    This version of PyFITS requires numarray version 0.3.4.

    Other changes include:

    1. Implement the new HDU hierarchy proposed earlier this year. This in turn reduces some of the redundant methods common to several HDU classes.
    2. Add 3 new methods to the Header class: add_history, add_comment, and add_blank.
    3. The table attributes _columns are now .columns and the attributes in ColDefs are now all without the underscores. So, a user can get a list of column names by: hdu.columns.names.
    4. The "fill" argument in the new_table method now has a new meaning:
      If set to true (=1), it will fill the entire new table with zeros/blanks. Otherwise (=0), just the extra rows/cells are filled with zeros/blanks. Fill values other than zero/blank are now not possible.
    5. Add the argument output_verify to the open method and writeto method. Not in the flush or close methods yet, due to possible complication.
    6. A new copy method for tables, the copy is totally independent from the table it copies from.
    7. The tostring() call in writeHDUdata takes up extra space to store the string object. Use tofile() instead, to save space.
    8. Make changes from _byteswap to _byteorder, following corresponding changes in numarray and recarray.
    9. Insert(update) EXTEND in PrimaryHDU only when header is None.
    10. Strip the trailing blanks for the comment value of a card.
    11. Add seek(0) right after the __buildin__.open(0), because for the 'ab+' mode, the pointer is at the end after open in Linux, but it is at the beginning in Solaris.
    12. Add checking of data against header, update header keywords (NAXIS's, BITPIX) when they don't agree with the data.
    13. change version to __version__.
    There are also many other minor internal bug fixes and technical changes.

    Version 0.6.2; February 12, 2002

    This version requires numarray version 0.2.

    Things not yet supported but are part of future development:

    • Verification and/or correction of FITS objects being written to disk so that they are legal FITS. This is being added now and should be available in about a month. Currently, one may construct FITS headers that are inconsistent with the data and write such FITS objects to disk. Future versions will provide options to either a) correct discrepancies and warn, b) correct discrepancies silently, c) throw a Python exception, or d) write illegal FITS (for test purposes!).
    • Support for ascii tables or random groups format. Support for ASCII tables will be done soon (~1 month). When random group support is added is uncertain.
    • Support for memory mapping FITS data (to reduce memory demands). We expect to provide this capability in about 3 months.
    • Support for columns in binary tables having scaled values (e.g. BSCALE or BZERO) or boolean values. Currently booleans are stored as Int8 arrays and users must explicitly convert them into a boolean array. Likewise, scaled columns must be copied with scaling and offset by testing for those attributes explicitly. Future versions will produce such copies automatically.
    • Support for tables with TNULL values. This awaits an enhancement to numarray to support mask arrays (planned). (At least a couple of months off).

Copyright  | Help  | Printable Page