Hubble Space Telescope Primer for Cycle 17

7.1 Routine Science Data Processing

Science data obtained with HST are sent to the TDRSS satellite system, from there to the TDRSS ground station at White Sands, New Mexico, then to the Sensor Data Processing Facility at Goddard Space Flight Center in Greenbelt, Maryland, and then finally to STScI. At STScI the production pipeline provides standard processing for data editing, calibration, and product generation. These functions, performed automatically, include the following:

• Reformatting and editing of data from spacecraft packet format to images and spectra.
• Performing standard calibrations (flat fields, wavelength calibrations, background subtraction, etc.) with currently available calibration files.
• Producing standard data output products (FITS files of raw and calibrated images, OMS [jitter and performance flags] files, and so on).

The standard calibrations performed on HST data, and the resulting output data products, are described in detail in the HST Data Handbook. Note that the Archive no longer saves the initial versions of the calibrated data (since 2000), nor the initial version of the uncalibrated FITS data (since 2005); rather, the production pipeline is run on-the-fly starting from the spacecraft packet format files, as described in Section 7.2.1.)

7.1.1 Space Telescope Science Data Analysis System (STSDAS)

STScI maintains a set of tools and support software used to calibrate and analyze HST data. One such tool is the Space Telescope Science Data Analysis System (STSDAS) and its accompanying package, TABLES. STSDAS provides access to all the existing calibration pipeline programs used by STScI to process all HST data so that HST observers can recalibrate their data, examine intermediate calibration steps, and re-run the pipeline using different calibration switch settings and reference data as appropriate. STSDAS also has various applications for the analysis of HST data as well as various utilities for manipulating and plotting data. The TABLES package facilitates the manipulation of FITS table data. STSDAS and TABLES are layered onto the Image Reduction and Analysis Facility (IRAF) software from the National Optical Astronomy Observatories (NOAO). Both packages run from within IRAF and are supported on a variety of platforms, although not all of the platforms that IRAF supports.

Much of the newer calibration and analysis software is written in Python, does not require IRAF, and is available as part of STScI Python library. PyRAF is an alternate, Python-based command line environment for IRAF that enables the new Python-based software to be used along with IRAF tasks using IRAF command line syntax. Stsci_python includes PyFITS, a module that provides Python programs the ability to read and write FITS files. The new Python environment allows users to manipulate and display data in a way not possible with IRAF that is more akin to how data can be manipulated and displayed by IDL (Interactive Data Language). The STScI Python environment described above is contained within the stsci_python package.

Detailed information on STSDAS, TABLES, PyRAF, PyFITS and other Python-based software, including the actual software, is available from the STScI Software Web page. Information about IRAF is available from the IRAF Web page.