Space Telescope Science Institute
Call for Proposals and HST Primer
help@stsci.edu
Table of Contents Previous Next Print


Hubble Space Telescope Primer for Cycle 22 > Chapter 7: Data Processing and the HST Data Archive > 7.1 Routine Science Data Processing

7.1 Routine Science Data Processing
Science data obtained with HST are sent to the TDRSS satellite system, from there to the TDRSS ground station at White Sands, New Mexico, then to the Sensor Data Processing Facility at Goddard Space Flight Center in Greenbelt, Maryland, and then finally to STScI.
At STScI, the production pipeline provides standard processing for data editing, calibration, and product generation. These functions, performed automatically, include the following:
Standard calibrations performed on HST data, and the resulting output data products, are described in detail in the HST Data Handbook.
For currently-active instruments, the Archive no longer saves initial versions of the calibrated data (since 2000), nor the initial version of the uncalibrated FITS data (since 2005). Instead, when a data request is submitted for all ACS, COS and WFC3, as well as STIS data obtained after Servicing Mission 4 (SM4), the Archive production pipeline processes the data “on-the-fly” using the best available reference files, starting with the spacecraft packet format (POD) files, as described in Section 7.2.1.
Data from first generation instruments (FOC, FOS, HSP, WFPC1), WFPC2, NICMOS, and STIS data obtained before SM4 have been calibrated with the best-available reference files; these calibrated data, as well as raw data and reference files for these instruments can be retrieved from the Archive.
7.1.1 Space Telescope Science Data Analysis System (STSDAS)
The Space Telescope Science Data Analysis System (STSDAS), and its accompanying package, TABLES, provides access to all the existing calibration pipeline programs used by STScI to process all HST data so that HST observers can recalibrate their data, examine intermediate calibration steps, and re-run the pipeline using different calibration switch settings and reference data as appropriate.
STSDAS also has various applications for the analysis of HST data as well as various utilities for manipulating and plotting data. The TABLES package facilitates the manipulation of FITS table data. STSDAS and TABLES were originally layered onto the Image Reduction and Analysis Facility (IRAF) software from the National Optical Astronomy Observatories (NOAO). Both packages run from within IRAF and are supported on a variety of platforms, although not all of the platforms that IRAF supports.
The new version of the calibration software, which no longer relies on IRAF, has been installed for Archive and pipeline operations. The IRAF-free version of CALACS has been released as part of a new HSTCAL package, a package which will allow users to compile and run the calibration software originally written in C without using or running IRAF. This version will replace the calibration software available under STSDAS, starting with CALACS for ACS.
Much of the newer calibration and analysis software is written in Python. It does not require IRAF, and is available as part of STScI Python library. PyRAF is an alternate, Python-based command line environment for IRAF that enables the new Python-based software to be used, along with IRAF tasks, with IRAF command-line syntax. STScI Python includes PyFITS, a module that provides Python programs the ability to read and write FITS files. The new Python environment allows users to manipulate and display data in a way not possible with IRAF that is more akin to how data can be manipulated and displayed by IDL (Interactive Data Language). The STScI Python environment described above is contained within the stsci_python package.
Detailed information on STSDAS, TABLES, PyRAF, PyFITS and other Python-based software, including the actual software, is available from the STScI Software webpage. Information about IRAF is available from the IRAF webpage.

Hubble Space Telescope Primer for Cycle 22 > Chapter 7: Data Processing and the HST Data Archive > 7.1 Routine Science Data Processing

Table of Contents Previous Next Print