Space Telescope Science Institute
HST Call for Proposals and Primer
help@stsci.edu
Table of Contents Previous Next Print


Hubble Space Telescope Primer for Cycle 21 > Chapter 7: Data Processing and the HST Data Archive > 7.1 Routine Science Data Processing

7.1 Routine Science Data Processing
Science data obtained with HST are sent to the TDRSS satellite system, from there to the TDRSS ground station at White Sands, New Mexico, then to the Sensor Data Processing Facility at Goddard Space Flight Center in Greenbelt, Maryland, and then finally to STScI.
At STScI, the production pipeline provides standard processing for data editing, calibration, and product generation. These functions, performed automatically, include the following:
Standard calibrations performed on HST data, and the resulting output data products, are described in detail in the HST Data Handbook. The Archive no longer saves the initial versions of the calibrated data (since 2000), nor the initial  version of the uncalibrated FITS data (since 2005). Instead, when a data request is submitted for all ACS, COS and WFC3, as well as all NICMOS and STIS data acquired after Servicing Mission 4, the Archive production pipeline processes the data “on-the-fly” using the best available reference files, starting with the spacecraft packet format files, as described in Section 7.2.1. Data from first generation instruments (FOC, FOS, HSP, WFPC1), WFPC2, and NICMOS, are maintained in a Static Archive.
7.1.1 Space Telescope Science Data Analysis System (STSDAS)
The Space Telescope Science Data Analysis System (STSDAS), and its accompanying package called TABLES, provides access to all the existing calibration pipeline programs used by STScI to process all HST data so that HST observers can recalibrate their data, examine intermediate calibration steps, and re-run the pipeline using different calibration switch settings and reference data as appropriate.
STSDAS also has various applications for the analysis of HST data as well as various utilities for manipulating and plotting data. The TABLES package facilitates the manipulation of FITS table data. STSDAS and TABLES were originally layered onto the Image Reduction and Analysis Facility (IRAF) software from the National Optical Astronomy Observatories (NOAO). Both packages run from within IRAF and are supported on a variety of platforms, although not all of the platforms that IRAF supports.
The new version of the calibration software, which no longer relies on IRAF, has been installed for archive and pipeline operations. The IRAF-free version of CALACS has been released as part of a new HSTCAL package, a package which will allow users to compile and run the calibration software originally written in C without using or running IRAF. This version will replace the calibration software available under STSDAS, starting with CALACS for ACS.
Much of the newer calibration and analysis software is written in Python. It does not require IRAF, and is available as part of STScI Python library. PyRAF is an alternate, Python-based command line environment for IRAF that enables the new Python-based software to be used, along with IRAF tasks, with IRAF command-line syntax. STScI Python includes PyFITS, a module that provides Python programs the ability to read and write FITS files. The new Python environment allows users to manipulate and display data in a way not possible with IRAF that is more akin to how data can be manipulated and displayed by IDL (Interactive Data Language). The STScI Python environment described above is contained within the stsci_python package.
Detailed information on STSDAS, TABLES, PyRAF, PyFITS and other Python-based software, including the actual software, is available from the STScI Software Web page. Information about IRAF is available from the IRAF Web page.

Table of Contents Previous Next Print