Space Telescope Science Institute
Call for Proposals and HST Primer
help@stsci.edu
Table of Contents Previous Next PDF


Hubble Space Telescope Primer for Cycle 23 > Chapter 7: Data Processing and the HST Data Archive > 7.1 Routine Science Data Processing

7.1
Science data obtained with HST are sent to the TDRSS satellite system, from there to the TDRSS ground station at White Sands, New Mexico, then to the Sensor Data Processing Facility at Goddard Space Flight Center in Greenbelt, Maryland, and then finally to STScI.
At STScI, the production pipeline provides standard processing for data editing, calibration, and product generation. These functions, performed automatically, include the following:
Standard calibrations performed on HST data, and the resulting output data products, are described in detail in the HST Data Handbook.
In late 2014, major upgrades to the Science Data Processing Pipelines (DP) and Data Distribution System (DADS) will be in progress. The upgrades will result in improved availability of all HST science data. When completed, fully calibrated data, uncalibrated data and reference files will be stored on disks for immediate download. Data will be reprocessed, as needed, when updated calibration reference files or improved calibration algorithms are received. Data from all non-active instruments (HSP, GHRS, FOC, FOS, WF/PC, WFPC-2 and NICMOS) will continue to be available for direct download. During the transition, Archive users will receive their data from the disks, when available, or through the On-the-Fly Reprocessing (OTFR) System. Both sources will provide up-to-date calibrated data.
7.1.1 Space Telescope Science Data Analysis System (STSDAS)
The Space Telescope Science Data Analysis System (STSDAS), and its accompanying package, TABLES, provides access to applications for the analysis of HST data as well as various utilities for manipulating and plotting data. The TABLES package facilitates the manipulation of FITS table data. STSDAS and TABLES were originally layered onto the Image Reduction and Analysis Facility (IRAF) software from the National Optical Astronomy Observatories (NOAO). Both packages run from within IRAF and are supported on a variety of platforms, although not all of the platforms that IRAF supports.
All the existing calibration pipeline programs used by STScI to process all HST data can found in the HSTCAL package, a package which allows users to compile and run the calibration software originally written in C without using or running IRAF. The latest versions of all the active instruments’ calibration software no longer relies on IRAF and has been installed as the HSTCAL package for Archive and pipeline operations. HST observers can use the programs in HSTCAL to recalibrate their data, to examine intermediate calibration steps, and to re-run the pipeline using different calibration switch settings and reference data as appropriate.
These HSTCAL software versions replace the calibration software available under STSDAS; specifically, CALACS for ACS, CALWF3 for WFC3, and CALSTIS for STIS.
Much of the newer calibration and analysis software is written in Python, including CALCOS for COS data. It does not require IRAF, and is available as part of the STScI Python library included and installed as part of Ureka releases.
PyRAF is an alternate Python-based command-line environment that enables users to run IRAF tasks and allows the new Python-based software to be used, along with IRAF tasks, with IRAF command-line syntax. Ureka includes astropy, a set of packages that provides Python programs the ability to read and write FITS files and read and manipulate the WCS information in image headers. The new Python environment allows users to manipulate and display data in a way not possible with IRAF that is more akin to how data can be manipulated and displayed by IDL (Interactive Data Language). The STScI Python environment described above is contained within the stsci_python package. Detailed information on STSDAS, TABLES, PyRAF, Ureka, and other Python-based software, including the actual software, is available from the STScI Software webpage. Information about IRAF is available from the IRAF webpage.

Hubble Space Telescope Primer for Cycle 23 > Chapter 7: Data Processing and the HST Data Archive > 7.1 Routine Science Data Processing

Table of Contents Previous Next PDF