2.2	Types of STIS Files

2.2.1 Trailer Files

2.2.2 Understanding Associations

The naming convention for STIS files is rootname_xxx.fits, where xxx is a three-character file suffix. The suffix identifies the type of data within the file. Table 2.1 lists the file suffixes for both uncalibrated and calibrated data files. Depending on the type of observation you have obtained, and therefore on the path it has taken through the calibration pipeline, you will find an appropriate subset of these files in your particular dataset. Table 2.2 shows the datasets that are produced by the calstis pipeline for different types of datasets.

Table 2.1: Data File Naming Conventions

Suffix	Type	Contents
Uncalibrated Science Data
_raw	image	Raw science1
_tag	table	TIME-TAG event list
_wav	image	Associated wavecal exposure
Uncalibrated Support Data
_asn	table	Association file
_epc	table	CCD housing temperature at 1.26 s intervals
_jif	image	2-D histogram of the _jit file
_jit	table	Spacecraft pointing data averaged over 3 s intervals
_lrc	image	Local rate check image
_lsp	text	LRC support file (header)
_pdq	table	Post observation summary and Data Quality
_spt	image	Support, planning & telemetry information
_trl	table	Trailer file; historical record of generic conversion and pipeline processing
_wsp	image	The _spt file for _wav (wavecal)
Calibrated Data
_flt	image	Flat-fielded science
_crj	image	Cosmic ray rejected, flat-fielded science (CCD data only)
_sfl	image	Summed flat-fielded science (MAMA data only)
_x1d	table	1-D extracted spectra for individual imsets: • Aperture extracted, background subtracted, flux and wavelength calibrated spectra
_x2d	image	2-D spectral and direct images for individual imsets: • Rectified, wavelength and flux calibrated first order spectra or • Geometrically corrected imaging data.
_sx1	table	1-D extracted spectra from summed (REPEATOBS) or cosmic ray rejected (CRSPLIT) images.
_sx2	image	2-D rectified direct or spectral images from summed (REPEATOBS) or cosmic ray rejected (CRSPLIT) images.

---

1

Raw data from non-automated GO wavecals, biases, darks, and flats, as well as from ACQs and ACQ/PEAKs, also have the _raw suffix.

Table 2.2: STIS Data File by Observation Type

Data Type		Header Keywords Determining Number of Imsets	Uncalibrated Files	Calibrated Files Containing Individual Imsets	Calibrated Files Containing Combined Imsets1
ACQ, ACQ/PEAK			raw
ACCUM: Single Imset Spectroscopic
	CCD 1st order MAMA 1st order MAMA echelle	NRPTEXP=1 & CRSPLIT=1 NRPTEXP=1 NRPTEXP=1	raw, wav raw, wav raw, wav	flt, x2d, x1d flt, x2d, x1d flt, ----, x1d
ACCUM: Single Imset Imaging
	CCD MAMA	NRPTEXP=1 & CRSPLIT=1 NRPTEXP=1	raw raw	flt, x2d flt, x2d
ACCUM: Multiple Imset Spectroscopic
	CCD 1st order MAMA 1st order MAMA echelle	NRPTEXP>1 or CRSPLIT>1 NRPTEXP>1 NRPTEXP>1	raw, wav raw, wav raw, wav	flt flt, x2d, x1d flt, ----, x1d	crj, sx2, sx1 sfl, sx2, 2 sfl
ACCUM: Multiple Imset Imaging
	CCD MAMA	NRPTEXP>1 or CRSPLIT>1 NRPTEXP>1	raw raw	flt flt, x2d	crj, sx2 sfl, sx2
TIME-TAG: Spectroscopic3
	MAMA 1st order MAMA echelle		tag, raw, wav tag, raw, wav	flt, x2d, x1d flt, ----, x1d
TIME-TAG: Imaging3
	MAMA		tag, raw	flt, x2d

---

1

Combined: summed if MAMA data, summed with cosmic ray rejection if CCD data.

2

Note that, while the pipeline does not produce a summed sx1 file for MAMA multiple imset spectroscopic datasets, you can easily do so by running x1d on the sfl file.

3

A single raw image is generated from the tag file for TIME-TAG data. For other options, see
Section 5.6.

2.2.1

Trailer Files

Each task in the calstis package creates messages during processing which describe the progress of the calibration. These messages are quite relevant to understanding how the data was calibrated, and in some of the cases, to determining the accuracy of the products.

When the data is processed in the archive pipeline, the output messages from the conversion of the telemetry data and the different calibration steps done by the calstis software are stored in a text FITS file known as the trailer file, with extension _trl. Each time the archive processes data before retrieval, the old trailer file is erased and a new one created using the results of the most recent processing performed. This is not the case when calstis is run in a user’s home environment. Calstis redirects the output of its many steps to the STDOUT, so when run on a personal machine, it will not override this _trl file but rather will direct the output to STDOUT.

The data on the telemetry conversion appears first in this table. In this section of the _trl file, there is also information relevant to the selection of the best reference files and the population of some of the header keywords. The second part of this file is the information on the calibration steps performed by the calstis pipeline, appearing in the order in which each step was performed. In this last section of the _trl file, the calstis steps are indicated by their module name (see Table 3.1 of this document).

The calstis messages provide information on the input and output files for each step, the corrections performed, information regarding the reference files used, and in the case of spectroscopic data, messages about the location of the extracted spectrum or shift correction applied to the data. Calstis also gives warnings in the event of failures of the software to locate the spectrum or when the appropriate correction to the data could not be applied. For more detailed information on the calibration steps and structure of calstis, please refer to Chapter 3.

2.2.2

Understanding Associations

An association is created when repeated exposures are obtained through CR-SPLITs or REPEATOBS, and when wavecal exposures are linked to science exposures. The repeated exposures in an association will appear in a single FITS file. You can recognize a file as part of an association because there will be a zero in the last position of the rootname (e.g., o3tt01010_raw.fits). The rootnames of the individual exposures in an association are contained in the association file, which has suffix _asn (e.g., o3tt01010_asn.fits). An association file holds a single binary table extension, which can be displayed with the IRAF tasks tprint or tread. The information within an association table shows how the associated exposures are related. Figure 2.1 illustrates the contents of the association table for a CRSPLIT=2 observation, with an associated wavecal.

Figure 2.1: Contents of Association Table

The association table above tells the user that the product, or dataset, will have the rootname o3tt01010, that there will be two science exposures contained in the o3tt01010_raw.fits file that are CR-SPLITs, and that a o3tt01010_wav.fits file should exist containing the contemporaneously obtained automatic wavecal. The o3tt01010_raw.fits file will contain six image extensions, one triplet of {SCI, ERR, DQ} for each exposure (see Section 2.3.1). The pipeline will calibrate these data as a unit, producing, in the case of CCD data, a single cosmic ray rejected image (rootname_crj.fits), its data quality and error images, and rectified spectra. Similarly, for REPEATOBS observations, in which many identical exposures are taken to obtain a time series, all the science data will be stored in sequential triplet extensions of a single FITS file (rootname_flt.fits). These will be processed through the calstis pipeline as a unit, with each image extension individually calibrated. The set of images will also be combined to produce a total time-integrated calibrated image.

Originally, only automatically inserted wavecals were associated and used for wavelength calibration in the OTFR pipeline. Although GOs could specify additional lamp exposures with the LINE, HITM1, or HITM2 lamps that could be used to supplement or replace the automatic wavecals, these were not used in the pipeline.

For a number of such spectroscopic datasets without auto-wavecals that were taken between STIS launch and the Side 2 failure in 2004, the associations were subsequently redefined to allow GO-specified wavecals to be used in place of the missing auto-wavecals. In these new associations, the GO-specified wavecal is treated in the same way as a standard auto-wavecal, although some parameters used for the exposure (e.g., aperture, exposure time, lamp, or lamp current) may differ from the default values that would have been used by the automatic wavecal.

It was also decided that it would be desirable to “associate” STIS IR fringe flats taken using the tungsten calibration lamp with the G750M or G750L science exposures for which they were taken. Instead of creating a formal association between such datasets, the name of the recommended fringe flat is placed in the FRINGFLAT header keyword in the extension header of the science data, and all tungsten lamp images taken contiguously with the science data (i.e., no MSM motion between exposures) are automatically delivered to the user whenever data are requested from the archive.

When suitable fringe flats are available, the recommended flat is chosen based on the aperture used for the fringe flat with the order of preference being 52X0.1, 52X0.05, 52X0.2, 52X0.5, 52X2, 0.3X0.09, and then any other. Should there be multiple suitable fringe flats taken using the same aperture, the one taken closest in time to the science exposure is preferred.

Chapter 3 of this document gives more information about the pipeline processing.