Uniform Recalibration of All STIS Data

Over the next few months we will perform a "final" recalibration of all STIS data obtained prior to the failure of the STIS Side 2 electronics on 2004-Aug-03. The October/November 2006 STAN describes the latest updates to the calibration reference files and the most recent changes to the CALSTIS code that have been delivered in support of this effort.

Currently when a user requests a STIS dataset from the HST archive, the On-The-Fly Reprocessing (OTFR) procedures recreate all the data files that will be delivered to the user (including the raw files) from the original raw telemetry data (POD files). This ensures that the data is processed with the most up-to-date versions of software and reference files, but does result in delays of several minutes or hours while the data is being processed.

As we perform the "final" reprocessing, copies of all the calibrated and uncalibrated data files will be archived, and in the future these archived copies will be made available to users. This will allow for significantly faster retrievals from the HST Archive, and will free up computing resources for OTFR requests for other instruments.

A number of other changes have been made in support of the final recalibration that affect both OTFR produced and archived versions of the calibrated data. Here we describe the most recent updates.

New supported grating/aperture combinations

The calibration rules for STIS data are designed to produce 1D and 2D extracted spectra only when such an extraction makes sense. For example, for "slitless" spectra, no extraction is attempted because the calibration will depend on where in the aperture the target (or targets) is located. However, a review of the available STIS spectroscopic data showed that there were a number of non-standard grating/aperture combinations that could sensibly be added to the list of combinations for which a full extraction could be done. The rules for deciding whether or not to fully calibrate a given dataset have been revised, and users can now obtain fully calibrated 1D and 2D spectral extractions for these datasets from the HST archive.

For example, for echelle observations done with the 31X0.05ND* apertures, spectral extractions were not originally done by the OTFR pipeline, because if an extended source were observed with such a long slit, the spectrum from different orders would overlap, making it impossible to define a unique 1D extraction. However, in practice all echelle observations with these long neutral density slits targeted point sources rather than extended objects, and order overlap is not a concern. The pipeline is now set to produce 1D extracted spectra for such observations (assuming other critera required for calibration, such as the presence of a wavecal exposure, are met).

IR fringe flat files

For G750L and G750M observations at wavelengths greater than 6000 angstroms, obtaining the best signal-to-noise requires the user to defringe the data using contemporaneously obtained fringe flat data and the IRAF defringing tasks in the STIS STSDAS software package. For each external G750L and G750M science exposure, the name of the affiliated fringe flat dataset (if it exists) is now put into the FRNGFLAT keyword in the header of the science exposure data file. When the uncalibrated data files for these science exposures are requested from the archive, the dataset named in the FRNGFLT keyword is now also automatically delivered.

GO wavecal files

To produce flux calibrated spectra, the STIS calibration pipeline needs a contemporaneous wavelength calibration spectrum. Without such a wavecal exposure, the zero-point uncertainties are too large to produce a reliable wavelength and flux calibrated spectrum. In some cases, observers were allowed to turn off the automatic wavecals and substitute user defined wavelength calibration exposures. Unfortunately, the archive has not been able to use these GO wavecals in the OTFR pipeline. To get fully calibrated spectra, users need to separately download the science and calibration exposures, reset the calibration flags in the file headers, and run the CALSTIS software locally.

As part of the final recalibration of STIS data, we have identified these GO wavecals, and will associate them with the appropriate science exposures so that they can be treated in the same way as the automatic wavecals. This will require redefining the data set names for many of these exposures. We anticipate that these new wavecal associations will be implemented early in 2007, after the other recalibration work has been finished.