|STIS Instrument Handbook for Cycle 27|
Here, we summarize briefly the basic reductions and calibrations that are performed in the STScI STIS pipeline, and summarize the effects that particular Phase II proposal parameter choices have on calibration. The material in this chapter is intended to provide only enough background to develop robust observing proposals. A series of STIS Instrument Science Reports and the STIS Data Handbook provide the more detailed information needed for analyzing your data.Science data taken by STIS are received from the Space Telescope Data Capture Facility and sent to the STScI pipeline, where the data are unpacked, keywords extracted from the telemetry stream, and the science data reformatted and repackaged into raw (uncalibrated) FITS1 files by the generic conversion process. All STIS data products are FITS files. The vast majority of the STIS data products are two-dimensional images that are stored in FITS image extension files as triplets of science, error, and data quality arrays. FITS image extensions offer great flexibility in data storage and allow us to package, together into one file, related science data that are processed through calibration as a single unit. The uncalibrated FITS files are passed through calstis, the software code that calibrates the data, producing calibrated FITS files. For more details on STIS data structure and the naming conventions for the uncalibrated and calibrated data products, see STIS Data Handbook.Calstis performs the following basic science data calibrations:
• Basic, two-dimensional image reduction producing a flat-fielded output image (rootname_flt.fits), which, depending on whether the data are from the MAMA or the CCD and whether they are imaging or spectroscopic data, includes the following: data quality initialization, dark subtraction, bias subtraction, non-linearity flagging, flat fielding, and photometric calibration.
• Two-dimensional spectral extraction producing a flux-calibrated, rectified spectroscopic image (usually rootname_x2d.fits for MAMA data, rootname_sx2.fits for CCD) with distance along the slit running linearly along the Y axis and wavelength running linearly along the X axis, for spectroscopic first-order mode data. See Table 2.2 in the STIS Data Handbook.
• One-dimensional spectral extraction producing a one-dimensional spectrum of flux versus wavelength (usually rootname_x1d.fits for MAMA data, rootname_sx1.fits for CCD), uninterpolated in wavelength space, but integrated across an extraction aperture in the spatial direction, for first-order and echelle spectroscopic data. See Table 2.2 in the STIS Data Handbook.
• Data taken in TIME-TAG mode are corrected for the Doppler shift from the spacecraft motion and output as an uncalibrated event stream by the pipeline in a FITS binary table (rootname_tag.fits). The time-tag data stream is also integrated in time to produce an uncalibrated ACCUM mode image (rootname_raw.fits) which is then passed through standard calibration. The odelaytime task in STSDAS can be used off-line to correct the TIME-TAG spacecraft times to heliocentric times.In addition, calstis performs two types of contemporaneous calibrations:
• For CCD exposures which have been CR-SPLIT or when multiple exposures have been taken, calstis combines the exposures, producing a cosmic ray rejected image (rootname_crj.fits) which is then passed through subsequent calibration (e.g., spectral extraction).
• For spectroscopic exposures, calstis processes the associated wavecal exposure (see Section Routine Wavecals) to determine the zero point offset of the wavelength and spatial scales in the science image, thereby correcting for thermal drifts and the lack of repeatability of the mode select mechanism. Whereas the uncalibrated science data are stored in the rootname_raw.fits file, the accompanying wavecal observations are stored in the file rootname_wav.fits.Figure 15.1 through Figure 15.3 show example output from the calstis pipeline. The calstis program propagates statistical errors and tracks data quality flags through the calibration process. Calstis is available to users as part of the AstroConda distribution so they can recalibrate their data as needed.2 Recalibration may be performed in its entirety in a manner identical to the pipeline calibration by using calstis, or modular components of calstis, such as basic two-dimensional image reduction (basic2d), two-dimensional spectral extraction (x2d), one-dimensional spectral extraction (x1d), or cosmic ray rejection (ocrreject). The calibration steps that calstis performs on a given set of science observations depends on the nature of those observations.3Between Spring 2001 and Fall 2016, calibrated data products for STIS were available through On-the-fly-reprocessing (OTFR), which replaced On-the-fly-calibration (OTFC). The OTFR system started with raw telemetry products, converted these to FITS files, and added the latest instrument calibrations.Between late 2006 and mid-2007, a comprehensive re-calibration of all STIS data was performed. The raw and calibrated data files were saved and used to produce a static version of the STIS archive. Since mid-2007, this static archive was used in place of the OTFR pipeline to satisfy requests for STIS data. This allowed significantly faster delivery of STIS data, but was otherwise transparent to the user. After SM4 and the STIS repair, OTFR was restarted for new data, but requests for pre-repair data have continued to be satisfied using the static archive.Since 2016, the MAST archive has changed from using OTFR to a pre-calibrated data cache for STIS to directly deliver raw and calibrated products without the need to wait for reprocessing and recalibration. When new reference files are submitted, the intent is that all affected data sets will be promptly reprocessed and the cache updated. This allows much faster access to HST data than is possible with OTFR, however, this may result in a delay between the submission of revised reference files and their application to the cached data. Users who require data calibrated with the most recently updated reference files may wish to consult with the STIS team to verify when calibrated products made using these updated files will be available.Users can manually update reference files in the header of the raw files using the crds.bestrefs script.Over the years, a number of changes have been made to calstis, and propagated to the pipeline, to handle temporal changes in instrument performance.
• In July 2001, STIS operation was resumed with Side-2 electronics after failure of the Side 1 electronics in May. A wider range in the operating temperature of the CCD detector with the Side-2 electronics has made it necessary to include a temperature-dependent scaling factor when producing and applying Side-2 CCD dark reference files. This is discussed more fully in STIS ISR 2001-03.
• Since September 2002, calstis has used epoch-selected darks for the NUV-MAMA, which are scaled by a factor depending on the time and detector temperature of the science data.
• Gradual changes in spectroscopic and imaging sensitivity have been measured over the lifetime of the STIS detectors. See STIS ISR 2004-04 for full details. Currently, the calculation for 1-D spectral fluxes (FLUX column in rootname_sx1.fits and rootname_x1d.fits files) includes a wavelength and time-dependent sensitivity (TDS) factor to correct for these changes. For imaging modes, the calculation of photometric keywords also takes these time-dependent sensitivity changes into account.
• Temporal changes in CCD spectroscopic count rates are dominated by the steadily decreasing charge transfer efficiency (CTE), whose dependence on the structure and count level of the source and background is complex. See Section 7.3.7 and Section 7.3.8 and STIS ISR 2003-03 and STIS ISR 2006-03 for additional discussion. Correction of extracted CCD spectral fluxes for these CTE effects is also implemented in the standard pipeline software.After the failure of the STIS Side-2 electronics on August 3, 2004 that rendered STIS inoperable, it was recognized that there would be a long and perhaps permanent hiatus in obtaining new STIS data, and it was decided to produce a “final” static calibration of all then existing STIS data. As part of the calibration closeout a number of improvements were made to the STIS pipeline calibration, including improvements to the calstis software and the STIS reference files. These calibration improvements include:
• Improved algorithm to correct for charge transfer efficiency (CTE) when extracting fluxes for first-order spectra: (STIS ISR 2006-01; also Goudfrooij et al. 2006, PASP, 118, 1455).
• Recommended fringe flat exposures are now delivered with most G750L and G750M data, and the name of this recommended fringe flat is put into the FRNGFLAT keyword in the data file header.
• Also implemented was association of GO-specified wavecals with spectroscopic observations that lacked auto-wavecals. Previously the pipeline could not produce wavelength or flux calibrated spectra for visits that had the automatic wavecals turned off, even if the observer included appropriate line-lamp exposures in place of the auto-wavecals. As part of the closeout, the data set associations were redefined to associate these separate lamp exposures with the corresponding science observations, allowing a more complete calibration. Note that this reassociation resulted in new names being assigned to these science data sets. For data acquired after SM4, GO wavecals are automatically associated with the science exposures and so this will not be an issue.
• After SM4 it was found that the scaling relation and the reference temperature changed for the dark normalization. As a result calstis was changed to incorporate these findings. calstis was using hard-coded parameters for the reference temperature and slope in order to correct the temperature dependence of the CCD dark rate. Analysis of SMOV and Cycle 17 calibration data shows that these values have changed. Rather than including time-dependent parameters in calstis, these are now included as header keywords in the dark reference files, and calstis has been modified to read and use these keyword values.Observers can retrieve HST data by using StarView Web or the HST archive Web interface to select specific datasets. One can choose where and how the data are delivered: on the archive computer staging disk for copy using anonymous FTP, directly sent to a home computer via FTP or SFTP, or for very large requests, sent on a medium of your choice (CDs, DVDs or tapes). One must be a registered archive user to be able to retrieve HST data (see http://archive.stsci.edu/hst/getting_started.html).Figure 15.1: Two-Dimensional RectificationFigure 15.2: Cosmic Ray RejectionFigure 15.3: One-Dimensional Spectral ExtractionThe calstis software is mostly written in C and uses open python in conjunction with a specially written I/O interface to the FITS data file.