Part II: STIS Data Handbook

4.1 Error Sources Associated with Pipeline Calibration Steps

---

In this section, we discuss sources of error that are associated with major steps in the STIS calibration pipeline (calstis). Note that these steps themselves were already described in Chapter 3 and will not be repeated here; this section will only describe specific issues related to the error budget of the resulting data which were not described before.

4.1.1 Readout Noise and A-to-D Conversion

Readout Noise (only relevant for CCD observations)

The readout noise is an unavoidable contribution to the total error budget of CCD observations. (MAMA observations do not suffer from readout noise.) It is linked to the readout process and there are no reduction steps that can minimize or remove it. The effect of the read noise to science measurements can however be minimized by keeping the number of pixels where signal is measured small and by minimizing the number of CCD readouts (while still allowing for good cosmic ray and hot pixel removal). The read noise is independent of position on the CCD, but it does vary with the gain setting. Furthermore, the read noise of the STIS CCD has been found to vary with time. The main changes occurred after Servicing Mission 3a (January 2000) and after the switch to the Side 2 electronics (July 2001). A table of read noise values of the STIS CCD as a function of time and gain setting can be found at the following URL:

http://www.stsci.edu/hst/stis/performance/monitoring/calibration/Monitors/Read-Noise

A-to-D Conversion (only relevant for CCD observations)

The analog information (electrons) accumulated in the CCD are converted into data numbers (DN) by an analog-to-digital converter (ADC). The STIS CCD camera employs a 16-bit ADC, which can produce a maximum of 2¹⁶ = 65, 535 DN. However, this is not a limitation for STIS CCD operations, because other factors set the maximum observable DN to lower levels in both supported gain settings. (For the CCDGAIN = 1 setting, the gain amplifier already saturates at ~33,000 e^-/pixel, while the full well of the CCD occurs at 144,000 e^-/pixel, which is before digital saturation would occur in the CCDGAIN = 4 setting.) The ADC produces only discrete output levels. This means that a range of analog inputs can produce the same digital output. This round-off error is called quantization noise (QN). It can be shown (Janesick, 2001, Scientific Charge Coupled Devices, SPIE Press) that the quantization noise is constant for a given gain setting when expressed in DN: QN(DN) = 12^-1/2. The quantization noise can be converted into noise electrons as follows: QN(e^-) = 0.288675*g, where g is the ATODGAIN (from the CCDTAB). The total noise (TRN) associated with CCD readout noise (RN) and quantization noise is obtained by adding the two figures in quadrature: TRN = (RN² + (0.288675*g)²).

4.1.2 Bias Subtraction (only relevant for CCD observations)

CCD bias frames were acquired daily for scientific calibration purposes and for monitoring detector performance. Every week multiple CCD bias frames were combined together into a reference superbias image. The combination removes the cosmic rays accumulated during the readout time and reduces the noise associated with the bias subtraction procedure to a level that is insignificant relative to the read noise associated with any single image.

The calstis calibration pipeline performs the bias correction in two steps (see Section 3.4): BLEVCORR subtracts the bias level from the overscan regions in two steps: it first measures the level and slope in part of the 19 pixel wide leading physical overscan region and then does the same in a 20 row wide virtual overscan region. After the BLEVCORR procedure, some structure remains in the bias frames. This structure includes real bias signal ("spurious charge") and also some dark current accumulated during the time required to read out the detector. However, this structure is stable on timescales of a few weeks. To remove this structure a superbias frame with high signal-to-noise ratio is subtracted from each science image in the BIASCORR step.

This process has been found to be highly stable and reliable. Details can be found in STIS ISR 97-09 and STIS ISR 98-31.

4.1.3 Dark Current and Hot Pixels

CCD Dark Current and Hot Pixels

Similar to the case of CCD bias frames, CCD dark frames were acquired on a daily basis, and combined together every week for use in the calstis calibration pipeline. The creation procedure of the CCD dark reference files involves two main steps:

1. Every month, a high signal-to-noise superdark frame was created from a combination of typically 40-60 "long" darks (dark frames with exposure time > 900 s). These monthly superdark frames are not actually delivered to the calibration data base, but used as "baseline" dark for the next steps.
2. All "long" darks taken during a given week were combined together (rejecting cosmic rays in the process), and normalized to intensity units of e^-/s. After that, the hot pixels in the monthly baseline dark (created as described above) are replaced by those of the normalized weekly superdark. These new 'hot' pixels have a value higher than "median dark current + 5 of the normalized weekly superdark." Previously hot pixels that have fully annealed out between the observing dates of the baseline and the new dark are assigned a value equal to that in a median filtered version of the baseline dark. The resulting dark has the high signal-to-noise ratio of the monthly baseline superdark, updated with the hot pixels of the current week. These "weekly darks" are the STIS CCD dark reference files used in the calstis pipeline (and are retrievable from the HST Archive).

We emphasize that the use of the correct dark reference file is important for most science applications. An incorrect dark reference file will likely introduce a poor dark correction. It will either leave too many hot pixels uncorrected and unflagged, or create many negative "holes" caused by the correction of hot pixels which were not actually hot in the science data (e.g., if the CCD was annealed in the meantime).

Finally, observers have the option to create "daily darks" for their CCD observations, i.e., CCD dark files for which the hot pixels have been optimized for any given date. These "daily darks" are not created within the automatic calstis pipeline, but observers have been provided with the daydark IRAF cl script within the STIS package of STSDAS as well as detailed instructions to create such files by themselves. Details are described on the following page of the STIS Web site.

http://www.stsci.edu/hst/stis/software/scripts/daydark.

In terms of long term trends of the CCD dark current rate, it is relevant to recall that the STIS CCD temperature was stabilized at -83 ºC between its installation onto HST on 1997-Feb-14 and the failure of the Side 1 electronics on 2001-May-16. During the Side 1 era, the CCD median dark rate rose from 0.0013 to 0.0043 e^-/s/pixel. CCD temperature cannot be stabilized with the Side 2 electronics, so thermal fluctuations perturbed the dark rate during the Side 2 era. These fluctuations are discussed in STIS ISR 2001-03. The calstis OTFR pipeline does scale the CCD dark current according to temperature, but users who require the best possible dark subtraction may improve upon the OTFR results by scaling their dark files themselves, using a log of CCD housing temperature that is finely sampled in time. This log is available at:

ftp://ftp.stsci.edu/pub/instruments/stis/CCD_Housing.tar.gz

Figure 4.1 shows the measured median CCD dark current as a function of time.

 
Figure 4.1: Median CCD Dark Current vs. Time
 

MAMA Dark Current

   

NUV-MAMA Dark Current

Most of the dark current in the NUV-MAMA detector is caused by phosphorescence of impurities in the MgF₂ detector faceplate. A simple model of the phenomenon has been developed by Jenkins and Kimble (private communication) that envisions a population of impurity sites each having three levels: (1) a ground state, (2) an excited energy level which can decay immediately to the ground state, and (3) a metastable level that is at an energy slightly below the one that can emit radiation. The metastable state can be thermally excited to the upper level, and this excitation rate is proportional to exp(-E/kT), where E is the energy difference between the levels. The behavior of the count rate versus temperature leads to an estimate of 1.1 eV for E. At a fixed detector temperature of 30 ºC, the time constant for the dark current to reach an equilibrium value is about 8 days. However, since the MAMA high voltage power supplies have to be shut down during orbits impacted by the South Atlantic Anomaly (SAA), the detector temperature varies from about 27 ºC to 40 ºC, and the dark current never actually reaches equilibrium.

MAMA temperatures cycle on a roughly daily time scale, being lowest just after the high voltage is turned on after an SAA passage. The dark current can be predicted with about 5% to 10% accuracy using the contemporaneous temperature of the NUV-MAMA charge amplifier recorded in the OM2CAT keyword in the _raw science file that is part of the standard data products. Originally the NUV-MAMA dark current was fit with the curve darkrate = 9.012 × 1020 × exp(-12710/T). This worked well for the first two years of STIS operations, but as the mean time-averaged temperature of the detector increased with time, this formula started to predict too high a dark current. Once this was realized, a more flexible fitting formula was implemented in the calstis pipeline. This updated formula for the dark current was

darkrate = norm × 1.805 × 1020 × exp[-12211.8 / max(T, Tmin)],

where both norm and Tmin are slowly varying functions of time that are empirically adjusted to give a good match to the observed dark rate, and which are tabulated in the temperature dependent dark correction table (_tdc) reference file. The uncertainty related to the application of the updated NUV-MAMA dark rate is 5% rms. Note however that this doesn't translate to any significant photometric uncertainty, since any residue of the dark subtraction process gets subtracted out during the extraction of imaging photometry or spectroscopic flux.

   

FUV-MAMA Dark Current

The FUV-MAMA dark current is very low indeed. In the early years of STIS operations, dark current values as low as 7 counts/s integrated over the whole detector were routinely encountered. However, there is also an intermittent glow that covers a large fraction of the detector (see Figure 4.2). The source of the dark current is not phosphorescence but is intrinsic to the micro-channel plate array. This glow can substantially increase the dark current over a large fraction of the detector, and this leads to count rates of up to 100 counts/s integrated across the face of the detector (which is of course still very low). During the first two years of STIS operations, this glow was only present intermittently, but since mid-1999 it showed up more often than not. This extra glow appears to be a function of both the detector temperature, and the length of time the detector has been on. (The STIS MAMA detectors were turned off daily during HST's passages through the South Atlantic Anomaly). However, the strength and frequency with which this glow appears increased throughout the lifetime of STIS. At later epochs, only the first non-SAA-impacted orbit each day could be reliably expected to be free of the extra glow.

 
Figure 4.2: Dark Current Variation Across FUV-MAMA Detector.
 

An example of the FUV-MAMA dark current variation across the detector can be seen in Figure 4.2, which is the sum of a number of 1380 second dark frames taken during periods of high dark current. The dark current in the lower right quadrant (pixels [900:1000,10:110] in IRAF notation) appears to be stable to within 10% over time. The dark current in the upper left quadrant (pixels [200:400,600:800]) varies with time and temperature. The total dark current can be approximated by the sum of a constant dark current plus a "glow" image, scaled to the net rate in the upper left quadrant. No reliable method has been found to predict the brightness of the glow with sufficient accuracy to subtract it from pipeline processed FUV-MAMA images, and so FUV-MAMA dark images provided by archive contain only the base dark current plus the mean values for any pixels flagged as hot pixels. For the FUV-MAMA, hot pixels are defined to be those that show an average dark current of more than 1 × 10^-4 counts/hi-res-pixel/s as measured over several months of dark monitor observations. In 1997, only 97 out of the more than 4 million pixels had a dark rate of more than 1 × 10^-4 counts/s. By the time of the Side 2 failure, more than 3,000 pixels were at this level.

Subtracting the extra glow requires identifying a region on the detector where the glow is strong, but the intrinsic target flux is know to be weak, and then using this region to scale and subtract a smoothed image of the glow. For long slit spectroscopic observations, the region of the detector occulted by one of the aperture bars may sometimes be useful for this purpose. Mean glow images for different epochs can be obtained from the STIS Web page at:

http://www.stsci.edu/hst/stis/performance/background,

and can be used for off-line data reduction if appropriate.

However, it is good to keep in mind that the FUV-MAMA dark current is so low that a typical STIS FUV-MAMA observation will have less than one count per pixel from the dark, so that many science projects will not require special data reduction efforts in this respect. Also, various standard measures of background (the median for example) are not good estimates of the dark when the data are quantized into just a few values. The best way to estimate the background is to identify hot pixels using the dark reference files, and then use an unclipped mean for the remaining pixels in a source- free region of the image.

4.1.4 Flat Fields

The flat field reference files currently in use by the calstis calibration pipeline were derived by several different methods, depending on the detector, the observing mode (imaging vs. spectroscopy), and (for some modes) the optical elements used. For many observing modes, there are two types of flat fields involved in the calibration: p-flats (the *_pfl.fits reference files) which correct for the pixel-to-pixel sensitivity variations, and l-flats (*_lfl.fits) which model the low-frequency variations in sensitivity over the field of view of the observing mode in question. Important features regarding the accuracy of the flat fields for different observing modes are described in the following sections.

CCD Imaging Flat Fields

• p-flat: The p-flat reference file for CCD imaging was created from several exposures of the internal tungsten lamp during ground calibration. The total accumulated intensity was at least 100,000 electrons/pixel, except for the borders of the image which are partially vignetted by the mode select mechanism (MSM). After image combination, the lamp illumination function was removed through division by a median-filtered version of the image (using a 55 × 55 pixel kernel). The vignetted borders of the image were flagged during this procedure (note that due to the intrinsic non-repeatability of the MSM, the exact vignetting of the borders of the image is different every time the MSM is moved to the CCD imaging mode). The structure of the CCD p-flat is shown in Figure 4.3. The most obvious features are circular dark patterns with typical diameters of ~20 pixels. These artifacts are shadows of dust on the CCD window, often called "dust motes". They are removed by the p-flat to well within 1%. On-orbit testing of CCD imaging p-flat images has shown that the p-flat is stable to within about 0.2% after correction for Poisson noise.
• l-flat: The low-order flat field reference file for CCD imaging was created from numerous images taken during the Hubble Deep Field South (HDF-S) campaign in October 1998. After bias subtraction and dark subtraction using dedicated high-signal-to-noise combinations of bias and dark files, the images were masked for cosmic rays and heavily masked around all sources detected in the individual images. The combined image was then fit by a 5-piece cubic spline function. The main purpose of this l-flat is a correction of the 5-10% sensitivity roll off near the edges of the 50CCD aperture which remains after application of the CCD imaging p-flat described above.

 
Figure 4.3: Pixel-to-pixel flat field for the STIS CCD with a stretch from 0.90 (black) to 1.10 (white). The intensity transfer table is shown in the bar at the bottom. The numerous ~20 pixel size dark circles in the field are caused by dust particles on the CCD window.
 

CCD Spectroscopic Flat Fields

• p-flats: P-flat reference files for the CCD spectroscopic modes were originally created from high S/N spectra of deuterium and tungsten lamps during ground calibration of STIS. They were first updated by in-flight data on 1999-Sep-20 (and several updates followed afterwards). Flat field data were taken in flight on a regular schedule and in every supported observation mode in order to enable one to identify any dependence on in-flight epoch and/or the grating used. Details of the flat field file creation procedure and analysis are published in STIS ISR 99-06. The main results of this analysis were:
  (a) The pixel-to-pixel flat field for the CCD has been found to be independent (to within the limits set by Poisson statistics) on wavelength, CCD gain setting, and signal level.
  (b) The structure of the L-mode flat fields is different from that of the M modes, due to the different optical magnification of the two sets of modes. Hence, flat fields have been derived separately for L- and M-modes.
  (c) A time dependence of the structure of the flat field was identified. The systematic (non-Poisson) effect is of order 0.3% per year. To keep this effect within 1%, spectroscopic CCD flat fields have been delivered for all spectroscopic CCD modes with USEAFTER dates of April 1997, July 1999, August 2000, and October 2001.
• l-flats: To probe the sensitivity as a two-dimensional function of wavelength and position along the slit, a number of calibration programs were performed in which a bright star was dithered up and down the length of the 52X2 slit. In addition, CCD sensitivity monitor measurements made after April 2002 routinely included measurements at both the central and E1 aperture positions on the slit (see Section 7.2.7 of the STIS Instrument Handbook). The ratio of sensitivities at the E1 position to the central row is therefore very well measured, while more limited data is available at other positions along the slit. The latter measurements feature 1 to 4 arcsec spacing along the cross-dispersion direction.
  
  L-flats were produced for the G140L, G230LB, G430L, and G750L gratings. The medium-resolution gratings lack sufficient data at most central wavelengths, while the G230L mode was found not to require a low-order flat field correction. The l-flats are defined to be unity at the row where the absolute sensitivity calibration is determined. The typical effect of applying the l-flats is of order 1-2% in sensitivity along the dispersion (depending on the grating in question) and 2-3% perpendicular to the dispersion. It should be kept in mind that the l-flat corrections are only strictly applicable to targets positioned along the center line of the slits. Targets that are significantly offset in the dispersion direction will not be corrected with the same degree of accuracy, especially near the extremes of the wavelength coverage of the grating used. More details are available in Proffitt (2005 HST Calibration Workshop).

MAMA Flat Fields

• p-flats: The MAMA flat fields are dominated by a fixed pattern that is a combination of several effects including "beating" between the micro-channel plate array and the anode pixel array and variations in the charge cloud structure at the anode. For this reason, the flat field correction for MAMA data is applied to the data in unbinned format (2048 × 2048 pixel²). Intrinsic pixel-to-pixel variations are 3.9% rms for the FUV-MAMA and 2.8% rms for the NUV-MAMA, respectively, in the format delivered by the calstis pipeline ("low-resolution pixels", 1024 × 1024 pixel²).
  
  The creation of high-signal-to-noise-ratio p-flats with the STIS MAMA detectors is done using the internal Krypton and Deuterium lamps. It is a rather long and tedious process, since individual exposures have to adhere to the global count rate linearity limit for MAMA detectors¹. For a typical 20 min exposure (which can be executed when HST's view is occulted by the Earth), this limits the accumulation of counts to only 230 counts/pixel, so that at least 50 such exposures need be combined (per detector) to achieve a signal to noise ratio of 100 (due to Poisson statistics) per low-resolution pixel. Since one also has to take exposures at several slit locations in order to illuminate regions of the detector that are normally occulted by the slit fiducial bars (see STIS Instrument Handbook, Chapter 7) and other scheduling limitations, it takes a full HST cycle to achieve this.
  
  The current MAMA p-flat reference files have been in use by the calstis pipeline since 2002-Nov-15, and have a signal-to-noise ratio of ~200 per pixel. A detailed analysis of the quality of the MAMA p-flats using well-illuminated spectra of flux standard stars (white dwarfs) shows that the rms of extracted spectra (summing 11 MAMA rows) is of order 1%. This is larger than the rms of the p-flats due to Poisson statistics, and it represents the current limit of the quality of flux calibration of individual targets from individual exposures (imaging or spectra). STIS users whose science requires higher signal-to-noise ratios have been directed to use multiple exposures using FP-SPLIT slits or spatial dithering along the slit (see STIS Instrument Handbook, Chapter 12).
• Spectroscopic l-flats: The need for L-flats was considered for the low-resolution spectral modes G140L and G230L, as described in detail above for the CCD spectral modes. It was determined that an l-flat correction was only required for the G140L mode.
• Imaging l-flat: MAMA imaging l-flats were determined from observations of the Galactic globular cluster NGC 6681 obtained at a large number of offset pointing positions (144 for the NUV-MAMA, 207 for the FUV-MAMA) using the "CLEAR" (25MAMA) filter. The l-flat consists of a two-dimensional, second-order polynomial surface fit to the ratios of large-aperture fluxes obtained for the same stars, normalized to unity in the inner 400 × 400 pixel² region. The results showed that an imaging l-flat is only needed for the FUV-MAMA, which features a significant ~22% peak-to-peak variation in sensitivity over the field of view (mainly due to variations towards the upper right corner and near the left-hand edge of the detector). The l-flat corrects this to 2% rms. In contrast, the sensitivity variations of the NUV-MAMA stay below 1% rms.

¹200,000 counts/s; given the lamp brightness, this can only be achieved with the G140M and G230M gratings and a narrow slit. Hence these p-flats are applied to both imaging and spectroscopic MAMA data.