Space Telescope Science Institute
STIS Data Handbook 2011
Table of Contents Previous Next Index Print

STIS Data Handbook > Chapter 3: STIS Calibration > 3.5 Recalibration of STIS Data

Sometimes the default pipeline calibration, performed shortly after the data were obtained from the telescope, is not the best possible calibration for your science program. There are a number of reasons why it may be desirable to recalibrate your data. The most likely reasons include:
More appropriate reference files have become available since the data were taken. CCD darks and biases are examples of reference files that are updated frequently, but they require some time to be installed in the pipeline. Further refinement of the dark files can be achieved by using daily dark exposures.
Contemporaneous CCD flat fields were obtained with the science data for G750L or G750M NIR observations to remove fringing.
Some steps need to be repeated with different input parameters. For example, you may wish to re-perform the cosmic ray rejection or the 1-D spectral extraction after adjusting the input parameters. The best target and background extraction regions for extracting 1-D spectra can depend on the science goals of the program.
Integer pixel dithers were performed along the slit to move hot pixels in a series of CCD spectroscopic exposures. These exposures are not associated and are thus not combined in pipeline processing. The _flt images must be aligned before they can be combined with ocrreject.
Spectral extractions will be made with heights less than the default size to isolate targets with small separations along the slit, or individual rows in a rectified spectral image of an extended target will be analyzed. More accurate spectral traces are needed to improve the extraction or rectification.
Be sure you are using the latest version of the calstis and STSDAS software. See for information about the latest release, and also see Section 3.6 of this chapter.
The simplest way to re-calibrate your data with the most appropriate reference files is to request the data from the archive again. However, to tailor the calibration to your individual preferences, it may be beneficial to run calstis yourself on your local machine, or to use tasks that improve the reference files or allow customized treatment of the data. The IRAF and PyRAF version of calstis, its constituent programs, and other STIS-specific tasks are available in the hst_calib.stis package in STSDAS.
The STIS calibration pipeline was designed to accommodate the need for full or partial recalibration. As mentioned at the beginning of this chapter, calstis is re-entrant, so that certain calibration steps can be performed outside of the pipeline, and others can be executed multiple times, depending upon the science goals.
Generally, the calibration switches in the header control the operations that calstis performs on the data. There are three basic ways to select which operations are performed during calibration:
Use one or more of the pipeline subset tasks or other STIS-specific tasks described below, managing the calibration through task parameters.
Run the calstis sub-tasks at the host level (i.e., outside of IRAF) using the command line switches and flags to control the processing.
This section describes the first two methods. In the end, the calibration switches in the headers of the calibrated data files will reflect the operations performed on the calibrated data and the reference files used.
You have chosen to fully recalibrate your STIS data. There is a certain amount of set up required for calstis to run properly. The operations mentioned in the checklist below will be described in detail in the following subsections:
Set the environment variable oref to point to your reference file directory. Note: you must do this before starting an IRAF session!
In an IRAF session, update the input data file headers (including reference file names) using chcalpar.
Run calstis or a subset of its constituent tasks.
Retrieve Reference Files
To recalibrate your data, you will need to retrieve the reference files used by the different calibration steps to be performed. The names of the reference files to be used during calibration must be specified in the primary header of the input files, under the section “CALIBRATION REFERENCE FILES.” Note that the data headers will be populated already with the names of the reference files used during pipeline calibration at STScI.
Chapter 1 of Part 1 of this handbook describes how to retrieve data and reference files via StarView and via the World Wide Web. To retrieve the best reference files for a given exposure via StarView, check "Best Reference Files" when selecting the data type(s) for retrieval. To retrieve them via the web tool, check "Best Reference Files" in the "Reference Files" section of the Retrieval Options form.
The STIS reference files are all in FITS format, and can be in either IMAGE or BINTABLE extensions. The names of these files along with their corresponding primary header keywords, extensions, and format (image or table), are listed in Chapter 2. The (somewhat obscure) rootname of a reference file is based on the time that the file was delivered to the Calibration Data Base System (CDBS).
Set up the Directory Structure for Running calstis
Before running calstis, you will need to define an environment variable to indicate the location of the directory containing the needed calibration reference files. The names of the calibration files are preceded with the logical path name “oref$” in the STIS science headers. Ordinarily you would define this directory in an IRAF session to be, for example, “/data/vega3/stis/cal_ref/” using the set command:
Note the trailing slash (/). However, calstis and all of its modules are actually foreign tasks and as such do not access IRAF environment variables. Therefore, before invoking the cl, you will need to define an environment variable from the host command line (see below) that is appropriate to your host machine. For Unix systems, the appropriate command for the example above is:
Then start IRAF. Note that an alternative to using the oref$ variable is specifying the full pathnames to the reference files in the science headers.
When running calstis or any of its modules, you must define environment variables (such as oref$) before starting the cl. It is not possible to define them within IRAF using the set command, nor is it possible to define them with an escape to the host level, such as:
!setenv oref /data/vega3/stis/cal_ref/
Edit the Calibration Header Keywords
To edit file headers in preparation for recalibration, use the STSDAS task chcalpar. The chcalpar task takes a single input parameter: the name(s) of the _raw data files to be edited. When you start chcalpar, the task automatically determines that the data are from STIS, determines the detector used and whether the observing mode was SPECTROSCOPIC or IMAGING, and opens one of four STIS-specific parameter sets (pset) that will load the current values of all the calibration related keywords. To edit the calibration keyword values:
Start the chcalpar task, specifying a list of images in which you want to change calibration keyword values. If you specify more than one image (using wildcards, for example) the task will read the initial keyword values from the first image in the list. For example, you could change keywords for all STIS _raw science images in the current directory (with initial values from the first image), using the command:
After starting chcalpar, you will be placed in eparam—the IRAF parameter editor; from there you will be able to edit the set of calibration keywords. Change the values of any calibration switches, reference files or tables to the values you wish to use for recalibrating your data.
Exit the editor when you are done making changes by typing “:q” two times. The task will ask if you wish to accept the current settings. If you type “y”, the settings will be saved and you will return to the IRAF cl prompt. If you type “n”, you will be placed back in the parameter editor to redefine the settings. If you type “a”, the task will abort and any changes will be discarded.
The parameter editor screen for STIS MAMA spectroscopy is illustrated in Figure 3.12.
Figure 3.12: Editing Calibration Keywords with chcalpar

Image Reduction and Analysis Facility
PACKAGE = ctools
TASK = ckwstis4
(dqicorr= perform) initialize data quality?
(lorscor= perform) convert to low-res?
(glincor= perform) correct global nonlinearity?
(lflgcor= perform) flag nonlinearity?
(darkcor= perform) dark correction?
(flatcor= perform) flat field correction?
(wavecor= perform) use wavecal?
(dispcor= perform) use dispersion solution?
(helcorr= perform) heliocentric correction?
(fluxcor= perform) convert to absolute flux?
(x2dcorr= perform) rectify 2-D spectral image?
(x1dcorr= perform) 1-D spectral extraction?
(backcor= perform) subtract background?
(rptcorr= omit) add individual repeat obs?
(bpixtab= oref$hcm14405o_bpx.fits) bad pixel table
(mlintab= oref$j9r16559o_lin.fits) MAMA linearity correction table
(darkfil= oref$q591955qo_drk.fits) dark reference file
(pfltfil= oref$mbj1658bo_pfl.fits) flat field reference file
(dfltfil= ) delta flat reference file
(lfltfil= oref$h2i1352bo_lfl.fits) low order flat reference file
(apertab= oref$n7p1032ao_apt.fits) aperture throughput table
(apdesta= oref$obm1723ro_apd.fits) aperture description table
(sptrcta= oref$q8l14504o_1dt.fits) 1-D spectrum trace table
(disptab= oref$m7p16110o_dsp.fits) dispersion coefficients table
(inangta= oref$h1v1541eo_iac.fits) incidence angle correction table
(lamptab= ) template lamp spectrum
(sdctab = oref$obm1723oo_sdc.fits) 2-D spectral extraction parameters
(phottab= oref$qb319512o_pht.fits) photometry calibration table
(xtracta= oref$j9h0900jo_1dx.fits) 1-D spectral extraction table
(instrum= stis) Instrument represented by this pset
(detecto= mama) Detector represented by this pset
(obstype= spectroscopic) Obstype represented by this pset
(Version= 5Sep2002) Date of Installation
(mode = al)

for HELP
It is also possible to use hedit to update the input file keywords. The example below illustrates how to turn on the bias correction switch and update the name of the bias image reference file for all STIS _raw images in the current directory that begin with the characters “o3y.”
cl> hedit o3y*_raw.fits[0] biascorr PERFORM up+
cl> hedit o3y*_raw.fits[0] biasfile "oref$new_bias.fits" up+
If you are changing keywords that reside in the FITS primary header unit with hedit, be sure to explicitly specify the primary header by appending “[0]” to the FITS file name.
The flags listed in Table 3.5 as controlling the basic calibration will usually be set to PERFORM, whether or not the data is fully calibrated. For data not fully calibrated by OTFR, some or all of the flags in the last column of Table 3.5 will be set to OMIT. To fully calibrate such data, the user can reset these flags in the primary extension header of the _raw file to PERFORM, and then run the calstis task with this _raw file as input. It is also possible to reset the necessary flags in the _flt or _crj file and then run calstis with this file as input to complete the steps that were omitted by the original calibration.
Table 3.5: STIS calibration switches commonly set to PERFORM
If WAVECORR is set to PERFORM, an appropriate STIS line lamp image should be named in the WAVECAL header keyword. This can be either the _wav file supplied with the dataset, or if a GO specified wavecal is used, the file name of the raw wavelength calibration lamp image should be put into this keyword.
The default calibration assumes that the target was centered in the aperture along the dispersion direction. If this is not correct, the wavelength scale will be incorrect, and this offset will apply the wrong sensitivity calibration at each wavelength. In such a case it may be better to perform only the basic 2-D calibrations using the calstis task, and to then perform the spectral extraction or rectification by running the x1d or x2d tasks independently, and taking the offset in the dispersion direction into account when running those tasks (see Section 5.5.2).
It is also possible to impose a wavelength offset by setting the WAVECORR keyword in the primary extension header to COMPLETE, and then setting the desired values for the SHIFTA1 and SHIFTA2 keywords in all science extension headers of the input image before running calstis. See the description of the WAVECORR task (Section 3.4.23) for an explanation of the SHIFTA1 and SHIFTA2 keywords. However, in the absence of a valid wavecal exposure, there is no straightforward way to determine what these values should be. In general they will be non-zero even in the absence of target offsets.
When running the calstis task itself, the calibration steps are controlled by setting the values of the calibration step keywords discussed in Section 3.4. These are set in the input data file’s primary extension header. However, when running the individual standalone tasks (e.g., basic2d or x1d), the calibration steps need to be turned on or off using the input parameters of each of the standalone tasks. The values of the calibration keywords in the data file’s header are usually ignored, unless they are set to COMPLETE. Most standalone tasks will not apply a correction if the header marks it in this way as already done.
However, regardless of whether calstis or one of the stand alone tasks is being used, all reference file names must be appropriately specified in the corresponding header keywords in the input data file’s primary extension header.
Selected portions of the pipeline can be executed with special tasks in the STSDAS stis package. The tasks that can be simply used in this fashion are listed in Table 3.6 below. See also Table 3.1 for the association between basic2d, ocrreject, wavecal, x1d, and x2d and the components of the calstis pipeline. See Chapter 5 for further information about some of these tasks. When you run these tasks individually, the calibration parameters usually read from the reference file must be entered either as command line arguments or via epar; however, most corrections will not be applied if the corresponding header keyword is already set to COMPLETE (see the Appendix of ISR STIS 98-26 and Section 6 of ISR STIS 98-10 for details).
The screen messages that appear when running any calstis module are equivalent to the trailer file contents delivered with the data.
The inttag task for TIME-TAG data will accumulate selected events from the raw event table, writing the results as one or more image sets (imsets) in a single, output FITS file. You can optionally specify an explicit starting time, time interval, and number of intervals over which to integrate, and the collection of imsets will be written to the output file, simulating a REPEATOBS ACCUM observation. Breaking the data into multiple, short exposures can be useful not only for variables but also to improve the flat fielding when the Doppler shift is significant. Once the new raw images have been created, it is straightforward to process them with calstis and to analyze the output image or spectra, as appropriate.
Table 3.6: calstis Pipeline Calibration Tasks
Radiation damage creates hot pixels in the STIS CCD Detector. Some of these hot pixels recover spontaneously. Many more have been repaired monthly by warming the CCD from its normal operating temperature to the ambient instrument temperature for several hours. Despite this annealing procedure, the number of permanently hot pixels has been increasing with time. The development of hot pixels is documented at:
Because the set of hot pixels changes continually, it is important to use the dark reference file prepared for the week of your observation. You should use the corresponding bias reference file, which corrects hot columns. If you requested your data close to the time of the observation, you should check whether new reference files applicable to your data have been made available since. If so, you may want to request your data from the archive again for automatic recalibration with the current reference files, since dark and bias correction occur so early in the calibration sequence.
The standard dark file subtracted from STIS data is based on a combination of several long dark files taken over an entire week. They usually don’t do a perfect job of subtracting hot pixels, both because the hot pixels can change on short time scales, and because some of the hotter pixels may have been saturated in the long dark images. It will therefore often be useful to construct a customized dark image for that particular day using short darks taken on the same day as your science image. Procedures for doing this are described at:
After you create your new daydark file, you should set the DARKFILE keyword in the [0] extension of the _raw files to reference this new dark file and then reprocess the data through calstis.
Even using a daydark will not give perfect removal of all bad pixels. There is no routine available that will automatically find and fix the bad pixels. It will often be necessary to identify them by hand and create a bad pixel mask. Once such a mask has been made, the IRAF routine fixpix may be useful. This task interpolates over bad pixels, and allows the bad pixel mask to be defined with a text file rather than a fits image. For data taken on the same day, most of the bad pixels will be in the same place in each image. It is therefore probably best to find the bad pixels in the _crj file, use fixpix to adjust the _crj fluxes and errors, and then run subsequent calibration and reduction programs on the corrected _crj file. For spectroscopic images of unresolved sources, interpolation within the spatially undersampled point spread function along the slit will be inaccurate, so fixpix should be used to interpolate along rows rather than along columns.
We have recommended that observers bypass the difficulties of calibrating hot pixels by using dither patterns that move the target by a few pixels from one exposure to the next. (See Section 11.3.5 in the STIS Instrument Handbook.) Spuriously high fluxes in hot pixels can then be rejected like cosmic rays when they are compared to normal fluxes in exposures made with offsets. See the following section (Section 3.5.4) for a discussion of aligning dithered images in preparation for cosmic ray rejection and for general information on handling and checking cosmic ray rejection.
For CCD datasets, calstis first runs the equivalent of basic2d with dqicorr, blevcorr, and biascorr set to PERFORM to initialize the ERR and DQ arrays, to trim the overscan regions and subtract the bias level measured there, and to subtract the bias image. It then performs cosmic ray rejection by running the equivalent of ocrreject. Next, the equivalent of basic2d is run with darkcorr and flatcorr set to PERFORM to perform dark subtraction and flat fielding. You can instead run basic2d just once on the raw data to perform all of its relevant tasks, then run ocrreject on the resulting _flt file, but this ordering of the tasks does not provide the optimum rejection of cosmic rays. The cosmic ray rejection is based on a "noise model." Part of that model is that the detected number of photons will follow a Poisson distribution. In order to compute the expected variation in detected photons at a pixel, i.e. to evaluate the noise model, you need a reasonable value for the number of electrons at that pixel. That estimate could be based on the median (for example) of the counts at that pixel in all the images that are to be combined. Because that value should reflect the number of detected photons, the bias level and bias image must first be subtracted from the images, because the electronic bias is an arbitrary offset. The data should not be corrected for dark counts or flat field, however. Dark counts do contribute to Poisson noise. The number of electrons to use for computing the Poisson distribution is the number that were actually detected, not the number corrected for variations in sensitivity. See ISR STIS 98-22 for further information on cosmic ray rejection. (Calstis was changed to perform the tasks in the order described here in July 1998, after the ISR was published.)
In cosmic ray rejection, one wants to strike a balance between failing to identify cosmic rays and clipping real flux from the target. The former lets spurious flux into the _crj image; the latter depresses the flux by systematically excluding values that are relatively high for that pixel because of noise fluctuations or small shifts of the target on the detector. Default values of cosmic ray rejection parameters to be used by calstis, or its component task ocrreject, are taken from the CRREJTAB reference file. If your _crj images still have many pixels affected by unrejected cosmic rays, you may want to try running ocrreject on the _flt files with different values for the rejection parameters, which will override the values taken from the CRREJTAB. Use crmask=yes to flag CRs in the input DQ images.
The optimum values of the cosmic ray rejection parameters depend on many factors, including the number of exposures to be combined, exposure time, signal-to-noise level, source structure, and the magnitude of the misalignment of the exposures due to drift of the target on the detector. (See ISR STIS 98-22.) For example, a small spatial misalignment of two exposures made with very high signal-to-noise may require one to increase the value of the parameter crsigmas (for a higher rejection threshold in sigmas) to avoid the systematic rejection of real flux. For a direct image or spectral image with only gradual spatial or spectral changes in flux from pixel to pixel, it may be possible to lower the rejection threshold to catch more cosmic rays without rejecting real flux. One may also need to change the value of crthresh, which sets a more stringent rejection level for the pixels surrounding a pixel that failed the crsigmas test. (e.g., for crthresh=0.5 and crsigmas=7, rejection occurs for surrounding pixels at a level of 0.5 * 7 sigma = 3.5 sigma out to a radius of crradius.)
If you want to use ocrreject to combine spectral images which have been dithered by a few pixels along the slit to move hot pixels, you should first apply sshift to the _flt files. This task will perform integer pixel shifts to produce aligned _sfl files. You will then need to set appropriate rejection parameter values for these files in ocrreject, since the CRREJTAB values depend on the value of CRSPLIT and the mean exposure time for an association of CRSPLIT exposures. The parameter values in the CRREJTAB can be used as a guide.
To see if real flux is being rejected by ocrreject, one can compare the _crj image to a summed image and look for systematically lower fluxes in the _crj image. Problems will tend to occur where the pixel-to-pixel flux changes are the greatest. For a direct image, examine the pixels near the centers of point sources. For a spectral image, examine the peak rows in the spectral image. One can also examine the DQ extensions of the _flt files after they have been flagged by ocrreject (crmask = yes). Systematic problems due to spatial misalignment are especially conspicuous in the DQ extensions of the _flt files of spectral images. Improperly flagged spectral images will have strings of DQ=8192 running along a row near the spatial peak of the spectrum, since that row will be less well centered on the target in one exposure than in another. (Remember to examine the DQ extension of the _flt files instead of the _crj file, since pixels in the latter will not be flagged for cosmic rays if their flux was computed from cosmic-ray-free pixels in any of the input exposures.)
Removing IR Fringes From Spectral Data
Detailed descriptions of the STIS tasks (normspflat, mkfringeflat, defringe, and prepspec) for using contemporaneous flat field images to remove IR fringing from STIS G750L and G750M spectra are given in STIS ISR 98-29. Note that this ISR was written before pixel-to-pixel flats were routinely subtracted from STIS spectroscopic data in the OTFR pipeline, and, as a result, includes some steps that are no longer always necessary.
Here we give a simplified and updated subset of these instructions for the most common case: a point source observed with the G750L.
The IR fringe flats may be taken with a variety of apertures. For point source fringe flats taken at or near a standard aperture position near the center of the detector, the best alignment between science image and fringe flat will usually be for fringe flats taken using the 0.3X0.09 aperture. For extended targets, or point sources located elsewhere along the slit, e.g., at the E1 aperture location near row 900, it will be necessary to use a long slit fringe flat, e.g, 52X0.1.
For G750L data, the defringing will usually be applied to the _crj image. In this example, the rootname of the fringe flat image is o8u201070, and the science image of the external target is o8u201060.
The normspflat task is used to prepare the fringe flat image and remove the low order lamp vignetting.
Fringe flat images taken with G750L include not only the IR fringing at wavelengths greater than 7500 Angstroms, but also some fringes at wavelengths less than 6000 Angstroms due to an order-sorter filter. Since these order-sorter fringes are already included in the sensitivity function, they should not be included in the fringe flat, and so these columns should be set to unity in the normalized fringe flat. This can easily be done by using the IRAF imcalc task to create a temporary array, and to then use the imcopy task to replace the appropriate columns in the fringe flat.
cl> imcalc o8u201070_nsp.fits temp_nsp.fits "if(x .lt. 250) then 1 else im1"
cl> imcopy temp_nsp.fits[1][1:250,*] o8u201070_nsp.fits[1][1:250,*]
The next task is to align and scale the IR fringes in the normalized flat to match those in the science image by using the mkfringeflat task.
It may be necessary to adjust a number of the input parameters of the mkfringeflat task to get proper alignment of the fringe flat and science image. See STIS ISR 98-29 for further details.
Then, to actually apply the fringe flat to remove the fringes from the science image:
The final output file will be the equivalent of the _crj file with the fringes removed. This file may be used as input to the x1d task to produce a final, fringe-subtracted 1-D spectrum.
To use an exposure other than the default wavecal, the user can either (1) put the name of the raw exposure file of the lamp image to be used into the WAVECAL keyword in the primary header of the _raw science image, make sure that the WAVECORR keyword is set to PERFORM, and then run calstis as normal, or (2) use the stis task wavecal, to measure the offset of the wavelength calibration image and populate the SHIFTA1 and SHIFTA2 keywords in the science file’s extension headers. For example, if o8mi02010 is the science observation and o8mi02ovq a corresponding wavelength lamp calibration image, the command would be
This will update the SHIFTA1 and SHIFTA2 keywords and set WAVECORR to complete. The modified science image file can then be processed with calstis and/or one or more of the stand alone STIS tasks, e.g.,
Note that the wavecal task is meant to update the headers of spectral images, prior to running the x1d or x2d tasks. It should not be used on images or tables (e.g., _sx1, _sx2, _x1d, or _x2d files) that have already had wavelengths assigned to individual pixels.
The orientation of spectral traces on the STIS detectors has been found to rotate slowly over time. Additionally, there is a small random offset in the orientation each time the MSM is positioned. The time-dependent component of the orientation has been calibrated for the most commonly used grating/central-wavelength combinations: G140L/1425, G230L/2376, G230LB/2375, G430L/4300, G750L/7751, G750M/6581, G750M/6768, and G750M/8561. Calstis has been modified to apply time-dependent rotation to spectral traces, and the evolutionary parameters for these modes were included in SPTRCTAB reference files that were delivered in August and October 2006. For other L and M modes with uncalibrated trace rotation, the PyRAF task mktrace can be used to generate a SPTRCTAB with accurately rotated traces for a specific science image. The task can also be applied to data taken with the calibrated modes to correct for the random offset in orientation due to the positioning of the MSM.
The mktrace task computes the trace of the science target using the default aperture location or accepting an input location. A linear fit is made to this trace. The user can choose to exclude portions of the trace from the fit because of velocity structure, low signal-to-noise, etc. A calibration trace is interpolated from the traces at the nearest rows in the SPTRCTAB, and a linear fit is made using the same ranges of columns selected for the fitting of the science trace. Image files of the target trace and calibration trace and of the linear fits are generated to assist the user in selecting the portions of the spectrum to include in the fits. The difference in the angle of the two linear fits is the correction that must be applied to the SPTRCTAB traces. A new SPTRCTAB is generated, applying this correction to all of the traces for that grating/cenwave combination, thus covering the entire detector. The keyword SPTRCTAB in the primary header of the science image is changed to the name of the new trace file, so that subsequent processing by calstis, x1d, or x2d will use the new file.

Table of Contents Previous Next Index Print