Sometimes the default pipeline calibration, performed shortly after the data were
obtained from the telescope, is not the best possible calibration for your science program. There are a number of reasons why it may be desirable to recalibrate your data. The most likely reasons include:
The simplest way to re-calibrate your data with the most appropriate reference files
is to request the data from the archive again. However, to tailor the calibration to your individual preferences, it may be beneficial to run calstis
yourself on your local machine, or to use tasks that improve the reference files or allow customized treatment of the data. The IRAF
version of calstis
, its constituent programs, and other STIS-specific tasks are available in the hst_calib.stis
package in STSDAS
The STIS calibration pipeline was designed to accommodate the need for full or
partial recalibration. As mentioned at the beginning of this chapter, calstis
is re-entrant, so that certain calibration steps can be performed outside of the pipeline, and others can be executed multiple times, depending upon the science goals.
Generally, the calibration switches in the header control the operations that calstis
performs on the data. There are three basic ways to select which operations are performed during calibration:
|Run the calstis
sub-tasks at the host level (i.e., outside of IRAF
) using the command line switches and flags to control the processing.
This section describes the first two methods. In the end, the calibration switches in the
headers of the calibrated data files will reflect the operations performed on the calibrated data and the reference files used.
You have chosen to fully recalibrate your STIS data. There is a certain amount of
set up required for calstis
to run properly. The operations mentioned in the checklist below will be described in detail in the following subsections:
|In an IRAF
session, update the input data file headers (including reference file names) using chcalpar
or a subset of its constituent tasks.
To recalibrate your data, you will need to retrieve the reference files used by the
different calibration steps to be performed. The names of the reference files to be used during calibration must be specified in the primary header of the input files, under the section “CALIBRATION REFERENCE FILES.” Note that the data headers will be populated already with the names of the reference files used during pipeline calibration at STScI.
Chapter 1 of Part 1 of this handbook describes how to retrieve data and reference
files via StarView and via the World Wide Web. To retrieve the best reference files for a given exposure via StarView, check "Best Reference Files" when selecting the data type(s) for retrieval. To retrieve them via the web tool, check "Best Reference Files" in the "Reference Files" section of the Retrieval Options form.
The STIS reference files are all in FITS format, and can be in either IMAGE or
BINTABLE extensions. The names of these files along with their corresponding primary header keywords, extensions, and format (image or table), are listed in Chapter 2
. The (somewhat obscure) rootname of a reference file is based on the time that the file was delivered to the Calibration Data Base System (CDBS).
Before running calstis
, you will need to define an environment variable to indicate the location of the directory containing the needed calibration reference files. The names of the calibration files are preceded with the logical path name “oref$
” in the STIS science headers. Ordinarily you would define this directory in an IRAF
session to be, for example, “/data/vega3/stis/cal_ref/
” using the set
Note the trailing slash (/). However, calstis
and all of its modules are actually foreign tasks and as such do not access IRAF
environment variables. Therefore, before invoking the cl
, you will need to define an environment variable from the host command line (see below) that is appropriate to your host machine. For Unix systems, the appropriate command for the example above is:
Then start IRAF
. Note that an alternative to using the oref$
variable is specifying the full pathnames to the reference files in the science headers.
To edit file headers in preparation for recalibration, use the STSDAS
. The chcalpar
task takes a single input parameter: the name(s) of the _raw data files to be edited. When you start chcalpar
, the task automatically determines that the data are from STIS, determines the detector used and whether the observing mode was SPECTROSCOPIC or IMAGING, and opens one of four STIS-specific parameter sets (pset
) that will load the current values of all the calibration related keywords. To edit the calibration keyword values:
|Start the chcalpar
task, specifying a list of images in which you want to change calibration keyword values. If you specify more than one image (using wildcards, for example) the task will read the initial keyword values from the first image in the list. For example, you could change keywords for all STIS _raw science images in the current directory (with initial values from the first image), using the command:
|After starting chcalpar
, you will be placed in eparam
parameter editor; from there you will be able to edit the set of calibration keywords. Change the values of any calibration switches, reference files or tables to the values you wish to use for recalibrating your data.
|Exit the editor when you are done making changes by typing “:q
” two times. The task will ask if you wish to accept the current settings. If you type “y
”, the settings will be saved and you will return to the IRAF cl
prompt. If you type “n
”, you will be placed back in the parameter editor to redefine the settings. If you type “a
”, the task will abort and any changes will be discarded.
Image Reduction and Analysis Facility
PACKAGE = ctools
TASK = ckwstis4
(dqicorr= perform) initialize data quality?
(lorscor= perform) convert to low-res?
(glincor= perform) correct global nonlinearity?
(lflgcor= perform) flag nonlinearity?
(darkcor= perform) dark correction?
(flatcor= perform) flat field correction?
perform) use wavecal?
perform) use dispersion solution?
perform) heliocentric correction?
perform) convert to absolute flux?
perform) rectify 2-D spectral image?
perform) 1-D spectral extraction?
perform) subtract background?
(rptcorr= omit) add individual repeat obs?
hcm14405o_bpx.fits) bad pixel table
oref$j9r16559o_lin.fits) MAMA linearity correction table
oref$q591955qo_drk.fits) dark reference file
oref$mbj1658bo_pfl.fits) flat field reference file
(dfltfil= ) delta flat reference file
oref$h2i1352bo_lfl.fits) low order flat reference file
oref$n7p1032ao_apt.fits) aperture throughput table
oref$obm1723ro_apd.fits) aperture description table
oref$q8l14504o_1dt.fits) 1-D spectrum trace table
oref$m7p16110o_dsp.fits) dispersion coefficients table
oref$h1v1541eo_iac.fits) incidence angle correction table
(lamptab= ) template lamp spectrum
oref$obm1723oo_sdc.fits) 2-D spectral extraction parameters
oref$qb319512o_pht.fits) photometry calibration table
oref$j9h0900jo_1dx.fits) 1-D spectral extraction table
(instrum= stis) Instrument represented by this pset
(detecto= mama) Detector represented by this pset
(obstype= spectroscopic) Obstype represented by this pset
(Version= 5Sep2002) Date of Installation
(mode = al)
It is also possible to use hedit
to update the input file keywords. The example below illustrates how to turn on the bias correction switch and update the name of the bias image reference file for all STIS _raw images in the current directory that begin with the characters “o3y
|cl> hedit o3y*_raw.fits biascorr PERFORM up+
cl> hedit o3y*_raw.fits biasfile "oref$new_bias.fits" up+
The flags listed in Table 3.5
as controlling the basic calibration will usually be set to PERFORM
, whether or not the data is fully calibrated. For data not fully calibrated by OTFR, some or all of the flags in the last column of Table 3.5
will be set to OMIT
. To fully calibrate such data, the user can reset these flags in the primary extension header of the _raw file to PERFORM
, and then run the calstis
task with this _raw file as input. It is also possible to reset the necessary flags in the _flt or _crj file and then run calstis
with this file as input to complete the steps that were omitted by the original calibration.
is set to PERFORM
, an appropriate STIS line lamp image should be named in the WAVECAL
header keyword. This can be either the _wav file supplied with the dataset, or if a GO specified wavecal is used, the file name of the raw wavelength calibration lamp image should be put into this keyword.
The default calibration assumes that the target was centered in the aperture along
the dispersion direction. If this is not correct, the wavelength scale will be incorrect, and this offset will apply the wrong sensitivity calibration at each wavelength. In such a case it may be better to perform only the basic 2-D calibrations using the calstis
task, and to then perform the spectral extraction or rectification by running the x1d
tasks independently, and taking the offset in the dispersion direction into account when running those tasks (see Section 5.5.2
It is also possible to impose a wavelength offset by setting the WAVECORR
keyword in the primary extension header to COMPLETE
, and then setting the desired values for the SHIFTA1
keywords in all science extension headers of the input image before running calstis
. See the description of the WAVECORR
task (Section 3.4.23
) for an explanation of the SHIFTA1
keywords. However, in the absence of a valid wavecal exposure, there is no straightforward way to determine what these values should be. In general they will be non-zero even in the absence of target offsets.
When running the calstis
task itself, the calibration steps are controlled by setting the values of the calibration step keywords discussed in Section 3.4
. These are set in the input data file’s primary extension header. However, when running the individual standalone tasks (e.g., basic2d
), the calibration steps need to be turned on or off using the input parameters of each of the standalone tasks. The values of the calibration keywords in the data file’s header are usually ignored, unless they are set to COMPLETE
. Most standalone tasks will not apply a correction if the header marks it in this way as already done.
However, regardless of whether calstis
or one of the stand alone tasks is being used, all reference file names must be appropriately specified in the corresponding header keywords in the input data file’s primary extension header.
Selected portions of the pipeline can be executed with special tasks in the STSDAS stis
package. The tasks that can be simply used in this fashion are listed in Table 3.6
below. See also Table 3.1
for the association between basic2d, ocrreject, wavecal, x1d
, and x2d
and the components of the calstis
pipeline. See Chapter 5
for further information about some of these tasks. When you run these tasks individually, the calibration parameters usually read from the reference file must
be entered either as command line arguments or via epar
; however, most corrections will not be applied if the corresponding header keyword is already set to COMPLETE
(see the Appendix of ISR STIS 98-26
and Section 6 of ISR STIS 98-10
The screen messages that appear when running any calstis
module are equivalent to the trailer file contents delivered with the data.
task for TIME-TAG data will accumulate selected events from the raw event table, writing the results as one or more image sets (imsets) in a single, output FITS file. You can optionally specify an explicit starting time, time interval, and number of intervals over which to integrate, and the collection of imsets will be written to the output file, simulating a REPEATOBS ACCUM observation. Breaking the data into multiple, short exposures can be useful not only for variables but also to improve the flat fielding when the Doppler shift is significant. Once the new raw images have been created, it is straightforward to process them with calstis
and to analyze the output image or spectra, as appropriate.
Radiation damage creates hot pixels in the STIS CCD Detector. Some of these hot
pixels recover spontaneously. Many more have been repaired monthly by warming the CCD from its normal operating temperature to the ambient instrument temperature for several hours. Despite this annealing procedure, the number of permanently hot pixels has been increasing with time. The development of hot pixels is documented at:
Because the set of hot pixels changes continually, it is important to use the dark
reference file prepared for the week of your observation. You should use the corresponding bias reference file, which corrects hot columns. If you requested your data close to the time of the observation, you should check whether new reference files applicable to your data have been made available since. If so, you may want to request your data from the archive again for automatic recalibration with the current reference files, since dark and bias correction occur so early in the calibration sequence.
The standard dark file subtracted from STIS data is based on a combination of
several long dark files taken over an entire week. They usually don’t do a perfect job of subtracting hot pixels, both because the hot pixels can change on short time scales, and because some of the hotter pixels may have been saturated in the long dark images. It will therefore often be useful to construct a customized dark image for that particular day using short darks taken on the same day as your science image. Procedures for doing this are described at:
After you create your new daydark file, you should set the DARKFILE
keyword in the  extension of the _raw files to reference this new dark file and then reprocess the data through calstis
Even using a daydark will not give perfect removal of all bad pixels. There is no
routine available that will automatically find and fix the bad pixels. It will often be necessary to identify them by hand and create a bad pixel mask. Once such a mask has been made, the IRAF
may be useful. This task interpolates over bad pixels, and allows the bad pixel mask to be defined with a text file rather than a fits image. For data taken on the same day, most of the bad pixels will be in the same place in each image. It is therefore probably best to find the bad pixels in the _crj file, use fixpix to adjust the _crj fluxes and errors, and then run subsequent calibration and reduction programs on the corrected _crj file. For spectroscopic images of unresolved sources, interpolation within the spatially undersampled point spread function along the slit will be inaccurate, so fixpix
should be used to interpolate along rows rather than along columns.
We have recommended that observers bypass the difficulties of calibrating hot
pixels by using dither patterns that move the target by a few pixels from one exposure to the next. (See Section 11.3.5 in the STIS Instrument Handbook
.) Spuriously high fluxes in hot pixels can then be rejected like cosmic rays when they are compared to normal fluxes in exposures made with offsets. See the following section (Section 3.5.4
) for a discussion of aligning dithered images in preparation for cosmic ray rejection and for general information on handling and checking cosmic ray rejection.
For CCD datasets, calstis
first runs the equivalent of basic2d
, and biascorr
set to PERFORM
to initialize the ERR and DQ arrays, to trim the overscan regions and subtract the bias level measured there, and to subtract the bias image. It then performs cosmic ray rejection by running the equivalent of ocrreject
. Next, the equivalent of basic2d
is run with darkcorr
set to PERFORM
to perform dark subtraction and flat fielding. You can instead run basic2d
just once on the raw data to perform all of its relevant tasks, then run ocrreject
on the resulting _flt file, but this ordering of the tasks does not provide the optimum rejection of cosmic rays. The cosmic ray rejection is based on a "noise model." Part of that model is that the detected number of photons will follow a Poisson distribution. In order to compute the expected variation in detected photons at a pixel, i.e. to evaluate the noise model, you need a reasonable value for the number of electrons at that pixel. That estimate could be based on the median (for example) of the counts at that pixel in all the images that are to be combined. Because that value should reflect the number of detected photons, the bias level and bias image must first be subtracted from the images, because the electronic bias is an arbitrary offset. The data should not be corrected for dark counts or flat field, however. Dark counts do contribute to Poisson noise. The number of electrons to use for computing the Poisson distribution is the number that were actually detected, not the number corrected for variations in sensitivity. See ISR STIS 98-22
for further information on cosmic ray rejection. (Calstis
was changed to perform the tasks in the order described here in July 1998, after the ISR was published.)
In cosmic ray rejection, one wants to strike a balance between failing to identify
cosmic rays and clipping real flux from the target. The former lets spurious flux into the _crj image; the latter depresses the flux by systematically excluding values that are relatively high for that pixel because of noise fluctuations or small shifts of the target on the detector. Default values of cosmic ray rejection parameters to be used by calstis
, or its component task ocrreject
, are taken from the CRREJTAB
reference file. If your _crj images still have many pixels affected by unrejected cosmic rays, you may want to try running ocrreject
on the _flt files with different values for the rejection parameters, which will override the values taken from the CRREJTAB
. Use crmask=yes
to flag CRs in the input DQ images.
The optimum values of the cosmic ray rejection parameters depend on many
factors, including the number of exposures to be combined, exposure time, signal-to-noise level, source structure, and the magnitude of the misalignment of the exposures due to drift of the target on the detector. (See ISR STIS 98-22
.) For example, a small spatial misalignment of two exposures made with very high signal-to-noise may require one to increase the value of the parameter crsigmas
(for a higher rejection threshold in sigmas) to avoid the systematic rejection of real flux. For a direct image or spectral image with only gradual spatial or spectral changes in flux from pixel to pixel, it may be possible to lower the rejection threshold to catch more cosmic rays without rejecting real flux. One may also need to change the value of crthresh
, which sets a more stringent rejection level for the pixels surrounding a pixel that failed the crsigmas
test. (e.g., for crthresh=0.5
, rejection occurs for surrounding pixels at a level of 0.5 * 7 sigma = 3.5 sigma
out to a radius of crradius
If you want to use ocrreject
to combine spectral images which have been dithered by a few pixels along the slit to move hot pixels, you should first apply sshift
to the _flt files. This task will perform integer pixel shifts to produce aligned _sfl files. You will then need to set appropriate rejection parameter values for these files in ocrreject
, since the CRREJTAB
values depend on the value of CRSPLIT
and the mean exposure time for an association of CRSPLIT
exposures. The parameter values in the CRREJTAB
can be used as a guide.
To see if real flux is being rejected by ocrreject
, one can compare the _crj image to a summed image and look for systematically lower fluxes in the _crj image. Problems will tend to occur where the pixel-to-pixel flux changes are the greatest. For a direct image, examine the pixels near the centers of point sources. For a spectral image, examine the peak rows in the spectral image. One can also examine the DQ extensions of the _flt files after they have been flagged by ocrreject
(crmask = yes
). Systematic problems due to spatial misalignment are especially conspicuous in the DQ extensions of the _flt files of spectral images. Improperly flagged spectral images will have strings of DQ=8192
running along a row near the spatial peak of the spectrum, since that row will be less well centered on the target in one exposure than in another. (Remember to examine the DQ extension of the _flt files instead of the _crj file, since pixels in the latter will not be flagged for cosmic rays if their flux was computed from cosmic-ray-free pixels in any of the input exposures.)
Detailed descriptions of the STIS tasks (normspflat
, and prepspec
) for using contemporaneous flat field images to remove IR fringing from STIS G750L and G750M spectra are given in STIS ISR 98-29
. Note that this ISR was written before pixel-to-pixel flats were routinely subtracted from STIS spectroscopic data in the OTFR pipeline, and, as a result, includes some steps that are no longer always necessary.
The IR fringe flats may be taken with a variety of apertures. For point source fringe
flats taken at or near a standard aperture position near the center of the detector, the best alignment between science image and fringe flat will usually be for fringe flats taken using the 0.3X0.09 aperture. For extended targets, or point sources located elsewhere along the slit, e.g., at the E1 aperture location near row 900, it will be necessary to use a long slit fringe flat, e.g, 52X0.1.
For G750L data, the defringing will usually be applied to the _crj image. In this
example, the rootname of the fringe flat image is o8u201070, and the science image of the external target is o8u201060.
task is used to prepare the fringe flat image and remove the low order lamp vignetting.
Fringe flat images taken with G750L include not only the IR fringing at
wavelengths greater than 7500 Angstroms, but also some fringes at wavelengths less than 6000 Angstroms due to an order-sorter filter. Since these order-sorter fringes are already included in the sensitivity function, they should not be included in the fringe flat, and so these columns should be set to unity in the normalized fringe flat. This can easily be done by using the IRAF imcalc
task to create a temporary array, and to then use the imcopy
task to replace the appropriate columns in the fringe flat.
It may be necessary to adjust a number of the input parameters of the mkfringeflat
task to get proper alignment of the fringe flat and science image. See STIS ISR 98-29
for further details.
The final output file will be the equivalent of the _crj file with the fringes removed.
This file may be used as input to the x1d
task to produce a final, fringe-subtracted 1-D spectrum.
To use an exposure other than the default wavecal, the user can either (1) put the
name of the raw exposure file of the lamp image to be used into the WAVECAL
keyword in the primary header of the _raw science image, make sure that the WAVECORR
keyword is set to PERFORM
, and then run calstis
as normal, or (2) use the stis task wavecal
, to measure the offset of the wavelength calibration image and populate the SHIFTA1
keywords in the science file’s extension headers. For example, if o8mi02010 is the science observation and o8mi02ovq a corresponding wavelength lamp calibration image, the command would be
This will update the SHIFTA1
keywords and set WAVECORR
to complete. The modified science image file can then be processed with calstis
and/or one or more of the stand alone STIS tasks, e.g.,
Note that the wavecal
task is meant to update the headers of spectral images, prior to running the x1d
tasks. It should not be used on images or tables (e.g., _sx1, _sx2, _x1d, or _x2d files) that have already had wavelengths assigned to individual pixels.
The orientation of spectral traces on the STIS detectors has been found to rotate
slowly over time. Additionally, there is a small random offset in the orientation each time the MSM is positioned. The time-dependent component of the orientation has been calibrated for the most commonly used grating/central-wavelength combinations: G140L/1425, G230L/2376, G230LB/2375, G430L/4300, G750L/7751, G750M/6581, G750M/6768, and G750M/8561. Calstis
has been modified to apply time-dependent rotation to spectral traces, and the evolutionary parameters for these modes were included in SPTRCTAB
reference files that were delivered in August and October 2006. For other L and M modes with uncalibrated trace rotation, the PyRAF
can be used to generate a SPTRCTAB
with accurately rotated traces for a specific science image. The task can also be applied to data taken with the calibrated modes to correct for the random offset in orientation due to the positioning of the MSM.
task computes the trace of the science target using the default aperture location or accepting an input location. A linear fit is made to this trace. The user can choose to exclude portions of the trace from the fit because of velocity structure, low signal-to-noise, etc. A calibration trace is interpolated from the traces at the nearest rows in the SPTRCTAB
, and a linear fit is made using the same ranges of columns selected for the fitting of the science trace. Image files of the target trace and calibration trace and of the linear fits are generated to assist the user in selecting the portions of the spectrum to include in the fits. The difference in the angle of the two linear fits is the correction that must be applied to the SPTRCTAB
traces. A new SPTRCTAB
is generated, applying this correction to all of the traces for that grating/cenwave combination, thus covering the entire detector. The keyword SPTRCTAB
in the primary header of the science image is changed to the name of the new trace file, so that subsequent processing by calstis
, or x2d
will use the new file.