Space Telescope Science Institute
Table of Contents Previous Next Index Print

NICMOS Data Handbook > Chapter 3: Calibration > 3.5 Recalibration

3.5 Recalibration
This section is intended to help you decide whether your data were calibrated with optimal calibration reference files and to help you decide whether you need to recalibrate your data.
In many cases, NICMOS calibrated data produced by the OPUS pipeline—the standard calibration—are adequate for scientific applications. However, as with all instruments, there is often room for improvement, and you may find it worthwhile or even necessary to reprocess your data, perhaps following additional procedures which are not part of the standard pipeline. As described at the start of this chapter, Cycle 7 and 7N data delivered to the observer or retrieved from the HST archive before 26 September 2001 were calibrated with the best, instrument configuration-specific reference files available at the time of the observation. However, updated or timely reference files sometimes do become available after the data were taken and first processed. As one important example, the DARK reference files were substantially updated and improved during and after Cycle 7. Also, after the installation of NCS, new sets of darks and flat fields were needed as a consequence of the higher operational temperature. Improved software for calibration (updates to calnica, as well as new tasks external to the standard pipeline, such as MultiDrizzle) has become available as our understanding of the instrument performance has increased with experience. For example, the ZSIGCORR step of calnica was added during Cycle 7, and BARSCORR was implemented in calnica version 3.3. As an example of a more recent change, the pipeline was updated in 2008-2009 to account for the temperature dependence of the dark and flat-field reference files as well as the temperature dependence of the photometry.
After 26 September 2001, NICMOS data retrieved from the Archive are automatically reprocessed by OTFR using the latest pipeline software and calibration reference files. Therefore if you have data retrieved prior to important updates in the pipeline, the easiest way to recalibrate is to retrieve the data again using the OTFR. However, there are still situations where you may wish to reprocess your data locally, using non-standard reference files or software tools. For example, reprocessing may be necessary to correct the NICMOS bias.
Moreover, NICMOS data are subject to a variety of anomalies which may complicate the task of data reduction. These are discussed extensively in Chapter 4. In many cases, procedures and software for handling these anomalies were not available when the observations were made or retrieved from the Archive. If you notice unusual features in your data (see, e.g., the checklist at the start of Chapter 4), or if your analysis requires a high level of accuracy, you may wish to explore whether a better set of calibration reference files exist than those that were used to process your data, or if additional processing steps may be needed. If better files are available or the calibration software has changed significantly, you may choose to recalibrate your data using the new files or software.
Finding that a calibration reference file has changed since your data were calibrated doesn’t always mean that you have to recalibrate. The decision depends very much on which calibration image or table has changed, and whether that kind of change to your data is likely to affect your analysis in a significant way. Before deciding to recalibrate, you might want to retrieve the new recommended reference file or table and compare it to the one used to calibrate your data at STScI in order to determine if the differences are important. You can use the table tools in the IRAF ttools package to manipulate and examine calibration tables. Reference files can be manipulated in the same way as your science data.
Finally, the observations may have been made in a non-standard way. Some of the input files (e.g. *_asn.fits) may require manual editing before recalibration.
This section describes the mechanics involved in actually recalibrating a dataset. As noted above, the simplest way to recalibrate your data is to retrieve it again from the HST Archive using OTFR. However, it is sometimes more convenient to simply reprocess your raw data locally using the latest pipeline software or reference files. In some cases, when you may wish to use special, customized reference files or processing software, local reprocessing is your only option.
The basic steps in recalibrating a dataset on your own computer are:
Set the desired calibration switches and reference file name keywords in the primary header of your raw (*_raw.fits) data file. These determine which steps will be executed by the calibration software and which reference files will be used to calibrate the data.
Assembling the Input Files
In order to recalibrate your data, you need to retrieve all of the reference files and tables that are used by the calibration steps you want to perform. The source of these files is the Calibration Database (CDBS) at STScI. A complete description of how to retrieve the reference files is given in Chapter 1 of the "HST Data Handbook".
Setting the Calibration Parameters
The calibration software is completely data-driven, meaning that the calibration steps to be carried out are determined by the values of the calibration switches and the calibration reference files keywords contained in the primary header of the file to be processed. An important step is then to set the calibration switches and reference file keywords in the primary header of your raw data file (*_raw.fits) to reflect how you want the data recalibrated and which reference files you want to use at each step in the process. This is done most easily with the chcalpar task in the hst_calib.ctools package of STSDAS or with the hedit task in the IRAF images package.
The calibration switch keywords reside only in the primary header of NICMOS FITS files. Therefore it is critically important to specify extension number zero when passing file names to tasks like hedit to modify these keywords. For example, to modify calibration keywords in the file n3xe01bhm_raw.fits, be sure to use the name n3xe01bhm_raw.fits[0] as input. If you specify any other extension number, the keywords you modify will end up getting written into the header of that extension instead, where calnica will not find them.
The chcalpar task takes a single input parameter—the name(s) of the raw data files to be edited. When you start chcalpar, the task automatically determines that the image data are from NICMOS and opens a NICMOS-specific parameter set (pset) that will load the current values of all the calibration-related keywords. To edit the calibration keyword values:
Start the chcalpar task, specifying the image(s) in which you want to change keyword values. Note that with chcalpar, it is not necessary to append the primary header extension [0] to the image name. If you specify more than one image, for example using wild cards, the task will read the initial keyword values from the first image in the list. For example, you could change keywords for all NICMOS raw science images in the current directory (with initial values from the first image), using the command:
After starting chcalpar, you will be placed in eparam [the IRAF parameter editor], and will be able to edit the set of calibration keywords. Change the values of any calibration switches, reference files or tables to the values you wish to use for recalibrating your data.
Exit the editor when you are done making changes by typing:q two times. The task will ask if you wish to accept the current settings. If you type “y”, the settings will be saved and you will return to the IRAF cl prompt. If you type “n”, you will be placed back in the parameter editor to redefine the settings. If you type “a”, the task will abort and any changes will be discarded.
As delivered from the archive, image header parameters which specify the names of the calibration reference files (e.g, FLATFILE, DARKFILE, etc.) take the form nref$name_ext.fits or ntab$name_ext.fits. The prefixes nref$ and ntab$ are environment variables pointing to the CDBS directories where the reference files reside at STScI. This can be convenient, and you may wish to keep your calibration reference files in a particular directory when reprocessing your data. However, because calnica is a stand-alone C program called by IRAF, the environment variables nref and ntab must be defined at the system host level, before starting an IRAF session. For UNIX environments, this is done with setenv nref /path/ and setenv ntab /path/, where you should specify the path to your reference file directory. Do not forget the trailing slash!
Running the Calibration Software
After you change the header keyword values for your raw data files, you are ready to recalibrate your data. To run calnica, type the name of the task followed by the names of the input raw data file and desired output calibrated data file. For example, to recalibrate the dataset n0g70106t, you could type:
or simply:
To run calnicb the name of the association table must be given as input:
To run calnicb on a subset of the *_cal.fits files, it is sufficient to edit the *_asn.fits table and remove the undesired files.
The calibration routines calnica and calnicb will not overwrite an existing output file. If the calibration tasks are run in the directory where the original calibrated files are located, a different output file name must be specified.
There are some calnicb processing parameters which reside in the association (*_asn.fits[1]) table header and not in any FITS file image header, and which therefore cannot be changed using chcalpar or hedit. In particular, to change the ILLMCORR and ILLMFILE parameters, you must use the parkey task on the *_asn.fits[1] file. As an example, to change the ILLMFILE, type: parkey /path/name_ilm.fits data_asn.fits[1] ILLMFILE, where /path/name_ilm.fits is the full name and path to the ILLMFILE, and data_asn.fits is the association table you are editing. In order to set ILLMCORR to OMIT and skip this processing step entirely, type: parkey OMIT data_asn.fits[1] ILLMCORR.

Table of Contents Previous Next Index Print