STScI automatically processes and calibrates all NICMOS data and
archives the data files resulting from pipeline processing in FITS format. If you have retrieved NICMOS files from the Archive, you will notice that their names look like this:
The first part of the file name (n3w2a1wqm
) is the rootname
, identifying the dataset to which the file belongs, the second (cal
) is the suffix
, identifying the type of data the file contains, and the third (fits
) indicates that this is a FITS format file. Chapter 2 of the "HST Data Handbook
" shows how to access the data contained in NICMOS FITS files, while Appendix B of the "HST Data Handbook
" explains how to decipher the rootnames of these files and explains why some of them are grouped into data associations
. This section describes the files that constitute a NICMOS dataset.
Each file in a NICMOS dataset has a three-character suffix that uniquely
identifies the file contents. The file name suffixes for NICMOS and the corresponding file contents are summarized below. Table 2.1
lists all of the files that the pipeline can produce. For some observing strategies not all of the processing steps are performed and only a subset of these files will be produced by the pipeline.
|Raw Science File
This FITS file contains the raw image data received from the spacecraft. One file per exposure is created (a MULTIACCUM exposure is considered a single exposure irrespective of the number of samples specified).
This FITS file contains supporting information about the observation, the spacecraft telemetry and engineering data from the instrument that was recorded at the time of the observation, including detector temperature measurements.
This file is a FITS binary table that contains the list of datasets making up an association.
|Calibrated Science File
This FITS file contains the calibrated science data for an individual dataset
, and is produced by the pipeline calibration task calnica
(see Chapter 3
). The input to calnica
are the _raw
images. For a MULTIACCUM exposure, this file contains a single science image formed by combining the data from all samples.
|Intermediate Multiaccum Science File
This FITS file is also produced by the pipeline task calnica
and contains the calibrated science data for all samples of a MULTIACCUM dataset before the process of combining the individual readouts into a single image has occurred. This file is only produced for MULTIACCUM observations.
These FITS files contain the composite target and, for chopped pattern sequences, background region images constructed by the pipeline task calnicb
for an associated set of observations (see Chapter 3
). The input to calnicb
are the calibrated _cal
images from calnica
and the _asn
association table. Target images are co-added and background-subtracted. The value of the last character of the rootname is 0 for targets, and 1 to 8 for background images. These files are only produced for an associated set of observations.
|Post-calibration Association Table
This table is produced by the pipeline calibration task calnicb,
the same as the association table _asn
, with the addition of new columns which report the offsets between different images of the mosaic or chop pattern as calculated by calnicb
, and the background levels computed for each image. This file is only produced for an associated set of observations.
|Trailer File (_trl):
This FITS ASCII table contains a log of the pipeline calibration processing that was performed on individual datasets and mosaic products.
|Processing Data Quality File
This FITS ASCII table provides quality information on the observation, mostly on pointing and guide star lock. Possible problems encountered, e.g., a loss of guide star lock or a guide star acquisition failure, are reported here.
|Temperature Telemetry Table
This FITS binary table is produced by the Engineering Data Processing System (EDPS) as an aid for use in calibration and temperature monitoring. Nine NICMOS Cooling System (NCS) mnemonics and five temperature sensors are bundled together into an STSDAS
FITS table. (Available after OPUS 15.2)
, and *_mos.fits
files are all defined as science data files
, as they contain the images of interest for scientific analysis.
The data for an individual NICMOS science readout consist of five
arrays, each stored as a separate image extension in the FITS file. The five data arrays represent:
A science data file can contain one or more imsets. For example, an
individual NICMOS exposure obtained with the ACCUM mode will generate a *_raw.fits
file with one imset; an individual MULTIACCUM exposure with n
readouts will generate a *_raw.fits
file (and, after calibration, an *_ima.fits
file) containing n
+ 1 imsets, including the zeroth readout. The *_cal.fits
files always contain one imset.
Although the five science, error, data quality, samples, and integration
time arrays associated with each imset are stored in a single file, they are kept separate as five individual FITS image extensions
within the file. The order of the images in the FITS files is listed in Table 2.2
and shown graphically in Figure 2.1
and Figure 2.2
. The examples given in Table 2.2
and Figure 2.2
refer to a MULTIACCUM image (multiple imsets), while Figure 2.1
refers to ACCUM and BRIGHTOBJ images (one imset, namely extensions 1 through 5). Each extension comes with its own header, and each FITS file contains, in addition, a global
or primary header
(primary header-data unit or HDU).
The only contents of the primary HDU are header keywords. There are
no image data in the primary HDU. The keywords in the primary header are termed global keywords
because they apply to the data in all of the file extensions. The organization and location of header keywords is explained in detail later in the chapter.
This image contains the data from the detector readout. In ACCUM and
BRIGHTOBJ modes the image received from the instrument is the result of subtracting the initial from the final readouts of the exposure. In MULTIACCUM mode the images received are the raw (unsubtracted) data corresponding to each detector readout. In raw datasets the science array is an integer (16-bit) image in units of DNs (counts). In calibrated datasets it is a floating-point image in units of DNs per second (count rates).
The error image is a floating-point array containing an estimate of the
statistical uncertainty associated with each corresponding science image pixel. This image is computed in the ground calibration pipeline task calnica
as a combination of detector read noise and Poisson noise in the accumulated science image counts (see Chapter 3
) and corresponds to 1σ
uncertainties. For calibrated MULTIACCUM images (i.e., *_cal.fits
files), the values of the error array are computed uncertainties in the count rates derived from the linear fit to counts vs. exposure time from the intermediate readouts.
This integer (unsigned 16-bit) array contains bit-encoded data quality
flags indicating various status and problem conditions associated with corresponding pixels in the science image. Because the flag values are bit-encoded, a total of 16 simultaneous conditions can be associated with each pixel. Table 2.3
lists the flag values and their meanings.
The SAMP image is an integer (16-bit) array containing the total
number of data samples that were used to compute the corresponding pixel values in the science image. For ACCUM and BRIGHTOBJ modes, the number of samples contributing to each pixel always has a value of 1 in the raw data file. For MULTIACCUM mode the sample values in the raw and intermediate data files are set to the number of readouts that contributed to the corresponding science image.
In MULTIACCUM calibrated data files (*_cal.fits
) the SAMP array contains the total number of valid samples used to compute the final science image pixel value, obtained by combining the data from all the readouts and rejecting cosmic ray hits and saturated pixels. In this case the sample array may have different values at different pixel locations (less than or equal to the total number of samples in the MULTIACCUM sequence), depending on how many valid samples there are at each location.
In the mosaic images (*_mos.fits
), the data in the SAMP array indicate the number of samples that were used from overlapping images to compute the final science image pixel value.
The TIME image is a floating-point array containing the effective
integration time associated with each corresponding science image pixel value. These data are always computed in the ground calibration pipeline for recording in the raw data file.
|For ACCUM and BRIGHTOBJ mode observations, each pixel has the
same time value. For MULTIACCUM observations each pixel for a given readout has the same time value in the raw and intermediate data. In these cases, to save on data volume, the image array is not created and the value of the time is stored in the header keyword PIXVALUE in the TIME image extension (see Table 2.5
In MULTIACCUM calibrated data files (*_cal.fits
) the TIME array contains the combined exposure time of all the readouts that were used to compute the final science image pixel value, after rejection of cosmic ray and saturated pixels from the intermediate data. As in the case of the SAMP array, the TIME array can have different values at different pixel locations, depending on how many valid samples compose the final science image in each pixel.
In mosaic images (*_mos.fits
), the TIME array values indicate the total effective exposure time for all the data from overlapping images that were used to compute the final science image pixel values.
, and the *_asc.fits
files are termed auxiliary data files
. They contain supporting information on the observation, such as spacecraft telemetry and engineering data, assessment of the quality of the observation, calibration information, and information on the associations present in the observations.
The association tables
, are FITS binary tables which are created when a particular observation generates an association of datasets (see the discussion of “Associations” in Appendix B of the "HST Data Handbook
"). In particular, the *_asn.fits
table is generated by OPUS1
, and contains the list of datasets which make up the association (e.g., from a dither or chop pattern). The *_asn.fits
tables are the inputs to the pipeline calnicb
, which creates the mosaiced or background subtracted images (*_mos.fits
files) from the input datasets. All the datasets must have been processed through the basic pipeline data reduction (calnica
) before being processed through calnicb
. In addition to the output science image(s), calnicb
produces another association table (*_asc.fits
), which has the same content as the *_asn.fits
table, along with additional information on the offsets used by the pipeline for reconstructing the science image and background values computed for each image. For mosaics (dither patterns), there is only one final image produced, with file name *0_mos.fits
. For chop patterns, in addition to the background-subtracted image of the target (*0_mos.fits
), an image for each background position is produced; the file names of these background images are *1_mos.fits
(a maximum of eight independent background positions is obtainable with the NICMOS patterns; see the NICMOS Instrument Handbook
The support files, *_spt.fits,
contain information about the observation and engineering data from the instrument and spacecraft that was recorded at the time of the observation. A support file can have multiple FITS image extensions within the same file; in the case of a MULTIACCUM observation there will be one extension for each readout (i.e., each imset) in the science data file. Each extension in the support file holds an integer (16-bit) image containing the data which populates the *_spt.fits
header keyword values.
The trailer files, *_trl.fits
, contain information on the calibration steps executed by the pipelines and diagnostics issued during the calibration. There is one *_trl.fits
file produced for each dataset, and, in the case of associations, one *_trl
file for each NICMOS product (i.e., each *_mos.fits
The processing data quality files, *_pdq.fits
, contain general information summarizing the observation, a data quality assessment section, and a summary on the pointing and guide star lock. They state whether problems were encountered during the observations, and, in case they were, describe the nature of the problem. There is one *_pdq.fits
file produced for each dataset, and, in case of associations, one *_pdq
file for each NICMOS product (i.e., each *_mos.fits
The temperature tables, *_epc.fits, contain a set of 14 mnemonics
(sensors) extracted from the engineering telemetry. The time range of the tables are over the duration of the corresponding observations. The selected sensor used to monitor the temperature of the NICMOS detectors is NDWTMP11, the temperature at the NIC1 mounting cup. A listing of the individual table names and their location within the FITS table can be obtained by using the IRAF/STSDAS
. (See NICMOS ISR 2003-008
for more details.) Mnemonics in the temperature table *_epc.fits, report temperature information within the NCS and five temperature sensors within the NICMOS dewar. Table 2.8
presents a listing of the 14 NCS and NICMOS telemetry temperature sensors.