|Space Telescope Science Institute|
|COS Data Handbook v.3|
3.7 This section contains a description of the COS reference files. See Figure 3.1 - Figure 3.6 for which modules use these files and Section 3.4 for explanations of how their contents are applied by those modules. The reference files are now described in the order they are called by the pipeline for the case of FUV TIME-TAG data (Figure 3.1).
• File Suffix: _badtThe BADTTAB reference file lists the start and end times of known bad time intervals. It is used by the BADTCORR calibration module to flag events in TIME-TAG events lists which occur during a bad time interval. In later processing the flagged events will be removed from the final calibrated data, and the exposure time header keyword, EXPTIME, updated. The bad time interval table consists of segment, start, and end columns (see Table 3.5). The segments columns can be populated with either FUVA, FUVB or ANY. The start and end columns are in Modified Julian Date.Table 3.5: BADTTAB Table Content
• File Suffix: _brfThe BRFTAB reference file is only applicable to FUV data and is used during pipeline processing in the TEMPCORR module to apply the thermal distortion correction. The FUV detector does not have physical pixels like a CCD. Instead, the x and y positions of detected photon events are obtained from analog electronics, which are susceptible to thermal changes. Electronic stim pulses are normally commanded during integration and are used as physical position reference points. To return the FUV data to a known physical space, the BRFTAB defines the stim pulse positions.The BRFTAB file consists of a primary header extension and a binary table extension. The table lists the stim pulse locations and search regions, and the active detector areas (Table 3.6).Table 3.6: BRFTAB Table Contents
X pixel coordinate (zero indexed) of stim pulse 11 X pixel coordinate (zero indexed) of stim pulse 22
• File Suffix: _geoThis file is only used for FUV data. The GEOFILE is used by the GEOCORR calibration module to perform the geometric correction. The analog nature of the XDL detector means that the physical sizes of the pixels vary across the detector. The geometric distortion maps are used to correct for this variation and to transform the data into a constant physical pixel size early in the data reduction calibration process. After the thermal correction has been applied and the detector digital span and position are adjusted to their reference values, as defined in the reference table, the geometric correction can be applied. This implies that all the files used to determine the geometric correction were initially thermally-corrected.Each geometric correction reference file contains four IMAGE extensions. There are two for each segment, and for each segment, there is one for each axis. At a given (X,Y) location in the thermally corrected COS data, the value at that location (corrected for binning and offset) in the geometric correction image gives the distortion to be subtracted from the X or Y coordinates. The order of the extensions are: 1 = X coordinate for FUVA, 2 = Y coordinate for FUVA, 3 = X coordinate for FUVB and 4 = Y coordinate for FUVB.
• File Suffix: _walkThe WALKTAB reference file is only applicable to FUV data and is used during pipeline processing in the WALKCORR module to correct the effects of walk. The COS FUV XDL detector is subject to gain sag, where as physical locations on the detector accumulate photon events, the pulse height of the electron cloud generated by the event becomes smaller, and the coordinates of the event are mis-registered. These effects depend on event pulse height.The current correction employed is a simple linear correction to registered Y location based on event pulse height, but the WALKTAB has the ability to correct both X and Y location based on arbitrary polynomials taking into account X location, Y location, and pulse height.The WALKTAB file consists of a primary header and a binary table extension. The table lists the coefficients of the polynomials in X and Y (Table 3.7). In order to determine how the coefficients will be used, see the WALKCORR section (Section 3.4.6).Table 3.7: WALKTAB Table Contents
3.7.5 DEADTAB: Deadtime Table
• File Suffix: _deadThe deadtAB reference file is used in the DQICORR: Initialize Data Quality File module, to obtain the true number of events received compared to the number of events appearing in the raw data files.There is one DEADTAB reference file for each of the NUV and FUV detectors. Each consists of a primary header and a binary table extension which contains the LIVETIME values for a given observed count rate (OBS_RATE) and segment. The livetime is defined as:
• File Suffix: _phaThe PHATAB reference file is only valid for FUV data, and is applied during the PHACORR step of calcos to filter non-photon events. The file consists of two header/data units, the first being the primary header, and the second a binary table (see Table 3.8). The table lists the lower and upper thresholds for valid individual pulse heights in TIME-TAG mode. In Time-Tag mode, each detector event has an associated pulse-height of 5 bits with values ranging from 0 to 31, The table also gives the minimum and maximum values for the location of the mean value of the pulse height distribution used in ACCUM mode. In ACCUM mode, a pulse height distribution histogram is generated for the whole exposure over the entire detector and downloaded as part of the science data file. The histogram includes all the digitized events for each segment independently of the currently defined subarrays. Note in ACCUM mode the pulse height is a 7 bit number with values ranging from 0 to 127.Table 3.8: PHATAB Table Contents
• File Suffix: _phfThis file is only used for FUV data, and is a 2D equivalent to the PHATAB. The PHAFILE is used by the PHACORR calibration module to filter non-photon events. If both a PHATAB and PHAFILE are available, the PHAFILE will be used.Each pulse height discrimination reference file contains four IMAGE extensions. There are two for each segment, containing the lower and upper PHA limits for each pixel. At a given (X,Y) location in the uncorrected COS data, the value at that location gives the lowest and highest (respectively) pulse height that will be treated as a valid photon event at that detector location.
• File Suffix: _flatFLATFILE provides a flat-field image which is used by the pipeline to remove the pixel-to-pixel variations in the detector. The FUV FLATFILE consists of a primary header and two 14000 x 400 IMAGE extensions, one for each segment. The NUV FLATFILE consists of a primary header and a 1024 x 1024 IMAGE extension.The FUV flat-field reference file corrects for grid wire shadows and for an effect of small-scale geometric distortion, and it is used for all three FUV gratings (G130M, G160M, and G140L).The NUV flat-field is a combination of internal and external deuterium flat field lamp exposures from thermal-vacuum testing which illuminate the portion of the detector where spectra fall. The data cover the following pixel region of the detector: x (dispersion): 0 to 1023, and y (cross-dispersion): 495 to 964. The rest of the detector, where flat field data are not available, has a value of 1.0. The bottom four and top three rows of the detector do not fit well with the rest of the detector and they are flagged in the data quality table.
• File Suffix: _lampThe LAMPTAB files consist of a primary header and a binary table extension which contains an extracted 1-D spectrum from the internal PtNe calibration lamp through the WCA aperture, for each grating, central wavelength, and FP-POS setting. It is used in the calcos pipeline to determine the pixel offset of the observed data. The structure of the template calibration lamp spectra table is shown in Table 3.9. The stepper motor offsets range from -2 to +1 and correspond to FP-POS settings of 1 to 4.Table 3.9: LAMPTAB Table Contents
• File Suffix: _dispThere are two DISPTAB files with similar formats, one for the NUV, and one for the FUV. They consist of a main header and a binary table in the second HDU. These tables provide the dispersion relations for each segment, aperture, optical element, and central wavelength. Each file has the format given in Table 3.10. The dispersion relation table gives a set of polynomial coefficients for computing wavelength from pixel number (see Oliveira et al., COS ISR2010-05 and 06 for details).Each row of the table gives a set of dispersion coefficients. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE, and APERTURE.Table 3.10: DISPTAB Table Format
For Px = the zero-indexed Doppler corrected pixel value in the dispersion direction, letPX’ = PX + (D_TV03 - D),λ(PX’) = COEFF + COEFF*PX’ + COEFF*PX’2 + COEFF*PX’3
• File Suffix: _1dxThere are two XTRACTAB files with similar formats, one for the NUV and one for the FUV. They consist of a main header and a binary table in the second HDU. These tables provide the information needed to extract the spectrum from a geometrically corrected image of the detector for each optical element and central wavelength. Each file has the format given in Table 3.11.Table 3.11: XTRACTAB Table Format
The spectral extraction of a source is performed by collapsing the data within a parallelogram of height HEIGHT that is centered on a line whose slope and intercept are given by SLOPE and B_SPEC. Similarly, two background spectra are determined by collapsing the data within parallelograms of height B_HGT1 or B_HGT2 centered on the lines defined by SLOPE and B_BKG1, and SLOPE and B_BKG2, respectively. The background spectra are then smoothed by a boxcar of width BWIDTH. These are then scaled and subtracted from the source spectrum.
• File Suffix: _burstThe BRSTTAB file is used for FUV data only. It provides the parameters needed to identify bursts. It consists of a primary header extension and a binary table extension with the columns listed in Table 3.12. Details of the burst rejection routine are given in Section 3.4.12.Table 3.12: BRSTTAB Table Contents
3.7.13 BPIXTAB: Bad Pixel Table
• File Suffix: _bpixThe data quality initialization table identifies rectangular regions on the detectors that are known to be less then optimal. The feature type describes the type of detector blemish enclosed within the bounding box and DQ is the quality value assigned to all events detected within the box. The regions were identified by visual inspection of the combined flat field data for each detector (and segment). The BPIXTAB files consist of a primary header and a binary table extension which consists of the columns listed in Table 3.13.Table 3.13: BPIXTAB Table Content
In the BPIXTAB table, the DQ field may be a logical OR due to several different values, each associated with a unique issue (see Table 2.19).3.7.14 GSAGTAB: Gain Sag Table
• File Suffix: _gsagThe gain sag reference table is only applicable for FUV data and it is used along with the bad pixel reference table (_bpix) in the DQICORR module. The table provides the locations of rectangular regions for portions of the FUV detector that have very low pulse height amplitude.After the primary header, each extension of the GSAGTAB is a binary fits table of the gain sagged pixels on the detector at a given voltage. During the pipeline processing, these extensions are selected depending on the SEGMENT and HVLEVEL. Each row in the table gives the location and data quality value for one rectangular region. The DATE column is used to select rows. A row will be used to flag a gain sagged region if the value in the DATE column is less than or equal to the exposure start time. For a description on the columns contained in the binary tables see Table 3.14.Table 3.14: GSAGTAB Table Format
3.7.15 SPOTTAB: Hotspot Table
• File Suffix: _spotThe hotspot table is only applicable for FUV data, and is used along with the bad pixel reference table (_bpix) and gain sag table (_gsag) in the DQICORR module. The table provides the start and stop times, locations and extents of hotspots, which are transient regions of high detector background.Format: The hotspot table is a FITS table with a primary header and 1 extension with optional EXTNAME = HOTSPOT. Each row has 9 columns: SEGMENT is the segment name the hotspot appears in (FUVA or FUVB). START and STOP are the MJD times of the start and stop of the hotspot. LX and LY are the (XCORR, YCORR) coordinates of the lower left corner of the rectangular hotspot region. DX and DY are the extent, in pixels, of the rectangular hotspot region. DQ is the value of the DQ flag to be applied to the region (see Table 2.19), and COMMENT is a comment string.Table 3.15: SPOTTAB Table Format
The hotspot is selected based on the value of SEGMENT, and then the START and STOP times are compared to the start and stop times of the good time intervals of the exposure being calibrated. If a hotspot overlaps any of the good time intervals, the region is added to the set of regions that are applied to create the DQ mask and against which each event is tested to assign a DQ value. The hotspot regions are flagged in the two-zone extraction module even if they are only in the outer zone, and they do not contribute to the summed spectra in the x1dsum file.
• File Suffix: _wcpThe WCPTAB file contains information relevant for the wavecal pipeline processing. It consists of primary header and a binary table extension which is described in Table 3.16. XC_RANGE is the maximum pixel offset to use when doing a cross correlation between the observed data and the template wavecal. That is, the observed spectrum should be shifted relative to the template by a number of pixels, ranging from -XC_RANGE to +XC_RANGE inclusive. XD_RANGE is half the search range for finding the spectrum in the cross dispersion direction. The search range is from B_SPEC - XD_RANGE to B_SPEC + XD_RANGE inclusive, where B_SPEC is the nominal location of the spectrum from the XTRACTAB table discussed below. BOX is the width of the boxcar filter for smoothing the cross-dispersion profile. RESWIDTH is the number of pixels per resolution element, and is assigned a value of 6.0 for the FUV detectors and 3.0 for the NUV detector.When applying the offsets found from the wavecals to the science data, it may happen that there was no wavecal at the same OSM position. In this case, the wavecal that was closest in time to the science observation may be used, with a correction for the difference in OSM positions. That correction is based on STEPSIZE, the number of pixels corresponding to one OSM step. There may be a check, however, to guard against using a wavecal that was taken too far away in time from the science observation. If the science observation and wavecal were taken more than MAX_TIME_DIFF apart, then the wavecal should not be used for that science observation.Table 3.16: WCPTAB Table Contents
• File Suffix: _photThere are two fluxTAB files with similar formats, one for the NUV, and one for the FUV. They consist of a main header and a binary table in the second HDU. These tables provide the information needed to convert from corrected detector counts to flux units of erg s-1cm-2A-1 for each segment, optical element, aperture, and central wavelength. Each file has the format given in Table 3.17.Table 3.17: fluxTAB Table Format
The units of the Sensitivity array are (count s-1 pixel-1)/(erg s-1 cm-2 Angstrom-1). For each segment, optical element, central wavelength setting, and aperture, these files contain arrays of wavelengths and sensitivities which can be interpolated onto the observed wavelength grid. The net counts can then be divided by the sensitivity curves to produce flux calibrated spectra.
• File Suffix: _tdsThere are two such files, one for the FUV and one for the NUV. They are only used for spectroscopic data. The files contain the information necessary to determine the relative sensitivity curve at any given time by interpolating between relative sensitivity curves given at fiducial times which bracket the observation, or else extrapolate the results from the last curve if the observation date is more recent than the last fiducial date. Interpolation data are provided for each segment, optical element, and aperture (see Table 3.18). Updated TDS plots are located at:Table 3.18: TDSTAB Table Format
For an observation obtained at time T, which lies between TIME[j] and TIME[j+1], the sensitivity curve used to calibrate the spectrum will be corrected by the following factor:The trace table gives the variation of the centroid of the spectrum as a function of column number (XCORR) in COS FUV data.Format: The file is a FITS table with a primary header and one extension. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE and APERTURE. Each row has 8 columns. DESCRIP supplies a short description, while TRACE_YLOC is the location of the center of the trace. TRACE is an array of 16384 floats where the index is the value of XCORR and the value is the offset to be subtracted from each event's YFULL value. The value of XCORR for each event is interpolated onto the TRACE array to give the value of the shift to be applied to the corresponding YFULL value of the event. ERROR is an array of 16384 floats that gives the statistical error of the TRACE measurement. Table 3.19 describes the column definitions.Table 3.19: TRACETAB Table Format
3.7.20 PROFTAB: Profile TableThe profile table gives the profile of a point source perpendicular to the dispersion direction as a function of column number (XFULL) in COS FUV data.Format: PROFTAB is a FITS table with a primary header and one extension with optional EXTNAME = PROFILE. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE and APERTURE. Each row has 8 columns. DESCRIP gives a short description of the row. CENTER is the measured centroid of the profile in the full-sized array in (XFULL, YFULL) coordinates. ROW_0 is the index of the first row of the profile in the full-sized array. In other words, if the profile has NROWS rows, it corresponds to rows with 0-based indices running from ROW_0 to (ROW_0 + NROWS - 1). PROFILE is the 2-d array of floats that gives the profile in the cross-dispersion direction for each column of data in (XFULL, YFULL) space (offset by ROW_0). During the ALGNCORR step, the flux-weighted centroid of the science data over 'good' rows and columns is calculated, and compared with the flux-weighted centroid of the profile contained in this reference file over the same rows and columns. The difference between these centroids is applied to the YFULL values of the events to align each set of science data to the same center. Table 3.20 describes the column definitions. The 2D spectral profiles contained in the PROFTAB for three settings (G140L/1280, G130M/1291, and G160M/1577) are given in Figure 3.17, Figure 3.18, and Figure 3.19.Table 3.20: PROFTAB Table Format
Figure 3.17: 2D Reference Profile for G140L/1280An image of the 2D reference profile in the PROFTAB for the G140L/c1280 setting at LP3 is show for each detector segment. The 80% and 99% enclosed energy contours that are currently used to define the inner and outer zones of the two zone extraction are also marked.Figure 3.18: 2D Reference Profile for G130M/1291The same as Figure 3.17, expect that the profiles shown here are for the G130M/1291 setting. Note that for this example the cross-dispersion profile is double peaked at many wavelengths due to the cross-dispersion astigmatism of the G130M grating.Figure 3.19: 2D Reference Profile for G160M/1577The TWOZONE extraction table contains the starting values for the object center and background regions, as well as the cumulative flux boundary values for the TWOZONE extraction.Format: TWOZXTAB is a FITS table with a primary header and one data extension. The row to be used is selected on SEGMENT, OPT_ELEM, CENWAVE and APERTURE. Each row has 16 columns. B_SPEC is the center of the science extraction aperture, and is used by the ALGNCORR step to get an initial guess for the location of the spectral trace. B_BKG1 and B_BKG2 are the center of the background regions, HEIGHT is the height of the target extraction region, and BHEIGHT is the height of the background extraction regions. BWIDTH is the width of the smoothing box used to smooth the background region in the extraction step.In the TWOZONE extraction step, the spectral profile in the PROFTAB is analyzed to determine the boundaries of INNER and OUTER zones. These boundaries are specified in terms of the cumulative flux enclosed. In the INNER region, the flux is summed within the region and any DQ flags are propagated to the extracted spectrum. In the OUTER region, the flux is also summed and added to the flux in the inner region, but any DQ flags in the outer region are not propagated to the final extracted spectrum unless they are in the DQ value SDQOUTER from the primary header. The columns LOWER_OUTER, UPPER_OUTER, LOWER_INNER and UPPER_INNER give the cumulative flux boundaries to be used in the two zone extraction. Typically the outer boundaries enclose 99% of the flux, while the inner boundaries enclose 80%.The YERRMAX column is used in the ALGNCORR step to test the statistical error in the calculation of the flux-weighted centroid of the science data. If this measurement is greater than the value of YERRMAX for that setting, the spectrum is deemed 'not found', and the location of the center of the reference profile is used instead. The PEDIGREE column gives the pedigree of the information in the row, with values that are typically INFLIGHT, GROUND or DUMMY.Table 3.21 describes the column definitions.Table 3.21: TWOZXTAB Table Format
The spectroscopic SPWCS table gives the parameters needed to populate the world coordinate keywords in the corrtag, counts, and flt files. There are entries for each SEGMENT, OPT_ELEM, CENWAVE, and APERTURE. The columns (see Table 3.22) are interpreted as follows. The detector coordinate system has two dimensions. Let the more rapidly varying axis be X and the less rapidly varying axis Y. The world coordinate system has three dimensions, the spectral coordinate, right ascension, and declination. The reference pixel is at approximately the middle of the detector. CTYPE1 can be WAVE to indicate that the wavelength is a linear function of pixel number, or it can be WAVE-GRI to indicate that the wavelengths should be computed by using the grating (“grism”) equation. In either case, the wavelengths are in vacuum. CRVAL1 is the wavelength at the reference pixel. CRPIX1 is the location of the reference pixel in the first axis (X); the location of the reference pixel in the second axis (Y) is gotten separately from the 1-D Extraction Parameters Table (XTRACTAB). CDELT1 is the dispersion in Angstroms per pixel at the reference pixel. At a single wavelength (nominally the wavelength at the reference pixel), a pixel when projected onto the sky would be approximately a rectangle. CDELT2 and CDELT3 are the sizes of that rectangle in the X and Y directions. SPECRES is the spectral resolution; this is only used for updating the archive search keyword of the same name. G is the groove density of the grating, e.g. 3.8E6 grooves per meter for G130M. SPORDER is the spectral order. This will usually be 1, but for G230L, stripe NUVC, SPORDER will be 2. ALPHA is the angle between the normal to the grating and the light that is incident onto the grating. THETA is the angle between two lines from the grating to the detector, the line to the reference pixel and the line that is perpendicular to the detector. Since the reference pixel is close to the middle of the detector, THETA will probably be close to zero.Table 3.22: SPWCSTAB Table Format