While STScI does not support IDL tasks, the FITS files generated by the OPUS and
calcos pipelines can be read directly into IDL by using the
mrdfits.pro task. This task and many other are available as part of a library of astronomical IDL routines available from:
STScI provides support for PyFITS, a module that provides an interface to FITS formatted files in the Python scripting language and PyRAF, the Python-based interface to IRAF. It is useful both for interactive data analysis and for writing analysis scripts in Python using FITS files as either input or output. More information, including the PyFITS handbook, can be found at
When calibrating a single spectroscopic exposure, the calcos pipeline creates a one-dimensional extracted spectra file, with suffix
x1d and a filename such as “
l9v220eqq_x1d.fits”.
COS x1d files are MEF format files and their data contents and extension formats are discussed in
Section 2.4.3. As with other COS data files, the primary [0] extension will contain only header information, but no data. The extracted spectra are stored in a single [SCI] extension as
multi-dimensional binary table. A standard FITS table consists of columns and rows forming a two-dimensional grid of cells; however, each of these cells can contain a data array, effectively creating a table of higher dimensionality. Tables containing extracted COS spectra take advantage of this feature and are
three-dimensional.
In order to analyze COS tabular spectral data with STSDAS tasks other than
sgraph and
igi (which have been appropriately modified to handle three-dimensional data tables), you will need to extract the desired arrays from the three-dimensional table. Two IRAF tasks, named
tximage and
txtable, can be used to extract the table-cell arrays. Complementary tasks, named
tiimage and
titable, will insert arrays back into table cells. The task
tscopy will copy rows, columns, and subsets of tables. To specify the arrays which should be extracted from or inserted into the table cells, you will need to use the
selectors syntax to specify the desired row and column. The general syntax for selecting a particular cell is:
intable.fits[extension number][ c:column_selector][r: row_selector]
intable.fits[keyword options][c: column_selector][r: row_selector]
|
A column selector is a list of column patterns separated by commas. The column pattern is either a column name, a file name containing a list of column names, or a pattern using the
IRAF pattern matching syntax (type "
help system.match" for a description of the
IRAF pattern matching syntax). To obtain a list of the column names, you can run the
tlcol task (type "
tlcol infile.fits").
A row selector can be used to specify a certain row in the table. For example, if you specify:
IRAF will extract data from the table stored in the first extension of
infile.fits, specifically from the columns labeled WAVELENGTH and FLUX, and will restrict the extraction to the row containing segment A data. Section 2.3.2 of the
Introduction to HST Data Handbooks describes the selectors syntax and provides a few examples using the selector syntax to plot COS spectra.
In PyRAF, it possible to dump the arrays of an
x1d file to an
ASCII file by using the tasks
txtable and
tdump. For example, to extract the WAVELENGTH, FLUX and ERROR columns of FUV
x1d file
l9v221fkq_x1d.fits, first use the
txtable task to convert the 3-D
x1d table to a 2-D table:
This will create two new 2-D tables, l9v221fkq_r0001.tab and
l9v211fkq_r002.tab, containing the FUVA and FUVB data respectively.
Then use the tdump task to dump the WAVELENGTH, FLUX and ERROR columns of the 2-D tables into one
ASCII file using the following commands:
Each row of each of the science extensions in an x1d file will contain the columns listed in
Table 2.5; a similar table, including array dimensions, can be displayed by using the task
tlcol (see Section 2.3.2 of the
Introduction to HST Data Handbooks).
When using many IRAF and
PyRAF routines with
x1d files as input, it will be necessary to specify the extension number of the file. For example, to plot flux vs. wavelength in an
x1d file, using the
sgraph task, the command needed would be:
cl> sgraph "l9v220eqq_x1d.fits[1][r:row=1] wavelength flux"
|
For FUV data, the x1d files contain both
1 segments A and B. For example, to plot flux vs. wavelength in an FUV
x1d file for segment A, using the
sgraph task, the command needed would be:
cl> sgraph "l9v220eqq_x1d.fits[1][r:row=1] wavelength flux"
cl> sgraph "l9v220eqq_x1d.fits[1][r:SEGMENT=FUVA] wavelength flux"
|
To plot flux vs. wavelength in an FUV x1d file for segment B, using the
sgraph task, the command needed would be:
cl> sgraph "l9v220eqq_x1d.fits[1][r:row=2] wavelength flux"
cl> sgraph "l9v220eqq_x1d.fits[1][r:SEGMENT=FUVB] wavelength flux"
|
For NUV data, the x1d files contain the three stripes A, B, and C. For example, to plot flux vs. wavelength in an NUV
x1d file for stripe A, using the
sgraph task, the command needed would be:
cl> sgraph "l9v220efq_x1d.fits[1][r:row=1] wavelength flux"
cl> sgraph "l9v220efq_x1d.fits[1][r:SEGMENT=NUVA] wavelength flux"
|
To plot flux vs. wavelength in an NUV x1d file for stripe B, using the
sgraph task, the command needed would be:
cl> sgraph "l9v220efq_x1d.fits[1][r:row=2] wavelength flux"
cl> sgraph "l9v220efq_x1d.fits[1][r:SEGMENT=NUVB] wavelength flux"
|
To plot flux vs. wavelength in an NUV x1d file for stripe C, using the
sgraph task, the command needed would be:
cl> sgraph "l9v220efq_x1d.fits[1][r:row=3] wavelength flux"
cl> sgraph "l9v220efq_x1d.fits[1][r:SEGMENT=NUVC] wavelength flux"
|
x1d files can also be displayed and analyzed using the
splot routine in the
noao.onedspec package of
IRAF or
PyRAF. This requires converting the
x1d binary table first into a standard
IRAF format (or OIF) which can be read by
splot. The standard format is usually
.imh format, containing the flux and wavelength information. The most convenient way of extracting the wavelength and flux information from the
x1d table is to use
tomultispec in the
stsdas.hst_calib.ctools package.
tomultispec takes rows from the
x1d table and converts them into standard
IRAF form. It fits a dispersion solution to each wavelength in the array and stores the solution in the header of the output
IRAF spectrum.
If a COS x1d table contains spectra from segments A and B of the FUV channel (as would be the case if
calcos were ran with both
_rawtag_a and
_rawtag_b files present), then the spectrum output by
tomultispec will consist of two spectra, one from each channel. Running
tomultispec on an NUV
x1d file will always produce a multispec file with three separate spectra, one for each stripe.
cl> tomultispec l9v220eqq_x1d.fits l9v220eqq_x1d.imh
|
This command will write the spectra from both channels to a single.imh file. However, a single channel can also be written if desired by specifying the row to be written. For example, to write spectra from FUV segment A use the following syntax:
Segment B would be denoted by row=2. For NUV data the three stripes are denoted as rows 1, 2, and 3 for Stripes A, B, and C respectively. The FLUX column is extracted by default, though other columns such as GROSS, NET, or ERROR could also be extracted.
If running tomultispec produces a segmentation fault, bus error, or other equivalent issue, the problem may be solved by setting a value for
IMDIR in login.cl (in particular, setting
IMDIR=HDR$ will generally ensure that
tomultispec functions as intended).
The IRAF spectrum can now be displayed using
splot:
cl> splot l9v220eqq_x1d.imh
|
Within splot, individual emission lines can be fit with commands like ‘k’ and composite lines can be deblended using ‘d’.
The x1dcorr module in
calcos is designed to extract flux calibrated 1-D spectra from corrected COS event lists (
corrtag files). This module is called by
calcos as part of standard pipeline processing; its functioning in that role is described in
Section 3.4.15.
For rawtag exposures the wavecal lamp exposures are taken either concurrently with the science
rawtag spectra (
TAGFLASH) or they are acquired as separate
rawtag spectra (
Auto or
GO wavecals
). For all science
rawaccum exposures the wavecals are acquired as separate
rawtag exposures.
The wavecal exposures are used by calcos to determine the location of both the wavecal image and the corresponding science image on the detector. The location may vary in a non-repeatable manner due to non-repeatability of the COS grating positions. When auto-wavecals are acquired as separate exposures they are taken close in time to the science exposures, with the grating in the same position as during the science exposure.
After processing data through calcos, you may decide that you need to shift the spectrum along the dispersion direction to correct offsets in the wavelength calibration. For example, wavelength calibration offsets may occur due to offsets of the target from the center of the PSA aperture (which can occur if the target acquisition was imperfect), or from repositioning of the grating due to thermal flexures. Assuming that
calcos has been run on the data and a residual wavelength offset has been found in the calibrated spectrum, the offset can be corrected by first calculating the number of pixels corresponding to the offset, then subtracting it from the raw position of coordinates along the dispersion direction. The shift is applied to the RAWX column of the
rawtag or
rawtag_(a,b) file. In
IRAF and
PyRAF the
tcalc task from the
ttools package can be used to apply the shift:
In the example above, the RAWX positions of the science spectrum in an FUV A rawtag file have been moved by
“SHIFT
” pixels. The shift is only applied to the parts of the science spectrum located between the pixels RAWY=Y1 and RAWY=Y2 along the cross-dispersion direction.
For spectra, a background region offset from the extraction region is used to determine the background. You can adjust the default parameters for this background region by first copying the _1dx reference file listed under the
XTRACTAB keyword in the primary header to a local directory, then adjusting the background parameters within the local version of the
_1dx reference file. Once you have adjusted the parameters to your satisfaction, edit the primary header of the
_rawtag file (with an
IRAF task such as
hedit) to indicate the path to the local version of the
_1dx file. You can then run
calcos with the updated background subtraction parameters.
The values of these parameters in the local _1dx file can be edited with the
tedit task in
IRAF/
PyRAF.
The task splice can be applied to combine overlapping extracted COS spectra (e.g. spectra taken with different central wavelengths). It takes into account the error (ERR) array as well as the data quality (DQ) array. Handling of the DQ array is important as it helps
splice perform the combination properly and avoid bad or noisy data in the output file arising from the large changes in throughput at the edges of the detector.
Please refer to the splice task help file for more useful information. If a multispec format spectrum is preferred for further analysis, the task
tomultispec can be run on the output file of the
splice task.
Running splice as mentioned above (rather than transforming individual
x1d fits tables into multispec format before combining them) has important advantages: it keeps the science data, error, and DQ arrays intact allowing for easier error analysis, and it does not have a limitation on the number of segments or wavelengths to include, a problem with the multispec format due to the limit on the size of the FITS header which requires fitting the wavelength scale with a function.
