|Space Telescope Science Institute|
|COS Data Handbook 2.00|
Sometimes the pipeline calibration performed shortly after the data were obtained from the telescope is not the best possible calibration for your science program. There are a number of reasons why it may be desirable to recalibrate your data. The most likely reasons include:
• Some steps need to be repeated with different input parameters. For example, you may wish to re-perform the 1-D spectral extraction with a smaller height in order to minimize the background, or you may wish to cut a TIME-TAG exposure into sub-exposures, in order to study time variability.The simplest way to recalibrate your data with the most appropriate reference files is to request the data from the archive again. However, to tailor the calibration to your individual preferences, it may be beneficial to run calcos yourself on your local machine, or to use tasks that improve the reference files or allow customized treatment of the data. Calcos can be imported and executed when running PyRAF or Python.
Be sure you are using the latest versions of the calcos and STSDAS software, COS calibration files, and raw data files (which list the latest reference files in their headers). STSDAS release information can be found at www.stsci.edu/resources/software_hardware/stsdas.Calcos contains provisions for recalibrating raw data. Users can specify the pipeline processing steps to be performed and select the associated reference files. However, calcos was not designed to run its various modules independently, i.e. it is not re-entrant. The pipeline flow is modified by setting calibration switches or reference file names and then rerunning the entire pipeline. The calibration switches in the headers of the calibrated data files will reflect the operations performed on the calibrated data and the reference files used.If you chose to recalibrate your COS data on your local machine, there is a certain amount of set up required for calcos to run properly. The operations mentioned in the checklist below will be described in detail in the following subsections:
• Set the environment variable lref to point to your reference file directory. Note: you must do this before starting a PyRAF session!
• Update the input data file headers (including reference file names). In IRAF, this would be done using thedit.
• Run calcos.Before running calcos, you will need to define an environment variable to indicate the location of the directory containing the needed calibration reference files. The names of the calibration files are preceded with the logical path name “lref$” in the COS science headers. Ordinarily you would define this directory in a PyRAF session to be, for example, “/data/vega3/cos/lref/” using the set command:
Note the trailing slash (/). However, calcos and all of its modules are actually foreign tasks and as such do not access PyRAF environment variables. Therefore, before invoking the cl, you will need to define an environment variable from the host command line (see below) that is appropriate to your host machine. For Unix/Linux/Mac systems, the appropriate command for the example above is:
Note that an alternative to using the lref$ variable is specifying the full pathnames to the reference files in the science headers.
When running calcos or any of its modules, you must define environment variables (such as lref$) before starting the cl. It is not possible to define them within IRAF using the set command, nor is it possible to define them with an escape to the host level, such as:
!setenv lref /data/vega3/cos/lref/To recalibrate your data, you will need to retrieve the reference files used by the different calibration steps to be performed. The names of the reference files to be used during calibration must be specified in the primary header of the input files, under the section “CALIBRATION REFERENCE FILES.” Note that the data headers will be populated already with the names of the reference files used during pipeline calibration at STScI.Chapter 1 of the Introduction to HST Data Handbooks describes how to retrieve data and reference files via the World Wide Web. To retrieve the best reference files via MAST, check “Best Reference Files” in the “Reference Files” section of the Retrieval Options form.The COS reference files are all in FITS format, and can be in either IMAGE or BINTABLE extensions. The names of these files along with descriptions of their contents are given in Section 3.7. The rootname of a reference file is based on the time that the file was delivered to the Calibration Data Base System (CDBS).To edit file headers in preparation for recalibration, use the STSDAS task thedit. The thedit task takes several input parameters: the name(s) of the raw data files to be edited, the header field to be edited, and the new value of the header field. It can be used to change the values of any calibration switches, reference files or tables to the values you wish to use for recalibrating your data. To edit the calibration keyword values:
1. Run the thedit task, specifying a list of files in which you want to change calibration keyword values. You may specify more than one file (using wildcards, for example) to be updated. For example, you could change the flat reference file to be used for all COS raw science files in the current directory using the command:
If you are changing keywords that reside in the FITS primary header unit with hedit or thedit, be sure to explicitly specify the primary header by appending “” to the FITS file name.Users may find it necessary to edit the input association file for calcos. Reasons for editing an association file might include the use of a different wavecal or to remove a compromised exposure from an association. One way to update an association file is to use the STSDAS task, tedit. For example, use the PyRAF task tprint to first look at the contents of association table, l9v221010_asn.fits.
To quickly see basic exposure information for each exposure listed in the association use the thselect command:
--> thselect l9v221*raw*fits \ filename,detector,aperture,opt_elem,cenwave,exptype,obsmode,fppos ’yes’To remove a member from association, l9v221010_asn.fits, and to use a different wavecal file, use the following commands in PyRAF.
Finally, reprocess the data by running calcos on the updated association file. See the following section for syntaxes of running calcos.Run CalcosUsers may choose any of three ways to run calcos using Python, PyRAF, or from the Unix/Linux/Mac command line. The input arguments and examples for each case are as follows:
1. To run calcos in Pyraf from the hstcos package:
Table 3.2: Arguments for Running calcos in PyRAF
Association table (asn) or individual raw file (rawtag, rawaccum) or corrtag file to be processed Verbose output from calcos Save temporary files: x1d_a, x1d_b, lampflash_a, and lampflash_b Have calcos find the spectrum location and centre the extraction box on that location
2. To run calcos in Python:
Table 3.3: Arguments for Running calcos in Python:
Association table (asn) or individual raw file (rawtag, rawaccum) to be processed Have calcos find the spectrum location and centre the extraction box on that location Save temporary files: x1d_a, x1d_b, lampflash_a, and lampflash_b
3. To run calcos from the Unix/Linux/Mac command line:
Table 3.4: Command-line Options for Running calcos in Unix/Linux/Mac:
Have calcos find Y location of spectrum To redirect the calcos STDOUT to a file use the following command:
While users are recommended to run calcos on association files, it is possible to run calcos with a single raw or corrtag file as the input. In this mode, calcos will always automatically process both segment files for FUV data if they both exist. For example if rootname_rawtag_a.fits is the input for calcos, then rootname_rawtag_b.fits will automatically be processed. The data from both segments will be calibrated and combined to create the final product, rootname_x1d.fits3.6.2 Using GO WavecalsThrough the use of associations, calcos also contains a provision to select wavecals other than the default for calibration of the science exposures. To use an exposure other than or in addition to the default wavecal, the user can add a row to the association table. The rootname (case insensitive) should be given in the MEMNAME column, the string EXP-GWAVE in the MEMTYPE column, and the value in the boolean MEMPRSNT column set to true (e.g. yes if you use the IRAF tedit task). Make sure that the WAVECORR keyword in the primary header of the raw science file is set to PERFORM, and then run calcos as normal. Note GO Wavecals can only be used with non tagflash data.