COS Data Handbook Version 2.0

COS detectors can be used in ACCUM or TIME-TAG modes, as described in Chapter 5 of the COS Instrument Handbook. In TIME-TAG mode, the position and detection time of every photon is recorded in an events list. Detection times are recorded with 32 millisecond precision, although events may be buffered for as long as 32 milliseconds prior to assignment of a detection time.

For TIME-TAG datasets, the HST archive returns a raw events list in a file with a _rawtag suffix. The _rawtag file is a FITS file with two binary table extensions. The first extension contains the events list and the last extension a list of good time intervals, indicating time intervals when events are valid.

An events list in a _rawtag file is a FITS binary table extension named EVENTS, containing four columns named TIME, RAWX, RAWY, and PHA. Note only FUV data will include the PHA columns in the _rawtag files.

The TIME in the events extension contains the time when each event was recorded, relative to the start time (MJD) of the exposure given in the EXPSTART keyword of the primary FITS header.

In TIME-TAG the RAWX column contains the pixel coordinate along the spectral axis where each event was recorded. Corrections to remove Doppler shifts introduced by the orbital motion of HST are applied by calcos and placed in the corrtag file. The correction depends on optical element and the projected orbital velocity of HST, which varies over the course of an observation. In ACCUM mode, this Doppler compensation is applied on orbit during an observation and is included in the RAWX column, but in TIME-TAG mode the uncorrected positions are downlinked and Doppler compensation is applied during ground processing. The RAWY column contains the pixel coordinate along the spatial, or cross-dispersion, axis. No Doppler compensation is applied. The PHA column (for FUV data only) contains the pulse height amplitude for each event as an integer on a 5-bit scale.

After all EVENTS extensions in a _rawtag file, there will be one final binary table extension named GTI, containing columns named START and STOP. There will be associated start and stop times for every uninterrupted observing interval during a planned exposure. For most datasets, there will be only one START and one STOP time encompassing all buffer dumps in an exposure. Multiple good time intervals are possible, however - for example, if guide star lock is lost. Times in START and STOP are expressed in seconds since the start time (MJD) of the exposure given in the EXPSTaRT keyword of the primary FITS header. The exposure start time (JD) is also provided in the EXPSTaRTj keyword of the primary FITS header. The start time is also provided in the In IRAF, good time intervals can be examined using the tprint task in the tables package:
where rootname must be replaced by the rootname of the _rawtag file being examined.

cl> tprint rootname_rawtag.fits[GTI]

Useful IRAF tasks for analyzing and manipulating data taken in the TIME-TAG observing mode are listed in Table 5.3.

Task	Purpose
tpar, tprint	Read the value for table header keywords
sgraph	Display the 1-D spectrum
tomultispec	Convert x1d spectrum into IRAF .imh spectrum
splot	Display an IRAF .imh spectrum
splice	Splice spectra together

To view a TIME-TAG file (_rawtag_(a,b) in the FUV, _rawtag in the NUV), open ds9, then choose ‘open’ from the menu bar at the top. The image will load but, save for a few pixels registering a value of 1, the remaining pixels will be zero.

Once the image is loaded, go to the menu item ‘bin’ and open the pull-down menu from that. From that pulldown menu you can choose the size of the image to view – generally you should make it as big as possible: 8192X8192 pixels for FUV data, 1024X1024 pixels for NUV.

NOTE: for the instructions below, the changes will not take effect until you click on the ‘Apply’ button.

Now choose ‘Binning Parameters’ from the ‘bin’ pulldown menu. This will open a new window with the binning parameters listed. You will notice right away that the Bin Columns are listed as TIME and RAWX. These are what is currently being displayed by ds9 (which is why the image looks so strange when initially loaded). However, what you really want is RAWX vs. RAWY, so change that in the pulldown menu under Bin Columns.

You can also set the blocking size of the image in the ‘Binning Parameters’ window – just type in ‘2’ in the Block field next to RAWX. By blocking this way along the dispersion direction, you can now see virtually all of the 16384 pixels along the dispersion direction. If you are looking at NUV data, then no additional blocking is needed – just leave the blocksize as 1, but choose the image size as 1024 pixels from the ‘bin’ pulldown menu.

Next, from the ‘Binning Parameters’ window choose the part of the spectrum to be centered on the middle of the dispersion direction by clicking on the button marked ‘center of data’. Now press ‘Apply’ on the binning parameters window to update the ds9 display.

The spectrum should now be displayed, with the dispersion direction running from left to right. To better see the data, choose ‘scale’ under the main ds9 menu bar, and from that pulldown menu choose a square root stretch and min/max range. You can now pan your cursor over the image, while holding the right button down on your cursor, until the contrast looks just right. If you would like to smooth the data a bit (this can be useful for bringing out fainter features and increasing signal to noise along the display), choose the ‘Analysis’ menu item under the main ds9 menu bar and select ‘smooth parameters’. A dialogue box will open, and from there you can set the number of pixels to smooth. Finally, you can also click on the ‘Color’ item on the ds9 menu bar and choose ‘invert color map’ to get an inverted color map.

You can also load a _corrtag_(a,b) table in ds9, but in this case the appropriate columns to display are XFULL and YFULL. Otherwise, the same ds9 commands apply as for _rawtag files. For both TIME-TAG and ACCUM spectroscopic data the _flt and _counts spectral images will load as simple 2-D images in ds9.

For both TIME-TAG and ACCUM imaging data The _flt and _counts images will load as simple 2-D images in ds9.

You can assign events registered during each time interval to a separate image in ds9, thereby creating a sequence of images which can be played as an animation. This can be useful in verifying the occurrence of lamp flashes in TAGFLASH data, in searching for the appearance of bursts in raw data, and so on. To bin the images in time, set up the image as described above – with RAWX and RAWY chosen in the ‘Binning Parameters’ dialogue box. At the bottom of the ‘Binning Parameters’ box is a parameter called ‘Bin 3rd Column’. Set the value of this parameter to TIME. Next, choose the number of bins you would like to divide the event file into under the ‘Depth’ parameter. Setting this value to 10, for example, will create 10 separate images, with the first one showing all events registered during the first (EXPTIME/10) seconds, the next one showing all events registered between (EXPTIME/10) and (2*EXPTIME/10) seconds, the next showing all events registered between (2*EXPTIME/10) and (3*EXPTIME/10), and so on up to EXPTIME. The ‘Min’ and ‘Max’ parameters let you choose the range of values in time to display – usually this is pre-set to 0 and EXPTIME, and can be left unchanged to bin the entire image as above. Select ‘Apply’ to do the binning.

Note that some time will be required to create the sequence of images, and that binning the events in time in ds9 is very memory intensive, and that it is easy to make ds9 crash if EXPTIME is large (for example >1000 seconds) and the number of bins in ‘Depth’ is set to a large value (for example 30). It is best to start with a small value for ‘Depth’ that works, then increase the value if needed.

After the binning is done, a new dialogue box will appear called ‘Data Cube’. Numbered from left to right will be the enumeration of the bins (in the example above from 1 to 10), along with a slider underneath. Click on ‘Play’ in that window to start the animation – it will play each of the binned images sequentially in the ds9 window. Again, the spacing between each of the bins will be (EXPTIME/10) in seconds, or (EXPTIME/Nbin), where Nbin is the number of bins.

In the animation, it should be possible to see the TAGFLASH spectrum appear and disappear as the sequence progresses. Obviously the sequence will show the flashes only if the keyword TAGFLASH=auto or TAGFLASH="uniformly Spaced" is in the header of the event file.

To exit from the animation, close the ‘Data Cube’ window, and then set the ‘Depth’ parameter in the ‘Binning Parameters’ dialogue box to zero, and click ‘Apply’. That will reset the image in ds9 to show all of the data again.

It is possible to bin in other parameters as well, such as PHA. The logic is the same as above.

As mentioned in Section 2.4.2, all corrtag files processed with calcos 2.14 or later contain a timeline extension. The timeline extension can be operated on by the timefilter module to exclude photon events that match user-specified patters in the time extension. The timefilter module is available as part of STSCI_PYTHON, and requires calcos version 14+ to work. The normal use of timefilter is to exclude daytime events in order to minimize the contribution of geocoronal lyman alpha or O I emission lines to your data. Timefilter will filter events according to a filter string passed to it.

The filter string consists of one or more filter conditions, separated by "and", "or", or "xor" (parentheses are currently unsupported). Each filter condition consists of a column name, a relation, and a cutoff value. Valid column names are "time", "longitude", "latitude", "sun_alt", "target_alt", "radial_vel", "shift1", "ly_alpha", "OI_1304", "OI_1356", and "darkrate" (see Table 2.3 for a description of the columns). Valid relations are '>', '>=', '<', '<=', '==', and '!='. Cutoff values are numerical values. In addition, it is possible to flag events based on one of the 32 SAA model contours with the filter condition "SAA #" where # is a number from 1 to 32. Events which match the filter string will be marked with the DQ flag 2048 (bad time interval), and will be excluded in the creation of flt and x1d files.

Timefilter can either modify an existing CORRTAG file in place, or create a new one, and it can be run in conjunction with splittag (although in that case, it is possible that some output files will contain no valid events at all). The file produced may be extracted with the x1dcorr task as usual. It is possible to remove any events filtered with timefilter by running "timefilter.py 'input_file.fits' '' reset" followed by "timefilter.py 'input_file.fits' '' clear".

The following examples show common uses of Timefilter:

timefilter.py test_corrtag_a.fits output_corrtag_a.fits 'sun_alt > 0'

timefilter xyz_corrtag_b.fits "" "sun_alt > -10 and ly_alpha > 2.5 or saa 31"

timefilter.py xyz_corrtag_b.fits '' reset timefilter.py xyz_corrtag_b.fits '' clear

Users may wish to process only sub intervals of TIME-TAG events, to look for variability in the data. One way to do this would be to divide an exposure up into several sub-exposures before re-processing by using the splittag program.

splittag operates on corrtag files, so you will need to retrieve the calibrated data (by using corrtag files, calcos is able to use the existing wavelength fits derived during the calibration process, and as such splitting data with lamp flashes will not result in wavelength calibration information being unavailable).

The splittag task is available as part of the STSDAS package within IRAF (stsdas.hst_calib.hstcos). It is a useful tool for dividing a COS time-tag exposure (FUV or NUV) into a series of sub-exposures with time intervals specified by the user. The task operates on the calcos corrtag files, copying rows from a corrtag file into one or more output files. The number of files depends on the number of time intervals specified by the user. The resulting corrtag sub-exposures can then be run separately through the x1dcorr task in stsdas.hst_calib.hstcos to extracted one-dimensional, flux-calibrated spectra (*_x1d.fits files) for each file.

The following keywords are modified when splittag copies the time columns to the new corrtag files: EXPTIME, EXPEND and EXPENDJ. The keywords EXPSTART and EXPSTRTJ, on the other hand, are not changed. The EXPTIME keyword in each of the new corrtag files will be set to the duration of the time interval being extracted, while the modified Julian date and Julian date in EXPEND and EXPENDJ will be set to the following:

where t_end(i) is the ending time of the ith desired sub-exposure. In addition to the updated keywords, splittag also produces updated GTI (good time interval) tables for each of the output corrtag files. The GTI intervals are specified relative to the times of the original corrtag file, such that the split corrtag files will not include events outside the GTI values. The EXPTIME keyword written to the corrtags affected by the GTI intervals is shortened accordingly.

There are two ways to run the splittag task: (1) specify a starting time, an increment, and an ending time, or (2) provide an explicit list of times (not necessarily adjacent to one another). In either case, the output corrtag files will have a root name specified by the user. If no root name is specified, the root name of the input corrtag will be used, appended with numbers 1,...N for N exposures.

The parameters input by the user for splittag include the following: an input corrtag file name, a root name for the output files, the starting time for the first event to be extracted, the time increment to be used in extracting the following intervals, and the ending time of the extraction. If option (1) from above is used, then the starting time and increment are specified, with the remaining parameters left at their indefinite values. This will extract however many corrtag files are needed until the ending time of the original exposure is reached. If option (2) is used, then the user can specify explicitly, in the form of start/stop pairs, which intervals are desired. For example, specifying time_list="0,20,100" will extract events in the range 0 < t < 20 seconds and output that to a corrtag file, then extract events in the range 20 < t < 100 seconds and write that to another file, and so on. splittag can also read in a text file with the start/stop pairs entered (using the format in the example above). In that case, all the start/stop pairs would be listed in one line in the text file, separated by either commas or spaces. If this option is used (i.e., the time_list parameter is set to point to the text file), then the starttime and increment parameters are ignored.