April 26, 2010

About This Article

CALCOS 2.12 went in OPUS on April 22, 2010. This was tested with OPUS 2010.1b build and installed days later after this version of OPUS was released. In this version of CALCOS, many changes to wavecal processing were made. For a complete list of the changes that went into these version, please check their release notes.

Relevant PRs
PR Updates
62062 CALCOS should populate photometric keywords for imaging data
62433 Modify wavecal processing in CALCOS.
62810 Check for integer division using '/' in calcos.
62895 Allow calcos to use a corrtag file as an input.
63497 Adjust ACS Operate<-->LVon recon times
63498 2009 ACS Path Times - No CCD cmding in SBC recons
63862 Calcos should check x1dcorr within the loop.
63863 Calcos should create x1dsum1, etc.
63864 Calcos should copy G140L wavecal info from A to B.
63865 Miscellaneous changes to calcos.
63866 Calcos uses a poor criterion for detecting a spectrum.
63867 Calcos is computing FUV wavecal wavelengths twice.
63868 Calcos needs to support multiple pulse height ranges.
63869 The calcos x1d table should have a gross counts column.
63870 Change numpy import statement in calcos.
64125 COS: Add keyword to provide the average deadtime correction factor
64187 Calcos should write a wavelength column to the corrtag table.
64189 Calcos should print the value of RANDSEED.
64190 Calcos should fit a quadratic to find the spectrum.
64193 Calcos should compress empty csum image.
64224 Calcos needs an option to not update keywords in the flt and counts files.
64233 Delete the SC baseline history files by adding their contents into the corresponding scs-set history files.
64275 Calcos differences in regions for wavecal shifts.
64419 Calcos move call to filter by pulse height.
64722 Bugs in calcos regarding DQ extension for NUV.
64723 Calcos should print the measured wavecal shift.

Relevant Trac Tickets

356, 394, 415, 417, 425, 446, 447, 463, 464, 465, 466, 466, 467, 468, 469, 470, 471, 480, 481, 482, 483, 484, 486, 491, 497, 513, 524, 525


  • There were cases where the flagged regions in the data quality (DQ) extension were much wider than they should have been. This was due to a bug in the code that widens the regions due to changes in the wavecal and Doppler shifts during the exposure.  Also, the broadening due to changes in the Doppler shift are no longer applied to the wavecal region.
  • The function to flag events if the pulse heights are out of bounds was moved to a point after geometric correction, because this should only be applied to data within the active area, and that region is not well defined until after geometric correction.
  • A new WAVELENGTH column has been added to the corrtag table. This gives the wavelength (Angstroms) of each event that is within the active area, if WAVECORR is PERFORM.  For events outside the active area, the wavelength will be zero.
  • A new GCOUNTS column has been added to the x1d table. This gives the gross counts, while the GROSS column contains count rate.
  • The average livetime factor (deadtime correction) that was applied to the data is recorded in keyword LIVETM (for NUV) or LIVETM_A and LIVETM_B (for FUV).  Keywords DEADRT* and DEADMT* were being updated in the primary header, but they are supposed to be in the extension header.
  • If an association has a product (i.e. MEMTYPE = PROD-FP), an x1dsumi.fits file (i = 1..4) will be created even if there is only one exposure with FPPOS = i.
  • X1DCORR may be set to PERFORM for one FUV segment and OMIT for the other.
  • A new GTI table (good time intervals) will only be added to the corrtag file if the GTI list was actually modified (e.g. due to FUV bursts).
  • When pseudo-random numbers are added to pixel coordinates for FUV data (RANDCORR), the seed (keyword RANDSEED) that was used to initialize the generator is now printed (followed by "(was -1)", if the value in the raw header was -1).
  • The hard-coded values used for populating photometry keywords PHOTFLAM, etc., for image data were updated based on in-flight data. The values for the BOA are still preliminary.
  • In the function for finding the location of a wavecal spectrum in the cross-dispersion direction, the test for whether the spectrum was found was improved, and a quadratic fit is now used for getting the location rather than using the nearest pixel to the maximum.

Many changes to wavecal processing were made:

  • For G140L data, the wavecal shift information from segment A is now copied and used to correct the data for segment B.
  • Emission lines in the wavecal spectrum that are on the edge of the detector (truncated by the edge) can interfere with finding the shift, because the truncated line isn't the same shape as the template line and because the mean position is off from what it would be if the entire line were present. For NUV data, a function was added that checks for truncated lines at the detector edges and sets the pixel values near the edge to zero if such a line is found.
  • The algorithm for finding the shift was based on minimizing chi square; however, there are many local minima of chi square, and the one corresponding to the actual shift is not always the absolute minimum. The RMS of the deviation between the wavecal spectrum and normalized template is a better (but still imperfect) indicator of which shift over the entire search range best matches the spectrum with the template, while chi square is more sensitive to small deviations from the best shift.
  • The following algorithm is now used:
    • The local minima of RMS vs shift are found. The 10 local minima with smallest RMS are selected. Some of these may then be rejected if the values of RMS in a wider neighborhood are not increasing. Of the points that remain, the one with smallest reduced chi square is selected. This gives the shift to the nearest integer. A quadratic fit to the neighboring five points of the reduced chi square is then used to improve the shift. Fits are done to 3, 4, 5, 6 and 7 points of the curve, and the scatter in the shift values for those fits are used as a check on the error estimate; if half the range of the scatter is larger than the error estimate based on fitting to five points, then the error estimate is overridden by half the range of the scatter. The error estimate is now printed to the trailer file along with the measured shift value.
  • A wavecal shift in the dispersion direction is flagged as "not found" if the reduced chi square is above a cutoff value. The cutoff was increased from 6 to 15 because there were many cases where the shift was correctly found even though chi square was large.
  • The information printed to the trailer now includes the shift that was actually measured and the value of column FP_PIXEL_SHIFT from the lamptab, rather than just the total shift (the measured shift plus FP_PIXEL_SHIFT).
  • The function to find the average shift of NUV stripes (for the case that one or two stripes were not found) was modified to include more reasonable (but still preliminary) values for parameters, based on SMOV data.


HST Help Desk 

Please contact the HST Help Desk with any questions.