STScI Logo

COS
CALCOS v2.17.3 Release Notes


---------------------CALCOS v2.17.3---------------------

This describes changes from CALCOS version 2.15.6 to 2.17.3.

Relevant Trac tickets and PRs:
Trac   PR
----  -----
 575  65353  Calcos should have an option to search for the target spectrum.
 645  67204  Add a TIMELINE extension to calcos corrtag files.
 719  68545  Calcos needs an option to use the default location for the
             target spectrum.
 720  68550  Calcos should accept data taken with different cenwaves.
 761  69143  Calcos should flag gain-sagged regions in FUV data.
 763  69206  Calcos should check for no rows in G140L lampflash table.
 764  69207  Calcos needs a utility function for segment-specific keywords.
 786  69401  Calcos should extrapolate the TDS correction.
 789  69437  Calcos should expand wildcards for the input file names.
 791  69679  Calcos should use the shift file if it's relevant.
 793  69730  In calcos, modify some pyfits.open calls.
 795  69737  Change the way calcos fixes APERTURE = "RelMvReq".
 805  69937  Account for the width of the OI 1304 line in calcos.
 806  69938  Inconsistent return value from calcos wavecal function.
 818  69977  Calcos should not look for wavecal offsets if wavecorr
             is not complete.
 829  70033  Calcos should check whether a keyword exists before deleting it.
 830  70036  Calcos should read the HVTAB and populate keywords.

Overview:

The TIMELINE table in the corrtag file has a new column, SUN_ZD, the angle
between HST and the Sun, as seen from the center of the earth.  This is
approximately equal to the zenith distance of the Sun as seen from HST,
except that the parallax of the Sun is not included.

A new reference table GSAGTAB may be specified.  This "gain sag" table is
a bad pixel table that flags regions of the FUV detector that have very low
gain.

A new high voltage history table (hvtab) can be read and used to populate
keywords HVLEVEL[AB].  The gain sag table now uses these keywords.

Previously, calcos would accept multiple input file names, but there was
a limit due to the line length.  The input file or files may now be
specified (in quotes) using wildcard characters.

The syntax for the command-line option --find has been changed; a value
should also be specified.  This option has to do with finding the target
spectrum in the cross-dispersion direction.  The value can be "yes", "no",
or a numerical value, a cutoff.  ("true" and "false" are aliases for "yes"
and "no" respectively.)  If a cutoff value is given, this means that the
spectrum will be extracted at the found location if the error estimate
for the location is less than or equal to the cutoff value.

When finding the cross-dispersion location of a spectrum, the quadratic
fit was being done to too few pixels (at least for FUV), resulting in a
poor value for the standard deviation of the fit.  The number of pixels
will now be based on the FWHM, or a minimum of five pixels.

The command-line option --shift  did not work for the case that the
raw file was not tagflash and there was no auto/GO wavecal.  The code
expected tagflash data, and the shift file would specify which lamp flash
was to be overridden.  This has been modified so that if a shift file
was specified for non-tagflash data, the given shift will be applied.

The extrapolation of the time-dependent sensitivity factors has been
modified.  Previously, if the time of observation were before the first
or after the last time in the TDS table, the factor would be extrapolated
with zero slope.  Now the extrapolation will be done using the slope at
the first or last time, as appropriate.  Note, however, that the TDS table
should cover all observations without extrapolation.

Changes were made to some pyfits.open statements to allow using memory
mapping.

When computing the count rate for an airglow line for the TIMELINE table,
a wider range of wavelengths is now used for the OI 1304 triplet.

A function used for wavecal processing could return the wrong number of
items if the input data contained no counts at all.  This could happen for
tagflash data, e.g. if the duration of a flash were extremely short.  This
has been fixed.

Details:

Some of the calls to pyfits.open were modified to change the mode from
"readonly" to "copyonwrite" and/or change the memmap flag.  The modified
files are:  accum.py, average.py, calcos.py, cosutil.py, extract.py,
fpavg.py, splittag.py, timetag.py, wavecal.py, x1d.py.
Trac #793, PR 69730

Changes to accum.py:

In the call to cosutil.updateDQArray in doDqicorr, bpixtab was dropped and
reffiles was added.
Trac #761, PR 69143

Changes to calcos.py:

shift_file was added to the calling sequences for Observation and its
subclasses.  This was done to cover the case that a shift file was specified
but the exposure has tagflash = NONE.  The tagflash flag will be reset to
True if the current root name and fpoffset are listed in the shift file (and
exptype is not WAVECAL).
Trac #791, PR 69679

Function uniqueInput was modified to expand environment variables and
wildcards in the input string.
Trac #789, PR 69437

Command-line options --find and --nofind (which did not have an associated
value) were replaced with --find value, where value could be "yes", "no",
or a numerical value, a cutoff.  ("true" and "false" are aliases for "yes"
and "no" respectively.)  The find_target argument to calcos used to be a
boolean flag, and now it's a dictionary containing a boolean flag and the
cutoff (if specified).
Trac #719, PR 68545

Function missingRefFiles now checks for a GSAGTAB, if dqicorr is perform.
Function mergeKeywords now calls extract.updateGsagComment to concatenate
the comments for keyword GSAGTAB for the segment A and segment B files.
Trac #761, PR 69143

Changes to concurrent.py:

Method outFlashSetup was renamed to setUpOutFlash for consistency with
other functions with verb/object names.  This was noticed while checking
out the problem that led to PR 69938.
Trac #806, PR 69938

processConcurrentWavecal was modified to set numflash to 1 (and to assign
dummy values for lamp_on, etc.) if no flash was detected but there is a shift
file.  Function dummyFlash was added to do this.
Trac #791, PR 69679

In copySegAtoSegB, the test on whether ofd[1].data is None was moved to
an earlier point, and if it is None, the function prints a warning message,
sets numflash to zero, and returns without trying to copy the data.
Trac #763, PR 69206

In the call to cosutil.updateDQArray in findShifts, bpixtab was dropped and
self.reffiles was added.
Trac #761, PR 69143

Changes to cosutil.py:

Function exptimeKeyword() was deleted, replaced by a new function
segmentSpecificKeyword().  This is now also called in doImageStat.
Trac #764, PR 69207

Functions getDQArrays, concatArrays, and findGSagExtn were added.  bpixtab
was dropped from the calling sequence of updateDQArray, and reffiles was
added.
Function getTable was modified to allow a two-element tuple as a value
in the filter dictionary.  In this case the first element is a numpy
function (e.g. np.less_equal) and the second is the numerical value with
which values in the table will be compared using the specified function.
This capability was added in order to select rows of a table based on a
column of MJD dates; if the date in a row of the table is less than or
equal to the specified value, the row will be selected.
Trac #761, PR 69143

Changes to extract.py:

Function fixApertureKeyword was changed to set APERTURE to just the
aperture name (assumed to be "PSA"), excluding the "-FUV" or "-NUV".
Trac #795, PR 69737

Function getTdsFactors was modified to change the extrapolation of the
TDS factors.  Instead of extrapolating with zero slope if the time of
observation is before the first or after the last time in the TDS table,
the extrapolation will now be done using the slope at the first or last
time.
Trac #786, PR 69401

The find_target argument was boolean flag, and now it's a dictionary
containing a boolean flag and the cutoff (if specified), based on the
"--find value" command-line argument to calcos.
Function checkLocation no longer returns find_target; instead, it updates
the "flag" entry in-place.
Function extractSegment checks both the flag and cutoff to determine
whether or not to use the found location.
Trac #719, PR 68545

The checkLocation function now only returns location and extrsize; the
"flag" member in the find_location argument may be modified in-place.

Function segmentSpecificKeyword() is now called instead of exptimeKeyword()
to get the exposure time keyword.
Trac #764, PR 69207

In the call to cosutil.updateDQArray in doExtract, bpixtab was dropped and
reffiles was added.  Function updateGsagComment was added; this concatenates
the comments for keyword GSAGTAB from two input files, one for segment A and
the other for segment B.
Trac #761, PR 69143

Changes to fpavg.py:

Function getInputInfo saves cenwave as a list, including all distinct values
of cenwave in the input files.  Function createOutput checks the length of
that list, an if it's greater than one, keyword cenwave will be deleted from
the primary header that's used as a template for creating the output file.
Trac #720, PR 68550

Changes to getinfo.py:

Function segmentSpecificKeyword() is now called instead of exptimeKeyword()
to get the exposure time keyword.
Trac #764, PR 69207

Function getRefFileNames was modified to get the name of the gain sag table.
Function getGeneralInfo was modified to get the FUV detector high voltage,
keyword DETHVLA or DETHVLB.
Trac #761, PR 69143

Changes to splittag.py:

Function segmentSpecificKeyword() is now called instead of exptimeKeyword()
to get the exposure time keyword.
Trac #764, PR 69207

Changes to timeline.py:

The AIRGLOW_WAVELENGTHS dictionary was modified to specify a tuple of
wavelengths for an airglow line, instead of just a single value.  This was
needed primarily for OI 1304, which is a triplet with a range corresponding
to about 380 pixels for an FUV M grating.  Function findPixelRegion uses
the minimum and maximum wavelengths of an airglow line or lines to find the
range of pixels on the detector.
Trac #805, PR 69937

Function timelineHDU now defines a new column, SUN_ZD, the angle between
HST and the Sun, as seen from the center of the earth.  This is approximately
equal to the zenith distance of the Sun, as seen from HST, except that the
parallax of the Sun is not included.
Function computeZD was created; this returns the angle from HST to an object
(in particular, the Sun), as seen from the center of the Earth.  This is
called in createTimeline to populate the new SUN_ZD column.
Trac #645, PR 67204

Changes to timetag.py:

Function getWavecalOffsets was modified; wavecorr was added to the calling
sequence, and a test on wavecorr was added to the test on obstype.
In doDqicorr, a check on minmax_doppler was added before calling
psaWcaBoundary.
Trac #818, PR 69977

Function segmentSpecificKeyword() is now called instead of exptimeKeyword()
to get the exposure time keyword.
Trac #764, PR 69207

In doDqicorr, printRef is also called for the GSAGTAB.  cosutil.getDQArrays
is now called to get the data quality arrays (combining bpixtab and gsagtab),
and the call to cosutil.updateDQArray was modified to include reffiles and
remove bpixtab.  The comment for keyword GSAGTAB is replaced by a string that
gives the number of the extension in the gain sag table that matches the
segment and detector high voltage.
Trac #761, PR 69143

Changes to wavecal.py:

Function ttFindWavecalSpectrum returns a tuple of four items (as of calcos
version 2.8.5), but if the input array xi has zero length,
ttFindWavecalSpectrum returned only three:  (0., {}, {}).  Now it returns
four items, and the two dictionaries (xd_shift and xd_locns) are populated
with None and 0 respectively for the current segment or for each stripe.
The dictionaries were populated because concurrent.findShifts expects an
entry for these dictionaries for each segment or stripe.
Function ttFindSpec now checks whether cosutil.centerOfQuadratic returns
None for y_temp.
Trac #806, PR 69938

Changes to x1d.py:

In function makeFltCounts, switches["wavecorr"] was added to the argument
list for getWavecalOffsets.
Trac #818, PR 69977

The command-line option --find was replaced with --find value, where value
could be "yes", "no", or a numerical value, a cutoff.  ("true" and "false"
are aliases for "yes" and "no" respectively.)  The find_target argument is
now a dictionary containing a boolean flag and the cutoff (if specified).
Trac #719, PR 68545

Function segmentSpecificKeyword() is now called instead of exptimeKeyword()
to get the exposure time keyword.
Trac #764, PR 69207

Changes to xd_search.py:

The findPeak function was modified to (sometimes) increase the range over
which the quadratic is fit.  For FUV data, the previous value of five pixels
was unreasonably small.  The FWHM is now initially determined by starting
with the location of the brightest pixel (instead of waiting until after
an accurate location had been found), and a quadratic is then fit to either
FWHM or 5 pixels, whichever is larger.  The code in findPeak to find the
limits of the slice to use for the fit was flawed; it could have resulted
in a negative lower limit for the slice.
Trac #575, PR 65353