Upreffile adds a directory name to the filename returned from getref, because directory names are not stored in the database. It was assuming all fits files go in the ?ref directories. But graph and component tables should be placed in the mtab directory. Addrefdir.x was modified to place these files properly.

Mkload has been modified so that if it finds more than one comparison file and at least one of them is a fits file, only the fits files are returned. This is a fix for the problem where a synphot fits file and its .tab equivalent both have the same useafter date, so both are chosen as comparison files. The modification was made to compfileqry.c.

3 April 2000 release

• Revised uniqname to generate new format throughput table names

  The naming convention for synphot throughput files was changed so that the extension _syn is added at the end of the root name. This means that the version number section of the root name is now second to last, rather than last. The changes are all in the file thrutab.c.

• Changed mkcomptab to add a phony useafter date to table header

  We finally realized the whole idea of selection of synphot tables by useafter dates is never going to work properly, so we made the decision to always use the most recent graph and component lookup tables. To support this decision, mkcomptab now always adds the phony useafter date of "April 1 200" to the table header. The changed file is comptabqry.c

• Changed the default value of ext option in mkcomptab

  The ext option contols which files are included in the component lookup table by only including files with extension specified by ext. The default value has been changed to fits.

• Restored stubbed out code to count image extensions

  The internal function count_reffile in cdbslib/reffile.c counts the number of groups or extensions in an image. It was stubbed out with a message: "The code in this function does not work currently because of a bug in the iraf interface. Send an error message instead." Apparently the bug has been fixed, so I removed the stub.

• Added script to update load files

  I wrote a script to update a batch of loadfiles at a time. The script either sets the value of a keyword if it is blank or adds it if it is missing. The script is addload.pl, in the util subdirectory.

16 February 2000 release

• Fixed DB server environment variable name

  The CDBS documentation stated that it was possible to override the default database server name by setting the environment variable CDBS_SERVER_2. The code actually was checking the environment variable CDBS_DBSERVER_2. I've changed the code to match the documentation. The changed files are include/dbquery.h and cdbslib/initquery.c.

• Modified mkcomptab to optionally produce fits table

  Mkcomptab produces the component lookup table from the "latest and greatest" throughput files in the CDBS database. Fits tables have been added to the database, but their useafter date is the same as the Stsdas binary tables they were produced from. Since the useafter date is the same, the component lookup table produced by mkcomptab contains both the stsdas and fits format table. To prevent this from happening, I have modified the query to only include files with a single user specified extension. By default this extension in .tab, but the user can produce a table of fits files by including the option ext=fits on the command line. The default extension can be set to .fits at a later date when fits becomes the default Synphot file format. (This will require a recompilation and redelivery.) The modified files are mkcomptab.h, mkcomptab.c, and comptabqry.c.

• Mkthrucat.pl utility script mdified

  The thrucat.dat file in the data subdirectory of CDBS associates a directory with a synphot throughput filename pattern. This file is used by mkcomptab because the CDBS database does not include the directory a synphot throughput file is located in. The script mkthrucat.pl builds this file from the contents of the throughput directories. The script has been modified so that it produces entries for files with both the .tab and .fits extensions.

19 November release

• Fixed date conversion in get_sybase_date

  The code in get_sybase_date which converts from 12 hour to 24 hour time was incorrect, because an if test was backwards. I have revised the code, which is in cdbslib/date.c

• Added check to configure script

  When make install was run twice in a row, it produced the warning message "Can't stat directory" for directories that had already been deleted. I added a -e check to configure.pl, the script called by make install, to stop it from trying to delete already deleted directories again.

• Fits_reffile prototype removed

  I removed the prototype of fits_reffile from reffile.c, as there was no function by that name. The prototype generated a warning message from gcc.

• Added filename function to certify

  I added the filename function to certify. It checks for the presence of characters that are illegal in filenames. It allows the presence of the string (initial) as a special case. To use it, add &FILENAME to the template file. I also wrote a utility script, setvalue.pl that allows you to set the value field of a number of keywords in a number of template files at the same time.

• Changed keyword change code to lowercase file names

  Getref was converting all keyword change values to upper case. I revised keychanges.c so that it calls is_reffile() to check if the keyword value is a reference file name and if so not convert the value to upper case. Is_reffile(), in isreffile.c, was also modified to recognize files with the wav extension as reference files.

• Upreffile changed to handle wavecal file names

  Upreffile contains the function is_reffile(), which determines if a value in the delta file is a reference file name. If upreffile determines a value is a reference file name, it adds the appropriate logical directory name before writing it to the header. Since wavecal file names look like reference file names, but should not have a directory added to the name, I modified isreffile.x to say that files with the extension "wav" are not reference file names.

26 August release

• Fixed calling sequence for c_iraferr

  Several calls to c_iraferr in reffile.c incorrectly had an argument. (The function takes no argument.) This is essentially harmless, but went undetected when the software was last released, as the function prototype had no arguments. (No argument means that the calling sequence is unspecified.) Michelle changed the calling sequence to have a single void argument, indicating that the function takes no arguments. This flushed out the errors in reffile.c, which are now fixed.

18 August release

• Server variable renamed

  I added the environment variable and C preprocessor symbol CDBS_SERVER_2. I added it because both my software, which only reads from the database, and the other CDBS software, which also writes to the database, were using the same symbol. All database reads should come from ZEPPO, while database writes should be done on CATLOG. Thus two different symbols should be used for the different software and since it was easier to change my software, I made the change. My software now only uses CDBS_SERVER_2. The files affected were include/dbquery.h, src/cdbslib/initquery.c, util/getref.cgi, util/defines.sh and util/defines.csh.

• Increased filename column length in mkcomptab

  The filename column was 40 characters wide in mkcomptab. This was not large enough to hold the filename when it was converted to fits. So I increased it to 50 characters, the same as the width of the field in the synphot_file database table.

• Modified write_sdastab for fits files

  To conform to CDBS conventions, fits tables should put their header keywords in the primary header. But the tables interface places them in the same extension with the table. I modified the code to check to see if the table is a fits file and if so, to close the table and reopen it using the primary header. To check the filename extension, I pulled the function fits_reffile out of reffile.c and converted it to a stand alone function, is_fitsfile.

• Yanked vms code from wild_expand

  Wild_expand does filename globbing. I yanked the code that supports vms, as it obscured the rest of the code and cdbs is never going to run on vms anyway. The changes are all in cdbslib/wildexpand.c

• Uniqname modified to support synphot fits files

  In order to support synphot fits files, I modified the code that renames synphot files. The only change for throughput files is that it ignores the extension when searching for throughput files with the same name. This ensures that new synphot files will be installed with a one higher version number if the previous file is an sdas table. The graph and component table names now resemble those for the new instruments (uniqroot_tmc.fits or uniqroot_tmg.fits).

Upref bug fixes

• Upreffile correctly tests to see if keywords should be deleted Before applying changes in the delta file, upreffile checks the current keyword value in the header to see if the delete is necessary. The check was incorrect when the operation was delete, resulting in some header keywords not being deleted. The new check does the delete if the header keyword is found and does not if it is not. The code was also modified to print a more descriptive message when querying the user to delete a keyword. All changes were in the file upheader.x
• Upreffile supports all data types

  Formerly upreffile wrote all keywords as text. This worked for reference files and switches, because they are all text strings. However, keyword changes can also be numbers or boolean values. To support these additional data types, I added a new procedure, datatype.x, that determines the data type of the value extracted from the delta file. This information is used by a revised version upr_putkey in upropen.x, which calls the appropriate header update procedure based on the file type and data type.

15 April 1999 release

• I modified getref so that switch keywords with a value of "skipped" are converted to "perform".
• I fixed a problem rdlines.c had in checking for buffer overflow. Formerly it called fgets and check for a trailing newline in the buffer. This test did not work when a text file was missing the trailing newline. I modified the test to also compare the length of the string to the size of the buffer.
• I changed upreffile so that it can update header keywords in tables as well as in images. This change was made to support updating table extensions in fits files. The new functions in upropen.x encapsulate file access to tables and image and present a format independent interface to the rest of the code. The functions in upreffile.x and upheader.x have been modified to call the procedures in upropen.x
• I replaced patmatch.c with a newer version based on an article by Kernigan and Pike in the April 1999 article of Dr. Dobb's Journal. Although the old version worked for the cases cdbs currently uses regular expressions for, I knew it was incorrect and would fail for some other cases. The new version should work for all regular expressions supported by patmatch.
• I renamed the database column okr_subset to okr_extname in keychanges.c.

14 Jan 99 release

• I added support for the ACS to the cdbs tasks.
• Getref has been modified so that if the recommended switch value is blank, nothing is written to the delta file. This is a slightly kludgey fix for a problem with stis wavecal switches.
• I moved the test versions of the cgi scripts on the ra server from cgi-bin to cgi-bin/bps.
• I wrote the script rungetref.sh to allow getref and upreffile to be run from the Unix command line.
• I updated the configuration script configure.pl to add a section to modify scripts (shell and Perl). It now configures rungetref.sh and loopfits.pl. In order to support configuring scripts, I added two new variables to the configuration files: SDAS_BIN and TABLES_BIN, the directories containing the sdas and tables executables.

14 Dec 98 release

This release addes a new column to the output of getref. The column is named source and contains an abbreviated name indicating the database table source of the keyword change. The latest version of upreffile can use this field to support partial updates from the delta file. Currently, the source column can contain three values. These are: obc, which is the source of the CALIBRAT keyword, okr, which is the source of changes to miscellaneous keywords, and ref, which is the source of calibration switches and reference file names.

Upreffile has been updated to use the information in the source column to do selective updates. Upreffile will only update a header keyword if the value in the source column matches one of the values in the source parameter. The source parameter is a comma separated list of sources to be processed by this task. The default value of this parameter includes all possible values of the source field.

The update to getref and upreffile have been designed so that the new upreffile can be used with the output of the new or old getref. Also, the old upreffile can be used with the output of the new or old getref. However, only the new upreffile and new getref allow using subsets of reference files. In all other cases, the entire file is used.

Configuration files have been added in the util directory for whaler and artichoke. Getref now compiles and runs successfully under Digital Unix, but the configuration file for whaler and artichoke will need to be checked, particulary the value of CDBSLIB, the top of the directory tree for CDBS reference files.

7 Dec 98 release

There are two significant changes in this release. The first set of fixes allow getref to work in an environment where some of the hst instruments support the otfc and others do not. The second set of changes is a new install script (util/configure.pl) and changes to the Makefiles to call it. This release also contains fixes for warning messages encountered compiling on Marvel and Whaler. The remaining problem with this version is that it does not link under Digital Unix, because of a problem with that linker.

To go along with the updated version of getref, upreffile has also been updated so that it accepts wild cards in the FILENAME column. If this column contains a wild card, all files that match the pattern will be updated by upreffile. This change was made to support hsp observations, as the extension for these files contains a wild card.

To install this version of cdbs, fetch the tar file from anonymous ftp, read the tar file, cd to src, type "make install" and then "make". The install script uses a configuration file named <machine>.cfg in the util directory, where <machine> is the name of the computer the installation is being done from. Before installing on a computer for the first time, review the existing configuration file or create a new one. The instructions in doc/install.tex explain the contents of the configuration file.

The configuration file deletes source code not used on a system, sets the value of CDBS_TOP in the shell scripts, updates the include file system.h, installs the cgi scripts, and creates the top level make file. It does not set the values of the other environment variables in defines.sh and defines.csh, as altering the master copy of these variables is a practice I would like to discourage.

23 Nov 98 release

This is the first official release of cdbs incorporating the changes required for the otfc. It incorporates an updated version of getref as well as the release of a new version of upref which accepts the new output of getref. The major change to getref is that its output file contains additional rows and columns. The new rows contain the calibration switches and other keyword changes. The other keyword changes will not be available, however, until the corresponding database table is populated. The new columns contain the fully qualified name of the image to be updated, the type of change (insert, update, or delete), and whether the keyword is a change to the original value (yes or no).

Upref has been modified to accept the output of getref. The most important change to upref are the changes to accept the new output and the deletion of the now superfluous task parameters template and addkey.

18 Sep 98 release

This version fixes problems that were encountered when building the software under Digital Unix. The file cdbslib/buildqry.c was modified to fix a compilation problem. The typedef va_list is defined as a structure under Digital's C compiler and the code was doing comparisons on variables of this type. The code was modified to eliminate the need for this comparison. The suffix rule .c.a was removed from the make files becuase it was not needed. The make files were also updated to link with ostdb version 1.8.

11 Aug 98 release

This version modifies getref to track more changes in the otfc database tables made by Lisa Gardner. The modified files are setcalibrate.c, getkeywords.c, whichfiles.c, and keychanges.c.

6 Aug 98 release

The only change in this release is to revise getref/keychanges.c to replace the database column name oke_keyword_name with oke_keyword. The change was made necessary because of a change to the database.

30 Jul 98 release

This release contains a modified version of getref intended for use in the otfc pipeline. It should not be installed on the hydra cluster until database changes necessary to support it have been made in the operational DADS database.

• Getref modifications

  The otfc database tables were modified, so getref was also modified to track the changes. The functions file_keywords and switch_keywords were merged into a new function, get_keywords. This new function searches the new database table otfc_cal_fields. The function which_files was also modified to call the new otfc_keyword_extensions table, which replaces otfc_calibrat_ext. Minor changes were made to the calling sequence of both modified functions. The global variable instrument was renamed inst_cdbs and inst_abbrev was renamed to inst_dads to better document their information: a list of instrument names as used in the cdbs and dads databases, respectively.

23 Jul 98 release

This release contains a modified version of getref intended for use in the otfc pipeline. It should not be installed on the hydra cluster until database changes necessary to support it have been made in the operational DADS database. Significant changes are:

• Getref rewrite

  Getref has been significantly modified to support the otfc pipeline. The output now includes information about the recommended calibration switches and other significant keyword changes in addition to the recommended calibration files previously reported. The output file now also contains the name of the image to be changed, including extension were necessary, and whether the change is an insert, update, or delete. The function getref_all has been entirely rewritten and a number of new functions have been added to perform the additional database queries required. The html output now contains any warning and informational messages generated during execution of the task.

• Loadfile library changes

  Seeveral new functions have been added. Sort_table will sort a table on a user specified column. The functions streq, strgt, and strlt can be used to compare two strings.

• Cdbslib library changes

  The function cgi_error now saves warning and informational messages in a buffer, so that they can be displayed in the html ouput The function(enquote) now checks to see if its input string contains quote marks and if so, surrounds the string with quotes of the opposite kind. If it contains both kinds of quotes, the quotes in the string are escaped with backslashes. Cdbs_error now sends its output to stderr rather than stdout.

7 Jul 98 release

• Case sensitivity bug in uniqname fixed

  The code in cdbslib/getirafdir.c constructs an iraf directory name that a reference file will reside in. It was failing when uniqname was asked to rename a throughput file. The throughput file had the value of the INSTRUME keyword in upper case. The upper case name was used to build the directory name, with predictably bad consequences. Adding a strlcase function call to convert the value of INSTRUME to lower case fixed the problem.

• Makefiles revised

  Macro definitions common to all the makefiles were placed in a new include file, Makefile.inc, in the src directory. Also, the list of Sun libraries was modified to be compiler independendent

1 April 98 release

• Certify bugs fixed

  A number of errors with certify were reported. The problem was traced to strsplit, a function widely used in all the cdbs code. This function splits a string into an array of strings on a delimeter specified by the user. The function was failing to terminate the last string in the array when there were more strings than would fit into the array. This lead to trailing characters such as blanks or newlines remaining on the last string, which in turn caused various errors when these string failed to match values they were supposed to. I modified the code to correctly terminate the last string in the array.

  In addition, I modified the rdlines function, which is used by all the code that reads ascii files except for load files, to trim off the trailing newline returned by fgets. Even though the revised strsplit catches existing errors, there may be cases where strsplit is not called and I can't think of any cases where the trailing newline is desired.

  This fix supersedes the fix to certify made in the March 26th release, so that fix was removed.

  Fixing strsplit caused query to break when passed an expression that included an equals sign, as the code which parses the task parameters in put_option called strsplit using an equals sign as a delimeter. I modified the code to no longer use strsplit.

• Getref link generation code fixed

  When getref produces html output, it writes the names of any reference files inside of links. When these links are followed, the produce getref detail output for that reference files. Getref was not generating links for fits reference files, because the code in is_reffile did not recognize them as valid file names. I have modified is_reffile to recognize fits file names.

  The results returned by the getref summary query sometimes contain non-filenames in columns containing file names. For example, "fill_me". I have added a call to is_reffile to check for these values so that they are no inappropriately surrounded by a link.

• Mkload bug fix

  Mkload was taking an error exit because the sql generated by the task contained a question mark where the observation mode clause should be. I traced the problem to mkloadcompfile.c, where the observation mode clause was not being initialized when the reference file had no observation mode fields. Thus it was possible for the unitialized clause to contain garbage characters, which it did in this particular case. A simple modification of the code makes sure the clause is always set to some valid value.

• Sybase date conversion problem fixed

  The format of dates returned by Sybase seems to have changed somewhat, which caused the code in date.c to indicate the date was invalid. I have modified the code to handle the case when there is no separation between the time and the meridian (am or pm) and thousandths of a second are included in the time.

30 March release

• Getref returns wrong files for phottab

  Because an array in global.h was missing an entry for the nicmos photometry table, it was not returning the correct list of reference files. I updated global.h with the correct list of files.

• Detail query did not work for new instruments

  The detail query was failing for the new instruments because getref could not retrieve the instrument code from fits file names. I modified geticode.c in getref so that it correctly retrieves the instrument code from fits files.

• Blanks in list of column names confuse query

  Query would not retrieve results from columns if their names were preceded by a blank. I added code to readcollist.c in query to delete superfluous blanks.

• Breakfname enhanced

  I pulled an improvement that I made to breakfname for the stis etc back into cdbs. Breakfname no longer writes a portion of a filename if the corresponding variable is set to NULL. This save a small amount of space and code, since the code no longer needs to, say, fetch the directory name and rootname when only the extension is needed.

26 March release

• Certify rule parsing bug

  The function parse_rule in certify was leaving on the trailing newline on the rule value. For the most part, this was innocuous, but it caused the code which parsed rule functions to fail because the trailing newline caused the name not to match the list of recognized rule function names. I added a call to strclean to delete superflous whitespace, including newlines.

20 March 98 release

• Source moved on ra

  I moved the directory tree for the cdbs code on ra from /data/valis1/bsimon/iraf/newcdbs to /data/valis1/bsimon/newcdbs. This required modifying the value of CDBSDIR in include/system.h and CDBSTOP in util/defines.sh and util/defines.csh.

• Getref bug in qry_obsmode

  The line which made an upper case, quoted version of the instrument name for the sql query mistakenly left off the array subscript. That is, it should have been instrument[code] but was mistakenly instrument. Since this is a serious bug, I put together this new release.

20 February 98 release

• The maximum number of task parameters was increased to 2000.
• The maximum length of cgi input was increased to 4000 characters.
• Getref has been modified so that the input area on the form for observation set names is a scrolling region. Observation set names can now be separated by white space in addition to commas.
• Several new reference files were added to the output. The wfpc2 output now contains the name of the graph and component tables and the stis output now contains the pct table.
• I modified loopfits so that files whose extension starts in c and end in d are not taken to be data files.
• I changed enquote so it does not convert strings to upper case and changed strucase and strlcase so they return the address of the string passed them.
• The functions in the file query.c were placed in the directories of the tasks that called them. They were also rewritten to call build_qry.
• I modified init_query.c so that zero length environment variables are treated as if they were undefined.
• I wote a new version of query that works both in command line mode and as a cgi script.
• The documentation is now also available in iraf help format.

3 December 97 release

• Bug in getref cgi script

  The getref cgi script was not generating text output when the radio button to select text output was pushed. I traced the problem to my failure to terminate the cgi buffer with an EOS character after reading it. This problem, in forminput.c, has now been fixed.

• Bug in expload

  Expload was getting a segmentation violation when its rules updated more than one column at a time. The problem was due to a mistake in building the action table in yyparse.c. The code which merged table rows was off by one in the row number. The bug has now been fixed.

21 November 97 release

• CDBS linked with OSTDB 1.7

  The CDBS tasks have been relinked with the latest version of the ostdb library, OSTDB 1.7. This required modifying the makefiles for the tasks that link with the database libraries.

17 November 97 release

• Datafile checking bug fixed

  The cdbs tasks skip over any image data files that may be present in the argument list. The function which skips over data files, isdatafile.c, was performing the check incorrectly. This has been fixed.

  dir(Getref now takes upper case names)

  Getref no longer chokes when the observation set name it is passed is in upper case. It now converts the name to lower case before using it.

7 November 97 release

• Old CDBS Tasks Moved

  Although the old cdbsutil package was deleted, several tasks within the package were still useful, so they were moved into other packages. Rdsiaf, which read the PDB Science Instrument Aperture File (SIAF) was moved into the tools package. Upreffile, which takes the output of getref and uses it to update an image header, was moved into the headers package. The uniq tasks in the tools package were removed, as they duplicate the functionality of uniqname in the new cdbs package.

• New CDBS Package

  The old cdbsutil package under stlocal has been remove and replaced by a new package where the new versions of the cdbs tasks are defined as foreign tasks.

• Makefiles

  I eliminated the non-standard makefiles. There now is only a single set of makefiles for the cdbs tasks, named Makefile, which should compile successfully on all Solaris 2.* machines with iraf and stdb.

• Mkcomptab

  A new optional parameter, date was added to the task. If date is supplied on the command line, only thoughput files whose useafter date is less than date will be included in the component lookup table. The useafter date stored in the table header is the last useafter date of any throughput table in the lookup table regardless of whetehr date is passed on the command line or not.

• Getref

  A new task named getref has been added to CDBS. This task retrieves the names of the best reference files to use for an observation and replaces the old CDBS task of the same name. Getref can be run from the command line or as a cgi script. For further information, see the documentation.

17 July 97 release

• Makefiles

  Compiling newcdbs under irafx requires special handling because the standard version of irafx does not include fitsio. So the makefiles must be modified to link with a non-standard library. I have created a modified set of makefiles, named makefile.hydra, that link with this non-standard library. Use of these makefiles is temporary, until a more up to date version of irafx is installed on hydra.

• Expload

  At the request of Colin Cox, I have modified expload so that it now looks for the rules file in the data directory. (This is the same directory which contains the template files, /data/cdbs1/tools/data/ on hydra.) The precise description of how expload searches for the rules file is: If the rules file name contains a directory, it looks for it in the specified location and nowhere else. If the rules file name does not contain a directory, it looks for it first in the current directory and if it does not find it there, it looks for it in the data directory. The major change to expload was to write a new function, openrules.c, which is called from parserules.c.

• Mkload

  Part of what mkload does is choose a default comparsion file for the reference file it is creating the load file for. It does this by retrieving the file from the database with the highest useafter date and same observation mode as the reference file. However, the reference file may include wildcards in its observation mode. These will not be matched in the database, since wildcards are expanded before the information is stored in the database. Mkload has been modified to exclude any observation mode field when matching the database if the field in the reference file has a wildcard in it. The result may include more than one reference file, in which case the comparison files and their useafter dates will be written as a comma separated list in the loadfile.

20 June 97 release

• Iraf

  Since a new version of iraf has been received (irafx), the cdbs tasks are now linked with its libraries rather than the older (iraft) libraries.

• Include files

  The header file which sets the location of the files that the cdbs tasks use, system.h, has been rearranged so that all the macros that need to be modified are placed at the beginning of the file and are controlled by an #ifdef.

• Cdbslib

  New functions have been added to parse the command line for optional arguments. Optional arguments follow the convention used by iraf: they are a name=value pair. Optional arguments with boolean values can also be passed as name+ or name-, which are equivalent to name=yes and name=no. Optional arguments are removed from the command line, parsed, and placed in an alist structure. The task then can search the alist by option name to retrieve the value. The new code to implement optional parameters is stored in option.c. Modifications were also made to main.c to process the command line.

  As a test of the new optional argument handling, the optional argument "v" was added to all the cdbs task. If v is set to yes, either by passing v+ or v=yes on the command line, the task will print out the version date. The code which handles this is in printver.c.

• Certify

  Certify was reporting that values of INDEF read from the database were not valid numbers. This is because the code which reads elements from a table converts INDEF values to the string "INDEF", which the C code is unable to recognize as a number. I added the new numeric conversion functions cvt_intval_rule and cvt_dblval_rule to ++rule.c in certify which recognize the string "INDEF" and convert it to a magic value.

  The string "INDEF" may now also be added to template files so certify can check for INDEF values explicitly. Making this change also tightened the numeric checking when parsing a template file, so that template files that worked in the past may now get an error message.

• Mkcomptab

  The width of the SDAS table generated by this task was 25 characters, one character too short to hold the widest possible string. So I increased it to 26 characters.

  I restored the code that writes the comment from the database into the table.