Space Telescope Users Committee
Pipeline General References and Procedures

2010/12/17 10:33:01.820 US/Eastern

Getting builds tested and installed in OPUS

The instrument teams, operations and DADS can make special requests, when necessary, through the Data Management Systems (DMS) Status Pipeline meeting. The Pipeline lead for the team will notify DMS that we are going to submit a delivery. In general the Pipeline meeting remain the coordinating body for providing the HST Mission Office input, when necessary, for this purpose.

In the case where we are short of resources or we are going through very high priorities, the HST Mission Office will review the list of OPUS/CALxxx/DADS changes that are requested and will set the priorities. This has not happened lately but things could change at any time.

All the request for pipeline changes (STSDAS or OPUS) should be done via PRs. The person requesting the change should provide a detailed information of the requested change and they way the change should be tested. See the PR Template for a guideline to the kind of information that should be included in the PR.

Type of PRs that should be filled

1. Change apertures from AVAILABLE to SUPPORTED
2. Add new reference files to the system
3. Add new apertures
4. Procedures for Changes in CALSTIS

Change apertures from AVAILABLE to SUPPORTED

What are the procedures for changing an aperture from "available" to "supported" in the system?

Three PRs must be filled: againts PROPINS, PP and OPUS.
Examples of these OPRs are:
PR 47451, for PROPINST: states it needs update of the table 10.4 of the Proposal Instructions
PR 42493, for PP request updates of the PP
PR 42407 or PR 47458 for OPUS: for update of the keyword_rules table.
Note that PR 42407 include the table used by PROPINS and that information not necesarily has to go in the OPUS PR.
The new aperture might need to be added to the ETCs and pysynphot. In this case you need to file PRs against ETC and STSDAS (for pysynphot).

There could be cases, however, when the reference files will not require to be used by the pipeline, or go in StarView; in those cases you don't need to fill all the above mentioned PR but only those that cover the systems where you want to have your reference files.
If you have any doubts about which PRs are needed, please contact the INS/CDBS Group (cdb_team_at_stsci.edu).

Examples are:

CDBS PR:
PR 42445 or PR 45386 for CDBS: this states that the new files should be defined in CDBS in order for them to be ingested in the database.

OPUS PR:
The OPUS PR covers instructions to OPUS on how to use the new reference files and on adding KEYWORDS to the data; so information related to KWDS now have to be put in the OPUS PR. Since in the past these were separated, here we show them in two separate PRs; however, keep in mind that from now on all the necessary information should be contained in the OPUS PR only.

PR 42444 or PR 45388 for OPUS: states that new header keywords for listing this files have to be populated in the primary header, that a new field has to be added to the CATALOG database.

PR 42442 or PR 45387 for KWDB (this information should go in the OPUS PR): states that new header keywords for listing this files have to be added to the primary header.

StarView web PR:
PR 45389 for STARVIEW: states that the new reference files have to be added to the STIS reference file screen.


ICD-47 PR:
IDC47 The document has to be updated too.

Add new apertures

To add new apertures to the system PRs have to filed against: PROPINS, PP, TRANS, SCIOPSDB and CDBS
Examples are:
PR 41708 for PROPINS: to add the new slits to the PropInst instructions table 10.4
PR 41713 for PP: to request changes to rules of the STIS Pre-Processor checks.
PR 41717 for TRANS: changes necessary for TRANS to generate auto/GO wavecals for the new slits.
PR 41715 for SCIOPSDB: to add new slits to the Slit Wheel Parameters Table.
PR 41783 for SCIOPSDB: to add new entries to the SIAF file.
PR 41661 for CDBS: to update the reference files rules to add the new apertures.

Procedures for changes in CALSTIS

Any change to CALSTIS should be detailed in an PR filed against STSDAS.

• The testing procedures of the given OPR should include verification of the scientific quality of the testing data.
  Example: PR 55998
• The testing data should be selected from the Test Suite, if possible. If data other than that of the test suite has to be used this should be moved temporarily to the test suite directory.
  If the test suite does not have any dataset of the kind needed for the present testing (e.g. a given opt_elem or obstype) it should be added, permanently, to the test suite. The request for this should go to Steve Slowinski (slowinski) and Mike Swam (mswam). It is preferable not to add more data to the test suite unless this is a new mode or a new problem case.
• If the changes made to CALSTIS can be verified through header keywords, those header keywords should be inspected in the testing. This should not be the only test done on the data. The resulting calibrated image or spectra should also be compared with the previous version of CALSTIS to verify that there were no differences or those are consistent with the expected changes.
• All the test suite should be tested in TUFNEL platform (until we change to the new Linux servers for the pipeline). The same results should be expected for the new CALSTIS run in Solaris and Linux.