[Top] [Prev] [Next] [Bottom]

45.6 Determining the "Best" Reference Files

We refer to a "best" reference file for a given dataset. This section explains what "best" means and how the best files can be determined for WF/PC-1 data, using both the WWW memos and StarView.

Calibration reference files are specific to each instrument configuration and observing mode; in addition, monitoring has shown that some instrumental properties change with time. Therefore, calibration reference files can be time-tagged, indicating that they should be used with data taken within a specific range of dates. The best set of reference files for calibrating the observation (according to the pipeline) will therefore depend on the instrument configuration and observing mode that was used and the date when the observation was taken.

StarView, the interface to the HST catalog and Archive, can be used to determine the current best available reference files for any dataset and to retrieve those files from the HST Archive (Chapter 1 describes how to use StarView to access the HST catalog and archive). The Calibration Reference screens (one per instrument) in StarView can be used to obtain a listing of the current best reference files for any dataset and a listing of the files that were originally used to calibrate the dataset by the RSDP pipeline. Figure 45.4 shows StarView being used to determine the best and used calibration reference files for one particular WF/PC-1 observation. Either the used or the recommended (or both) reference files can be marked for retrieval and fetched from the Archive using FTP, as shown in the StarView tutorial in Chapter 1.

45.6.1 When is "Best" Not the Best?

The current limitation of StarView, and the reference files chosen automatically by the pipeline, is that only one "best" reference file can be specified. Frequently, there are several possible reference files that are appropriate and the files used may not be the optimal calibration files for some purposes, depending upon the scientific objectives.

For this reason, we highly recommend checking the Reference File and other memos on WWW, at:

http://www.stsci.edu/ftp/instrument_news/WFPC/
wfpc1_memos.html
This memo suggest possible alternate reference files that can be used for recalibration. These memos summarize the file names and histories of available reference files, as well provide more detailed information about the nature and quality of each reference file. For high accuracy in the science analysis, it is best to try using several different reference files to calibrate the data, to see which ones give the best results and to empirically determine the degree of uncertainty in the data attributable to the choice of reference file.

Figure 45.4: Finding Best Reference File Using -StarView

45.6.2 Choosing Among Available Flats

A history of all flatfield reference files generated and delivered to the HST 
Archive is maintained in the Reference File memo on WWW, at:

http://www.stsci.edu/ftp/instrument_news/WFPC/
wfpc1_memos.html
This file is updated as new files are added. The flatfields listed in the memo include those used in the pipeline as well as some alternate flatfields, which were never used in the pipeline but which may be of use in recalibration. The Reference File memo provides a single one-line history comment for all files; for example, some reference file changes only involved keyword changes to the header and do not affect the data. On the other hand, some flatfield reference files were mere placeholders until data could be obtained and a flatfield generated and delivered. These placeholder flatfield files contained only values of 1 in order to prevent the routine pipeline processing from crashing yet avoid applying an improper correction. The same wfpc1_memos WWW site also provides links to more detailed memos concerning specific reference files listed in the Reference File memo, such as the Closure Flatfields, the Super-Sky (MDS) Flatfields, and the Delta Flat Corrections.


See Chapter 1 for information about how to retrieve the flats (and other reference files) from the HST Archive. See Chapter 46 for more information on improving the calibration of WF/PC-1 data.

There were two main epochs of flatfields, dubbed "SV" and "non-SV" (Science Verification), and "closure" flats. The SV flats were generated from data taken between July 1991 and January 1992 and delivered to STScI in early 1992 by the WF/PC-1 Investigation Definition Team (IDT) (see OV/SV Report). All of these flats were reformatted and installed in CDBS (the Calibration Data Base); camera and filter combinations are given in Table 45.3. The STScI WF/PC-1 group generated and installed into the pipeline, flatfields for the non-SV camera and filter combinations (also referred to as "Cycle 1"); these are listed in Table 45.4.



SV -Flatfield -Camera and Filter Combinations

Camera

Filters

WF

F230W

F284W

F336W

F375N

F439W

F487N

F502N

F547M

F555W

F622W

F631N

F656N

F658N

F673N

F702W

F785LP

F889N

PC

F230W

F284W

F336W

F469W

F487N

F502N

F517N

F555W

F656N

F658N

F664N

F673N

F702W

F718M

F785LP

F889N



Non-SV Flatfield Camera and Filter -Combinations

Camera

Filters

WF

F492M

F502N

F569W

F588N

F606W

F673N

F675W

F725LP

F791W

F814W

F850LP

F875M

F889N

F1042M

F1083N

PC

F368M

F413M

F517N

F569W

F588N

F606W

F631N

F648M

F664N

F675W

F718M

F725LP

F791W

F814W

F850LP

F875M

F1042M

Earthcal observations for the final Cycle 2 and 3 closure flatfield set were taken after August 2, 1993. The closure flats were generated in the same manner as previous cycles, using the IRAF/STSDAS tasks streakflat and normclip. In addition, the STSDAS wfixup task was used to interpolate across known bad pixels (the fixed pixels are flagged in the flatfield DQF file, suffix b6h/b6d). As for all reference files, users are encouraged to refer to the HISTORY records in the file headers for details about the generation of the file. The closure flats were installed in CDBS, the Reference File Memo updated, and a detailed Closure Flats Memo posted on WWW; filter and camera combinations are given in Table 45.5.


Closure flatfields are not only useful for 1993 WF/PC-1 data, but along with an appropriate delta flat, are useful for recalibrating older data as well.



Closure (Cycle 3) Flatfield Camera and Filter Combinations

Camera

Filters

WF

F336W1

F368M2

F375Mab

F413M34

F439Wd

F487Nab

F502Nab

F517Nc

F547Mcd

F555Wc

F569Wcd

F588Nab

F606Wd

F622Wcd

F631N

F648Mc

F673Nb

F702Wd

F785LPd

F791Wc

F850LPd

F889Nab

F1042Mab

PC

F336W

F368Ma

F413Ma

F439Wa

F492Mcd

F517Nab

F547Mb

F555Wcd

F569Wc

F606Wcd

F622Wcd

F648Mc

F664Nb

F673N

F675Wcd

F702Wcd

F718Mc

F725LPc

F785LPc

F791Wc

F814Mc

F875Mab

F889Ncd

F1042M

1 Multiple versions of flatfields available: one taken without the F8ND filter and one taken with F8ND.

2 Multiple versions of flatfields available: one taken without the F122M filter and one taken with F122M.

3 Flatfield generated from data taken only with neutral density filter F122M.

4 Flatfield generated from data taken only with neutral density filter F8ND.

Particularly for science data taken in the same epoch as the streakflats (August 2, 1993 to December 1993), the closure flats will generally be the best choice for flatfielding. For data taken between the August 7, 1992 and August 2, 1993 decontaminations, the closure flatfields may be the better choice, as long as an appropriate delta flat is used as well. The older Cycle 0 and 1 flatfields may be most appropriate for data taken before February 1992, before the appearance of the "permanent measles", since the closure flats do contain the measle features. Delta flats, discussed in the next section, may be used to correct any epoch of flatfield to any science data epoch.

In addition, some of the closure flats are considered to be "high-fidelity", that is, they are flatfields where various features (such as the 30% intensity gradient due to the F122M filter) in the input earthcals have been removed; in addition, an attempt was made to remove the ~5% artifacts present in some earthcals due to short exposure reciprocity failure. Filter and camera combinations for high fidelity flatfields available from the HST Archive are given in Table 45.6. Please refer to the Closure Flat memo on WWW for names and details of specific flatfields; the memo outlines the history of all closure flatfields as well as the the procedure for creating additional high fidelity flatfields if needed:

http://www.stsci.edu/ftp/instrument_news/WFPC/wfpc1_memos.html



Closure "High Fidelity" Flatfield Camera and Filter Combinations

Camera

Filters

PC

F336W

F368M

F413M

F439W

F492M

F517N

F547M

F555W

F569W

F606W

F622W

F648M

F664N

F673N

F675W

F702W

F718M

F725LP

F785LP

F791W

F814W

F875M

45.6.3 Choosing and Generating Delta Flats

Observations taken during one epoch (time period between decontamination events) and processed with flatfield reference files created from data taken during a different epoch may require a delta flat correction to correct for QE changes that occurred during the decontaminations1. A delta flat, generated from ratios of INTFLATs, can also be used to reduce, although not eliminate, the effects of the measles contamination. In practice, since the pipeline flatfields are inverse flats (calwfp multiplies the science data by the FLATFILE specified in the .d0h header), the delta flat correction is the ratio of an INTFLAT of the same epoch as the flatfield calibration file divided by an INTFLAT taken close in time and wavelength to the science observation to be corrected. The calibrated pipeline image is multiplied by the appropriate delta flat: F = CxD where F is the final image, C is the calibrated pipeline product image (.c0h) which has had the flatfield reference file applied, and D is the delta flat.

The INTFLATs were taken by illuminating the backside of one of the two shutter blades (A or B) with internal lamps. Due to variations in the shutter blade reflectivities, all INTFLATs used in a delta flat must have been taken with the same shutter blade (given by SHUTTER keyword). Since the flatfields and measles features have been found to depend on wavelength, the delta flat (or INTFLATs) closest in filter to the observation will provide the best possible correction. Note that prior to the development of the persistent measles in February 1992, regular INTFLATs were being obtained using a relatively small number of filters; therefore, INTFLATs for the baseline epoch, and thus the resulting delta flats, are available for only a small number of filters. However, after February 1992, the delta flat proposals were greatly expanded in order to obtain INTFLATs in nearly 20 filters in each camera; these may allow the correction of the closure flatfields to data taken prior to August 1993.

Delta flats that have been generated were archived into CDBS and can be retrieved from the HST Archive; delta flat file suffixes are r8h/b8h for the data file and its data quality file, respectively. Listings of the file names can be found in the delta flat memo that is currently available through the World Wide Web at:

http://www.stsci.edu/ftp/instrument_news/WFPC/
wfpc1_memos.html
For some epochs, there are two types of delta flats available for use, average delta and individual delta flats. The average delta flats were created by averaging all the individual delta flats taken between decontaminations (dates of all decontaminations are listed in the delta flat file). These average delta flats are partially corrected for cosmic rays, and therefore, should be used whenever possible. The closure delta flats, generated from INTFLATs taken August 8,1992 through November 1993, have all been cosmic ray corrected using the STSDAS crrej task and wfixup to average across questionable pixels. Note that a subset of the closure flats already include the appropriate delta flat correction; the Closure Flat Memo flags these appropriately.


Custom delta flats can be generated from internal flats observations retrieved from the HST Archive using StarView.

One use of the delta flats is to back-correct the Closure flats for use with earlier Cycle science data. For example, to adjust a hi-fi Cycle 3 flat for use with Cycle 1 data, find 2-3 INTFLATs for a nearby filter near the time of the science observations, and find 2-3 INTFLATs taken during Cycle 3 (after Aug 2,1993). After retrieving and recalibrating these INTFLATs with the best reference files, removing cosmic rays and combining (e.g., with STSDAS crrej ), the Cycle 1 "high-fidelity" flatfield is created by simply multiplying the Cycle 3 hi-fi flatfield by the ratio of the cleaned internals (e.g., with IRAF imcalc). Since the flatfields used by calwfp are inverse flats, the ratio is Cycle 3 divided by Cycle 1, that is:

Hi-Fi (Cycle 1) = Hi-Fi (Cycle 3) * (clean Cycle 3 intflat) / (clean Cycle 1 intflat)



[Top] [Prev] [Next] [Bottom]

1 Decontamination events occurred January 1991, May 1991, July 1991, February 1992, March 1992, August 1992, and August 1993 (see Table 46.1 for details).

stevens@stsci.edu
Copyright © 1997, Association of Universities for Research in Astronomy. All rights reserved. Last updated: 01/14/98 16:04:33