HST Data Handbook for WFPC2

2.1 Data Files and Suffixes

---

HST data files are distributed by the Archive in FITS format, either via tape or electronically (see chapter 1 of the HST Introduction). For historical reasons, WFPC2 files are provided in "waivered FITS" format, which is not the same as the extension FITS format of the more recent STIS and NICMOS instruments. (See chapter 2 of the HST Introduction) The waivered FITS format WFPC2 files can be converted into the so-called GEIS format using the STSDAS task strfits. Users unfamiliar with GEIS format or the strfits task should consult chapter 2 of the HST Introduction; in addition, the WFPC2 Data Analysis Tutorial provides cookbook examples of converting and working with WFPC2 images.

A Note about HST File Formats

Data for WFPC2 and other older instruments (FOC, FOS, FGS, GHRS, HSP, WF/PC1) are generated in the so-called GEIS, or Generic Edited Information Set, format. An HST-specific format, a GEIS image consists of a header file and a binary data file; one of the primary reasons it was used for the early generations of HST instruments was that it easily accommodated multiple images, each assigned to its own group, within a single GEIS file. A WFPC2 exposure, consisting of four images, one from each camera, could be stored in a single GEIS file.

However, GEIS format is also machine-dependent and is therefore not ideal for archival storage and distribution. For this reason, a specialized FITS format was developed, referred to as "waivered" FITS. Since waivered FITS format can only be a single file, multiple group data, such as in WFPC2 exposures, must be stored as a 3-dimensional image. For example, a typical WFPC2 observation file contains four 800x800 pixel images (one for each chip); each of the four images is stored in its own group in one GEIS format file, while the waivered FITS format consists of the four images in an 800x800x4 (3-d) cube.

Observers who intend to use IRAF and STSDAS tasks for their WFPC2 data analysis should first convert from FITS format images to GEIS format. Many IRAF and STSDAS tasks do not work properly on the waivered FITS format WFPC2 files, particularly for groups 2 through 4.

Several years after the implementation of the waivered FITS format, new enhancements were developed for FITS format data, providing significant flexibility. In particular, FITS files can now contain multiple image extensions, each with its own header, size, and datatype. This type of FITS has been dubbed "extension" FITS and has now been adopted as the standard for current and future HST data (e.g., STIS, NICMOS, ACS).

The example below illustrates the use of the STSDAS catfits task, which can be used to ascertain the FITS format of a file; the WFPC2 image u5kl0101r is in waivered FITS format (note the dimensions) while the NICMOS image is in the new extension FITS format. Please refer to chapter 2 of the HST Introduction for a more extensive comparison of the various formats.

cl> catfits u5kl0101r_c0f.fits EXT# FITSNAME FILENAME EXTVE DIMENS BITPI OBJECT 0 u5kl0101r_c0f u5kl0101r_cvt.c0h 800x800x4 -32F 1 TABLE u5kl0101r_cvt.c0h.tab 49Fx4R cl> catfits n3zd0wetq_ima.fits EXT# FITSNAME FILENAME EXTVE DIMENS BITPI OBJECT 0 n3zd0wetq_ima n3zd0wetq_ima.fits 16 1 IMAGE SCI 1 256x256 -32 2 IMAGE ERR 1 256x256 -32 3 IMAGE DQ 1 256x256 16 4 IMAGE SAMP 1 16 5 IMAGE TIME 1 -32 6 IMAGE SCI 2 256x256 -32 7 IMAGE ERR 2 256x256 -32 8 IMAGE DQ 2 256x256 16 ... [lines removed to improve readability] 55 IMAGE TIME 11 -32

 

To avoid confusion for experienced WFPC2 observers, WFPC2 data continues to be delivered from the Archive in waivered FITS format. Note, however, that since observers are becoming more familiar with extension FITS format in the new instruments, adding an extension FITS option to WFPC2 Archive retrieval formats is being considered. Work is underway to upgrade IRAF and STSDAS tasks to handle WFPC2 data in both GEIS and extension FITS format (this is needed before the Archive can begin to deliver WFPC2 data in extension FITS format), though the changes will not guarantee that the tasks will work properly on waivered FITS data. This work will require the release of some new conversion tasks (fits2geis and geis2fits) and changes to the Archive to make extension FITS format an option for retrieving WFPC2 data. In any event, the waivered FITS format will not disappear, extension FITS format merely would become a new option for data retrieval and analysis. Any such future enhancements will be clearly advertised via the Newsletters and WWW pages.

GEIS Files

After conversion to GEIS format, a directory listing (type dir within IRAF) will show that the files have a nine-character rootname and a three-character suffix (also called the "extension"). By definition, all files for a single exposure will have the same rootname. The image rootnames (e.g., u2850303p)are constructed from several pieces: the first character represents the instrument used (always U for WFPC2), followed by a three character program ID (unique to each proposal), a two character observation set ID (often identical to the visit number), a two character observation number, and a single character for the data transfer path. For each instrument, the suffix uniquely identifies the file contents. The WFPC2 suffixes are listed in table 2.1.

Files whose suffix ends with the letter "h" (e.g., u2850303p.c1h) are the ASCII header files. The header files contain keywords that describe the parameters used to take the observation, the processing of the data, the properties of the image, and a description of the group keywords. Files whose suffix ends in the letter "d" (e.g., u2850303p.c1d) are binary data files; these files contain the data as well as values of the group keywords. A single GEIS image is composed of a header and data pair (e.g., the files u2850303p.c1h and u2850303p.c1d together represent a single image). Note that in FITS format, useful for network transfers, header and data are combined into a single file; the name of the FITS file becomes: rootname_suffix.fits, with the letter "f" replacing "h" or "d" in the suffix (e.g., u2850303p_c1f.fits). GEIS files may be converted back to waivered FITS format by using the STSDAS task stwfits, see chapter 3 of the HST Introduction. As noted in chapter 3 of the HST Introduction, you should set your 'tmp' directory to "./", imtype to ".hhh", and imdir to "HDR$" as shown below:

cl> set tmp = "./" cl> set imtype = "hhh" cl> set imdir = "HDR$"

 

You can also make these settings in your login.cl or loginuser.cl files. (Note that most STSDAS tasks will not work properly if you have imtype set to "imh".)

Table 2.1: WFPC2 Dataset Suffixes and File Sizes

GEIS Format Suffix	Waivered Fits1 Format Suffix	File Contents	Approximate Size
Raw Data Files
.d0h/.d0d	_d0f.fits	Raw science data	5 MB
.q0h/.q0d	_q0f.fits	Data quality for raw science data	5 MB
.x0h/.x0d	_x0f.fits	Extracted engineering data	100 kB
.q1h/.q1d	_q1f.fits	Data quality for extracted engineering data	100 kB
.shh/.shd	_shf.fits	Standard header packet containing observation parameters	30 kB
.pdq2	_pdq.fits	Text file containing jitter statistics and other data quality information	20 kB
.dgr	_dgr.fits	Text file listing the values of the group header keywords in the raw image.	5-10 kB
Calibrated Data Files
.c0h/.c0d	_c0f.fits	Calibrated science data	10 MB
.c1h/.c1d	_c1f.fits	Data quality for calibrated science data	5 MB
.c2h/.c2d	_c2f.fits	Histogram of science data pixel values	220 kB
.c3t	_c3t.fits	The output throughput table generated based upon the PHOTMODE group header keyword.	10-30 kB
.cgr	_cgr.fits	Text file listing the values of the group header keywords in the calibrated image.	5-10 kb
.trl	_trl.fits	Trailer file	10-30 kB

1 Observers using IRAF tasks for their WFPC2 data analysis should first convert their images to GEIS format as many tasks do not work properly on the waivered FITS format files from the Archive. 2 PDQ files and Observation Logs (jitter) files, see appendix C, are not automatically distributed as part of the OTFR products but must be specifically requested, see chapter 1 of the HST Introduction. The jitter files are discussed in appendix C.

 

A single WFPC2 exposure is obtained as four images, one image for each CCD chip. GEIS files use group format to keep all of the data from a given HST exposure together in a single image file, see chapter 2 of the HST Introduction. The data corresponding to each sub-image for the WFPC2 are stored sequentially in the groups of a single GEIS image. The header file for an image contains the information that applies to the observation as a whole (i.e., to all the groups in the image); this information is viewable by simply paging the header. The group-specific (that is, chip-specific) keyword information is stored in two places. The descriptions for each keyword can be found in the main header, while the keyword values are stored with the group data itself in the binary data file; group keyword values are only accessible via specialized software, such as the STSDAS tasks hedit, hselect, or imhead.

WFPC2 images are normally four-group images. The first group is used for the planetary camera and groups 2, 3, and 4 are used for the wide field cameras. If only a subset of chips are read out, only a subset of groups will be present. The keyword, DETECTOR, gives the chip used for a specific group (1 through 4 for PC1, WF2, WF3, and WF4, respectively). For example, if you have a single-group image (not uncommon before the installation of the solid state recorder) and would like to know which chip was used for the observation, check the DETECTOR keyword. The total number of groups within the image is stored in the keyword, GCOUNT.