STScI

HST Data Handbook for WFPC2

TOC PREV NEXT INDEX PDF

2.3 GEIS File Format


The HST-specific Generic Edited Information Set (GEIS) format1 is the standard format for reducing data from FOC, FOS, FGS, GHRS, HSP, WF/PC-1, and WFPC2. All HST images in GEIS format consist of two components: a header file and a separate binary data file, both of which should reside in the same directory. GEIS header files, whose suffixes end in "h" (e.g., w0lo0105t.c1h), consist entirely of ASCII text in fixed-length records of 80 bytes. These records contain header keywords that specify the properties of the image itself and the parameters used in executing the observation and processing the data. GEIS binary data files, whose suffixes end in "d" (e.g., w0lo0105t.c1d), contain one or more groups of binary data. Each group comprises a data array followed by an associated block of binary parameters called the Group Parameter Block (GPB). The sizes and datatypes of the data arrays and group parameters in each group of a GEIS file are identical. Figure 2.2 depicts the structure of a GEIS data file graphically.

The binary content of GEIS files is machine dependent. Copying GEIS files directly from one platform to another (e.g., from a VAX to a Sun) may result in unreadable data.

Figure 2.2: GEIS File Structure
 

2.3.1 Converting FITS to GEIS

The STScI archive stores and distributes datasets from FOC, FOS, FGS, GHRS, HSP, WF/PC-1, and WFPC2 in a special archival FITS format. We highly recommend that users convert these datasets back into their native GEIS format before working with them. Your data must be in GEIS format for you to use many of the STSDAS software tools developed specifically for analysis of these data. It is important to use the strfits task found in stsdas.fitsio or in tables.fitsio to perform the conversion from archival FITS format to the GEIS format because the data-processing pipeline employs a special convention for mapping GEIS files to FITS format. While other FITS readers may be able to read portions of the data correctly, they are unlikely to reconstruct the entire data file properly.

To recreate the original multigroup GEIS file using strfits, you must first type:
cl> set imtype=hhh

 

This command tells IRAF to write output files in GEIS format. You then need to set the strfits parameters xdimtogf and oldirafname both to "yes". For example, after you have set imtype = hhh, you can convert the FITS file *_hhf.fits into the GEIS format files *.hhh and *.hhd by typing:
cl> strfits *_hhf.fits "" xdim=yes oldiraf=yes

 

2.3.2 GEIS Data Groups

One of the original advantages of GEIS format noted in Section 2.1 was that it could accommodate multiple images within a single file. This feature is useful because a single HST observation often produces multiple images or spectra. For example, a single WF/PC-1 or WFPC2 exposure generates four simultaneous images, one for each CCD chip. Likewise, the FOS and GHRS obtain data in a time-resolved fashion so that a single FOS or GHRS dataset comprises many spectra-one corresponding to each readout. The data corresponding to each sub-image (for the WF/PC-1 or WFPC2) or each sub-integration (for the FOS or GHRS) are stored sequentially in the groups of a single GEIS binary data file. The header file corresponding to this data file contains the information that applies to the observation as a whole (i.e., to all the groups in the image), and the group-specific keyword information is stored in the group parameter block of each data group in the binary data file.

The number of groups produced by a given observation depends upon the instrument configuration, the observing mode, and the observing parameters. Table 2.2 lists the contents and the number of groups in the final calibrated image for the most commonly-used modes of each instrument which uses the GEIS data format.

Table 2.2: Groups in Calibrated Images, by Instrument and Mode
Instrument Mode Number of Groups Description
FGS All 7 FGS data are not reduced with IRAF and STSDAS. Therefore, FGS groups have different meaning than for the other instruments.
FOC All 1 All FOC images have only a single group.
FOS ACCUM n Group n contains accumulated counts from groups (subintegrations) 1, 2, ... n. The last group is the full exposure.

RAPID n Each group is an independent subintegration with exposure time given by group parameter EXPOSURE.
HSP All 1 HSP datasets always have only a single group that represents either digital star (.d0h, .c0h), digital sky (.d1h, .c1h), analog star (.d2h, .c2h), or analog sky (.d3h, .c3h).
GHRS ACCUM n Each group is an independent subintegration with exposure time given by group parameter EXPOSURE. If FP-SPLIT mode was used, the groups will be shifted in wavelength space. The independent subintegrations should be coadded prior to analysis.

RAPID n Each group is a separate subintegration with exposure time given by group parameter EXPOSURE.
WF/PC-1 WF 4 Group n represents CCD chip n, e.g., group 1 is chip 1 (unless not all chips were used). Group parameter DETECTOR always gives chip used.

PC 4 Group n is chip n + 4, e.g., group 1 is chip 5. If not all chips were used, see the DETECTOR parameter which always gives the chip used.
WFPC2 All 4 Planetary chip is group 1, detector 1. Wide Field chips are groups 2-4 for detectors 2-4. If not all chips were used, see the DETECTOR keyword.

 

2.3.3 Working with GEIS Files

This section briefly explains how to work with information in GEIS header and data files.

GEIS Headers

Header keyword information relevant to each group of a GEIS file resides in two places, the header file itself and the parameter block associated with the group. Because GEIS header files are composed solely of ASCII text, they are easy to print using standard Unix or VMS text-handling facilities. However, the group parameters are stored in the binary data file. To access them you need to use a task such as imheader, as shown in sectionPrinting Header Information.

You can use the IRAF hedit task to edit the keywords in GEIS headers. While it is possible to edit GEIS header files using standard Unix and VMS text editors, you must maintain their standard 80-character line length. The hedit task automatically preserves this line length. If you need to add or delete group parameters, you can use the STSDAS groupmod task in the stsdas.hst_calib.ctools package. The STSDAS chcalpar task, described in more detail in the Calibration chapters for each instrument's data handbook, is useful for updating header keywords containing calibration switches and calibration reference files.

Always edit headers using tasks like hedit, eheader, and chcalpar. Editing headers with a standard text editor may corrupt the files by creating incorrect line lengths.

GEIS Data Files

Numerous IRAF/STSDAS tasks exist for working with GEIS images (see chapter 3 of the HST Introduction). Most of these tasks operate on only one image at a time, so you usually need to specify which group of a GEIS file is to be processed. If you do not specify a group, your task will choose the first group by default.

   

Specifying a Group

To specify a particular group in a GEIS file, append the desired group number in square brackets to the file name (e.g., z2bd010ft.d0h[10]). For example, to apply the imarith task to group 10 of a GEIS image, type the following (always refer to a GEIS file by its header file name, i.e. *.??h, even though mathematically you are operating on the data portion):
cl> imarith indata.hhh[10] + 77.0 outdata.hhh

 

This command will add 77.0 to the data in group 10 of the file indata.hhh, and will write the output to a new single-group file called outdata.hhh. Any operation performed on a single group of a multigroup GEIS file results in an output file containing a single group.

   

Specifying an Image Section

If you wish to process only a portion of an image, you can specify the image section after the group specification in the following manner:
cl> imarith indata.hhh[2][100:199,200:399] * 32.0 outdata.hhh

 

This command extracts a 100 by 200 pixel subsection of the image in the second group of the file indata.hhh, multiplies this data by a factor of 32.0, and stores the result in a new output file, outdata.hhh, which is a 100 by 200 pixel single group GEIS file.

   

Printing Header Information

As discussed in the previous section, the task imheader extracts and prints information about the GEIS image. This task reports the image name, dimensions (including the number of groups), pixel type, and title of the image when it is run in default mode. For example:
cl> imhead indata.hhh
    indata.hhh[1/64][500][real]: INDATA[1/64]

 

The output line indicates that indata.hhh is a multigroup GEIS file which contains 64 groups of images, each consisting of a spectral array 500 pixels in length. The data type of the values is real (floating point). Note that since no group designation was provided, the task defaulted to the first group. To reveal more information regarding group 10, you can type:
cl> imhead indata.hhh[10] long+ | page

 

which will generate a long listing of both the ASCII header parameters in the *.hhh file and the specific group parameters for group 10 from the *.hhd file.

   

Other Group-Related Tasks

Currently, IRAF or STSDAS tasks cannot process all the groups in an input image and write the results to corresponding groups in an output image. However, there are several STSDAS tasks, particularly in the toolbox.imgtools and hst_calib.ctools packages, that simplify working with group format data. Please refer to chapter 3 and the STSDAS User's Guide for more details about working with GEIS images.

2.3.4 The "waiver" FITS format

Although "waiver" is not quite the accurate or good word for the intended purpose, for historic reasons it has stuck and will be reluctantly adopted. However, in the past, a grammatically incorrect word "waivered" had been used.

The "waiver" FITS format was developed when the HST archive needed a format to store and distribute the data products in a machine-independent medium for the community, at a time before FITS image extension was standardized. As a result, the "waiver" FITS format was adopted as a compromise.

Since, at the time, FITS could only have a single image while the HST data (in GEIS format) may have several images as multiple groups in one file, the idea is to stack the images of different groups together as a new dimension in the FITS image. As for group parameters, they are put in an ASCII table and the table becomes the first (and only) extension of the FITS file.

For example, the WFPC2 pipeline generates the science data as a GEIS file of 4 groups, each is an 800x800 image corresponding to one of the 4 detectors. When this GEIS file is converted to the "waiver" FITS file, the FITS file has an image of 800x800x4 (a three-dimensional image!) at its primary HDU. Similarly, an FOS GEIS file may have 40 groups, each group is a 1-D image (spectrum) of the size 2064. The waiver FITS file then will have one 2-D image of the size 2064x40, at its primary HDU. In the case of WFPC2, the first extension of the waiver FITS file will be an ASCII table containing 4 rows; each row corresponds to a group. The value of each group parameter is under a column named after the group parameter, i. e. the value of the group parameter CRVAL1 of the 2nd group will be at the 2nd row, under the column named "CRVAL1". In other words, the ASCII table has as many rows as there are groups in the original GEIS file, and as many columns as group parameters.

Although, in theory, certain IRAF/STSDAS tasks can directly access the data in the "waiver" FITS file, e.g. to display the 2nd "group" of a WFPC2 image:
st.> display u67m0206r_c0f.fits[0][*,*,2]

 

will work, while most tasks, especially those specific to HST instruments, can not. It is therefore HIGHLY recommended that all waiver FITS files are converted back to the GEIS format, by using the task strfits, before further processing and analysis with IRAF/STSDAS tasks.

1 GEIS files are also commonly referred to as STSDAS images.


Space Telescope Science Institute
http://www.stsci.edu
Voice: (410) 338-1082
help@stsci.edu
TOC PREV NEXT INDEX PDF