STScI Logo

Hubble Space Telescope
The NICMOS 'Pedestal' Effect

The NICMOS "Pedestal" Effect

This memo describes our current understanding of the "pedestal" effect
in NICMOS images. Observers are encouraged to read the on-line image
anomalies page at the STScI NICMOS web site. Questions about anomalies
in general should be addressed to your contact scientist. Questions 
specific to the software may be addressed to the authors of the programs.

Work on the "pedestal" effect is ongoing, and we hope that within the next
year or so we'll have a definitive solution to this problem. At that time
we'll pull together a full Instrument Science Report (ISR) (which will be
available on the web) and circulate a notice through the ST Analysis News-
letter (STAN) which NICMOS observers get monthly via email.

I. WHAT IS IT?

A big problem in reducing NICMOS images is the removal of the "DC
pedestal", a time varying bias level which appears in all NICMOS images. 
The pedestal is also quadrant-dependent. Due to its random nature, it
cannot be removed by current pipeline processing, but must be removed at 
the data-analysis level. Quadrant-dependent means that the 4 quadrants of
a NICMOS camera will have different levels (which usually shows as a
different degree of "darkness" of each quadrant in a science image);
the levels change from exposure to exposure. 

The most obvious effect of the pedestal is to leave a residual flat-field 
signature in the science images, because the image processing is applying 
flat-fields to a component which does not follow the quantum efficiency 
variations of the pixels. The pedestal, therefore, compromises photometry,
isophotal studies, etc.

This problem was much more severe in the early months of NICMOS operation.
A change in the instrument software that took effect on Aug. 21, 1997 has
minimized (but not eliminated) this effect.

Another effect that looks like pedestal is "residual shading".
This is a bias effect; like the pedestal, it is a noiseless component, 
but, unlike the pedestal, the shading varies ACROSS each quadrant. This is 
why none of the "pedestal removal" techniques described below can detect it. 
The residual shading is due to a temperature-dependent change of the bias 
level of NICMOS; thus sun-angles, seasons, level of detector activity, SAA, 
all affect the shading level. There is also a secular component which the 
NICMOS group is now characterizing and, hopefully, at the end of the NICMOS 
lifetime we will have time-flagged calibration darks to reduce the images. 
Luckily, the residual shading is generally a small effect, and although its 
presence leaves residual flat-field fluctuations, these are much smaller 
than the fluctuations given by the pedestal.

Thus there is usually more than one thing going on in the data that needs 
fixing. In addition, recently acquired data when processed with old dark
reference files has residual shading (because the shading is changing with
time/temp). This leaves a *non-constant* and *non-linear* signal across
quadrants and through a MultiAccum sequence. You should not ignore this
and simply try to compute and subtract a constant from the end of your
exposure. Some investigators have had some success with a separate shading 
fit and removal before doing the more common pedestal fit and removal.

Hopefully we may be able to ease the situation by soon coming up with time-
dependent darks in which the shading matches the science images. Then if you 
reprocess using the newer darks, all that should be left to worry about is 
the normal pedestal.

A final complication is that much of what routines like those described below
are probably measuring is *not* bona fide pedestal, but rather a signature of 
a mismatch between the flatfielding lamp and the sky. It may be possible in 
the future to make some "super-sky" images in at least the most popular 
filters, but this has not been done yet.

II. PEDESTAL REMOVAL:

One of the best methods available for removing the pedestal
in a semi-automatic way is to use the code developed by Roeland Van
der Marel; both code and instructions are available from the Web:
http://sol.stsci.edu/~marel/software.html

Roeland's program (unpedestal) is packaged for UNIX workstations. It
takes as input IRAF format (.imh and .pix pairs) although the NICMOS
group has a cl script to translate the .fits files for use by this program,
and to translate the products back into fits. This program works iteratively,
by using unsharp masking to remove large diffuse objects from the field and 
then measuring and subtracting a constant (per quadrant) times the flat field
image until the residual is minimized. This works well on diffuse sources but
can give non-optimal results on images with lots of point sources.

Another removal code has been written by Mark Dickenson (med@stsci.edu).
Mark may be able to provide the IRAF script to run this code on request.
This program (pedsky) starts with the _raw.fits data and interfaces directly
with calnica, again iteratively solving for the sky. This program is very good
at eliminating point sources before removing the pedestal, but doesn't do
as good a job on extended sources.

A third script (which calls IDL) has been written by Eddie Bergeron. This
program runs well at STScI but is not intended for use at other institutions.
Eddie's programs (pedmed, for images with minimal thermal contribution) and 
pedtherm (for images with a thermal component) measures and subtracts the 
median of each quadrant before the flatfielding step. This program is very 
sensitive to the amount of real signal in the images, and so works best with 
sparse fields. You may want to try this program if you visit STScI to reduce 
your data.

It may be possible to remove the pedestal unambiguously as part of the calnica
processing. Development of this technique, and/or incorporation of one or more
of the above techniques into STSDAS, is a major goal of NICMOS software
development over the next year.

As you can imagine, the major complication arises when the NICMOS frames 
contain extended objects or crowded fields; in this case the identification 
and subsequent removal of the pedestal level can be tricky. Roeland's code 
has provisions for both cases (in addition to the "easy" one of sparsely 
populated fields).

Many attempts at removing pedestal are not very successful. Eddie's
programs work ok at the short wavelengths, but the thermal wavelengths are 
often more difficult because they were being affected by the thermal back-
ground, so it is harder to get the pedestal level just right.

Roeland's code works fine for sparse fields or images with low gradients but 
you have to deal with each case individually. There seems to be no optimal 
set of parameters that always do the job, at least for the case of one 
extended, dominant source that almost fills the chip.

III. OTHER WORKAROUNDS

If you have data that has contemporaneous background measurements (generally
recommended for observations at "thermal" wavelengths) the background and
pedestal can be removed almost completely in the standard background sub-
traction process. This may be because although the pedestal is time variable, 
the time constant is rather long and object/sky pairs have approximately the 
same effect.

Another way to minimize pedestal (and dark) problems is to reduce your data 
using the first or second read as the "zeroth" read. You will lose some 
integration time but this should not be a concern for longer exposures. 
Note that if you do this you will have to massage your darks in a similar 
way (i.e. shift all the multiaccum images "down" by a place or two). This 
method does not work in every case and is difficult to implement-- you will 
probably only want to try this as a last resort.

IV. BASIC STEPS for the cleaning and stacking of the NICMOS images using
Roeland's code.

The application of the code is not necessarily straightforward to
observations of crowded fields; sometimes the presence of the extended
target/multiple objects confuses the algorithm which is supposed to
identify the pedestal and you end up with a final image which looks
like junk. Of course, the jumbling is not automatic and sometimes the
code works well even in the presence of such "confusing" objects. It
is worth trying the code directly on the images the first time; you may
be lucky and, if it works, will save some time. If the code doesn't
work, the best way is to create a mask to mask away the object(s).  So
the general approach looks like the following:

1. Make sure you have the *_cal.fits images produced by the pipeline; 
Roeland's software works on those (after they have been translated into 
IRAF *.imh images).
2. Create mask images for the objects in the field.
3. Feed both *_cal.imh and the flatfield images and the masks to Roeland's 
software. As you 
will see from the instructions, there are other input parameters which 
must be set up; this will be an exercise of trial/error, but one thing 
that you want to input is the correction done with the "pedestal 
multiplied by the flatfield". 
4. Once you get the outputs, there is one more step: you want to use the 
IRAF task "background" to remove the residual shading; you want to set
up the task to remove the background along columns for NIC2 (axis=2) or rows 
for NIC1 (axis=1) with a low order polynomial (a straight line is what you 
want).
5. Finally, now you can use your most loved programs for registration and 
stacking of the images. 

The final images obtained with the 5 steps above should be science-grade ones.

A few general remarks can be made:

- Often the "best" method to bring the quadrants to a common level using 
Roeland's code (i.e. the method that makes the result look nicest, without 
edges at the quadrant borders) is a simple linear interpolation over just 
a few columns/rows. 

That is to say, set MORCONT=-1, ix0=2, and ix1=3 or 4. That is especially true
if the galaxy nucleus with its steep brightness gradient sits close to a 
quadrant border. This seems a little frustrating, but we could not find any 
other set of parameters that has a similar success rate....