One of the fundamental IR data reduction steps in calwf3 is the process of "up-the-ramp" fitting, through which a final value is determined for each image pixel by performing a linear fit to the accumulating signal in the stack of non-destructive reads in an IR MultiAccum exposure. During this CRCORR step, data from individual reads can be rejected from the fit based on their associated Data Quality (DQ) flags. The choice of which DQ flag values to consider as bad and therefore reject from the fit is based on the setting of "BADINPDQ" in the Cosmic Ray Rejection parameters reference table (CRREJTAB). Until recently this value, as used in the STScI pipeline processing, had been set to a combination of DQ flags 4 (bad/dead pixel), 8 (deviant zero-read value), 16 (hot pixel), 32 (photometrically unstable), and 256 (saturated). Any pixel that is dead, deviant in the zero read, hot, or unstable will naturally have that condition in all readouts of an exposure and therefore all of the samples would be rejected from the up-the-ramp fitting process. In cases where all samples for a pixel are rejected, the calwf3 CRCORR routine sets the output value in the _flt.fits SCI array to zero. Thus leaving a large number of "holes" in the final output image.
A new version of the IR CRREJTAB went into use in the STScI production pipeline on June 11, 2010, in which the "BADINPDQ" DQ mask value has been reset to include only the DQ value 256. This has the effect of causing the calwf3 CRCORR routine to no longer reject all the samples for dead, hot, and unstable pixels, but rather it makes a best-effort attempt to fit the data for those pixels and record the result in the output _flt.fits SCI image. These populations of pixels will therefore no longer be zeroed out in the _flt.fits file. The DQ values associated with these pixels are recorded in the DQ array of the _flt.fits file.
By no longer blanking out these pixels, users are given the freedom to decide on their own whether or not to use the values in subsequent processing and analysis. It is therefore extremely important to make use of the DQ array values in the _flt.fits files when, for example, performing photometry or using these files as input to other processing such as MultiDrizzle. To be most conservative, one could choose to reject any pixel that has a non-zero DQ value. Alternatively, one could choose, for example, to reject only hot pixels (DQ=16), but use those flagged with any other DQ value.If you would like to use the new IR CRREJTAB to reprocess your own existing datasets, you can download the file u6a1748ri_crr.fits from the HST archive.