2.4 Data Reduction

Automated reduction systems must carry out, not only advanced scientific methods to generate useful information, but also the routine tasks an astronomer must perform to correct for imperfections in the data. Outlined below are these ritual tasks, along with the cause which forces their use.

2.4.1 Bias Frames

The read-out signals of all CCDs have a fixed direct current (DC) off-set level. A negative off-set has to be applied to stop the random fluctuations, that are inherent to the read-out process, causing a negative input to the analogue to digital converter (ADC). If plate A of a parallel plate capacitor is held positive relative to plate B, then the potential across the plates is measured to be +x volts. The read-out capacitor of a CCD however, has plate A held at ground - i.e., zero volts - to allow discharge once it has been recorded. This should always be more positive than its associated plate B that has the accumulated electron build-up on it (i.e., negative charge) however, “ground” is not always zero volts. Due to stray charges which, can be positive or negative, plate A, the ground plate, can sometimes become negative with respect to plate B, especially if the pixel being read-out has only a few electrons and hence a small charge. This would allow a negative input to the ADC, giving a negative pixel value or, depending on the ADC used, a positive value equal in size to that of its negative counter part. The fixed off-set or “bias” level applied to the read-out signal eliminates this possibility by ensuring that plate B always has a relatively large and negative charge on it compared with the ground plate (plate A). While this DC off-set has stopped one problem, unless it is corrected for the image frames will be incorrect. This bias correction is achieved through use of a bias frame, or bias-strips.

A bias frame is a zero second exposure with the CCD shutter closed, the resulting frame contains charges that are a remnant only of the bias off-set. The bias frame can then be subtracted from the object image leaving data,

(2.1)

where o_ij is a pixel in the observed image frame, b_ij is the corresponding pixel in the bias-frame and a_ij is the corresponding pixel in the resultant de-biased frame. In practice several bias frames are taken and combined using a median filtering technique (see Section 2.4.6) which reduces the read-noise element of the bias frame, but leaves any spatially coherent pixel-to-pixel bias levels untouched.

A bias frame created by an efficient CCD should show no fixed-pattern structure and be dominated by random noise. If this is the case however, then the subtraction of a bias frame can add noise to the frame and it is better to subtract a mean bias level value which can be calculated from the bias strips generated by scanning.

Under-Scanning and Over-Scanning

Under-scanning and over-scanning is the act of reading-out the CCD without clocking the data and continuing to read-out the CCD when the entire array has been read-out, respectively. This results in a border of pixels around the image frame that contains only bias levels - bias strips. These bias strips can be averaged to determine the mean bias level of the frame and it is sometimes enough to subtract this numerical value rather than a bias frame:

(2.2)

where <s> is the value calculated from the bias strips.

2.4.2 Dark Current

The atoms within the crystal lattice of a solid (the arrangement of the atoms within the material), are constantly moving, these tiny oscillations increase in amplitude with increased temperature. If the temperature is sufficient then an electron-hole pair can be generated by an electron becoming completely detached from its atom. This gives rise to noise in the pixel as the electron is indistinguishable from the incident radiation of the source being observed. The dark current is calculated using:

(2.3)

Where N₀ is the dark current in nAcm^-2 at room temperature (typically 1.8 × 10^-9 A/cm² (McLean, 1997) - for 30m pixels), d_pix is the pixel size in cm, T is the operating temperature in K, E_G is the band gap energy in eV and k is Boltzman’s constant in the form 8.62 × 10^-5 eV/K, where the energy-gap of a material varies with temperature and is given by:

(2.4)

This effect is the reason why CCDs are connected to cryogenic cooling systems whilst in operation. Whilst dark current is not a problem for short integrations, it can become a problem for long observations.

To overcome dark current “dark frames” are taken, these are long integrations taken with the shutter closed. This generates an image that after having been de-biased leaves only the signal generated by the thermal motion of the crystal lattice of the semi-conductor; the dark frame is then subtracted from the image in the same way as a bias frame. Modern CCDs have now advanced to a stage where dark currents have been reduced to value where removal via dark frames is only required for images with integration times of the order of hours. The LT CCD will operate at a temperature of 120K and will have a dark current of 2 × 10^-3 electrons pixel^-1hour^-1 (Pittock, 1998).

2.4.3 Flat Fields

Most CCDs do not have uniform response across their pixel matrix. Due to variations in semi-conductor quality, differential sensitivity of pixels, bad pixels and even dust lying on the detector or other optical surface (e.g., filters, windows), a uniformly illuminated CCD will produce a non-uniform image frame. It is a simple matter however, to correct for this non-uniformity. If a CCD is illuminated uniformly for a given time the result is called a “flat-field” - an image of the non-uniformity of the CCD. This flat-field can be divided in to the image frame pixel-by-pixel to produce a uniform and normalised image. If a_ij is a pixel in bias subtracted image frame and f_ij is the corresponding pixel in the flat-field, then the process of correcting for this non-uniformity or “flat-fielding” is described by:

(2.5)

where is the mean of the entire flat-field and normalises the resultant file. In effect the resultant file has been divided by unity and the image amplitude has been normalised. CCDs are highly wavelength dependent, with the different energy photons being absorbed at different depths in the semi-conductor. This requires a flat-field to be obtained for each pass-band or colour an observer uses. There are two types of flat-field image, dome flats and sky flats.

Dome flats are obtained by integrating on the inside of the telescope dome which has been illuminated by a bright continuum source free of emission lines. The two disadvantages to dome flats are (i) the wavelength distribution of the sky is generally not matched by that of the lamp. This effect is more important for observations made through broad band filters, than narrow band filters and can lead to fringing (see Section 2.4.4). (ii) Light reflected from the dome is incident on the telescope at a different angle to light from the sky. This difference does not affect pixel-to-pixel sensitivity variations but can affect the vignetting and the shape of images caused by dust (Davenhall et al., 1999).

Sky flats are obtained by integrating at twilight on an area of sky that is still much brighter than any of its constituent stars and tend to be favoured by astronomers as they more accurately reflect the conditions the CCD will be observing. In practice many flat-fields are taken and median filtered (see Section 2.4.6) together to give the best possible frame. Sky flats present their own problems however as the interior of the dome is illuminated by the twilight sky, thus sky flats can inherit some of the vignetting and dust effects present in dome flats.

Flat-fielding also corrects for other effects: (i) Dust particles on the CCD, these appear as small dark features with the same percentage absorption on all flat-fields. (ii) Dust on the filters. These appear as torus-shaped features, due to the fact that they are out of focus as seen by the CCD. They are the same for each exposure in a given filter and (iii) vignetting, the dimming of objects near the edge of the field of view, caused by out of focus obstructions in the light path.

2.4.4 Fringes

Interference fringes on CCD images are dependent upon the passband of observation and the thickness of the CCD substrate. Multi-path interference effects in CCDs can modulate sensitivity. With broadband filters the effect may be negligible, but with narrow-band illumination, or where the light contains a strong component at a single wavelength, as in spectroscopy, the effects can be more serious.

Low energy hydroxyl (OH) narrow emission lines are the principal cause of fringing, these lines will often fall within the bandwidths of broad band filters, occurring almost exclusively in the I ~ (7000 9000)Å & Z ~ (8000 10,000)Å bands.

Fringing occurs in thinned back-illuminated CCDs due to the phenomenon of internal reflection. The thicker the substrate of the CCD the fewer photons are able to internally reflect and cause fringe patterns. CCDs with small pixel sizes are more prone to fringing, as destructive interference that would remove the fringing is less likely.

Fringe removal is carried out by the subtraction of a “fringe-frame” in similar way that a bias-frame is used. This fringe frame is constructed from several images which, have been taken using the same telescope and telescope setup, the same filter, observing similar magnitude objects and integrated for a similar time scale - frames must all be time normalised before commencement. These images are then median filtered together as described in section 2.4.6. This process produces the fringe frame. Fringes are not however present in the featureless spectra of dome flats and as such they may not be appropriate to use when fringing, due to night sky emission lines, is present.

This frame must then be normalised to the same intensity level as the image frame and then subtracted from it. The normalisation is performed by taking intensities from a number of dark fringed areas of the image frame, the intensities of the corresponding areas on the fringe frame are then recorded and the differences calculated. The average change in intensity is then used to normalise the fringe frame before it is subtracted from the image frame. This process is highly iterative and can take several attempts before successful de-fringing is accomplished.

2.4.5 Photon Noise

One source of noise which can not be reduced is photon noise, which is due to the Poissonian nature of counting photons. The error in the signal is proportional to the square root of the signal.

2.4.6 Median Filtering

Median Filtering is a non-linear filtering technique used to remove pixel “spikes” (McLean, 1997). The technique takes pixel a_ij of every frame being used and stacks them in order of increasing intensity (I_min I_max) the middle value of the stack is then selected as the value of the frame being constructed. The resulting frame removes any abnormalities evident in only a few spurious frames. Each frame needs to be time normalised before the process begins.