ASTR 3520: APO Echelle Data Reduction Guide

Warnings:
1. Do not copy images with 0 size to your working directory, they will cause errors.
2. If you use wildcards as specified below, be careful how you name your files.
3. The below commands are strongly order-dependent. Be VERY careful with wildcards if you mix up order or have to go back to a previous step.

General Instructions

Just like with other spectra and images, echelle data requires ccd processing and flat fielding. As with other spectra, we need to correct for chip sensitivity using a standard, but this is less important over the small ranges of individual echelle orders. We do need a calibration lamp to determine what our wavelength scale is, and this is the most difficult part of both longslit and echelle spectroscopy. Thankfully, John Barentine provided us with an identified cal lamp, so dispersion correction is not difficult with IRAF's automation.

Below is a set of commands used to batch-process a set of images. I have reduced the data from UT071031, but because it was done in batch mode it shouldn't be trusted. These commands show you exactly how everything was done, but many of the steps listed in the batch section have been done for you and do not need to be repeated.

In /home/shared/astr3520/ref/, there are two spectra and a database folder containing their aperture lists and the linelists. These references should be used as listed below - JBrefspec is a magnified standard star reference to use for aperture tracing on the science files, and JBtharspec should be used to extract the thorium argon spectra and identify lines. JBtharspec can also be used as a wavelength reference as long as it is used carefully - it is from many years ago and it's possible things could have changed (though I think it's pretty much correct).

Command summary:

Packages you will need:
noao
imred
ccdred required for ccdproc
echelle required for apall
proto Only needed for bad pixel mask generation

Example for evega.0039.fits from UT071031:

ccdproc evega.0039.fits ccdtype="" zerocor=no darkcor=no flatcor=no fixfile="echmask.pl" biassec="[2100:2128,2:2027]" trimsec="[21:2048,1:2048]" order=3 niterate=3 int-

hedit evega.0039.fits DISPAXIS 1 add+

magnify evega.0039.fits evega.0039.rs.fits xmag=1 ymag=4

apall evega.0039.rs.fits references=JBrefspec.fits interactive- find- recenter+ resize+ edit- trace- fittrace- extract+ extras- lower=-2 upper=2 b_sample="-6:-3,3:6" b_naverage=1 width=12 radius=15 shift- ylevel=.05 t_order=5 t_sample="200:1850,*" t_naverage=3 t_niterate=3

imarith evega.0039.rs.ec.fits / eQuartzSUM.rs.ec.fits evega.0039.rs.ec.flat.fits

refspectra evega.0039.rs.ec.flat.fits answer="yes" references="JBtharspec.ec" sort="" group=""

dispcor evega.0039.rs.ec.flat.fits evega.0039.rs.ec.dispcor.fits

Individual Example for the Quartz Lamp:

ccdproc eQuartz*.fits ccdtype="" zerocor=no darkcor=no flatcor=no fixfile="echmask.pl" biassec="[2100:2128,2:2027]" trimsec="[21:2048,1:2048]" order=3 niterate=3 int- takes care of basic CCD correction (bias, bad pixels)

imarith eQuartz.0033.fits / 200 eQuartzBLUEdiv200.fits

Divide the two quartz lamps by their exposure times to get flats (eQUARTZ_OPEN.0030.fits has 1s exposure time).

imarith eQuartzBLUEdiv200.fits + eQUARTZ_OPEN.0030.fits eQuartzSUM.fits

Because they are in different filters, they cover different orders, so the two lamps should be combined.

hedit eQuartzSUM.fits FILTER "Blue+Open" and the header updated

magnify eQuartzSUM.fits eQuartzSUM.rs.fits xmag=1 ymag=4 makes image larger (rs=rescale)

apall eQuartzSUM.rs.fits references=JBrefspec interactive+ find- recenter+ resize- edit+ trace- fittrace- extract+ extras- lower=-2 upper=2 b_sample="-6:-3,3:6" b_naverage=1 width=12 radius=15 shift- ylevel=.05 t_order=5 t_sample="200:1850,*" t_naverage=3 t_niterate=3 **INTERACTIVE** task to apply apertures from a reference, then check to make sure they're correct

Batch example / ref file creation

flatcombine e-Bias.0*.fits output="e-Bias.flatcombine.fits" ccdtype="" process=no subsets=no

!echo "788 788 803 2000\n1683 1683 664 2000" > badcols create badcols file

text2mask text="badcols" mask="echmask" ncols=2128 nlines=2048 create bad pixel mask

ls e*.fits | grep -E "[0-9]\{4\}.fits" > rawlist create list of raw images

ccdproc @rawlist ccdtype="" zerocor=no darkcor=no flatcor=no fixfile="echmask.pl" biassec="[2100:2128,2:2027]" trimsec="[21:2048,1:2048]" order=3 niterate=3 int- apply CCD processing to raw images

cat rawlist | grep -iv bias > echellespeclist

hedit @echellespeclist DISPAXIS 1 add+ add dispersion axis to spectra (essential for use of apall!)

ls e*.fits | grep -i thar > tharlist

apall @tharlist references=JBtharspec.fits interactive- find- recenter+ resize+ edit- trace- fittrace- extract+ extras- lower=-2 upper=2 b_sample="-6:-3,3:6" b_naverage=1 width=12 radius=15 shift- ylevel=.05 t_order=5 t_sample="200:1850,*" t_naverage=3 t_niterate=3

ls e*.ec.fits | grep -i thar > tharspeclist

ecreidentify @tharspeclist reference="JBtharspec.ec" cradius=1 threshold=100

cat echellespeclist | grep -iv thar | grep -iv quartz | grep -iv flat | grep -iv bias > premagnifylist

!sed s/.fits/.rs.fits/ premagnifylist > postmagnifylist

magnify @premagnifylist @postmagnifylist xmag=1 ymag=4

apall @objectlist references=JBrefspec interactive- find- recenter+ resize+ edit- trace- fittrace- extract+ extras- lower=-2 upper=2 b_sample="-6:-3,3:6" b_naverage=1 width=12 radius=15 shift- ylevel=.05 t_order=5 t_sample="200:1850,*" t_naverage=3 t_niterate=3

!sed s/.fits/.ec.fits/ objectlist > objspeclist

!sed s/.fits/.flat.fits/ objspeclist > flattedlist

imarith @objspeclist / eQuartzSUM.rs.ec.fits @flattedlist

refspectra @flattedlist answer="yes" references="JBtharspec.ec" sort="" group=""

!sed s/flat/dispcor/ flattedlist > dispcorlist

dispcor @flattedlist @dispcorlist

!sed s/.rs.ec.dispcor./.singlespec./ dispcorlist > singlespeclist

scombine @dispcorlist @singlespeclist group="images" combine="sum" **THIS COMMAND DOESN'T WORK AS IS: it requires some sort of rejection routine that I have not been able to pin down.

Detailed explanation of individual commands

imarith [file1] [operation] [file2 or number] [outputfile] performs the designated mathematical operation on the desired files
flatcombine [filelist] output=[outfile] is a flat-specific version of imcombine. It defaults to "average" with "avsigclip", or average sigma clipping, to remove outliers from the set of images.
text2mask text=[textfile] mask=[outfile] generates a bad pixel mask. In the case of the APO Echelle, they tell us exactly what bad pixel mask to use.
ccdproc [filename] fixfile=[fixfilename] biassec=[range] trimsec=[range] order=[ordernum] niterate=[niter] Automated CCD processing package uses a bad pixel mask (fixfilename) to eliminate bad pixels and a bias segment to determine what the chip bias is and remove it. trimsec specifies a range to remove from the image. The ordernum and niterate parameters are for fitting the bias from the small "overscan" regions used to determine it.
apall [filename] reference=[refspec] uses the reference spectrum specified to extract apertures from the 2D echelle image. With the parameters used here, it does not automatically find new apertures, but will recenter and (except for the quartz lamp) resize the apertures.Most of the parameters specified are used for tracing the aperture and are not strictle necessary, but are included here because they were suggested in the APO Echelle Reduction guide
ecreidentify [filename] reference=[refspec] uses the reference spectrum with identified lines to identify the wavelengths of the extracted apertures. ecreidentify is only necessary here if ThAr lamps other than those provided are going to be used; in this example it is not used.
magnify [filename] [outname] xmag=[nx] ymag=[ny] simply magnifies the image by some factor while conserving flux. This is used to reduce aliasing that results from the narrow apertures - see the echelle reduction guide.
respectra [filename] reference=[refname] adds the reference spectrum to the header file of a target file. This step is necessary to use dispcor
dispcor [in] [out] applies the dispersion correction specified in the reference spectrum, i.e. it changes units from pixels to wavelength
scombine [in] [out] combine=[combtype] attempts to combine all of the individual spectral orders into a single spectrum using either sum, average, or median combine. However, what we REALLY want is a "max" combine that will take the maximum of the two spectra at each point, and I don't know how to do that yet.
hedit [filename] [parameter] [value] add=[y/n] modifies the header of a fits file. It is a very simple process to do this, but hedit preserves the fits format (which affects how different programs - e.g. ds9 - read and display the file) when making edits and prevents you from overwriting key parameters. We use "DISPAXIS 1 add+" to say "change the value of DISPAXIS to 1, and add the parameter DISPAXIS if it does not exist". DISPAXIS=1 means the X axis. You can see in the echelle images that the x axis is the wavelength axis and the y axis is the "order" axis.

Log of work for UT110702:

In this example, I extracted a different set of Quartz lamp flat field images, then attempted to modify that image so that it would mask out the regions of lowest sensitivity on the Echelle detector. However, this did not work very well
flatcombine eblueflat* output="BLUEFLAT.comb.fits" ccdtype="" process- subset- flatcombine eredflat* output="REDFLAT.comb.fits" ccdtype="" process- subset- imarith BLUEFLAT.comb.fits / 240 BLUEFLAT.div240.fits imarith REDFLAT.comb.fits / 70 REDFLAT.div70.fits imarith REDFLAT.div70.fits + BLUEFLAT.div240.fits FLAT.fits magnify FLAT.fits FLAT.rs.fits xmag=1 ymag=4 apall FLAT.rs.fits references=JBrefspec interactive+ find- recenter+ resize- edit+ trace- fittrace- extract+ extras- lower=-2 upper=2 b_sample="-6:-3,3:6" b_naverage=1 width=12 radius=15 shift- ylevel=.05 t_order=5 t_sample="200:1850,*" t_naverage=3 t_niterate=3 imcopy FLAT.rs.ec.fits FLAT.rs.ec.rp.fits imreplace FLAT.rs.ec.rp.fits value=1000 lower=0 upper=20 imreplace FLAT.rs.ec.rp.fits[*,1:70] value=1000 lower=0 upper=100 imreplace FLAT.rs.ec.rp.fits[*,7:20] value=1000 lower=0 upper=400 imarith @objspeclist / FLAT.rs.ec.rp.fits @flattedlist

ASTR3520 home

Page written by