|
|
This page describes changes to the IRS pipeline that accompany changes to the overall Spitzer pipeline. Roughly every six months, the pipeline number (the "S-number") for all new Spitzer data in the archive is increased. New data will appear in the archive with a different S-number than old data. Parallel to these changes, the IRS pipeline may be improved and those improvements are described below. However, old IRS data will not be reprocessed with new pipeline versions unless the are substantial changes in the data quality or the calibration files. Users that have already downloaded data do not need to download it again when new pipeline is released. When pipeline changes result in substantial improvements to the IRS data, all past campaigns will be reprocessed with the newest pipeline.
S18.0.1 (Release Date: Jun 08)
- Spectral table files are now provided also in FITS format
S17.2.0 (Release Date: Apr 08)
- The LH spectral extraction algorithm and corresponding flux calibration has been updated the flux values (in e/sec) for LH delivered by the Extract module (Regular or Optimal) are now divided by the height (measured along the spectrum in pixels) of the extraction window, as set by the wavsamp wavelength calibration file. This change improves the order matching in *_extract.tbl spectra, but does not have much effect on the final, flux calibrated, *_spect.tbl spectra delivered by Tune. These updates do not pertain to any of the other modules.
S17.0.0 (Release Date: Jan 08)
- Errors due to readnoise and Poisson noise are now incorporated in the
pipeline uncertainties, and are propagated during slope estimation.
Uncertainties are now propagated during background subtraction (bksub).
- A new slope estimation module was developed. Slopes (fluxes) are now
computed via linear fits to entire ramp, as oppossed to individual slope
fitting to consecutive samples.
- Better treatment of ramp slopes after a radhit. The S16 pipeline
discards all samples of a ramp following a radhit. This leads to improvement
in the signal-to-noise for low illumination levels, but to a slight (10%)
increase in noise in high resolution data, in the presence of many radhits
(several hundred radhits per array). For S17, the ramp following a radhit
is discarded only if the slope is <500 e-/s (before the radhit).
- Darks are now campaign-dependent for LL, built from data taken in a
symmetric moving window 5 campaigns wide. This improves the cosmetic
appearance of non-background subtracted data but results in negligible
differences for sky subtracted data.
- The LL bias and temperature were changed in campaign 45 to reduce the
number of rogue pixels. The decrease in gain resulting from the change is
more than compensated by the reduction in rogue pixels, leading to a significant
improvement in overall S/N (10-30%). New darks, nonlinearity coefficients, flats,
and flux calibrations were derived for the new settings and used to process
the S17 post-44 campaign LL data. Additional improvement was made in the
post-44 flatfield spatial correction, leading to better nod and order matching.
- The PU imaging mode has been recalibrated in order to reduce the zeropoint
uncertainty, which is now 2%. The zeropoint is now tied to an infinitely large
aperture. More details are given in the IRS Data Handbook (http://ssc.spitzer.caltech.edu/irs/dh/)
S16.1.0 (Release Date: May 07)
- During slope fitting in SL, SH, and LH, all samples are discarded following a
radhit. This implementation is already active for LL, since S15.
The reason for this change is that the slopes of faint ramps can
change by factors of 30% or more following a radhit (w.r.t. before the
radhit).
- The radhit module now allows to skip the first sample of a ramp during its
detection process. Therefore the module now avoids the spurious detection
of radhits in the second sample of a ramp. These spurious detections were
caused by the first sample being depressed with respect to the rest of
the ramp slope, in modules such as SL. This implementation of skipping of
the first sample does not apply to 4-sample ramps, because the radhit
module needs a minimum of 4 samples to work.
- All data taken from IRS campaign 40 onward (15 Apr 2007) will be processed with S16.1.0. Most previous data remain S15, with the exception of LH IRS campaigns 29 to 34 and campaign 36, as well as SL IRS campaigns 38 and 39. See the archive to Important Notes to Observers, note from 31 Jul 2007.
S16.0.0 (Release Date: Apr 07)
No changes.
S15.3.0 (Release Date: Feb 07)
All IRS data have been re-processed with S15.3.0 and are available for download via Leopard.
Listed here are the most significant changes in this version.
- To reflect all the pipeline changes, new flux calibration polynomials
(fluxcons) were derived for all modules. Among other things, the new fluxcons
corrected the 5% red tilt in LL2 present in S14 (see ssc.spitzer.caltech.edu/irs/features.html, "Low-Res Nonrepeatability").
LL1 spectra now have slopes that are steeper (redder) by 3% over the spectral range, with respect to S14.
- New flatfields were built for all modules
a) The spatial behavior in all the flatfields was improved. Observations in two different nods now match to better than 2%.
b) SH and LH flatfields are now corrected using extended zody observations.
c) SL and LL interference fringes reduced to 1-2% level for sources well centered on the slit.
d) Bad regions in LH flat masked out to remove "bumps" at 20 micron.
- Updated SH, SL, and LL wavsamp files
a) SH: The wavelength solution updated in all orders. The largest changes occurred at orders 19 and 20 (<0.5 pix). Spatial profiles changed in orders 11-15 only.
b) SL, LL: Updated spatial profile for all orders. The change is especially apparent in the LL2 ridge location. No wavelength changes were made, beyond very small shifts arising from the spatial profile change.
- BCD products contain new header keywords VHELCORR and VLSRCORR to allow
correction for the orbital velocity of Spitzer
The values of VHELCORR and VLSRCORR are to be added to velocities derived from IRS spectra, to obtain
final velocities in the Heliocentric or LSR frames respectively. These keywords
will be of interest only to observers measuring velocities with high precision from
Short- High or Long-High spectra.
- Revised uncertainty propagation
a) Suppressed propagation of the correlated component of uncertainty during residual droop correction.
b) During 2-D coaddition of BCDs (which produces the postBCD products *coa2d.fits and *bksub.fits),
the output uncertainty (*c2unc.fits and *bkunc.fits) is now the root-sum-square of the input BCD uncertainties,
divided by the number of coadded BCDs. Formerly (S14) the output uncertainty was the trim-standard deviation of the input BCDs.
c) The resulting uncertainties do not yet include all the systematics (photon noise per sample, readnoise, etc) associated with
the processing.
- Spectrum extraction heights and widths
a) SL and LL default extraction height changed from 1 pixel to native wavsamp height, to better follow spectrograph resolution.
b) SL3 default extraction aperture widened to 5.3333 pixels at 8.0 microns, to match SL2.
- Updated SL, LL, and SH super-darks
New observations of the dark spot were obtained at times when the zodiacal emission was lowest.
This change is most apparent for LL, which used to have a mixture of high and low zody data.
- Radhit rejection in LL ramps
All samples of a ramp following a radhit are discarded during slope estimation in long-low data.
This results in improved signal-to-noise in faint long-low data. Future pipelines
will implement the rejection in other modules.
- The calibration files for the Peak Up imaging mode have changed
a) Absolute calibration values and aperture corrections have been updated, which
results in changes in measured photometric values of <15%.
b) New PRFs have been provided for APEX.
c) The bit masks have been updated to better interact with the mosaicker from MOPEX.
d) Fixed bug in error propagation. Users still need to remember the systematic zeropoint
uncertainty for themselves.
S14.0.0 Partial Reprocessing (Release Date: Jul 06)
All IRS data taken during or after IRS Campaign 30 (i.e., after the
DATE_OBS keyword = 2006-04-16T00:00:00) have been processed with the
S14.0.0 pipeline. Except for the special cases mentioned below, the
pipeline is identical in every way to that of S13.2.0 and S13 versions
of SPICE will work seamlessly with S14 data. (No changes have been made
to fluxcon files or other SPICE related files for S14).
Special reprocessing of IRS campaigns back to IRS Campaign N2
Two targeted changes have been made to archived data for S14 which will
lead to improvements in the quality of these data.
- All IRS Long-Low (Channel 2) data, extending back to Campaign N2 in
IOC, have been reprocessed through S14.0.0 to improve the linearity
correction for saturating sources. The largest changes (15-20% in some
cases) will be found for targets with flux densities well above 1 Jy, in
regions of the spectrum where the ramps approach or exceed saturation.
Targets with fluxes less than 1 Jy will, in general, be less affected
unless long ramps were used leading to saturated/near saturated values.
The improvements manifest themselves as smoother spectral continua in
the 20-30 micron range, and reduced noise in this region. In all cases
the apparent flux density in the affected region of the spectrum will go
down, since the linearity correction has been reduced. Users should not
be worried that they have mixed S13 and S14 data for targets involving
observations with both SL and LL prior to Campaign 30. This is to be
expected since the S13 and S14 pipelines are identical for SL (as they
are for SH and LH).
- All IRS Peak-up imaging AORs were reprocessed through S14.0.0 to
ensure that improved 16 and 22 micron PRF files (Point-source Response
Function) were correctly included with archived products. Previous
versions of the pipeline had provided for the user a preliminary version
of the PRF which could have led to problems with automated extraction of
sources in MOPEX. The reprocessing and archive to these new PRF files
ensures that the automated calibration, if run by the user, will lead to
results consistent with those fluxes derived by manual extraction.
Further refinements of the red PRF are underway and an even better
version will be available in S15.
S13.2.0 (Release Date: Jan 06)
All IRS data taken in Nominal Ops campaigns 1-27, and SV campaigns N2-R
have been re-processed with S13 and are in [the process of being]
archived. Check this page
to see when your favorite campaigns have been archived and are
available for download via Leopard.
There are a number of significant changes to these data processed
through the S13 pipeline. These are:
Wavelengths
- Wavelength calibration updated for all Short-Hi orders.
- Spatial centering of the wavelength solution was changed for most
modules.
- Largest LH shift is 0.5 pixels for order 11.
- Largest SH shift is 0.3 pixels for order 20.
- 0.68 pixel shift of LL2.
- LL slit length was increased to 34 pixels.
- The sampling of the LL and SL "bonus" (order 3) wavelength
solution
was changed to match the gridding of order 1.
- LH wavelength samples were adjusted to ensure none fall completely
off the detector array.
- The sign of the spatial offset in the b?_wavsamp_offset.fits files
were changed for SL, LL and SH, to properly align with the sense of the
PA_SLT keyword in the FITS headers.
- The scale of the LH b3_wavsamp_offset.fits file was updated to
reflect the correct value of 4.5 arcseconds per pixel.
Flat Fields
Completely new (and improved) flat fields have been created for all
modules, including a new flat field that applies specifically for LH
data taken after the bias change to the detector at the beginning of
IRS Campaign 25.
Flux Calibration
Significant refinements have been made to the Flux calibration in all
modules, with new fluxcon tables
for each module. As a result of the change in bias made for the LH
module at the beginning of IRS Campaign 25, we now provide a different
fluxcon table for data taken from Campaign 25 onwards.
As a result of improved flux calibration, we discovered a phenomenon we
have called the "teardrop"
effect in the SL module. The SL fluxcon tables differ from previous
pipeline processing, in that the teardrop enhancement is not fitted out
by the fluxcon tables. The resulting fluxcon table is much more linear
over the SL1 module. Users are warned that as a result of the new
fitting procedure, the teardrop effect will be more obvious than it was
in past processing (previous processing partly masked the effect of the
teardrop to the detriment of our flux calibration). The cause of the
teardrop effect is not fully understood yet by our team.
Linearity Coefficients post LH bias-change
We have modified the linearity coefficients use to correct
non-linearities in the ramps for data taken in LH after the bias change
(Campaign 25 and onwards).
Browse-quality Automatic Background Subtraction for Low-res
modules in Stare-mode only
We have implemented an automated background subtraction process for all
low-res data taken in staring mode. The process operates only on
coadded data, and subtracts adjacent nod positions. Spectra are
extracted automatically from these new products. (See IRS DATA Handbook
for details). We caution users that this process is not necessarily the
optimal way to do background subtraction in a number of cases. Firstly
if the source is extended over a significant part of the slit then
obviously the automated nod subtraction will not work. Secondly, for
very bright point sources, the extended psf from one nod position could
extended far enough at the long wavelength end of the spectrum to
compromise the background subtraction. The greatest benefit of the
automated subtraction will be for faint point-sources with no other
confusing source in the slit. Useful filenames have the following
suffixes: bksub.tbl (irs-tuned final nod-subtracted spectrum),
bksub_unc.fits (uncertainty file). If you have downloaded
your S13 data prior to 02 Feb 2006, it may have been missing the
post-BCD background subtracted staring mode data. If this is the
case, and these data are needed, you should download the files again.
Coordinates and wavsamp_offset files
S13 SPICE has the capability to view the coordinates of sources which
are within the slit. For this purpose, it utilizes
the b?_wavsamp_offset.fits and b?_wavsamp_omask.fits cal files. The cal
files, specifically the b?_wavsamp_offset.fits files, which you get
when downloading your pipeline processed
data from the archive have the incorrect sign convention which will
result
in the coordinates being incorrect. Therefore, S13
SPICE should only be used with the cal files which come with the SPICE
release rather than the archived cal files.
Row-Droop Correction Applied to SL module
As discussed in detail in the notes for the S12.0 pipeline release
(below) we were not correcting fully for the effects of row-droop. This
problem has been fixed in S13. The correction is only applied to SL
because of the high charge loading on the SL peakup arrays. In the
other modules, no correction is necessary.
Browse-quality IRS Peak-up Imaging through the MOPEX image mapper
Unlike S12, we now pass data that was obtained under IRS Peak-up
Imaging mode through
the MOPEX mosaicing software and generate browse-quality peak-up
images. The files are named with the following suffixes: b_mos.fits
and r_mos.fits (blue and red peakup mosaics), b_unc.fits and
r_unc.fits (uncertainty files) and b_cov.fits and rr_cov.fits (coverage
maps similar to those seen in SPOT for IRAC and MIPS imaging).
S12.0 (Release Date: Jun 05)
All IRS data taken in Nominal Ops campaigns 1-20, and SV campaigns N-R
have been re-processed with S12 and are in the process of being
archived. Check on this page
to see when your favorite campaigns have been archived and are available
for download via Leopard.
Memo on IRS uncertainties
(October 2005)
There are three significant changes to the data processing
for S12. These are:
- Wavelengths: The wavelength solutions (wavsamp files) for all modules
have been changed. This was primarily to correct some inter-order offsets
that were apparent in the S11 data. These changes include wavelength
shifts (e.g. for SL1) and changes to the dispersion (e.g., for SH, LH LL
and SL2). Lines will shift (from their S11 positions) by 0.01-0.14
microns, depending upon wavelength and module. Observers are encouraged
to check their S12 data, and report any remaining wavelength discrepancies
to the SSC help desk.
- Flux calibration: Small changes to the Short-High and Long-High flux
calibration (fluxcon files) were made in accordance with the wavelength
changes described above. These should reduce order to order variations
seen in some S11 data.
- Short-Low stray light: The correction for stray light in the Short-Low
module (see the Observers Manual and Pipeline Handbook) has been modified
to include only an exponential fall-off from the Peak-up arrays for S12.
Previously, this stray-light correction also included a small "pedestal"
term in addition to the exponential. The removal of this pedestal
occasionally caused unwanted horizontal striping in Short-Low data. These
stripes should not be present in your S12 data. Observers should be
aware that although the net effect of this modification is positive (data
processed through the S12 are of generally superior quality to S11), the
fact that we no longer remove a "pedestal" means that a minor effect
called "row droop" will be present in the final spectra. As we discuss
below, this will have almost no impact on users if they perform a
background subtraction of SL data (this is highly recommended in most
observing strategies). However, in the unlikely event that background
subtraction is not performed, observers should be aware of the possible
consequences of uncorrected row-droop.
"Row-droop" is an additive (pedestal-like) signal which can affect all
the ramps along a given row of an image by a constant factor which is
proportional to the total charge seen on that row of the detector. The
Short-low module is the most likely detector affected by the problem
because the peak-up arrays can contain a significant signal--and this
signal can cause an increase in the flux seen along that entire
row--especially when observing medium and high background scenes. Because
we are no longer removing a pedestal from SL data in the stray-light
correction, row-droop will be present in your S12 Short-Low BCD data at
some level of significance. It will manifest itself as an apparent
increase in the flux in SL2 and SL1 in regions where the peak-up arrays
affect those rows. In contrast, the pedestal will NOT BE present in rows
between the two peak-up arrays--an effect that could cause the observer to
think they had faint absorption there. The spectral region that might show
an apparent flux deficit is between 5.5-5.9 microns and 11.2-11.8
microns--the wavelength regions corresponding to the rows in SL2 and SL1
that are between the peakup arrays. Sky subtraction will remove this
effect. However, observers of extended sources should take care in
interpreting low-level spectral variations at these wavelengths in their
Short-Low S12 data if no appropriate sky subtraction is performed. We
will characterize and remove this pedestal in future pipeline releases.
Observers are encouraged to compare their SL data before and after sky
subtraction to gauge the relevance of this effect on their science.
S11.4.0 (Release Date: Feb 05)
Most of the changes were uplink-related (e.g., Spot-related) and
thus do not affect the data. However, the LH dark
calibration files (which are updated every campaign) have changed and so
depending on whether or not you have data from the IRS LH module, you
might have updated dark files.
S11.0.2 (Release Date: Dec 04)
Complete reprocessing of IRS campaigns using S11.0.2 software.
Modifications include:
- Completely revised Droop Coefficients for all modules.
- Completely revised linearity corrections for all modules.
- Newly derived Flatfields/2dspectral response correction (Flats).
- Newly derived Fluxcon tables (Flux conversion for 1-d spectral extraction).
- All LH data are now processed with campaign derived darks
(to partially mitigate against time-variable warm pixels).
- Fix to stray-light correction alogorithm and operation of module that
was causing problems "banding" across spectral orders.
- Fix to "bright source drop-out" problem that cause trouble with 4-sample ramps of very bright
sources. This problem caused sources which were only measured with two independent samples
to have spectra whose fluxes were lower than they should have been. this problem was fixed in
this release.
- Fix to "*_spec2.tbl files not being archived" problem.
S11.0 (Release Date: Nov 04)
Complete reprocessing of IRS campaigns using S11 software.
Modifications include:
- Addition of new Peak-up mapping pipeline to support
GO2 Peak-up mapping capability.
- Fix to "high-flux drop out" problem seen in 4-sample spectral
observations in which the 3rd and 4th samples are saturated because
the target is so bright. RADHIT and IMAGEST modules can now handle a
single-slope measurement required to capture information from the
unsaturated samples. This problem ONLY affects 6-s RAW data on extremely
bright sources.
- All S11 data incorporates newly measured "Droop" coefficients and
refined and improved non-linearity corrections for all 4 modules.
- As a result of 3, new flat field/spectral response corrections and
flux conversion tables are also applied.
S10.5 (Release Date: Sep 04)
A complete reprocessing of IRS campaigns using the existing S10 pipeline
but with the following change resulting from new calibration products:
- Improved performance of pipeline slope estimates in regions close to
A/D saturation.
S10.5 (Release Date: 8 Aug 04)
A complete reprocessing of IRS campaigns incorporating the following
changes:
- Fix to ramp timing problem for slope estimates for spectral data.
- New higher-quality "flat-fields" (i.e. spatial flats/2-d RSRFs)--
improvement mainly in spatial quality along the slit.
- New associated fluxcon tables.
S10.0 (Release Date: 14 Apr 04)
Complete processing of IRS campaigns using best available calibration
products available at that time.
- Nod to nod variations in flat fields no better than 7-10%.
- Problem with spectral data slope estimatation especially for very
short ramps. This problem was fixed in S10.5.
S9.5 (Release Date: 20 Feb 04)
- Incrementally improved calibration products after analysis of
post-launch solar-flare activity and in-flight performance.
|
