VLA CASA Pipeline-CASA4.5.3: Difference between revisions
Created page with "== Introduction == == Data == == Overview of the Pipeline procedures == == Pipeline Requirements == == Running the Pipeline == == Assessing the Weblog == == Pipeline Ou..." |
|||
Line 1: | Line 1: | ||
== Introduction == | == Introduction == | ||
• With the start of Jansky VLA Full Operations (January 2013), we started a | |||
new operational model: | |||
– Deliver flagged and calibrated visibility data | |||
– You will self-calibrate and image visibility data to meet science goals, using | |||
resources at home institution or NRAO computing resources | |||
• Automated pipeline should run correctly on all “standard” Stokes I science | |||
SBs; “standard” means: | |||
– 128 MHz spws, but may work on other set-ups as well | |||
• Some constraints on strength of calibrators needed | |||
– Contains correctly labeled and complete scan intents | |||
• Current versions available: | |||
– “scripted” pipeline is a collection of python scripts that use CASA tasks | |||
wherever possible, but also uses toolkit calls; readable and easy to modify | |||
– CASA integrated pipeline is compatible with ALMA pipeline infrastructure, | |||
improved diagnostics in weblog, used as real-time pipeline since Sep 2015 | |||
= | |||
• Real-time pipeline: | |||
– Minimal human intervention | |||
• Pipeline is run automatically on every science SB as it completes (not just | |||
“continuum”) | |||
– Pipeline output undergoes quality assurance checks by NRAO staff upon | |||
request; reports generated are archived as pipeline products | |||
• At your home institution: | |||
– Instructions for installation and operation of the VLA CASA Calibration | |||
Pipeline are available at | |||
https://science.nrao.edu/facilities/vla/data-processing/pipeline | |||
• Uses CASA 4.3.1, similar to current real-time pipeline | |||
• CASA 4.5.2 currently being validated (you are helping with this!) | |||
• Scripted pipelines for CASA versions through 4.5.0 also available | |||
– Provides more flexibility in how to use the pipeline, options suitable for | |||
spectral line datasets, mixed correlator set-ups, multi-band observations, etc. | |||
– Working to incorporate these into the CASA integrated pipeline | |||
= | |||
“Standard” Stokes I science SB means: | |||
– 128 MHz spws, but may work on other set-ups as well | |||
• Can work for narrower BWs, depends on the strength of the calibrators | |||
• Heuristics currently make some assumptions about the strength of the calibrators, | |||
in particular, the delay calibrator | |||
– Contains correctly labeled and complete scan intents | |||
• And also that the observation has been set up correctly! | |||
• Will the pipeline work for you? | |||
– The pipeline successfully completes on ~95% of all science SBs observed on | |||
the VLA; whether the output can be used for science depends on the science | |||
goal, and whether the observation was correctly set up | |||
• Pipeline includes Hanning smoothing, RFI flagging, and weight calculations that may | |||
not be appropriate for spectral line projects (but can modify scripted pipeline) | |||
• No polarization calibration (yet) but can use pipeline output as starting data for | |||
pol. cal. | |||
• Will probably work well for data taken since May 2012, may work for earlier EVLA | |||
data, likely that extra flagging may be needed in these cases | |||
= | |||
Calibrator strength: | |||
– Conservative limit on | |||
strength of BP and | |||
complex gain calibrators | |||
can be derived from | |||
requirement for initial gain | |||
calibration to work at high | |||
end of Q-band | |||
– Heuristic for delay | |||
calibration currently | |||
requires the SNR=3 limit | |||
on initial gain calibration | |||
per integration | |||
== Data == | == Data == |
Revision as of 22:28, 2 May 2016
Introduction
• With the start of Jansky VLA Full Operations (January 2013), we started a new operational model: – Deliver flagged and calibrated visibility data – You will self-calibrate and image visibility data to meet science goals, using resources at home institution or NRAO computing resources • Automated pipeline should run correctly on all “standard” Stokes I science SBs; “standard” means: – 128 MHz spws, but may work on other set-ups as well • Some constraints on strength of calibrators needed – Contains correctly labeled and complete scan intents • Current versions available: – “scripted” pipeline is a collection of python scripts that use CASA tasks wherever possible, but also uses toolkit calls; readable and easy to modify – CASA integrated pipeline is compatible with ALMA pipeline infrastructure, improved diagnostics in weblog, used as real-time pipeline since Sep 2015
=
• Real-time pipeline: – Minimal human intervention • Pipeline is run automatically on every science SB as it completes (not just “continuum”) – Pipeline output undergoes quality assurance checks by NRAO staff upon request; reports generated are archived as pipeline products • At your home institution: – Instructions for installation and operation of the VLA CASA Calibration Pipeline are available at https://science.nrao.edu/facilities/vla/data-processing/pipeline • Uses CASA 4.3.1, similar to current real-time pipeline • CASA 4.5.2 currently being validated (you are helping with this!) • Scripted pipelines for CASA versions through 4.5.0 also available – Provides more flexibility in how to use the pipeline, options suitable for spectral line datasets, mixed correlator set-ups, multi-band observations, etc. – Working to incorporate these into the CASA integrated pipeline
=
“Standard” Stokes I science SB means: – 128 MHz spws, but may work on other set-ups as well • Can work for narrower BWs, depends on the strength of the calibrators • Heuristics currently make some assumptions about the strength of the calibrators, in particular, the delay calibrator – Contains correctly labeled and complete scan intents • And also that the observation has been set up correctly! • Will the pipeline work for you? – The pipeline successfully completes on ~95% of all science SBs observed on the VLA; whether the output can be used for science depends on the science goal, and whether the observation was correctly set up • Pipeline includes Hanning smoothing, RFI flagging, and weight calculations that may not be appropriate for spectral line projects (but can modify scripted pipeline) • No polarization calibration (yet) but can use pipeline output as starting data for pol. cal. • Will probably work well for data taken since May 2012, may work for earlier EVLA data, likely that extra flagging may be needed in these cases
=
Calibrator strength: – Conservative limit on strength of BP and complex gain calibrators can be derived from requirement for initial gain calibration to work at high end of Q-band – Heuristic for delay calibration currently requires the SNR=3 limit on initial gain calibration per integration