https://casaguides.nrao.edu/api.php?action=feedcontributions&user=Emomjian&feedformat=atomCASA Guides - User contributions [en]2024-03-29T06:57:51ZUser contributionsMediaWiki 1.38.6https://casaguides.nrao.edu/index.php?title=MG0414%2B0534_P-band_Spectral_Line_Tutorial_-_CASA_5.7.2&diff=30011MG0414+0534 P-band Spectral Line Tutorial - CASA 5.7.22020-12-22T22:49:57Z<p>Emomjian: </p>
<hr />
<div><!--By Bjorn Emonts--><br />
<br />
<b>This CASA Guide is for Version 5.7.2 of CASA.</b> <br />
<br />
{| class="wikitable"<br />
|-<br />
| style="background: red; color: white;" | '''DISCLAIMER'''<br />
|-<br />
| The following guide is for users who are experts in data reduction using CASA. If you are a beginner or novice user, please review other CASA guides found at [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials VLA Tutorials]. <b> Note: This guide may take up to 2-days to complete </b><br />
|}<br />
<br />
== Overview ==<br />
<br />
This tutorial describes how to use CASA 5.7.2 to reduce spectral-line data in the low-frequency P-band of the VLA (230&ndash;470 MHz). The goal is to make an image cube containing HI 21cm absorption against the strong radio continuum of gravitationally lensed radio galaxy MG0414+0534. As a result of the high redshift of z=2.6365, the HI absorption signal in MG0414+0534 is redshifted to an observed frequency of 390.597 MHz. The HI absorption in MG0414+0534 was previously imaged with the VLA by Moore, Carilli & Menten 1999 ([http://iopscience.iop.org/article/10.1086/311818/meta ApJ, 510, 87]), before the upgrade to the WIDAR system.<br />
<br />
=== Observing strategy ===<br />
<br />
To perform P-band spectrsocopy, there are three important considerations for planning the observations:<br />
<br />
* Use a bandpass calibrator that is strong enough to accurately calibrate the frequency-dependent gain variations. This is particularly important for most HI 21cm absorption projects, which are typically performed against strong radio continuum sources. As a rule-of-thumb, use ''t''<sub>cal</sub> > ''t''<sub>obj</sub> × (''S''<sub>obj</sub> / ''S''<sub>cal</sub>)<sup>2</sup>, with ''t'' the exposure time and ''S'' the source flux density. This rule-of-thumb serves to avoid introducing excessive additional noise in the spectrum.<br />
<br />
* Additionally, if very high spectral dynamic range is needed (i.e., when expecting a ratio between the detection limit and the radio continuum of about 1:10,000 or more), consider observing a bandpass calibrator several times during your run to be able to correct for time-varying bandpass changes, which scale with the continuum emission in the target field. See the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/calibration Calibration section] of the Guide to Observing with the VLA for more information.<br />
<br />
* Use a bandwidth that is wide enough to perform accurate self calibration. Using a wide bandwidth for self-calibration is important for fields with relatively weak continuum sources. For strong continuum sources, a narrower bandwidth can be used to avoid excessive RFI. <br />
<br />
The P-band test data on MG0414+0534 that we use in this tutorial were obtained using a large bandwidth. This is to ensure that good bandpass solutions can be procured, and that self-calibration can accurately be performed. However, due to the strong radio continuum of MG0414+0534 (3.3 Jy at 390 MHz), and the large amounts of RFI across the entire band, we only use a small fraction of the total band for data reduction and analysis in this tutorial.<br />
<br />
== Obtaining the data ==<br />
<br />
<font color=red>'''This 15 GB data set can be downloaded by clicking [https://casa.nrao.edu/Data/EVLA/MG0414/MG0414_d1_data.ms.tgz here].'''</font> The filename should be MG0414_d1_data.ms.tgz (add the ".tgz" if this is missing). We are providing this starting data set, rather than the original data set for two reasons. First, many of these initial processing steps can be rather time consuming (> 1 hr). Second, while necessary, many of these steps are not fundamental to the calibration and imaging process, which is the focus of this tutorial.We will use test data that was taken in a hybrid configuration when the VLA array was moved from B-config to A-config. <br />
<br />
The entire data set can be downloaded from the [https://archive.nrao.edu/archive/advquery.jsp NRAO archive] by searching for the following Project Code: <br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
TSUB0001<br />
</pre><br />
</div><br />
<br />
This returns a long list with test-data that are publicly available. Our observations were performed on 14 Sept 2016:<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
TSUB0001.sb32720781.eb32763188.57645.263958564814<br />
</pre><br />
</div><br />
<br />
Note that this observation was duplicated on 15 September 2016. <!-- This second day of observations can be found under filename:<br />
TSUB0001.sb32720781.eb32773507.57646.24443702547 --><br />
<br />
The 15 GB data set from the download has already had online flagging and Hanning smoothing applied, and the smaller data set split out from the larger data set. If you wish, you can download the SDM from the NRAO archive in order to apply the online flags and run Hanning smoothing on the large data set before splitting out the smaller data set to work on. To apply the on line flag use the task {{flagcmd}}; to apply the Hanning smoothing use the task {{hanningsmooth}} on scans 7 to 16 and field 3 and 4; and to split the data, use the task {{split}}. For the purpose of this tutorial, we only reduce and image the first data set. Then, you will then need to split only the spw 17, and only channels 20 to 485 to remove the borders. <br />
<br />
<!-- The second data set can be reduced in an identical way (pending the flagging, given that the RFI conditions most likely changed). During the imaging stage you can then combine both reduced data sets. --><br />
<br />
<!-- To download the data, fill in your email, select either the "SDM-BDF (all files)" or “MS” option, and check the box next to the data sets that you want. The practical difference between selecting the MeasurementSet (MS) and Science Data Model (SDM) is that for MS the first step in the data reduction described below, namely reading in the SDM data to save it as a MeasurementSet, is already done by the archival engine. Note that you can also opt to download the data as a tar-file by tickking the appropriate box.<br />
<br />
Optionally, you can request to discard data marked as ‘bad’ by ticking the box “Apply Telescope Flags”, which gets rid of data taken during times of instrument calibration, shadowing, slewing, etc. However, it also applies all flags on the science data that were automatically created during the observations. Therefore, a safer option is to not apply any flags before downloading the data, but first inspect and subsequently apply the flags generated during the observations using <i>flagcmd</i>, as part of the data reduction plan.<br />
<br />
Click "Get My Data" will forward you to the next page, where you should choose the delivery method (either downloading over the internet, or sending home a hard-drive with the data). If you opt to retrieve the data over the internet, wait until you get an email confirming that the data is ready for download. --><br />
<br />
== Examining the Data ==<br />
<br />
=== Loading data into CASA ===<br />
<br />
We will have to untar the 15 GB data set that you've downloaded.<br />
<br />
<pre style="background-color:lightgrey;"><br />
# in a terminal, outside of CASA:<br />
tar -xzf MG0414_d1_data.ms.tgz<br />
</pre><br />
<br />
Please use CASA 5.7.2 for this tutorial (typing ''casa -ls'' in a linux window shows the available versions and the current version; to explicitly change the version type, e.g., ''casa -r 5.7.2-4.el7'')<br />
<br />
<source lang='bash'><br />
casa<br />
</source><br />
on the command line. This should start a CASA interactive python (iPython) session, and open a separate log window. To guarantee that the below mentioned procedure for data reduction and imaging works, make sure you are using CASA version 5.7.2. While older versions may work as well for the purpose of this tutorial, it is good to read instructions on how to [https://casa.nrao.edu/casa_obtaining.shtml download and install] the latest version of CASA.<br />
<!-- <br />
We will begin by importing our data from the binary format (SDM-BDF) into the MeasurementSet format, which is the standard for CASA data. For this, we use <i>importevla</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
importevla(asdm='TSUB0001.sb32720781.eb32763188.57645.263958564814', vis='MG0414_d1.ms', flagpol=False, applyflags=False, savecmds=True, outfile='flagfile.txt')<br />
</source><br />
<br />
<!-- In this case, we do not apply the flags that were created as part of the observations, but we will write them out to a <i>flagfile.txt</i> file. We will inspect and apply the flags as follows:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagcmd(vis='MG0414_d1.ms', action='plot')<br />
flagcmd(vis='MG0414_d1.ms', action='apply')<br />
</source><br />
<br />
As an alternative to <i>flagcmd</i>, given that we saved the flags to <i>flagfile.txt</i>, we can also use <br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1.ms', mode='list', inpfile='flagfile.txt', action='apply')<br />
</source><br />
<br />
to apply the flags.<br />
<br />
One of the flag commands is to <i>clip</i> all data points that have a 0-value, as these most likely have not recorded any science data. This is normally done with the settings <i>correlation=`ABS_RR’</i> and <i>correlation=`ABS_LL’</i>. Because the VLA P-band and 4-band systems use linear XX and YY polarisation, rather than the circular RR and LL polarisation of the other VLA bands, you will encounter an error message when running <i>flagcmd</i>. We can get around this issue by manually clipping the 0-data for the XX and LL polarization with <i>flagdata</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1.ms', mode='clip', correlation='ABS_XX,ABS_YY', clipzeros=True, action='apply')<br />
</source><br />
--><br />
<br />
It is a good practice to read the Operator Log that was created by the duty operator during the observations. This log can provide additional information on data that should be flagged manually during the data reduction stage. The Operator logs can be found at [http://www.vla.nrao.edu/cgi-bin/oplogs.cgi Ops Logs form entry], and this is the [http://www.vla.nrao.edu/operators/logs/2016/9/2016-09-14_0620_TSUB0001.pdf link for the log of the observation] (in PDF format).<br />
<br />
=== Inspecting the observation info ===<br />
<br />
We will inspect the content of our data set using CASA task {{listobs}}.<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='MG0414_d1_data.ms')<br />
</source><br />
<br />
The task plots an overview of the data in the CASA logger. Using the parameter <i>listfile=`listobs.txt’</i>, this information is written out to the file <i>listobs.txt</i>: <br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='MG0414_d1_data.ms', listfile='listobs.txt')<br />
</source><br />
<br />
The content is included below:<br />
<br />
<small style="font-size:62%;"><pre><br />
Observer: . Frazer Owen Project: uid://evla/pdb/1695465 <br />
Observation: EVLA<br />
Data records: 991814 Total elapsed time = 9390 seconds<br />
Observed from 14-Sep-2016/06:43:48.0 to 14-Sep-2016/09:20:18.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
14-Sep-2016/06:43:48.0 - 06:48:42.0 7 0 3C48 33861 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
06:56:21.0 - 07:08:39.0 8 1 MG0414+0534 82702 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:08:42.0 - 07:28:36.0 9 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:28:39.0 - 07:48:33.0 10 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:48:36.0 - 08:08:30.0 11 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:10:30.0 - 08:14:30.0 12 0 3C48 26549 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
08:16:30.0 - 08:34:27.0 13 1 MG0414+0534 124918 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:34:30.0 - 08:54:21.0 14 1 MG0414+0534 139347 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:54:24.0 - 09:14:18.0 15 1 MG0414+0534 139048 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
09:16:21.0 - 09:20:18.0 16 0 3C48 26295 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
(nRows = Total number of rows per scan) <br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C48 01:37:41.299431 +33.09.35.13299 J2000 0 86705<br />
1 NONE MG0414+0534 04:14:37.800000 +05.34.41.99999 J2000 1 905109<br />
Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 EVLA_P#A0C0#17 466 TOPO 386.312 15.625 7281.2 389.9453 12 XX XY YX YY<br />
Sources: 2<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 3C48 0 - - <br />
1 MG0414+0534 0 - - <br />
Antennas: 27:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 ea01 W24 25.0 m -107.38.49.0 +33.53.04.0 -2673.3457 -1784.6009 10.4844 -1604008.744400 -5042135.825100 3553403.710800<br />
1 ea02 W56 25.0 m -107.44.26.7 +33.49.54.6 -11333.1991 -7637.6832 15.3636 -1613255.391400 -5042613.097800 3548545.906000<br />
2 ea03 W40 25.0 m -107.41.13.5 +33.51.43.1 -6377.9680 -4286.7773 8.2312 -1607962.451800 -5042338.218100 3551324.962400<br />
3 ea04 E04 25.0 m -107.37.00.8 +33.53.59.7 102.7973 -63.7800 -2.6177 -1601068.806000 -5042051.932700 3554824.838800<br />
4 ea05 E36 25.0 m -107.33.20.2 +33.52.34.3 5761.3733 -2704.6731 33.0133 -1596127.730800 -5045193.742100 3552652.419700<br />
5 ea06 N24 25.0 m -107.37.16.1 +33.55.37.7 -290.3584 2961.8653 -12.2425 -1600930.069900 -5040316.395500 3557330.390000<br />
6 ea07 N16 25.0 m -107.37.10.9 +33.54.48.0 -155.8512 1426.6377 -9.3934 -1601061.954400 -5041175.875300 3556058.026700<br />
7 ea08 W64 25.0 m -107.46.20.1 +33.48.50.9 -14240.7524 -9606.2900 17.0885 -1616361.575500 -5042770.516600 3546911.419900<br />
8 ea09 N40 25.0 m -107.37.29.5 +33.57.44.4 -633.6056 6878.5897 -20.7984 -1600592.749000 -5038121.341300 3560574.826900<br />
9 ea10 E20 25.0 m -107.35.43.6 +33.53.29.9 2082.1759 -987.0087 9.4361 -1599340.800100 -5043150.963000 3554065.231500<br />
10 ea12 N32 25.0 m -107.37.22.0 +33.56.33.6 -441.7248 4689.9674 -16.9328 -1600781.044500 -5039347.439100 3558761.527100<br />
11 ea13 E28 25.0 m -107.34.39.3 +33.53.04.9 3732.7776 -1757.3308 21.4271 -1597899.895900 -5044068.684700 3553432.450200<br />
12 ea14 E08 25.0 m -107.36.48.9 +33.53.55.1 407.8283 -206.0315 -3.2236 -1600801.931400 -5042219.382600 3554706.429400<br />
13 ea15 N64 25.0 m -107.37.58.7 +34.02.20.5 -1382.3871 15410.1326 -40.6450 -1599855.687000 -5033332.368600 3567636.606800<br />
14 ea16 W72 25.0 m -107.48.24.0 +33.47.41.2 -17419.4641 -11760.2694 14.9442 -1619757.299900 -5042937.656400 3545120.392300<br />
15 ea17 N56 25.0 m -107.37.47.9 +34.00.38.4 -1105.2042 12254.2800 -34.2710 -1600128.378100 -5035104.139200 3565024.633900<br />
16 ea18 W48 25.0 m -107.42.44.3 +33.50.52.1 -8707.9181 -5861.7682 15.5302 -1610451.901900 -5042471.123800 3550021.073100<br />
17 ea19 W16 25.0 m -107.37.57.4 +33.53.33.0 -1348.7121 -890.6209 1.2959 -1602592.853500 -5042054.992400 3554140.702800<br />
18 ea20 E16 25.0 m -107.36.09.8 +33.53.40.0 1410.0403 -673.4656 -0.7790 -1599926.104100 -5042772.977200 3554319.801100<br />
19 ea21 E24 25.0 m -107.35.13.4 +33.53.18.1 2858.1906 -1349.1352 13.7186 -1598663.082000 -5043581.391200 3553767.014100<br />
20 ea22 W32 25.0 m -107.39.54.8 +33.52.27.2 -4359.4399 -2923.1314 11.7614 -1605808.634100 -5042230.084000 3552459.197800<br />
21 ea23 E12 25.0 m -107.36.31.7 +33.53.48.5 848.6994 -411.6226 -2.7575 -1600416.518000 -5042462.430500 3554536.041700<br />
22 ea24 W08 25.0 m -107.37.21.6 +33.53.53.0 -432.1080 -272.1514 -1.5061 -1601614.083200 -5042001.656900 3554652.505900<br />
23 ea25 N08 25.0 m -107.37.07.5 +33.54.15.8 -68.9088 433.1829 -5.0674 -1601147.942500 -5041733.833600 3555235.947000<br />
24 ea26 E32 25.0 m -107.34.01.5 +33.52.50.3 4701.6503 -2209.7119 25.1976 -1597053.124400 -5044604.675000 3553058.992700<br />
25 ea27 N48 25.0 m -107.37.38.1 +33.59.06.2 -855.2719 9405.9407 -25.9485 -1600374.881000 -5036704.201700 3562667.858900<br />
26 ea28 N72 25.0 m -107.38.10.5 +34.04.12.2 -1685.6797 18861.8306 -43.5015 -1599557.928700 -5031396.353400 3570494.736800<br />
</pre></small><br />
<br />
<!-- === Hanning smoothing the data ===<br />
<br />
When plotting the data in plotms (plotting amplitude against frequency), it is immediately clear that our observing band contains lots of Radio Frequency Interference (RFI), which is unavoidable at these low frequencies. The situation actually looks worse than it is, because ‘Gibbs ringing’ causes the blending of strong RFI signal into adjacent channels (Fig.1). To reduce the effect of Gibbs ringing, we first Hanning smooth the data (please note that this can take a while to run on the entire 142 Gb data set):<br />
<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='MG0414_d1_data.ms', datacolumn='data', outputvis='MG0414_d1_hanning.ms')<br />
</source><br />
<br />
[[File:Pband_hanning.png|400px|thumb|right|Fig.1 Reducing the effects of Gibbs ringing by Hanning smoothing the data. Shown is only a small fraction of the band, both before and after Hanning smooth. The amplitude scale is the same in both plots.]]<br />
<br />
=== Splitting the data ===<br />
<br />
Upon further inspection in “plotms” (using ea01, field 0, corr xx,yy, and coloraxis baseline), it is clear that there is a lot of RFI across most of the 64 MHz (or 43,000 km/s) band that we used for the observations. Luckily, the region around the expected HI absorption line at 390.6 MHz is relatively clean of RFI. Because our target is strong enough in the continuum to perform a good self-calibration, we select this relatively clean part of the band centered around the expected HI line. Note that there were two IFs with different spectral resolution included in the observations. We need only one of the two IFs, and choose the one with the highest spectral resolution. Moreover, we only need the sources 3C48 (which we will use for bandpass, gain and flux calibration) and our target MG0414+0534, hence we select field=’3,4’. This approach has the additional advantage that we reduce our data volume a lot, bringing it down to about 15 Gb.<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_hanning.ms', outputvis='MG0414_d1_data.ms', datacolumn='data', field='3,4', spw='17:20~485', keepflags=False)<br />
</source><br />
<br />
<font color=red>'''This 15 Gb data set can be downloaded by clicking [https://casa.nrao.edu/Data/EVLA/MG0414/MG0414_d1_data.ms.tgz here].'''</font> The filename should be MG0414_d1_data.ms.tgz (add the ".tgz" if this is missing), and the file can be unpacked by typing <i>tar xzf MG0414_d1_data.ms.tgz</i><br />
--><br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white;"<br />
| style="border: 1px solid darkgray;" | [[File:MG0414_plotants_5.4.0.png|200px|thumb]] || style="border: 1px solid darkgray" | [[File:Ant_logplot_5.4.0.png |200px|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 1a. Plotants plot for <br />MG0414. || Figure 1b. Plotants plot <br />for MG0414 with a log scale.<br />
|}<br />
<br />
Next we look at a graphical plot of the antenna locations and save a hard copy<br />
in case you want it later using CASA task {{plotants}} (see Figures 1a and 1b). This will be useful for selecting a reference antenna. Note that the VLA was in a period of re-configuration when these MG0414 data were taken, so the antenna configuration is not a standard VLA configuration. Typically a good choice is an antenna close to the center of the array (see Figure 1b). Unless it shows problems after inspection of the data, we provisionally choose ea04. (Note: clicking on any image will open up a larger version.)<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis='MG0414_d1_data.ms',figfile='ant_locations.png')<br />
<br />
# In CASA<br />
# If you would like to see another view of the antennas you may plot them on a log plot<br />
plotants(vis='MG0414_d1_data.ms',logpos=True,figfile='ant_logplot.png')<br />
<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
<!-- NOTE: when revising for CASA release 5.3.0, EM wants to revisit how flagging is approached in the guide. Some of the flagging tasks, such as running rflag and flagging out antennas ea09, 14, and 17 should be introduced earlier - perhaps in this section - so that major flagging is done once and then the user can proceed to the self-calibration and imaging section. Tony --><br />
<br />
[[File:Pband_flagchan.png|400px|thumb|right|Figure 2. Flagging of calibrator 3C48. Shown are the XX and YY polarization on antenna ea01, both before (left) and after (right) flagging the bad channels. The colors represent the different baselines with ea01.]]<br />
<br />
We will now perform a more in-depth flagging of the bad data. Our strategy is to first focus on the calibrator source (in this case only 3C48). If the calibrator data are flagged well, we should be able to perform an accurate calibration of the target data. Later in this document we will then flag the calibrated target data using an automated flagger. <br />
<br />
When plotting amplitude vs. frequency in CASA task {{plotms}}, we see that there are still two ranges of channels filled with RFI throughout the observing run (see Figure 2). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',antenna='ea01',xaxis='chan',yaxis='amp',correlation='xx,yy',coloraxis='baseline', scan='7',averagedata=True, avgtime='1e9')<br />
</source><br />
<br />
We will flag this data with the CASA task {{flagdata}}. Because we are flagging channels on all scans of the bandpass calibrator 3C48, these channels cannot be calibrated accurately, so we flag these channels also in the data of the target.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', spw='0:148~155', action='apply')<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', spw='0:51~59', action='apply')<br />
</source><br />
<br />
The data on our bandpass+gain+flux calibrator 3C48 now look clean, with no dead or misbehaving antennas. We can repeat the same {{plotms}} call as above: <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',antenna='ea01',xaxis='chan',yaxis='amp',correlation='xx,yy',coloraxis='baseline', scan='7',averagedata=True, avgtime='1e9')<br />
</source><br />
<br />
We now start with the calibration.<br />
<br />
== Calibration ==<br />
<br />
Many of the steps below are based on the CASA tutorial on [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials#3C129_P-band_continuum_imaging_data_reduction_tutorial reducing VLA P-band continuum data].<br />
<br />
=== <i>A priori</i> Antenna Position Corrections ===<br />
<br />
We start the calibration by obtaining the latest set of antenna-position corrections compared to the stored values that were derived at the start of the observing period. For this you need to run CASA task {{gencal}} with parameter <i>caltype='antpos'</i>. (Be sure to have internet connection in order to grab the antenna position corrections.)<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='antpos', caltable='antpos.cal')<br />
</source><br />
<br />
The output is a list of antenna position corrections that are written to the calibration table <i>antpos.cal</i>:<br />
<br />
<small style="font-size:62%;"><pre><br />
2017-07-25 19:54:51 INFO gencal Determine antenna position offests from the baseline correction database<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea01 : 0.00200 0.00000 0.00200<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea02 : 0.00210 0.01800 0.00430<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea03 : -0.00700 0.01600 -0.02000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea06 : 0.00200 -0.00200 0.00500<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea08 : -0.01290 0.01280 0.02920<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea09 : -0.01200 -0.01300 0.02400<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea15 : -0.00500 -0.01000 0.01100<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea16 : 0.01100 -0.01500 -0.01200<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea17 : -0.01300 0.00000 0.05000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea18 : -0.00600 0.02200 -0.02000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea22 : 0.00600 0.00000 0.00000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea25 : -0.00140 0.00140 -0.00140<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea27 : -0.00400 -0.00500 0.03100<br />
2017-07-25 19:54:52 INFO calibrater Beginning specifycal-----------------------<br />
2017-07-25 19:54:52 INFO Creating KAntPos Jones table from specified parameters.<br />
2017-07-25 19:54:52 WARN NB: This EVLA dataset appears to fall within the period<br />
2017-07-25 19:54:52 WARN + of semester 16B during which the online tropospheric<br />
2017-07-25 19:54:52 WARN + delay model was mis-applied.<br />
2017-07-25 19:54:52 WARN A correction for the online tropospheric delay model error WILL BE APPLIED!<br />
2017-07-25 19:54:52 WARN Marking antpos caltable to turn ON the trop delay correction.<br />
</pre></small><br />
<br />
Note that these observations were taken during a period in which the atmospheric delay terms were calculated incorrectly. In CASA versions 4.7.1 and up a [https://science.nrao.edu/facilities/vla/data-processing/vla-atmospheric-delay-problem correction] for this term is taken into account automatically when running CASA task {{gencal}} with parameter <i>caltype='antpos'</i>.<br />
<br />
=== Ionospheric calibration ===<br />
<br />
At frequencies below 5 GHz, ionospheric effects may need to be corrected. These corrections become important for frequencies < 1 GHz. CASA’s strategy is to obtain information on the total electron content (TEC) for the date of observations, which is based on data from the [https://science.nrao.edu/facilities/vlba/publications/memos/sci/gps_ion/node3 global navigation satellite system (GNSS)]. A series of CASA images in the form of a 24 hour movie of the TEC, as function of longitude and latitude, is then generated and stored as ''filename.IGS_TEC.im'', with a corresponding TEC error movie named ''filename.IGS_RMS_TEC.im'' (these movies can be viewed in CASA’s {{viewer}}. In order to download the data, you need to be connected to the internet:<br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='MG0414_d1_data.ms', doplot=True)<br />
</source><br />
If, for any reason, the file cannot be retrieved from the server automatically, you can also download this specific TEC file for the tutorial [http://casa.nrao.edu/Data/EVLA/MG0414/igsg2580.16i here].(Note: tec_maps is currently not working in CASA versions prior to 5.7.2.)<br />
<br />
<br />
The information regarding the TEC is sparse and either active or direction-dependent ionospheric conditions may not be corrected very well. Also, the online TEC information improves with time, hence the quality of these data is best about two weeks after the observations.<br />
<br />
[[File:Pband_TECIM.png|200px|thumb|right|Figure 3. Total Electron Content (TEC) as functions of time for the day and location of the observations. The time of the observing run is shown in red. We can also see this plot on the log scale, depending on the last plotants inputs.]]<br />
<br />
An ionosphere correction table is subsequently generated using CASA task {{gencal}}, in which the projected line-of-sight TEC (which depends on the zenith angle) is sampled for all times in the observation and stored in a standard CASA caltable. Figure 3 shows a plot that is generated for this caltable and can be opened using xv.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='tecim', caltable='tecim.cal',infile=tec_image)<br />
</source><br />
<br />
See the CASA Cookbook for a [https://casa.nrao.edu/casa_cookbook.pdf#subsection.4.3.9 detailed description] on ionosphere corrections in CASA (thanks are due to Jason Kooi, University of Iowa, for his contributions).<br />
<br />
=== Calibration of requantizer gains ===<br />
<br />
Next we will calibrate the requantizer gain levels, which are the visibility amplitudes that were set as the input of the WIDAR correlator. Requantizer scans are added to a P-band observation to optimize the digital power in each spectral window which, in turn, maximizes the signal-to-noise of each window. Because there is a significant variation in power across the 240 MHz of P-band, and because some spectral windows may experience a high power due to strong RFI, setting the requantizer levels during the observations may improve the quality of the data, including the shape of the bandpass across multiple spectral windows. As part of the test observations of MG0414+0534, the very strong source Cygnus-A was observed at the start of the run. Re-setting the requantizer levels both before observing Cygnus-A and before targeting the other (much weaker) sources was essential to optimize the digital power and avoid correlation errors. Although we do not use Cygnus-A as part of this tutorial, and we use only a single spectral window, performing calibration of the requantizer gains is still good practice for P-band spectral line reduction using CASA task {{gencal}}.<br />
:<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='rq', caltable='rq.cal')<br />
</source><br />
<br />
=== Initial absolute flux density scale calibration ===<br />
<br />
<br />
The next step is to set the absolute flux levels of our flux calibrator 3C48 (field='0' from our {{listobs}} output), for which a well-known model exists using CASA task {{setjy}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
setjy(vis='MG0414_d1_data.ms', field='0', scalebychan=True, standard='Perley-Butler 2017', listmodels=False, usescratch=False)<br />
</source><br />
<br />
It is crucial for spectral line work to set the parameter ''scalebychan=True'' to ensure that the absolute flux level is calculated per channel and correctly interpolated across the observing band. If ''scalebychan=False'', then only a single value per spectral window is calculated, resulting in a step function in flux between spectral windows.<br />
<br />
=== Delay calibration ===<br />
<br />
The delay of each antenna, for each polarization and each spectral window, is now determined. Doing this on a short scan of the primary calibrator is generally sufficient and will correct for both internal (e.g., electronics, cables) and external (e.g., ionosphere) effects. We select ea04 as reference antenna as it produced good data, has baselines to all other used antennas in the array, and is located in the center of the array. We also need to apply, "on-the-fly," the calibration tables for the antenna positions, the total electron content, and the requantizer levels using the CASA task {{gaincal}}. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='delays.cal', field='0', selectdata=True, timerange='06:43:48~06:48:42', solint='inf', refant='ea04', gaintype='K', gaintable=['antpos.cal','tecim.cal','rq.cal'])<br />
</source><br />
<br />
[[File:Pband_delaycal_5.4.0.png|200px|thumb|right|Figure 4. Delays for each antenna and each polarization. Plot made with CASA task {{plotms}}.]]<br />
The calibration solutions can be plotted using CASA task {{plotms}} (see Figure 4):<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='delays.cal',xaxis='antenna1',yaxis='delay',field='0',coloraxis='corr')<br />
</source><br />
<br />
<br />
Note that any delays deviating by more than 30 nsec should be treated with caution and, if needed, flagged.<br />
<br />
=== Bandpass calibration ===<br />
<br />
Next step is to calibrate the bandpass, which corrects for the frequency dependent gain variations. Before determining the frequency dependent gains, we first calibrate the phases in time, deriving a solution for each integration, to make sure that we can derive accurate bandpass solutions when integrating over a full scan on the bandpass calibrator 3C48. This is done using CASA task {{gaincal}}:<br />
<br />
[[File:396px-Pband BPcal.png|200px|thumb|right|Figure 5. Effects of bandpass calibration on the frequency dependent amplitudes (top) and phases (bottom) across the observing band. (For corrections applied using the CASA task {{applycal}} see Figure 8a below.) The various colors show the different baselines with reference antenna ea04. The plots were made in CASA task {{plotms}} and in the ''Axes'' tab choosing the option ''Data Column: data''.]]<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='bpphase.gcal', field='0', spw='0:250~300', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintype='G', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal'])<br />
</source><br />
<br />
We use these phase solutions to perform the actual bandpass calibration. In this case, we have scans of our bandpass calibrator 3C48 taken at three different times during the observation. By interpolating the bandpass solutions in time, we try to minimize any possible time-varying bandpass effects and improve our spectral dynamic range. Bandpass calibration is performed in CASA by the task {{bandpass}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
bandpass(vis='MG0414_d1_data.ms', caltable='bandpass.cal', field='0', spw='0', solint='inf', combine='', refant='ea04', solnorm=False, bandtype='B', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bpphase.gcal'])<br />
</source><br />
<br />
To interpolate between bandpass scans, it is essential to specify parameter <i>combine=' '</i>. The reason being that the default of <i>combine='scan'</i>, when coupled with the parameter <i>solint='inf'</i>, would cross the scan boundaries to form one single solution for the bandpass. Only when combining <i>solint='inf'</i> with <i>combine=' '</i> is a bandpass solution obtained for each individual scan on 3C48. The bandpass plots can be viewed using CASA task {{plotms}} (see Figure 5).<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='amp',scan='7',correlation='xx,yy', coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='phase',scan='7',correlation='xx,yy', coloraxis='baseline')<br />
</source><br />
<br />
=== Gain calibration ===<br />
<br />
After the bandpass calibration, we perform a phase calibration of our data. Because we apply the bandpass solutions "on-the-fly," we can use the full bandwidth to determine the phase solutions. We will create two different calibration tables to correct for the time-varying phases. First we calibrate the phases on each interval with the CASA task {{gaincal}} parameter of ''solint'' set to ''int'': <br />
<br />
<source lang='python'><br />
# In CASA <br />
gaincal(vis='MG0414_d1_data.ms', caltable='intphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
</source><br />
<br />
[[File:NewFig6_plot.png|200px|thumb|right|Figure 6. Effects of phase calibration. (For corrections applied using the CASA task {{applycal}} see Figure 8b below.) The various colors show the different baselines with reference antenna ea04.]]<br />
<br />
The sole purpose of generating phase solutions on each 10 second interval is to be able to accurately correct for the time-varying amplitudes as we will see below. This method, however, is not ideal for interpolating the phase solutions in time. For that we do a second phase calibration, this time averaging the signal across the phase calibrator scans with the CASA task {{gaincal}} parameter of <i>solint='inf'</i>:<br />
<br />
<source lang='python'><br />
# In CASA <br />
gaincal(vis='MG0414_d1_data.ms', caltable='scanphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='inf', combine='', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
</source><br />
<br />
Note that an alternative approach to deriving phase solutions per scan is to use the CASA task {{smoothcal}} to average the phase corrections obtained every 10s with <i>solin='int'</i> over the full duration of a single scan. Generate a phase plot using CASA task {{plotms}} with the parameters below (see Figure 6):<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='phase',coloraxis='baseline')<br />
</source><br />
<br />
=== Scaling the Amplitude Gains ===<br />
<br />
[[File:NewFig7_plot.png|200px|thumb|right|Figure 7. Effects of amplitude and flux density scale calibration. The green points are the YY products on an offending antenna ea17, which we will flag in our target data. (For corrections applied using the CASA task {{applycal}} see Figure 9 below.)]]<br />
<br />
<br />
We now perform a calibration of the amplitude variations in time. For this, we will apply the phase solution obtained in each 10 second interval via CASA task {{gaincal}}: <br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='amp.gcal', field='0', spw='0', selectdata=False, solint='inf', combine='', refant='ea04', calmode='ap', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','intphase.gcal'])<br />
</source><br />
<br />
Normally this procedure will only calibrate the relative amplitudes, and the CASA task {{fluxscale}} is required to place the data on an absolute flux scale. However, because our flux-calibrator is also our bandpass and gain calibrator, the absolute flux scale will already be ok, so we can skip this step. Generate an amplitude gain plot using CASA task {{plotms}} (see Figure 7) with the parameters below:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='amp',coloraxis='baseline')<br />
</source><br />
<br />
=== Applying the calibration ===<br />
<br />
We have now finished the calibration of the data. We will apply the calibration tables that were derived from 3C48 to both the calibrator 3C48 itself and our target MG0414+0354. The CASA task {{applycal}} creates a '''corrected''' data column where the calibrated data are stored.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_data.ms', field='0', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], parang=True, calwt=False, applymode='calflagstrict', flagbackup=True)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_data.ms', field='1', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], calwt=False, applymode='calflagstrict', flagbackup=True)<br />
</source><br />
<br />
Sometimes after calibrating the data it is easier to see any offending antennas or baselines. Replotting with <i>ydatacolumn='corrected'</i> will produce the corrected calibration plots (Figures 8a, 8b). <br />
<br />
<source lang='python'><br />
# In CASA for corrected amplitude and phase vs frequency figure 8a<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='amp',scan='7',ydatacolumn='corrected',correlation='xx,yy', coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='phase',scan='7',ydatacolumn='corrected',correlation='xx,yy', coloraxis='baseline')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA for corrected amplitude and phase vs time figure 8b<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy', xaxis='time',yaxis='amp',ydatacolumn='corrected',coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='phase',ydatacolumn='corrected',coloraxis='baseline')<br />
</source><br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:center;"<br />
|- style="background: white; vertical-align: top;"<br />
| style="border: 1px solid darkgray;" | [[File:Pband_BPcal_corrected.png|thumb]] || style="border: 1px solid darkgray" | [[File:Pband_phase_cal_corrected.png|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 8a. Effects of bandpass <br />calibration on the frequency <br />dependent amplitudes (top) and <br />phases (bottom) across the <br /> observing band. The plots were <br />made in CASA task {{plotms}} and <br />in the ''Axes'' tab choosing option <br />''Data Column: corrected''. || Figure 8b. Effects of phase calibration.<br /> The various colors show the different <br />baselines with reference antenna <br />ea04. The plot was made in CASA task {{plotms}} and in the ''Axes'' tab choosing <br />option ''Data Column: corrected''.<br />
|}<br />
<br />
The calibrated data of the primary calibrator 3C48 show that the YY amplitudes of ea17 are off during the second part of the run (see the green points with low values in figure 7). Because calibration solutions are antenna based, to get the best calibration, flag any poor data and re-run the calibration process. A currently known issue is that CASA task {{gaincal}} will not find good solutions if any correlation, including cross-correlation, in the data is completely flagged. In this case, there is no difference between flagging only the YY polarization of ea17 or the entire antenna.<br />
Antenna ea17 will be flagged using CASA task {{flagdata}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea17', action='apply')<br />
</source><br />
<br />
<br />
<font size="+1">'''Then run the calibration process again starting with the <i>Delay calibration</i>:'''</font> <br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='delays.cal', field='0', selectdata=True, timerange='06:43:48~06:48:42', solint='inf', refant='ea04', gaintype='K', gaintable=['antpos.cal','tecim.cal','rq.cal'])<br />
<br />
gaincal(vis='MG0414_d1_data.ms', caltable='bpphase.gcal', field='0', spw='0:250~300', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintype='G', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal'])<br />
<br />
bandpass(vis='MG0414_d1_data.ms', caltable='bandpass.cal', field='0', spw='0', solint='inf', combine='', refant='ea04', solnorm=False, bandtype='B', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bpphase.gcal'])<br />
<br />
gaincal(vis='MG0414_d1_data.ms', caltable='intphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
<br />
gaincal(vis='MG0414_d1_data.ms', caltable='scanphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='inf', combine='', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
<br />
gaincal(vis='MG0414_d1_data.ms', caltable='amp.gcal', field='0', spw='0', selectdata=False, solint='inf', combine='', refant='ea04', calmode='ap', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','intphase.gcal'])<br />
</source><br />
<br />
The new calibration tables have overwritten the previous tables. Once you've re-run the calibration, you need to apply it. <br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_data.ms', field='0', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], parang=True, calwt=False, applymode='calflagstrict', flagbackup=True)<br />
<br />
applycal(vis='MG0414_d1_data.ms', field='1', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], calwt=False, applymode='calflagstrict', flagbackup=True)<br />
</source><br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white; vertical-align: top;"<br />
| style="border: 1px solid darkgray;" | [[File:Pband_ampcal_corrected_CASA5.5.0.png|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 9. Effects of amplitude and <br />flux density scale calibration. The green points <br />(the YY products of antenna ea17) <br />have been flagged out. The plot was <br />made in CASA task {{plotms}} and in <br />the ''Axes'' tab choosing option <br />''Data Column: corrected''.<br />
|}<br />
<br />
And then, plot the corrected amplitudes vs time with ea17 flagged (see Figure 9):<br />
<br />
<source lang='python'><br />
# In CASA for corrected amp vs time with ea17 flagged out figure 9<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='amp',ydatacolumn='corrected',coloraxis='baseline')<br />
</source><br />
<br />
Having re-run the calibration steps with ea17 flagged, we now split the calibrated data of our target into a new measurement set, including only the XX and YY polarization products. Using CASA task {{split}} the calibrated target data are placed into the '''data''' column of the new measurement set:<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_data.ms', outputvis='MG0414_d1_calibrated.ms', datacolumn='corrected', field='1', correlation='xx,yy', keepflags=False)<br />
</source><br />
<br />
== Imaging and Self-calibration ==<br />
<br />
We now have a calibrated data set that has all calibration applied except for possible self-calibration. Self-calibration relies on the target itself to better calibrate antenna-based gains (phases and amplitudes) as a function of time. It can be applied to targets that are strong in the continuum or have a strong enough spectral line to derive good phase and, if desired, also amplitude solutions on short (order of minutes) timescales. The procedure relies on an initial model of the target, which you then use iteratively to improve the gains and hence the quality of the image itself. Here we show the various steps in self-calibrating our data based on the strong continuum emission of MG0414+0354. We refer to a [https://casaguides.nrao.edu/index.php?title=First_Look_at_Self_Calibration dedicated CASA guide] on self-calibration for further details.<br />
<br />
If strong RFI is present in the target data, it may be necessary to flag your data before proceeding with self-calibration. This is not the case with MG0414+0354.<br />
<br />
We create an initial model of our target field using CASA task {{tclean}}. Noticed that we will exclude from the continuum image the channels that we have flagged and the channels where we expected to detect the line:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_calibrated.ms', datacolumn='data', imagename='MG0414_d1_cont_R03', spw='0:4~50;60~147;156~250;350~460', specmode='mfs', niter=100, threshold='1.5Jy', imsize=[256,256], weighting='briggs', robust=0.3, savemodel='modelcolumn')<br />
</source><br />
<br />
The information in the MODEL column consist of visibilities that present a model of the target field. CASA task {{tclean}} produces a ''.model'' image of the model that you can visualize with {{viewer}} or CASA task {{plotms}}. It is essential that only components that you know for sure represent the continuum in the target field are taken into account in self-calibration. If this is not the case, you probably cleaned too deep and included noise features or artifacts (e.g., from the beam-pattern) in the model. Self-calibration would permanently lock these as features in your data, so this should be avoided. On the other hand, it is equally crucial to ensure that the model is not empty, as this will result in your target being shifted to the phase center.<br />
<br />
Next, we derive phase corrections to this model. Because our target is strong enough to do self-calibration on it, we should get enough S/N per baseline on a 60 sec interval, using CASA task {{gaincal}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_calibrated.ms', caltable='sc1.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='60s', combine='', minsnr=3.0)<br />
</source><br />
<br />
This creates a new phase-calibration table that we call ''sc1.gcal''. We now apply this calibration table to the calibrated data in order to improve the phase-only solutions using CASA task {{applycal}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_calibrated.ms', field='0', gaintable=['sc1.gcal'], interp='linear')<br />
</source><br />
<br />
Because self-calibration is an iterative process, we want to repeat this step again in order to further improve the gain calibration. For that, we first split the calibrated data with the self-calibration solution applied to the DATA column of a new measurement set, again using CASA task {{split}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_calibrated.ms', outputvis='MG0414_d1_sc1.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
Now we can use this newly created measurement set to make a better continuum image and subsequently perform the next round of self-calibration:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc1.ms', datacolumn='data', imagename='MG0414_d1_cont_sc1.R03', spw='0:4~50;60~147;156~250;350~460', specmode='mfs', niter=500, threshold='500mJy', imsize=[256,256], weighting='briggs', robust=0.3, savemodel='modelcolumn')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_sc1.ms', caltable='sc2.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='30s', combine='', minsnr=3.0)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_sc1.ms', field='0', gaintable=['sc2.gcal'], interp='linear')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_sc1.ms', outputvis='MG0414_d1_sc2.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc2.ms', datacolumn='data', imagename='MG0414_d1_cont_sc2_R03', spw='0:4~50;60~147;156~250;350~460', specmode='mfs', niter=1000, threshold='100mJy', imsize=[256,256], weighting='briggs', robust=0.3, savemodel='modelcolumn')<br />
</source><br />
<br />
Now that we have a good model of the continuum in the field of MG0414+0354, we will attempt a final self-calibration using both the phases and amplitudes via CASA task {{gaincal}} :<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_sc2.ms', caltable='sc3ap.gcal', field='0', spw='0', refant='ea04', calmode='ap', solint='30s', combine='', minsnr=3.0, solnorm=True)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_sc2.ms', field='0', gaintable=['sc3ap.gcal'], interp='linear')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_sc2.ms', outputvis='MG0414_d1_sc3ap.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc3ap.ms', datacolumn='data', imagename='MG0414_d1_cont_sc3ap_R03', spw='0:4~50;60~147;156~250;350~460', specmode='mfs', niter=1000, threshold='100mJy', imsize=[256,256], weighting='briggs', robust=0.3, savemodel='modelcolumn')<br />
</source><br />
<br />
The output continuum image from the last iteration of CASA task {{tclean}} continues to be an improvement over previous imaging, but also shows that we are not likely to gain further improvement with additional rounds of self-calibration (you can inspect this yourself by comparing the images created across the different iterations in {{viewer}}). This indicates that we now have a calibrated data-set of MG0414+0354 that is almost ready for final imaging. <br />
<br />
However, before imaging, we need to flag the target data to remove poor-quality data. First, we run an automated flagging routine using CASA task {{flagdata}} with the parameter <i>mode='rflag'</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_sc3ap.ms', mode='rflag', datacolumn='data')<br />
</source><br />
<br />
[[File:MG0414 d1 sc3ap baselines CASA5.5.0.png|right|thumb|400px|Figure 10. Display of time vs. baseline for the MG0414+0354_d1_sc3ap.ms measurement set in {{viewer}}, with the animation axis set to 'channel'.]]<br />
<br />
<br />
After having ran this automated flagging, we inspect the data via the CASA task {{viewer}} by displaying ''time'' vs. ''channel'' for the different baselines, or you may display ''time'' vs. ''baseline'' for the different channels. Keep in mind that when the data are properly calibrated, it is easier to recognize low-level imperfections which may have to be flagged to improve the image quality. In this case, the noisier baselines are seen with lighter colors (see Figure 10). Hovering over these baselines will report a number like 8-16, 8-17, etc. in the {{viewer}}. From this we can tell that baselines 8-# and 12-# (which correspond to antennas ea09 and ea14) have significantly higher noise than the other antennas, something which was not picked up by the automated flagging routine. Additionally, for a number of baselines, the data quality looks relatively poor. We flag these antennas and baselines before imaging the data via the CASA {{flagdata}} task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea09', datacolumn='data')<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea14', datacolumn='data')<br />
</source><br />
<br />
To flag offending individual baselines, in CASA task {{flagdata}} you may use <i>antenna='antenna&antenna(&antenna...)'</i>. <br />
<br />
[[File:Spectral_Profile-MG0414_d1_line_sc3ap_vel_R03_5.5.0.png|200px|thumb|right|Figure 11. Spectral line profile with ea09, ea14, and ea17 flagged out but re-calibration has not been run.]]<br />
<br />
Flagging data can be an iterative process. Often, the better the data are calibrated, the easier it is to recognize low-level imperfections in the data. To optimize the data reduction, you can script the calibration process, and opt to flag the data also in the original data set (i.e., before calibration), after which you run the entire calibration again.<br />
<br />
When you flag these antennas you must go back and rerun the calibration before the final imaging, starting at the '''<i>Delay calibration</i>''' section. If you rerun the calibration you will either have to rename your ms's created from split during self calibration or delete them in order to redo self calibration as split will not overwrite a ms name. If you do NOT re-run the calibration sequence and proceed ahead with the final {{tclean}} run then inspect the line-data cube (see below), you will get a plot that should look similar to Figure 11.<br />
<br />
Finally, performing the calibration sequence again after flagging antennas ea09 and ea14, we image the data using the CASA task {{tclean}}. A 0.5Jy threshold was chosen for this command by interactively cleaning a single channel of the final selfcal image and taking the 5σ of the noise levels once the image was cleaned. With an ideal threshold found, we can now clean all channels (this process can take hours to complete):<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc3ap.ms', datacolumn='data', imagename='MG0414_d1_line_sc3ap_vel_R03', spw='0', specmode='cube', outframe='bary', veltype='optical', restfreq='390.600GHz', niter=10000000, threshold='0.05Jy', imsize=[256,256], pbcor=True, weighting='briggs', robust=0.3, perchanweightdensity=True, savemodel='none')<br />
</source><br />
<br />
We can inspect the resulting line-data cube in the {{viewer}} (Figure 12). The data cube contains both continuum and line emission, which can optionally be separated using the CASA task {{uvcontsub}}. The HI absorption feature is clearly present (see Figure 13).<br />
<br />
To see your spectral line, open the last image of the source. Go to a channel that shows the source, then zoom in on the source. Click the ''Point marking'' icon, find the brightest pixel in the source and click on it. Now click on the ''spectral profile tool'' button to get the single point profile of the spectrum. Here you will see a spectrum with 2 dips that drop to zero; these dips are a part of the flagging we have done. You will need to zoom in in order to notice any spectral lines due to the auto scaling; zoom in by clicking on the graph and creating a yellow box then release.<br />
<br />
<gallery mode="packed" style="text-align:left" heights=375px><br />
<br />
File:Image-MG0414_d1_line_sc3ap_vel_R03_recal_5.5.0.png|Figure 12. Final cleaned image cube of MG0414.<br />
File:MG0414_d1_line_sc3ap_vel_R03_recal_5.7.0.jpg|Figure 13. Test-data of the redshifted HI 21cm absorption line on top of the radio continuum in MG 0414+0354; see More, Carilli & Mention 1999 (Apj, 510, 87) for the original detection. On top: The Single Point Spectral Profile with the auto scaling. Noticed the tip, make a tight horizontal yellow box around the main signal, like shown in figure. On Bottom: The same after we zoom it. <br />
<br />
</gallery><br />
<br />
Alternative Tip: You can click in the save icon (top left) in the Spectral Profile tool and export the spectral profile as a text file. And then, you can use any program you find appropriate to make the plot. You can see small differences in the continuum flux level, depending on the pixel you have selected to see the line profile. However, you should be able to identify the line in any of the bright central pixels.<br />
<br />
<!-- --[[User:bemonts|Bjorn Emonts]] <br />
-- 5.0 corrections: [[User:kradford| Karlee Radford]] (2018-03-21)<br />
-- Edits [[User:tperreau| Tony Perreault]] (26 Apr 2018)<br />
-- Edits [[User:Hmedlin | Heidi Medlin]] (27 April 2018)<br />
-- Yet more edits [[User:tperreau| Tony Perreault]] (2 May 2018)<br />
-- And yet more edits [[User:tperreau| Tony Perreault]] (May the 4th be with you 2018)<br />
-- 5.4.0 updates: [User:kradford| Karlee Radford]] (2018-10-11)<br />
-- 5.5.0 updates: [User:asobotka| Alex Sobotka]] (3 July 2019)<br />
-- 5.7.0 updates: [User:pbeaklin| Pedro P.B. Beaklini]] (29 September 2020)<br />
-- Changed last figure: [User:pbeaklin| Pedro P.B. Beaklini]] (7 October 2020)<br />
-- 5.7.2 updates [User:pbeaklin| Pedro P.B. Beaklini]]. Cheking tec_map (21 December 2020) <br />
--><br />
<br />
{{Checked 5.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=29804Karl G. Jansky VLA Tutorials2020-11-10T16:56:33Z<p>Emomjian: </p>
<hr />
<div><br />
Disclaimer: These tutorials provide guidelines to help users become familiar with VLA data reduction in [https://casa.nrao.edu/ CASA]. They offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results demonstrated here are specific to the given data sets. The VLA is an extremely flexible instrument, and the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
''For data reduction of pre-EVLA upgrade VLA data, we provide separate, [https://casaguides.nrao.edu/index.php/Pre-upgrade_VLA_Tutorials pre-upgrade VLA data reduction tutorials]''<br />
<br />
''In addition to the current VLA CASA Guides below, we also provide links to [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials-archived older versions of the VLA CASA guides].''<br />
<br />
<br />
<!--<br />
--><br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== High frequency (36GHz), spectral line data reduction: Carbon Star IRC+10216 ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.7.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 6cm Continuum Imaging, Mosaicking: Supernova Remnant 3C391 ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.7.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA continuum data, image a mosaic of the region in Stokes I. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Polarization Calibration based on CASA pipeline standard reduction: The radio galaxy 3C75 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php?title=CASA_Guides:Polarization_Calibration_based_on_CASA_pipeline_standard_reduction:_The_radio_galaxy_3C75-CASA5.6.2 Polarization Calibration based on CASA pipeline (3C75)] (CASA 5.6.2) <br />
<br />
* This tutorial requires to download a dataset <font color=red>10GB</font> in size<br />
<br />
This tutorial demonstrates continuum calibration with the standard VLA pipeline and focuses on linear polarization calibration based on pipeline products, as well as full Stokes wide-field imaging and analysis. It includes:<br />
* Instructions on how to execute the VLA pipeline<br />
* How to modify pipeline products for polarization calibration<br />
* Linear Polarization Calibration<br />
* Wide-field Polarization Imaging<br />
* Self-calibration<br />
* Analysis of Polarization Products<br />
</blockquote><br />
<br />
=== P-band continuum imaging data reduction tutorial: 3C129 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.7.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== MG0414+0534 P-band Spectral Line Tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.7.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15GB</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
=== HI 21 cm Spectral Line Data Reduction Tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/HI_21cm_(1.4_GHz)_spectral_line_data_reduction:_LEDA_44055-CASA5.7.0 HI 21cm Spectral Line (CASA 5.7.0)]<br />
<br />
* This tutorial is based on an archival data-set of 85 Gb in size. <br />
<br />
This HI 21cm spectral line tutorial is aimed at calibrating and making an image cube of the dwarf galaxy LEDA4 4055. It includes:<br />
* Finding the HI 21 cm spectral line in LEDA 44055<br />
* Basic flagging <br />
* Splitting out the HI Line<br />
* Spectral-line imaging<br />
* Continuum Subtraction<br />
<br />
</blockquote><br />
<br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.7.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags <br />
* Shadowing, zeroes, quacking, flagmanager, Hanning smoothing<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging (plotms, msview)<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.7.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
=== Self-calibration of VLA Data ===<br />
<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Selfcalibration_Tutorial VLA Self-calibration Tutorial] (CASA 5.7.0)<br />
* This tutorial requires to download a dataset <font color=red>19GB</font> in size<br />
<br />
This tutorial demonstrates how to perform self-calibration on a VLA data set, covering aspects such as how to choose the time interval, signal-to-noise ratio, and combination over spectral windows and polarizations.<br />
</blockquote><br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.7.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.7.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Simulation Guide ==<br />
<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/Protoplanetary_Disk_Simulation_-_VLA VLA Protoplanetary Disk Simulation Guide] (CASA 5.7.0)<br />
* This tutorial does not require any download. <br />
<br />
The VLA Protoplanetary Disk Simulation Guide shows examples on how to simulate VLA data based on model images. The tutorial will explain the main simulation tools, apply different parameters to simulations, and compares the results to the VLA exposure calculator. <br />
</blockquote><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 5.6.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots, tables, and Quality Assurance (QA) metrics. <br />
</blockquote></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLBI_Tutorials&diff=28195VLBI Tutorials2019-10-21T16:23:39Z<p>Emomjian: </p>
<hr />
<div><br />
== Tutorials for Very Long Baseline Interferometry ==<br />
<br />
CASA is offering increased VLBI support through a collaboration with the Joint Institute for VLBI ERIC (JIVE). The below VLBI CASA tutorials introduce the CASA environment and touch upon the visualization, calibration and imaging capabilities available, along with the scripting methods so that you can create your own calibration scripts for your own data.<br />
<br />
VLBI processing in CASA has been extensively tested with EVN and VLBA continuum data. Other use cases and telescopes are still being verified. Feedback on results (also successful ones) and missing or optimizing functionality is very welcome at casa_feedback_at_nrao.edu.<br />
<br />
For the Event Horizon Telescope (EHT) project, one of three certified pipelines is based on CASA. This 'rPicard' pipeline has since evolved into a generic all-purpose pipeline for VLBI processing. While rPicard is not supported by the CASA team, users can find more information in [https://arxiv.org/abs/1905.01905 this paper].<br />
<br />
<br />
== <font size = 3> '''CASA tutorials''' ==<br />
<br />
<font size = 2><br />
CASA tutorial: [http://www.jb.man.ac.uk/DARA/unit4/Workshops/EVN_continuum.html ENV Continuum Imaging & Calibration]<br />
<br />
CASA tutorial: [http://www.jb.man.ac.uk/DARA/unit4/Workshops/Spectral_line.html EVN HI Spectral Line]<br />
<br />
Credits: These CASA tutorials represent publicly available materials developed by scientists from the University of Manchester, the Joint Institute for VLBI ERIC (JIVE), and the University of Pretoria/SARAO through funding from Development in Africa with Radio Astronomy (DARA). The materials are based upon a workshop developed by Anita Richards (University of Manchester) in 2016. The subsequent update has been conducted by Jack Radcliffe (SARAO/ University of Pretoria), which has been influenced by Des Small's (JIVE) [http://jive.nl/~small/FringeFitting/n14c3_tutorial.html EVN CASA tutorial], based on AIPS material provided by Minnie Mao. When starting with archival EVN observations, this [http://jive.nl/~small/FringeFitting/n14c3_tutorial.html EVN CASA tutorial] explains the required steps for converting the data into a Measurement Set (in the EVN Continuum tutorial these steps are not complete). Thanks to Ilse van Bemmel (JIVE) for making these materials available for the CASA Guides.<br />
<br />
<br />
== <font size = 3> '''AIPS tutorials''' ==<br />
<br />
<font size = 2><br />
While implementation of VLBI data processing is an ongoing effort in CASA, we also include below two AIPS tutorials on this topic.<br />
<br />
AIPS tutorial: [https://casaguides.nrao.edu/index.php/AIPS-Simple-Self-Cal Simple Self-Calibration]<br />
<br />
AIPS tutorial: [https://casaguides.nrao.edu/index.php/AIPS-Spectral_Lines_and_Astrometry Spectral Lines and Astrometry]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=27936Karl G. Jansky VLA Tutorials2019-07-12T16:10:44Z<p>Emomjian: </p>
<hr />
<div><br />
Disclaimer: These tutorials provide guidelines to help users become familiar with VLA data reduction in [https://casa.nrao.edu/ CASA]. They offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results demonstrated here are specific to the given data sets. The VLA is an extremely flexible instrument, and the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
''For data reduction of pre-EVLA upgrade VLA data, we provide separate, [https://casaguides.nrao.edu/index.php/Pre-upgrade_VLA_Tutorials pre-upgrade VLA data reduction tutorials]''<br />
<br />
''In addition to the current VLA CASA Guides below, we also provide links to [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials-archived older versions of the VLA CASA guides].''<br />
<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== High frequency (36GHz), spectral line data reduction: Carbon Star IRC+10216 ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.5.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 6cm Continuum Imaging, Mosaicking: Supernova Remnant 3C391 ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.4.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA continuum data, image a mosaic of the region in Stokes I. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== P-band continuum imaging data reduction tutorial: 3C129 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.4.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.5.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.5.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags <br />
* Shadowing, zeroes, quacking, flagmanager, Hanning smoothing<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging (plotms, msview)<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.5.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.5.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.5.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Simulation Guide ==<br />
<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/Protoplanetary_Disk_Simulation_-_VLA VLA Protoplanetary Disk Simulation Guide] (CASA 5.5.0)<br />
* This tutorial does not require any download. <br />
<br />
The VLA Protoplanetary Disk Simulation Guide shows examples on how to simulate VLA data based on model images. The tutorial will explain the main simulation tools, apply different parameters to simulations, and compares the results to the VLA exposure calculator. <br />
</blockquote><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 5.4.1 / 5.4.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots, tables, and Quality Assurance (QA) metrics. <br />
</blockquote></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Flagging-CASA5.5.0&diff=27875VLA CASA Flagging-CASA5.5.02019-07-05T20:50:56Z<p>Emomjian: </p>
<hr />
<div>* '''This CASA guide is designed for CASA 5.5.0''' <br />
* Printing Disclaimer: When printing, CASA commands in this guide may be cropped depending on your web browser text size and printer settings.<br />
<br />
<br />
== Overview ==<br />
<br />
<div style="text-align: justify;"><br />
This CASA guide covers online data flagging, shadowing, zero-clipping, and quacking. It also covers auto-flagging RFI (Radio Frequency Interference) via TFCrop (Time-Frequency Crop) and rflag.<br />
<br />
We will be utilizing data taken, with the Karl G. Jansky Very Large Array, of [http://simbad.u-strasbg.fr/simbad/sim-id?Ident=SNR+G055.7%2B03.4&NbIdent=1&Radius=2&Radius.unit=arcmin&submit=submit+id G055.7+3.4.], which is a supernova remnant. The data were taken on August 23, 2010 in the first D-configuration for which the new wideband capabilities of the WIDAR (Wideband Interferometric Digital ARchitecture) correlator were available. The 8-hour long session included all available 1 GHz of bandwidth in L-band, from 1&ndash;2 GHz in frequency., although in this guide we will use dataset composed of only 4 spectral windows (128 MHz bandwidth each). <br />
<br />
The guide will often reference the CASA documentation archives which are available [https://casa.nrao.edu/casadocs/casa-5.5.0 online].<br />
<br />
<br />
== Obtaining the Data ==<br />
<br />
A copy of the data can be downloaded from [http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.tar.gz http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.tar.gz] <font color=green>(5.0 GB)</font>. These data are already in the CASA Measurement Set (MS) format and were prepared specifically for this guide. <br />
<br />
The following explains how the data were processed to achieve the above MS file, as it has implications for later flagging steps. <br />
<br />
Note that commands with a grey background do not need to be run in CASA.<br />
<br />
The data were first loaded from the [http://archive.nrao.edu NRAO archive]. On the archive search form, we specified '''Archive File ID''' to be ''AB1345_sb1800808_1.55431.004049953706'' and selected '''SDM-BDF dataset (all files)''' as the data download option on the following page. Be aware that the full dataset is 170GB!<br />
<br />
Once the SDM-BDF was downloaded, we created an MS with the task {{importasdm}}:<br />
<br />
''importasdm(asdm='AB1345_sb1800808_1.55431.004049953706', vis='SNR_G55.ms', process_flags=True, tbuff=1.5, applyflags=False, <br />
ocorr_mode='co', savecmds=True, outfile='SNR_G55.ms.onlineflags.txt', flagbackup=False)''<br />
<br />
process_flags=True: This parameter controls the creation of online flags from the Flag.xml SDM table.<br />
It will create online flags in the FLAG_CMD sub-table within the MS (more on this later).<br /><br />
tbuff=1.5: This parameter adds a time buffer padding to the flags in both directions to deal with timing mismatches.<br />
This is important for VLA data taken before April 2011. This value should be set to 1.5x integration time.<br />
This particular observing session had 1 second correlator integration time.<br /><br />
applyflags=False: We will apply these flags later in the tutorial.<br /><br />
ocorr_mode ='co': Only choosing to import the cross-only (co) correlations.<br /><br />
savecmds=True: Save the online-flag commands to an ASCII file.<br /><br />
outfile='SNR_G55.ms.onlineflags.txt': This will create a text file with a list of online flags that can be applied to the data.<br />
The online flags within the file will include the time buffer requested in the ''tbuff'' parameter. <br />
Mainly created for demonstration purposes.<br />
<br />
The data were split from the original MS and time-averaged using the CASA task {{split}}. <br />
<br />
''split(vis='SNR_G55.ms', outputvis='SNR_G55_10s.ms', field='3~5', spw='4~5,7~8', timebin='10s', <br />
antenna='!ea06,ea17,ea20,ea26', datacolumn='data', keepflags=False)''<br />
<br />
field='3~5': We only include the complex gain calibrator, flux density/bandpass calibrator, and the target source at this stage.<br /><br />
spw='4~5,7~8': Several spectral windows have been removed which were heavily impacted with RFI, which neither auto-flagging nor<br />
hand flagging could remedy. We are keeping spectral windows 4,5,7,8.<br /><br />
timebin='10s': Time-averaged to 10-seconds, to reduce size and processing time when running tasks. <br /><br />
antenna='!ea06,ea17,ea20,ea26': Several antennas have been removed, which at the time of the observations, did not have an L-Band<br />
receiver installed (please see the [[#Identifying_Problematic_Antennas_from_the_Operator_Logs | Identifying Problematic Antennas from the Operator Logs]] section).<br />
The exclamation point (!) before ea06 is called a negation operator. It will exclude all the antennas<br />
provided here, as long as they are separated by commas. For more on CASA syntax, please see this page<br />
on [https://casa.nrao.edu/casadocs/casa-5.5.0/calibration-and-visibility-data/data-selection-in-a-measurementset].<br />
<br />
== Unpacking the Data == <br />
<br />
Once you've downloaded the MS file from the link above, unzip and untar the file from a terminal window (before starting CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf SNR_G55_10s.tar.gz<br />
</source><br />
<br />
This will create a directory called "SNR_G55_10s.ms" <font color=green>(5.5GB)</font>, which is the MS.<br />
<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing '''casa''' on a terminal command line. If you have not used CASA before, some helpful tips are available on the [[Getting Started in CASA]] page.<br />
<br />
This guide has been written for CASA version 5.5.0. Please confirm your version before proceeding by checking the message in the command line interface window or the CASA logger after startup.<br />
<br />
For this tutorial, we will be running tasks using the ''task (parameter = value)'' syntax. All parameters not explicitly called by this manner will use their default values.<br />
</div><br />
<br />
== Preliminary Data Evaluation == <br />
<br />
As a first step, use {{listobs}} to have a look at the MS:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='SNR_G55_10s.ms', listfile='SNR_G55_10s.listobs')<br />
</source><br />
<br />
*listfile: Creates a text document ''SNR_G55_10s.listobs'' with the summary of the data set. <br />
<br />
The ''SNR_G55_10s.listobs'' text document can be opened with your favorite text viewer.<br />
<pre style="white-space: pre-wrap;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/akapinsk/CASAguides-5.5.0/flagging/SNR_G55_10s.ms MS Version 2<br />
================================================================================<br />
Observer: Dr. Sanjay Sanjay Bhatnagar Project: uid://evla/pdb/1072564 <br />
Observation: EVLA<br />
Data records: 2732400 Total elapsed time = 26926 seconds<br />
Observed from 23-Aug-2010/00:56:36.0 to 23-Aug-2010/08:25:22.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
23-Aug-2010/00:56:36.0 - 00:58:06.0 14 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
00:58:06.0 - 00:59:36.0 15 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
00:59:36.0 - 01:01:05.0 16 0 J1925+2106 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:01:05.0 - 01:02:35.0 17 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:02:35.0 - 01:04:05.0 18 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:04:05.0 - 01:05:34.0 19 0 J1925+2106 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:05:34.0 - 01:07:04.0 20 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:07:04.0 - 01:08:34.0 21 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:08:34.0 - 01:10:04.0 22 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:10:04.0 - 01:11:34.0 23 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:11:34.0 - 01:13:03.0 24 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:13:03.0 - 01:14:33.0 25 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:14:33.0 - 01:16:03.0 26 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:16:03.0 - 01:17:33.0 27 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:17:33.0 - 01:19:02.0 28 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:19:02.0 - 01:20:32.0 29 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:20:32.0 - 01:22:02.0 30 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:22:02.0 - 01:23:32.0 31 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:23:32.0 - 01:25:01.0 32 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:25:01.0 - 01:26:31.0 33 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:26:31.0 - 01:28:01.0 34 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:28:01.0 - 01:29:31.0 35 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:29:31.0 - 01:31:00.0 36 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:31:00.0 - 01:32:30.0 37 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:32:30.0 - 01:34:00.0 38 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:34:00.0 - 01:35:30.0 39 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:35:30.0 - 01:36:59.0 40 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:36:59.0 - 01:38:29.0 41 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:38:29.0 - 01:39:59.0 42 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:39:59.0 - 01:41:29.0 43 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:41:29.0 - 01:42:58.0 44 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
[....]<br />
08:11:54.0 - 08:13:24.0 305 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:13:24.0 - 08:14:54.0 306 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:14:54.0 - 08:16:24.0 307 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:16:24.0 - 08:17:54.0 308 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:17:54.0 - 08:19:23.0 309 2 0542+498=3C147 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:19:23.0 - 08:20:53.0 310 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:20:53.0 - 08:22:23.0 311 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:22:23.0 - 08:23:52.0 312 2 0542+498=3C147 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:23:52.0 - 08:25:22.0 313 2 0542+498=3C147 9108 [0,1,2,3] [10,10,10,10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
(nRows = Total number of rows per scan) <br />
Fields: 3<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 D J1925+2106 19:25:59.605371 +21.06.26.16218 J2000 0 391644<br />
1 NONE G55.7+3.4 19:21:40.000000 +21.45.00.00000 J2000 1 2277000<br />
2 N 0542+498=3C147 05:42:36.137916 +49.51.07.23356 J2000 2 63756<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 Subband:0 64 TOPO 1256.000 2000.000 128000.0 1319.0000 4 RR RL LR LL<br />
1 Subband:2 64 TOPO 1384.000 2000.000 128000.0 1447.0000 4 RR RL LR LL<br />
2 Subband:1 64 TOPO 1648.000 2000.000 128000.0 1711.0000 8 RR RL LR LL<br />
3 Subband:0 64 TOPO 1776.000 2000.000 128000.0 1839.0000 8 RR RL LR LL<br />
Sources: 12<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1925+2106 0 - - <br />
0 J1925+2106 1 - - <br />
0 J1925+2106 2 - - <br />
0 J1925+2106 3 - - <br />
1 G55.7+3.4 0 - - <br />
1 G55.7+3.4 1 - - <br />
1 G55.7+3.4 2 - - <br />
1 G55.7+3.4 3 - - <br />
2 0542+498=3C147 0 - - <br />
2 0542+498=3C147 1 - - <br />
2 0542+498=3C147 2 - - <br />
2 0542+498=3C147 3 - - <br />
Antennas: 23:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 ea01 W09 25.0 m -107.37.25.2 +33.53.51.0 -521.9416 -332.7766 -1.2001 -1601710.017000 -5042006.925200 3554602.355600<br />
1 ea02 E02 25.0 m -107.37.04.4 +33.54.01.1 9.8240 -20.4293 -2.7806 -1601150.060300 -5042000.619800 3554860.729400<br />
2 ea03 E09 25.0 m -107.36.45.1 +33.53.53.6 506.0564 -251.8670 -3.5825 -1600715.950800 -5042273.187000 3554668.184500<br />
3 ea04 W01 25.0 m -107.37.05.9 +33.54.00.5 -27.3562 -41.3030 -2.7418 -1601189.030140 -5042000.493300 3554843.425700<br />
4 ea05 W08 25.0 m -107.37.21.6 +33.53.53.0 -432.1167 -272.1478 -1.5054 -1601614.091000 -5042001.652900 3554652.509300<br />
5 ea07 E05 25.0 m -107.36.58.4 +33.53.58.8 164.9788 -92.8032 -2.5268 -1601014.462000 -5042086.252000 3554800.799800<br />
6 ea08 N01 25.0 m -107.37.06.0 +33.54.01.8 -30.8810 -1.4664 -2.8597 -1601185.634945 -5041978.156586 3554876.424700<br />
7 ea09 E06 25.0 m -107.36.55.6 +33.53.57.7 236.9058 -126.3369 -2.4443 -1600951.588000 -5042125.911000 3554773.012300<br />
8 ea10 N03 25.0 m -107.37.06.3 +33.54.04.8 -39.0773 93.0192 -3.3330 -1601177.376760 -5041925.073200 3554954.584100<br />
9 ea11 E04 25.0 m -107.37.00.8 +33.53.59.7 102.8054 -63.7682 -2.6414 -1601068.790300 -5042051.910200 3554824.835300<br />
10 ea12 E08 25.0 m -107.36.48.9 +33.53.55.1 407.8285 -206.0065 -3.2272 -1600801.926000 -5042219.366500 3554706.448200<br />
11 ea13 N07 25.0 m -107.37.07.2 +33.54.12.9 -61.1037 344.2331 -4.6138 -1601155.635800 -5041783.843800 3555162.374100<br />
12 ea15 W06 25.0 m -107.37.15.6 +33.53.56.4 -275.8288 -166.7451 -2.0590 -1601447.198000 -5041992.502500 3554739.687600<br />
13 ea16 W02 25.0 m -107.37.07.5 +33.54.00.9 -67.9687 -26.5614 -2.7175 -1601225.255200 -5041980.383590 3554855.675000<br />
14 ea18 N09 25.0 m -107.37.07.8 +33.54.19.0 -77.4346 530.6273 -5.5859 -1601139.485100 -5041679.036800 3555316.533200<br />
15 ea19 W04 25.0 m -107.37.10.8 +33.53.59.1 -152.8599 -83.8054 -2.4614 -1601315.893000 -5041985.320170 3554808.304600<br />
16 ea21 E01 25.0 m -107.37.05.7 +33.53.59.2 -23.8638 -81.1510 -2.5851 -1601192.467800 -5042022.856800 3554810.438800<br />
17 ea22 N04 25.0 m -107.37.06.5 +33.54.06.1 -42.6239 132.8436 -3.5494 -1601173.979400 -5041902.657700 3554987.517500<br />
18 ea23 E07 25.0 m -107.36.52.4 +33.53.56.5 318.0509 -164.1850 -2.6957 -1600880.571400 -5042170.388000 3554741.457400<br />
19 ea24 W05 25.0 m -107.37.13.0 +33.53.57.8 -210.0959 -122.3887 -2.2577 -1601377.009500 -5041988.665500 3554776.393400<br />
20 ea25 N02 25.0 m -107.37.06.2 +33.54.03.5 -35.6245 53.1806 -3.1345 -1601180.861480 -5041947.453400 3554921.628700<br />
21 ea27 E03 25.0 m -107.37.02.8 +33.54.00.5 50.6641 -39.4835 -2.7273 -1601114.365500 -5042023.151800 3554844.944000<br />
22 ea28 N08 25.0 m -107.37.07.5 +33.54.15.8 -68.9057 433.1889 -5.0602 -1601147.940400 -5041733.837000 3555235.956000<br />
</pre><br />
<br />
<div style="text-align: justify;"><br />
* J1925+2106, field ID 0: Phase (complex gain (amplitude and phase)) calibrator;<br />
* G55.7+3.4, field ID 1: The supernova remnant;<br />
* 0542+498=3C147, field ID 2: Flux density/bandpass calibrator.<br />
[[Image:plotAnts.png|300px|thumb|right|'''Figure 1''' <br />Antenna plot showing the configuration during the observing session.]]<br />
<br />
We can also see that these sources have associated scan intents that indicate their function in the observations. Note that you can select sources based on their intents in certain CASA tasks. The various scan intents in this data set are:<br />
<br />
* CALIBRATE_PHASE indicates that this is a scan to be used for complex gain (amplitude and phase) calibration;<br />
* OBSERVE_TARGET indicates that this is the science target;<br />
* CALIBRATE_AMPLI indicates that this is to be used for flux density calibration, and; <br />
* CALIBRATE_BANDPASS indicates that these scans are to be used for bandpass calibration.<br />
<br />
Note that more recent data sets may have different scan intents assigned to calibration scans.<br />
<!-- Also note that 3C147 is used for both flux density and bandpass calibration.--><br />
<br />
It's important to be aware that the antennas have both a name and an ID associated with them. For example antenna ID 15 is named ea19 (the ''ea'' stemming from the Expanded VLA project). When specifying an antenna within a task parameter, we will primarily reference them by their ''ea'' name. <br />
<br />
We can see the antenna configuration for this observing session by using {{plotants}}. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis='SNR_G55_10s.ms', figfile='SNR_G55_10s.plotants.png')<br />
</source><br />
<br />
Figure 1 the antenna configuration for our observations; antennas ea01, ea03, and ea18 were on the extreme ends of the west, east, and north arms, respectively. <br />
<br />
<!-- This is really not for this guide, so commented out for the time being. ADK, Oct 2018--><br />
<!-- The antenna position diagram is particularly useful as a guide to help determine which antenna to use as the reference antenna later during calibration. Antennas on station 8 of each arm (N08, E08, W08) do not get moved during array re-configurations; they can, therefore, at times be good choices as reference antennas. In this case, we'll probably want to choose something closer to the center of the array with no shadowing.--><br />
<br />
<br />
=== Identifying RFI with Plotms() ===<br />
[[Image:CASA5.4.0-FLAG-Plotms-amp-time.png|300px|thumb|right|'''Figure 2''' <br />Plotms image of supernova remnant data, showing strong RFI through the observation (x axis) and affecting a number of spectral windows (data are colorised by spw).]]<br />
<br />
The data can be inspected with the {{plotms}} task. <br />
<br />
It is worth first having a look at the data, separately per each field, in <b>amplitude vs. frequency</b> (to identify narrow-band RFI) and <b>amplitude vs. time</b> (intermittent RFI). <br />
<br />
To start with, let's look at the supernova remnant observations (field='1') for all scans (default) and all antennas (default):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', field='1', xaxis='time', yaxis='amp', coloraxis='spw')<br />
</source><br />
<br />
We can immediately see that there is plenty of RFI throughout the observations, affecting a number of spectral windows (data are colorised by spectral window; Figure 2). The RFI is discerned by the large spikes in amplitude.<br />
<br />
To narrow down where the RFI occurs let's now have a look at the data in amplitude vs frequency, plotted separately per each scan:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', scan='30,75,120,165,190,235,303', antenna='ea24', xaxis='freq', <br />
yaxis='amp', coloraxis='spw', iteraxis='scan')<br />
</source><br />
[[Image:amp_v_freq_rawdata.png|300px|thumb|right|'''Figure 3''' <br />Plotms image of scan 190 showing strong RFI spikes in amplitude for several spectral windows.]]<br />
* coloraxis='spw': Parameter indicates that a different color will be assigned to each spectral window.<br />
* antenna='ea24' : Here we chose only information for antenna ea24.<br />
* iteraxis='scan': Parameter tells plotms to iterate over scans, that is to display a new plot for each scan. <br />
<br />
The command above will set the {{plotms}} for iteration, but if you want to set this option from within {{plotms}}, go onto the Plot tab (horizontal), and then Page tab (vertical). You will now find Iteration options. Set Axis to the parameter you want to iterate through. Iterate by clicking on the green arrow buttons on the toolbar within the {{plotms}}. <br />
<br />
Progressing through to scan 190 , we can see there is significant time and frequency variable RFI present in the observing session. To determine which spectral windows are affected, click on the <b>Mark Regions</b> tool at the bottom of the {{plotms}} GUI (the open box with a green plus sign) and use the mouse to select a few of the highest-amplitude points in each of the spectral windows (Figure 3). Click on the <b>Locate</b> button (magnifying glass on white background). Information about the selected areas should now display in the logger window. For example:<br />
<br /><br />
</div><br />
<pre><br />
INFO PlotMS Frequency in [1.30517 1.31368] or [1.32584 1.33313] or [1.68146 1.69544], Amp in [0.132051 0.223077] or [0.124359 0.188034] or [0.139316 0.237179]:<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea01@W09 & ea24@W05[0&19] Spw=0 Chan=27 Freq=1.31 Corr=LR X=1.31 Y=0.139029 (110/144/110)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea03@E09 & ea24@W05[2&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.145391 (623/144/623)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea04@W01 & ea24@W05[3&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.166033 (879/144/879)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea04@W01 & ea24@W05[3&19] Spw=0 Chan=37 Freq=1.33 Corr=LR X=1.33 Y=0.131179 (918/144/918)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea07@E05 & ea24@W05[5&19] Spw=0 Chan=27 Freq=1.31 Corr=RL X=1.31 Y=0.14596 (1389/144/1389)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea07@E05 & ea24@W05[5&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.142578 (1391/144/1391)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea09@E06 & ea24@W05[7&19] Spw=0 Chan=27 Freq=1.31 Corr=RR X=1.31 Y=0.141398 (1900/144/1900)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea12@E08 & ea24@W05[10&19] Spw=0 Chan=37 Freq=1.33 Corr=LR X=1.33 Y=0.133163 (2710/144/2710)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea13@N07 & ea24@W05[11&19] Spw=0 Chan=37 Freq=1.33 Corr=RR X=1.33 Y=0.129079 (2964/144/2964)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea15@W06 & ea24@W05[12&19] Spw=0 Chan=27 Freq=1.31 Corr=RR X=1.31 Y=0.135901 (3180/144/3180)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea16@W02 & ea24@W05[13&19] Spw=0 Chan=27 Freq=1.31 Corr=RR X=1.31 Y=0.17187 (3436/144/3436)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea16@W02 & ea24@W05[13&19] Spw=0 Chan=27 Freq=1.31 Corr=LR X=1.31 Y=0.155113 (3438/144/3438)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea16@W02 & ea24@W05[13&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.142598 (3439/144/3439)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea19@W04 & ea24@W05[15&19] Spw=0 Chan=27 Freq=1.31 Corr=LR X=1.31 Y=0.154241 (3950/144/3950)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea19@W04 & ea24@W05[15&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.178607 (3951/144/3951)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea19@W04 & ea24@W05[15&19] Spw=0 Chan=37 Freq=1.33 Corr=RR X=1.33 Y=0.126879 (3988/144/3988)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea21@E01 & ea24@W05[16&19] Spw=0 Chan=27 Freq=1.31 Corr=RL X=1.31 Y=0.144868 (4205/144/4205)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea21@E01 & ea24@W05[16&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.158044 (4207/144/4207)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea22@N04 & ea24@W05[17&19] Spw=0 Chan=37 Freq=1.33 Corr=RR X=1.33 Y=0.18105 (4500/144/4500)<br />
<snip><br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:08.0 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RR X=1.686 Y=0.195303(10060/169/4428)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:08.0 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RL X=1.686 Y=0.19185 (10061/169/4429)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:08.0 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.169598 (10068/169/4436)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea08@N01 & ea24@W05[6&19] Spw=2 Chan=19 Freq=1.686 Corr=LR X=1.686 Y=0.157531(7246/170/1614)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea10@N03 & ea24@W05[8&19] Spw=2 Chan=19 Freq=1.686 Corr=LL X=1.686 Y=0.15168 (7759/170/2127)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea15@W06 & ea24@W05[12&19] Spw=2 Chan=19 Freq=1.686 Corr=LR X=1.686 Y=0.202071(8782/170/3150)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea18@N09 & ea24@W05[14&19] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.159526 (9300/170/3668)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea19@W04 & ea24@W05[15&19] Spw=2 Chan=19 Freq=1.686 Corr=RR X=1.686 Y=0.192973(9548/170/3916)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea19@W04 & ea24@W05[15&19] Spw=2 Chan=19 Freq=1.686 Corr=LR X=1.686 Y=0.155768(9550/170/3918)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea19@W04 & ea24@W05[15&19] Spw=2 Chan=19 Freq=1.686 Corr=LL X=1.686 Y=0.147995(9551/170/3919)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RR X=1.686 Y=0.199225(10060/170/4428)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RL X=1.686 Y=0.193489(10061/170/4429)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.179036 (10068/170/4436)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea24@W05 & ea25@N02[19&20] Spw=2 Chan=19 Freq=1.686 Corr=RL X=1.686 Y=0.156535(10573/170/4941)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea24@W05 & ea25@N02[19&20] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.140293 (10580/170/4948)<br />
INFO PlotMS Found 287 points (287 unflagged) among 202752 in 0.01s.<br />
</pre><br />
<div style="text-align: justify;"><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:RFI_1244-1372.png|380px|thumb|right|'''Figure 4a''' <br />Amplitude vs. frequency plot of a portion of the L-band spectrum, showing RFI spikes from different sources.]]<br />
| [[Image:amp_v_freq.spw0_Scan190.png|330px|thumb|right|'''Figure 4b''' <br />Amplitude vs. frequency plot of L-Band, spectral window 0, for scan 190. Generated by running the command "plotms(vis='SNR_G55_10s.ms', spw='0', scan='190', xaxis='freq', yaxis='amp')" within CASA.]]<br />
|}<br />
<br />
Reviewing the log output, we see that spw 0 and 2 are affected the most by RFI. We also get the corresponding frequency where the RFI is present, which appears to be 1.31 and 1.33 GHz for spectral window 0, and 1.686 GHz for spectral window 2. This information is important, especially when flagging data interactively as we will be doing towards the end of this guide.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', spw='0', scan='190', xaxis='freq',yaxis='amp')<br />
</source><br />
<br />
The VLA has obtained RFI sweeps over its full frequency range. The results are available on the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/rfi Radio Frequency Interference website.] Most of the RFI features can be identified as radar, communications, satellites, airplanes, or birdies (i.e., RFI generated by the VLA electronics itself). For our observations, we inspect the L-band specific RFI plots [https://science.nrao.edu/facilities/vla/observing/RFI/L-Band website] and find that the 1310 and 1330 MHz features are due to FAA ASR radars, while the one at 1686 MHz is most likely due to a GOES weather satellite. We can compare the plots of the FAA ASR radar from the website (Figure 4a), with that of spw 0 (Figure 4b), and see that the spikes are practically identical and fall within the same frequencies.<br />
<br />
=== Identifying RFI with MSview() ===<br />
<br />
Similar to using plotms() for the ms file editing, one can also use the [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] task.<br />
<br />
MSview() displays data in a 3D fashion, where the intensity of the plotted data points represents amplitude, phase, or their derivatives, while x and y axes of the display can be set to any of the following: time, baseline, channel, correlation or spectral window. For the introduction to the [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] tool see its dedicated [https://casaguides.nrao.edu/index.php/Data_flagging_with_viewer guide] (note that msview() can also be invoked as viewer(), as it is done in the linked guide).<br />
[[Image:CASA5.4.0-flagging-msview-load-data.png|300px|thumb|right|'''Figure 5''' <br /> Loading data into msview(). The red box marks where data preselection can be made.]]<br />
<br />
<b>Note:</b> msview() will not handle large data volumes very well, and it may crash if your ms file is too large. If that happens try preselecting a fraction of your data for viewing. Also, sometimes restarting CASA may help. <br />
<br />
Since the task does not allow for much data selection in the command line, open [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] tool first, and load and select the data from its graphical interface:<br />
<br />
<source lang="python"><br />
# In CASA<br />
msview()<br />
</source><br />
<br />
Once the tool opens, it will prompt you for data upload. At the same time as selecting the ms file to open, choose which field, spw etc you would like to load from that ms file as shown in Figure 5, and hit Raster Image button. Only the preselected data will be available in the viewer, and further selection for viewing can be done via Data Display Options window (wrench button, within [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()]) under its ''display axes'' tab. Note that the ''ms and visibility selection'' tab will inform you about the overall shape of your data, even if you have not loaded the full ms file.<br />
<br />
To identify RFI in [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()], one can look at the data in multiple ways. For example, let's plot <b>time vs. channel</b> for a single spectral window. We can set these selections in the Data Display Options (click on the Wrench button) as shown in Figure 6. The selection window will appear (highlighted with red box in Figure 6). We plotted here spw=2, which corresponds to the orange-coloured spw from Figure 3. The blue coloured pixels are fairly well behaved data, while the green and red pixels are the data affected by the RFI. We can narrow down where that RFI occurs by hovering cursor over the RFI areas (white box/magenta circle in Figure 6) and details on affected baselines, scans, etc will be listed in the Cursors window (highlighted with magenta box in Figure 6). Note that [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] refers to the antennas via their ID numbers and not the ''ea'' names (compare with the listobs output and commentary in section [[#Preliminary_Data_Evaluation | Preliminary Data Evaluation]]). Also, note that here only RR polarisation is plotted, and you will need to check also LL (as well as RL and LR if you have and use them in further data processing).<br />
<br />
One can also have a quick look at which spectral windows most RFI occurs, and when. For example, let's plot <b>channel vs. spectral window</b> and iterate over time Figure 7). You can choose iteration by selecting animation axis under the ''display axes'' tab within the Data Display Options (marked with red box in Figure 7). To inspect how the RFI changes with time use the iteration buttons within the Animators panel (marked with yellow box in Figure 7). Again, the blue colour marks the well behaved data here; you can change the colour scheme in the Data Display Option window.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:CASA5.4.0-flagging-Msview-time-chan-spw2.png|400px|thumb|right|'''Figure 6''' <br /> Time vs channel plot visualised with msview(). The data selection window for viewing is highlighted with the red box. The magenta box gives information on pixels which we hovered over with a cursor (white box/magenta circle) and which are affected by RFI.]]<br />
| [[Image:CASA5.4.0-flagging-Msview-chan-spw-iterate-time.png|400px|thumb|right|'''Figure 7''' <br /> Channel vs spectral window plotted with msview(). The data selection window for viewing is highlighted with the red box. The yellow box indicates buttons to play a movie over the iteration axis, which here is set to time.]]<br />
|}<br />
<br />
== Identifying Problematic Antennas from the Operator Logs ==<br />
<br />
We first check the operator log for the observing session to see if there were any issues noted during the run that need to be addressed. The logs are available from the [http://www.vla.nrao.edu/cgi-bin/oplogs.cgi VLA operator log website]. Here is the link for the [http://www.vla.nrao.edu/operators/logs/2010/8/2010-08-23_0005_AB1345.pdf observing log] file for our observing session.<br />
<br />
The log has various pieces of information including the start/end times for the observing session, frequency bands used, weather, baseline information for recently moved antennas, and any outages or issues that may have been encountered. We see that antenna ea07 may need position corrections (see one of our calibration tutorials, [https://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.5.0 IRC+10216], on how to fix this), and several antennas (ea06, ea17, ea20, and ea26) are missing an L-Band receiver. Antenna 07 position corrections will need to be applied during data calibration and will not be covered in this tutorial. Additionally, the antennas with missing receivers were already removed from the dataset used here (as explained in the above section Obtaining the Data), so we can continue with online flagging.<br />
<br /><br />
<br /><br />
<br />
== Online Flags ==<br />
<br />
At the time of importing from the SDM-BDF raw data to a MS, we chose to process the online flags from the Flags.xml file to the FLAG_CMD sub-table within the MS. We also created the ''SNR_G55.ms.onlineflags.txt'' file, a plain text document that includes the list of online flags. Note that this step is not necessary if you choose to apply online flags at the time of the data download from the archive.<br />
<br />
The Flags.xml file holds information of flags created during the observing session, such as subreflector issues and antennas not being on source. <br />
We will now apply these online flags to the data by employing {{flagcmd}}, but first, let's create a plot of the flags to get an idea of what will be flagged.<br />
<br />
[[Image:flaggingreason_vs_time.png|300px|thumb|right|'''Figure 8''' Online Flags]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagcmd(vis='SNR_G55_10s.ms', inpmode='table', reason='any', action='plot', plotfile='flaggingreason_vs_time.png')<br />
</source><br />
<br />
Open the ''flaggingreason_vs_time.png'' with your favorite image viewing program. Figure 8 shows several instances of online flagging. Most notably, ea28 and ea08 had some subreflector issues throughout the observing session. Online flags are instances of possible missing data, including:<br />
<br />
* ANTENNA_NOT_ON_SOURCE <br />
The VLA antennas have slewing speeds of 40 degrees per minute in azimuth and 20 degrees per minute in elevation. Some antennas are slower than others and may take a few more seconds to reach the next source. The antennas can also take a few seconds to settle down due to small oscillations after having slewed. <br />
<br />
* SUBREFLECTOR_ERROR <br />
The Focus Rotation Mount (FRM), located at the apex of the antenna, is responsible for focusing the incoming radio signal to the corresponding receiver. At times there can be issues with the focus and/or rotation axes.<br />
<br />
Now that we've plotted the online flags, we will apply them to the MS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagcmd(vis='SNR_G55_10s.ms', inpmode='table', reason='any', action='apply', flagbackup=False)<br />
</source><br />
<br />
* inpmode='table': Will use the online-flags imported to the FLAG_CMD sub-table from the Flags.xml file. This was done during the importing of our data with {{importasdm}}. Note that the FLAG_CMD sub-table already includes a 1.5 second time buffer, as was requested during the importing of the data (parameter ''tbuff=1.5'').<br />
<br />
The CASA logger should report the progress as the task applies these flags in segments. Once finished, it will report the percentage of flagged data. The terminal window will display warnings about the four missing antennas; this is expected as we had previously removed the antennas during the split task.<br />
<br />
Alternatively, we could have applied the online flags from the text file (''SNR_G55.ms.onlineflags.txt'') created during the importing of the data. The one caveat when it comes to applying flags from a list is not being able to un-apply them (setting parameter ''action='unapply' ''within {{flagcmd}}. Transferring the flags to the FLAGS_CMD table ''before'' applying them ''does'' allow for the use of the un-apply feature. For convenience, we provide the command for applying online flags from a list (don't run the following command in CASA):<br />
<br />
<pre><br />
flagcmd(vis='SNR_G55_10s.ms', inpmode='list', inpfile='SNR_G55.ms.onlineflags.txt', reason='any', action='apply', flagbackup=False)<br />
</pre><br />
<br />
== Shadowed Antennas ==<br />
<br />
The VLA D-configuration is the most compact; there may be instances where one antenna blocks, or shadows another. For more on shadowing, please refer to the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/dynsched#section-1 antenna shadowing] section in the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide Guide to Observing with the VLA]. Generally, observing sources at 40 degrees elevation and higher will result in less shadowing. We will create a plot of elevation vs. time by using the {{plotms}} task and note the elevation of the radio sources we observed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', xaxis='time', yaxis='elevation', antenna='0&1;2&3', spw='*:31',<br />
coloraxis='field', title='Elevation vs. Time', plotfile='Elevation_vs_Time.png')<br />
</source><br />
<br />
[[Image:Elevation_vs_Time.png|300px|thumb|right|'''Figure 9''' Plot of elevation vs. time.]]<br />
<br />
Figure 9 shows that most of the sources were at or above 40 degrees in elevation; accordingly, we expect a minimal amount of shadowing. The flux density / bandpass calibrator 3C147 was observed towards the end of the session, which could result in some shadowing of antennas due to its low elevation of about 20&ndash;22 degrees. <br />
<br />
Task {{flagdata}} can determine and flag shadowed antennas by their location and observing direction:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s.ms', mode='shadow', tolerance=0.0, flagbackup=False)<br />
</source><br />
* ''mode='shadow''': has a subparameter ''tolerance=0.0'' that controls how many meters of shadowing overlap is allowed<br />
<br />
The logger will report on the percentage of flagged data due to shadowing. As expected in this observing session, there does not appear to be much data affected by shadowing (approx. 0.4%).<br />
<br />
== Zero-Amplitude Data ==<br />
<br />
In addition to shadowing, there may be times during which the correlator writes out pure zero-valued data. In order to remove this bad data, we run {{flagdata}} to remove any pure zeroes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s.ms', mode='clip', clipzeros=True, flagbackup=False) <br />
</source><br />
* ''mode='clip''': is used to flag all values below a given threshold. With ''clipzeros=True'' this mode will flag exact zero values that are sometimes being produced by the VLA correlator.<br />
<br />
Inspecting the logger output generated by {{flagdata}} shows that there is a small quantity of zero-valued data present in this MS.<br />
<br />
== Quacking ==<br />
<br />
Now we utilize the flagdata task one more time to run quacking mode.<br />
<br />
It's common for the array to settle down at the start of a scan. Quacking is used to remove data at scan boundaries, the same edit can be applied to all scans for all baselines. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s.ms', mode='quack', quackinterval=5.0, quackmode='beg', flagbackup=False)<br />
</source><br />
<br />
*quackmode='beg': Data from the start of each scan will be flagged.<br />
*quackinterval=5.0: Flag the first 5 seconds of every scan.<br />
<br />
<br />
== Backup Data - Flagmanager ==<br />
<br />
Flags can be backed up in a file ''ms.flagversions'' that is related to the MS. For our MS, the related flag backups are stored in ''SNR_G55_10s.ms.flagversions''. <br />
<br />
'''Note: flags that are applied to the data are contained in the MS;''' '''''ms.flagversions''''' '''only contains backup flags that can be restored if required.'''<br />
<br />
Most flagging tasks, like {{flagdata}}, offer a ''flagbackup'' parameter that controls whether or not the current flags are being backed up in ms.flagversions before new (additional) flags will be applied. Backup flags can also be manually saved to or restored from ms.flagversions using the {{flagmanager}} task. <br />
<br />
Now that we've applied online flags, clipped zero amplitude data, removed shadowed data, and quacked the data, we will create a backup of the flags by setting the parameter ''versionname='after_online_flagging' '': <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='SNR_G55_10s.ms', mode='save', versionname='after_online_flagging')<br />
</source><br />
<br />
The version we just created is saved inside the SNR_G55_10s.ms.flagversions directory. From here onward, if we make a mistake, we can always revert back to this flag version of the MS by setting parameter ''mode='restore' '' and providing the version name we want to revert back to, such as:<br />
<br />
<pre><br />
flagmanager(vis='SNR_G55_10s.ms', mode='restore', versionname='after_online_flagging')<br />
</pre><br />
<br />
The flagmanager() can also be run on calibration tables. In this instance just specify the table name as the ''vis'' parameter and run is as above.<br />
<br />
== Benefit of Hanning Smoothing ==<br />
<br />
Strong RFI sources can give rise to the Gibbs phenomenon. This is seen by ringing: a zig-zag pattern across the channels that neighbor the strong, usually narrow, RFI. To remedy this ringing across the frequency channels, we employ the Hanningsmooth algorithm via the {{hanningsmooth}} task (an implementation based on {{mstransform}}). Hanning-smoothing applies a triangle kernel across the pattern which diminishes the ringing, reducing the number of channels that may look bad and get flagged. Note that this smoothing procedure will also decrease the spectral resolution by a factor of two.<br />
<br />
The task requires the creation of a new MS, which we will call ''SNR_G55_10s-hanning.ms''. <br />
<br />
<b>Note: As Hanning-smoothing will remove amplitude spikes, it is not recommended for spectral analysis related science (e.g. strong narrow maser lines).</b><br />
<br />
Let's create before (Figure 10a) and after images (Figure 10b) with the {{plotms}} task to see the effect. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', scan='190', antenna='ea24', spw='0~2',<br />
xaxis='freq', yaxis='amp', coloraxis='spw', title='Before Hanning',<br />
correlation='RR,LL', plotrange=[1.2,1.8,-0.01,0.25],<br />
plotfile='amp_v_freq.beforeHanning.png')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
hanningsmooth(vis='SNR_G55_10s.ms', outputvis='SNR_G55_10s-hanning.ms', datacolumn='data')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190', antenna='ea24', spw='0~2',<br />
xaxis='freq', yaxis='amp', coloraxis='spw', title='After Hanning',<br />
correlation='RR,LL', plotrange=[1.2,1.8,-0.01,0.25],<br />
plotfile='amp_v_freq.afterHanning.png')<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:amp_v_freq.beforeHanning.png|350px|thumb|right|'''Figure 10a''' <br />Plot of amplitude vs. frequency before Hanning smoothing data.]]<br />
| [[Image:amp_v_freq.afterHanning.png|350px|thumb|right|'''Figure 10b''' <br />Plot of amplitude vs. frequency after Hanning smoothing data.]]<br />
|}<br />
<br />
Figure 10b shows the effect of applying Hanning-smoothing. Notice that single channel RFI has spread into three channels. This spreading of RFI to other channels will ultimately result in a little more data being flagged.<br />
<br />
== Automatic RFI Excision ==<br />
<br />
Now that we're done with online flagging, we can move on to removing some of the RFI present with auto-flagging algorithms used within {{flagdata}}, TFCrop and rflag. For further details on the two algorithm based tasks we will employ, please see the [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/EVLAWorkshop_RFI_2016_UR.pdf RFI presentation] given at the 5th VLA Data Reduction Workshop.<br />
<br />
=== TFCrop ===<br />
<br />
Task {{flagdata}}'s TFCrop is an algorithm that detects outliers in the 2D time-frequency plane and can operate on uncalibrated (non-bandpass corrected) data. TFCrop will iterate through segments of time and undergo several steps in order to find and excise different types of RFI:<br />
<br />
Step 1: Detect short-duration RFI spikes (narrow- and broad-band). <br /><br />
Step 2: Search for time-persistent RFI. <br /><br />
Step 3: Search for time-persistent, narrow-band RFI. <br /><br />
Step 4: Search for low-level wings of very strong RFI. <br /><br />
<br />
[[Image:amp_v_freq_before_tfcrop.png|350px|thumb|right|'''Figure 7''' <br />Amplitude vs. Frequency plot of Scan 190, before the TFCrop algorithm is applied.]]<br />
We will apply the auto-flagging TFCrop algorithm for each spectral window. With TFCrop, it's a good idea to first inspect a small portion of data and review what and how much will be flagged. Once you are comfortable with the results, you can apply the algorithm to the remaining data. We will walk through the first spw and then include the remaining spws with the same command. First plot the corrected data for scan 190 (Figure 11) with all spectral windows so we can compare the data before and after TFCrop.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190', antenna='ea24', xaxis='freq', iteraxis='scan', yaxis='amp',<br />
ydatacolumn='data', plotfile='amp_v_freq_before_tfcrop.png', title='Before TFCrop',<br />
coloraxis='spw', plotrange=[1.2,2,-0.01,0.25])<br />
</source><br />
<br />
The following are a set of {{flagdata}} commands which have been found to work reasonably well with these data. Take some time to play with the parameters, and the plotting capabilities, as this will be important when you are working with your own data later on. Because we set parameters ''display='both' ''and ''action='calculate', '' the flags are displayed but not actually written to the MS. This allows one to try different sets of parameters before actually applying the flags to the data.<br />
[[Image:Tfcrop.png|350px|thumb|right|'''Figure 12''' <br />TFCrop flagging results. Top row is before, and bottom row is after TFCrop has applied flagging.]]<br />
<br />
Some representative TFCrop plots are displayed (Figure 12). Each column displays an individual polarization product; all four polarizations are, from left to right, RR, RL, LR, and LL. The first row shows the data with current flags applied and the second includes the flags generated by flagdata. The x-axis is channel number (the spectral window ID is displayed in the top title) and the y-axis of the first two rows is all integrations included in a time segment, set by the parameter ''ntime''. These are the data considered by the TFCrop algorithm during its flagging process, and changes in ''ntime'' will have some (relatively small) effect on what data are flagged. <br />
<br />
Each plot page displays data for a single baseline and time segment, with colours representing the intensity of the visibilities (amplitude) in the heat colour scheme ranging from black for lowest amplitude points to white for highest amplitude data points. The flagged, or to be flagged, visibilities are masked in blue. The buttons at the bottom allow one to step through baseline (backward and forward), spw, scan, and field while Stop Display will continue the flagging operation without the GUI, and Quit aborts the run.<br />
<br />
''' spw 0 '''<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='tfcrop', spw='0',<br />
datacolumn='data', action='calculate',<br />
display='both', flagbackup=False)<br />
</source><br />
<br />
The first two scans (14,15) are fully flagged (hence displayed as completely blue). Click on Next Scan until you reach scan 17 where it will be easier to see the flagging the algorithm performs. The top row shows current flags already applied to the data. The bottom row illustrates the flagging that can be applied with TFCrop, represented by the additional blue areas. Iterate through several scans and baselines. Once you're done, click on the Quit button on the bottom right corner. Let's now apply these flags by changing the action parameter.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='tfcrop', spw='0',<br />
datacolumn='data', action='apply',<br />
display='', flagbackup=False)<br />
</source><br />
<br />
The logger will report the percentage of flagged data in the table selection. We can now apply the TFCrop algorithm to the remaining spectral windows. <br />
<br />
''' spw 1, 2, 3 '''<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='tfcrop', spw='1~3',<br />
datacolumn='data', action='apply',<br />
display='both', flagbackup=False)<br />
</source><br />
Iterate through several scans and baselines and, once you're ready to apply the flags, click on Stop Display. After TFCrop has gone through and flagged some of the worst RFI, we can inspect the log report and take note of how much has been flagged.<br />
<br />
We can also use {{plotms}} to review the effects of using TFCrop. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190', antenna='ea24', xaxis='freq', iteraxis='scan', yaxis='amp',<br />
ydatacolumn='data', plotfile='amp_v_freq_after_tfcrop.png', title='After TFCrop',<br />
correlation='RR,LL', coloraxis='spw', plotrange=[1.2,2,-0.01,0.25])<br />
</source><br />
<br />
Compare the plots before applying TFCrop (Figure 11) to the plot after applying TFCrop (Figure 13). Figure 13 shows great improvements, especially for spectral window 0 and 2, which had some of the worst RFI.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:amp_v_freq_before_tfcrop.png|350px|thumb|right|'''Figure 11''' <br />Amplitude vs. Frequency plot of Scan 190, before the TFCrop algorithm is applied.]]<br />
| [[Image:amp_v_freq_after_tfcrop_Scan190.png|350px|thumb|right|'''Figure 13''' <br />Plot of amplitude vs. frequency showing the effects of applying TFCrop to our data, for scan 190.]]<br />
|}<br />
<br />
''' Flagging Summary '''<br />
<br />
We now calculate the amount of data flagged so far in our Measurement Set by using the {{flagdata}} task with parameter ''mode='summary' ''and assign the returned Python dictionary to the variable flagInfo. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagInfo = flagdata(vis='SNR_G55_10s-hanning.ms', mode='summary')<br />
</source><br /><br />
<br />
Now we parse the dictionary to print some of the flagging statistics, first per source, then per spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
print("\n %2.1f%% of G55.7+3.4, %2.1f%% of 3C147, and %2.1f%% of J1925+2106 are flagged. \n" % <br />
(100.0 * flagInfo['field']['G55.7+3.4']['flagged'] / flagInfo['field']['G55.7+3.4']['total'], <br />
100.0 * flagInfo['field']['0542+498=3C147']['flagged'] / flagInfo['field']['0542+498=3C147']['total'], <br />
100.0 * flagInfo['field']['J1925+2106']['flagged'] / flagInfo['field']['J1925+2106']['total']))<br />
</source><br /><br />
<br />
Note: IPython currently prevents copy/pasting several lines of code. To remedy this, we will use the %cpaste command to allow multiple lines to be placed in the terminal. Make sure you also copy the "--" characters to close the %cpaste input.<br />
<br />
<source lang="python"><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
print("Spectral windows are flagged as follows:")<br />
<br />
for spw in range(0,4):<br />
print("SPW %s: %2.1f%%" % (spw, 100.0 * flagInfo['spw'][str(spw)]['flagged'] / flagInfo['spw'][str(spw)]['total']))<br />
--<br />
</source><br />
<br />
We can now move to the other auto-flagging algorithm, rflag.<br />
<br />
=== RFlag ===<br />
<br />
RFlag, like TFCrop, is an autoflag algorithm which uses a sliding window statistical filter. Data are iterated through in segments of time where statistics are accumulated and thresholds calculated. <br />
<br />
'''WARNING:''' It is important to not assume that rflag can run on your target. This is especially important for spectral lines, as it may mistake them for interference. <br />
<br />
'''IMPORTANT:''' Task {{rflag}} should be executed <u>on calibrated data only</U>. For more details, please see Emmanuel Momjian's presentation on [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/E_Momjian_Data_Reduction_Techniques_Calibration_2016.pdf VLA Data Reduction Techniques and Calibration] given during the [https://science.nrao.edu/science/meetings/2016/vla-data-reduction 5th VLA Data Reduction Workshop].<br />
<br />
<div style="background-color: #eeeeee; margin: 40px"><br />
<div style="background-color: #eeeeee; margin: 43px"><br />
<BR><br />
==== Bandpass calibration (to demonstrate RFlag) ====<br />
<br />
For the purpose of demonstrating the functionality of RFlag on our data, we need to perform bandpass calibration first. Here, we just briefly outline the bandpass calibration process. For detailed calibration tutorial see [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials#VLA_Data_Reduction_Tutorials VLA Data Reduction Tutorials].<br />
<br />
First, run {{gaincal}} to solve for an initial set of antenna-based phases over a narrow range of channels. To find channels for each spectral window that are relatively RFI-free over the course of the observing session, look at the data with {{plotms}}. Use only a narrow channel range.<br />
<!-- <br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
<br />
In order to get the best possible result from the automatic RFI excision with {{flagdata}}'s mode ''rflag'', we will first apply bandpass calibration to the MS. Since the RFI is time-variable, using the phase calibration source to make an average bandpass over the entire observing session will mitigate the amount of RFI present in the calculated bandpass. For the final calibration, we will use the designated bandpass source 3C147, which will give a much higher signal to noise in the bandpass. Since 3C147 was only observed in the last set of scans, it doesn't sample the time variability and would not provide a good average bandpass.<br />
<br />
Since there are likely to be gain variations over the course of the observing session, we will run {{gaincal}} to solve for an initial set of antenna-based phases over a narrow range of channels. Those solutions will be applied to the data when the bandpass solutions are determined. While amplitude variations will have little effect on the bandpass solutions, it is important to solve for these phase variations with sufficient time resolution to prevent decorrelation when vector averaging the data in computing the bandpass solutions.<br />
<br />
In order to choose a narrow range of channels for each spectral window that are relatively RFI-free over the course of the observing session, we can look at the data with {{plotms}}. Note that it's important to solve only for phase using a narrow channel range, since an antenna-specific delay will cause the phase to vary with respect to frequency over the spectral window, perhaps by a substantial amount.<br />
--><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='42,65,88,11,134,157', antenna='ea24', xaxis='channel', <br />
yaxis='amp', iteraxis='spw')<br />
</source><br />
<br />
Iterating over each spectral window plot, search for channel ranges with stable amplitudes. An example of a few appropriate channel ranges include: SPW 0:20~24, SPW 1:49~52, SPW 2:38~41, SPW 3:41~44.<br />
<br />
Using these ranges, run {{gaincal}} to calculate phase-only solutions that will be used as input during the initial bandpass calibration. Remember: the calibration tables we are creating now are so that we can use an automatic RFI flagging algorithm. The final calibration tables will be generated later, after automated flagging. Here are the inputs for the initial pre-bandpass phase calibration:<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='SNR_G55_10s-hanning.ms', caltable='SNR_G55_10s-hanning.initPh', field='J1925+2106', <br />
solint=' int ', spw='0:20~24,1:49~52,2:38~41,3:41~44', refant='ea24', minblperant=3, <br />
minsnr=3.0, calmode='p')<br />
</source><br />
<!--<br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
<br />
* caltable='SNR_G55_10s-hanning.initPh': this is the output calibration table that will be written.<br />
* field='J1925+2106': this is the phase calibrator we will use to calibrate the phases.<br />
* solint=' int ': we request a solution for each 10-second integration.<br />
* spw='0:20~24,1:49~52,2:38~41,3:41~44': note the syntax of this selection: a ''''':''''' is used to separate the SPW from channel selection and '''''~''''' is used to indicate an inclusive range. <br />
* refant='ea24': we have chosen ea24 as the reference antenna after inspecting the antenna position diagram (see above). It is relatively close to the center of the array. Since the antennas in the very center have a high probability of being shadowed by nearby antennas, choose one that is not directly in the center. This is most important for the more compact (D, C) array configurations.<br />
[[Image:gain.phase_v_time.AllAnt.plotcal.png|300px|thumb|right|'''Figure 10b''' <br /> Gain Phase vs. Time for spw0, for all antennas, with plotsymbol='-', in order to connect the data points and inspect for phase jumps.]]<br />
* minblperant=3: the minimum number of baselines which must be present to attempt a phase solution.<br />
* minsnr=3.0: the minimum signal-to-noise a solution must have to be considered acceptable. Note that solutions which fail this test will cause these data to be flagged downstream of this calibration step.<br />
* calmode='p': perform phase-only solutions.<br />
<br />
Note that a number of solutions do not pass the requirements of the minimum 3 baselines (generating the terminal message''' "Insufficient unflagged antennas to proceed with this solve."''') or minimum signal-to-noise ratio (outputting''' "n of x solutions rejected due to SNR being less than 3 ..."'''). The logger output indicates 320-324 solutions succeeded out of 387 attempted. <br />
[[Image:gain.phase_v_time.plotcal.png|300px|thumb|right|'''Figure 10a''' <br /> Gain Phase vs. Time for spw0, for four antennas.]]<br />
--><br />
[[Image:CASA5.4.0-Plotms-pahse-time.png|300px|thumb|right|'''Figure 14''' <br /> Gain Phase vs. Time for spw0, for four antennas.]]<br />
<br />
It is always a good idea to inspect the resulting calibration table. We can do this with {{plotms}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.initPh', xaxis='time', yaxis='phase', <br />
iteraxis='spw', antenna='ea01,ea05,ea10,ea24', coloraxis='Antenna1')<br />
</source><br />
<br />
<br />
Figure 14 shows the phases do not change much over the course of the observing session. <br />
<!-- <br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
We deliberately chose to only plot four antennas in our last call to plotcal, as plotting all of them together will make the plot look convoluted which, in turn, makes it difficult to determine any phase jumps that may be present. To remedy this, we will employ plotcal again with the plotsymbol parameter set to connect the data points.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='SNR_G55_10s-hanning.initPh', xaxis='time', yaxis='phase', <br />
iteration='spw', antenna='', plotsymbol = '-', plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
Figure 10b may look to have phase jumps, but the V-shapes present are the program attempting to wrap the data points. As a user exercise, plot the phase from -180 to 0, and 0 to +180 to get the stable lines.<br />
--><br />
<br />
Since the plots look fairly reasonable, we will now create a time-averaged bandpass solution for the phase calibration source using the {{bandpass}} task.<br />
<br />
<source lang="python"><br />
# In CASA<br />
bandpass(vis='SNR_G55_10s-hanning.ms', caltable='SNR_G55_10s-hanning.initBP', <br />
field='J1925+2106', solint=' inf ', combine='scan', refant='ea24', <br />
minblperant=3, minsnr=10.0, gaintable='SNR_G55_10s-hanning.initPh', <br />
interp='nearest', solnorm=False)<br />
</source><br />
<!--<br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
<br />
* solint=' inf ', combine='scan': the solution interval of infinite, up to the boundaries controlled by the combine parameter; this requests that the solution intervals be combined over scans, so that we will get one solution per antenna.<br />
* gaintable= 'SNR_G55_10s-hanning.initPh': we will pre-apply the initial phase solutions.<br />
* interp='nearest': by default, {{bandpass}} will use linear interpolation for pre-applied calibration. However, we want the <i>nearest</i> phase solution to be used for a given time.<br />
<br />
Again, we can see that a number of solutions have been rejected by our choice of <tt>minsnr</tt>.<br />
--><br />
<br />
Now, inspect the resulting bandpass plots with plotms (Figures 15a,b,c). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.initBP',xaxis='freq', yaxis='amp', iteraxis='antenna', <br />
coloraxis='corr',gridrows=2)<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:CASA5.4.0-Plotms-amp-freq-ea01-02.png|300px|thumb|right|'''Figure 15a''' <br /> Bandpass for antennas ea01 and ea02.]]<br />
| [[Image:CASA5.4.0-Plotms-amp-freq-ea03-04.png|300px|thumb|right|'''Figure 15b''' <br /> Bandpass for antennas ea03 and ea04.]]<br />
| [[Image:CASA5.4.0-Plotms-amp-freq-ea05-07.png|300px|thumb|right|'''Figure 15c''' <br /> Bandpass for antennas ea05 - ea07 (note ea06 is flagged).]]<br />
|}<br />
<br />
<br><br />
</div><br />
</div><br />
<br />
==== Interactive flagging of bad solutions (calibration tables) ====<br />
<br />
[[Image:CASA5.4.0-Plotms-amp-freq-ea01-05-flagging.png|300px|thumb|right|'''Figure 16''' <br /> Interactive plotms flagging of the offset points for ea01 and ea05 in the calibration table.]]<br />
We notice that antennas ea01 and ea05 have a point that is offset from the rest (Figures 15a,c). Let's plot just these two antennas, locate the point on the plot, and flag it interactively through {{plotms}}. The interactive flagging within plotms will not create a backup, therefore it may be wise to use {{flagmanager}} before doing so (see Section [[#Backup_Data_-_Flagmanager |Flagmanager]] earlier in this tutorial). <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='SNR_G55_10s-hanning.initBP', mode='save', versionname='before_flagging')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.initBP',xaxis='freq', yaxis='amp', iteraxis='antenna', <br />
antenna='ea01,ea05', coloraxis='corr', gridrows=2)<br />
</source><br />
<br />
We will highlight the points by clicking on the <b>Mark Region</b> button, drawing boxes over the points, and clicking on the <b>Flag</b> button. Before doing this, one could also get information on the points by clicking on the Locate button, which will display details about the highlighted regions (Figure 16). After having flagged the offset points, your plots should update.<br />
<br />
<div style="background-color: #eeeeee; margin: 40px"><br />
<div style="background-color: #eeeeee; margin: 43px"><br />
<br><br />
<br />
==== Applying calibration ====<br />
<br />
We will now apply the bandpass calibration table using {{applycal}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='SNR_G55_10s-hanning.ms', gaintable='SNR_G55_10s-hanning.initBP', calwt=False)<br />
</source><br />
<br />
This operation will flag data that correspond to flagged solutions, so {{applycal}} makes a backup version of the flags prior to operating on the data. Running this task might take a little while.<br />
<br />
<br><br />
</div><br />
</div><br />
<br />
==== Running RFlag on bandpass calibrated data ====<br />
[[Image:amp_v_freq.beforeRFlag.png|320px|thumb|right|'''Figure 17''' <br />Amplitude vs. Frequency plot of scan 190, before RFlag is applied.]]<br />
<br />
Now that we have bandpass-corrected data, we will run {{flagdata}} in rflag mode.<br />
<br />
We will use flagdata with parameters ''mode='rflag','' and ''action='calculate' ''to first review the amount of data to be flagged. We will also change the parameter ''datacolumn='corrected' ''since we've applied the bandpass corrections to the MS and, in the process, created the ''corrected_data'' column in the MS table. <br />
<br />
''' spw 0 '''<br />
<br />
First, let's create a plot of our corrected data before we apply RFlag (Figure 17).<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190' , antenna='ea24', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', coloraxis='spw', title='Before RFlag', <br />
plotfile='amp_v_freq.beforeRFlag.png') <br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='0', datacolumn='corrected', <br />
action='calculate', display='both', flagbackup=False)<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:rflag_default_values.png|400px|thumb|right|'''Figure 18''' <br />RFlag with Default Parameters for scan 22 (use the next scan, next baseline, and next field buttons to see this specific screen).]]<br />
| [[Image:rflag_freq2.5_time3.5.png|400px|thumb|right|'''Figure 19''' <br />RFlag with freqdevscale=2.5, and timedevscale=3.5 for scan 22. We can see more data are being flagged with these more stringent values.]]<br />
|}<br />
<br />
After reviewing the calculated flags, we can see that a lot of RFI is being missed (Figure 18). We will need to modify the default parameters for this spectral window. Click on Quit on the lower right hand side of the window. Let's review the parameters for {{flagdata}} and modify them:<br />
<br />
<source lang="python"><br />
# In CASA, get the last called parameter values for flagdata<br />
tget flagdata<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, inspect the inputs<br />
inp<br />
</source><br />
<br />
The default ''timedevscale=5.0'' and ''freqdevscale=5.0'' parameters are not flagging enough bad data. We can provide more stringent values to the time/freq deviation scales parameters. A value smaller than the default (5.0) will flag more data in either the time or frequency axes, as can be seen on the plots in the displayed window. A value larger than the default will result in less flagged data. <br />
<br />
Since we know spw 0 has lots of RFI, we will change parameters ''freqdevscale=2.5'' and ''timedevscale=3.5''. We will also change parameter ''action='apply'.'' Note that we can always decide that this still isn't good enough and click on <b>Quit</b> to change our parameter values (will not perform flagging). Once the window opens, proceed to scan 22 (Figure 19), review what will be flagged, and click on <b>Stop Display</b> (as we did during TFCrop) to allow the task to continue with the flagging.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='0', datacolumn='corrected', <br />
freqdevscale=2.5, timedevscale=3.5, action='apply', display='both', flagbackup=False)<br />
</source><br />
<br />
Although RFlag has done a pretty good job of finding the bad data, some still remains. One way to excise the remaining bad data is to use the parameter ''mode='extend' ''feature of flagdata, which can extend flags along a chosen axis and removes islands of small data patches in the midst of flagged data. We first extend the flags across polarization so if any one polarization is flagged, all data for that time / channel will be flagged:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='0', extendpols=True, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
Now we extend the flags in time and frequency using the parameters ''growtime'' and ''growfreq''. For this data, the rflag algorithm seems most likely to miss RFI that should be flagged along more of the time axis. We will try with parameter ''growtime=50.0'', which will flag all data for a given channel if more than 50% of that channel's time is already flagged, and parameter ''growfreq=90.0'', which will flag the entire spectrum for an integration if more than 90% of the channels in that integration are already flagged. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='0', growtime=50.0, growfreq=90.0, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
''' spw 1, 2, and 3 '''<br />
<br />
Now, let's work on SPW's 1,2, and 3. We've chosen parameter ''display='both' ''in order to first review the flags to decide if too much or too little is being flagged. We can always click on Quit and decide to change parameter values for ''freqdevscale'' and ''timedevscale'' as we did with spectral window 0. Note that spw 2 has more stringent parameter values compared to spw 1 and 3, as it contains more RFI. To allow the flagging to continue, click on <b>Stop Display</b>.<br />
<br />
<source lang="python"><br />
# In CASA, run RFlag on spectral windows 1 & 3<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='1,3', datacolumn='corrected', <br />
freqdevscale=4.0, timedevscale=4.0, action='apply', display='both', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in polarization for spectral windows 1 & 3<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='1,3', extendpols=True, action='apply', <br />
display='', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in time and frequency for spectral windows 1 & 3<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='1,3', growtime=50.0, growfreq=90.0, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, run RFlag on spectral window 2 with more RFI, hence smaller frequency and time deviation scale values<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='2', datacolumn='corrected', <br />
freqdevscale=2.5, timedevscale=2.5, action='apply', display='both', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in polarization for spectral window 2<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='2', extendpols=True, action='apply',<br />
display='', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in time and frequency for spectral window 2<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='2', growtime=50.0, growfreq=90.0, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
Let's create an after RFlag plot to see the improvements (Figure 20). Compare the resulting plot to the before TFCrop plot (Figure 17).<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190' , antenna='ea24', xaxis='freq', yaxis='amp', <br />
ydatacolumn='corrected', coloraxis='spw', title='After RFlag', plotfile='amp_v_freq.afterRFlag.png',<br />
plotrange=[1.2,2,0,2.5])<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:amp_v_freq.beforeRFlag.png|350px|thumb|right|'''Figure 17''' <br />Amplitude vs. Frequency plot of scan 190, before RFlag is applied.]]<br />
| [[Image:amp_v_freq.before_after_RFlag.png|350px|thumb|right|'''Figure 20''' <br /> Plot of amplitude vs. frequency of scan 190, after having run RFlag.]]<br />
|}<br />
<br />
<b>Summary of flagdata modes tfcrop and rflag</b><br><br />
Tfcrop algorithim iterates through the data in sections of time, creates a 2D time-frequency (x-y) plane with amplitude in the z axis, one for each baseline and correlation-expression to flag outliers. <br />
RFlag algorithm also iterates through the data in sections of time, but compares local rms versus global statistics of the data to remove outliers. For more info on TFCrop and RFlag, please see the [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/EVLAWorkshop_RFI_2016_UR.pdf RFI presentation] given at the 5th VLA Data Reduction Workshop.<br />
<br />
{| class='wikitable'<br />
|-<br />
!Case<br />
!Tfcrop<br />
!RFlag<br />
|-<br />
| Strong peaks in RFI ||works well.||works well, though continuous RFI (time or frequency) needs tuning.<br />
|-<br />
|Noisy RFI ||Not good for noisy, low level RFI.||works well.<br />
|-<br />
|Broadband RFI ||Not robust, but possible by tuning the polynomial fit (maxnpieces).|| Good if RFI looks noisy. Continuous RFI needs work.<br />
|-<br />
|Non-calibrated data ||Yes, because it can fit for the bandpass.||No, it needs a flat base. Calibration is essential.<br />
|-<br />
|Extended emission|| Yes, since each baseline is treated separately.||high flux on short baselines biases the statistics (use uvrange, or operate on residual data).<br />
|}<br />
<br />
== Interactive Flagging ==<br />
<br />
To flag data interactively (manually), one can use {{plotms}}. We will be scanning for data points that look odd. Please note that flagging data through plotms will not create a backup, so it's important to use the flagmanager() before deciding to mark your regions for flagging purposes. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='SNR_G55_10s-hanning.ms', mode='save',<br />
versionname='before_interactive_flagging')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='50~100', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', coloraxis = 'spw')<br />
</source><br />
<br />
Figure 21 (next section) shows some points in green that have higher amplitudes than the rest. Use the <b>Mark Regions</b> button (square with green plus sign) to highlight the region, and the <b>Locate</b> button (magnifying glass with white page) to obtain information on the data points. We can now decide to flag all of the highlighted area by clicking on the <b>Flag</b> button (red flag). The region is now flagged.<br />
<br />
<br />
Similarly, one can also perform interactive flagging with {{msview}}. Again, the flagging is performed directly on the data, and so it is beneficial to execute flagmanager as above before embarking on interactive flagging with msview(). <br />
<br />
Open msview() and load the data as described in section [[#Identifying_RFI_with_MSview | Identifying RFI with MSview]]:<br />
<br />
<source lang="python"><br />
# In CASA<br />
msview()<br />
</source><br />
<br />
Select what you would like the axes to display (by default msview will plot time vs baseline), and using zoom tool (magnifying glass button with plus sign) zoom in on the data chunks you would like to inspect and potentially flag. Now, draw a region around the bad data using the <b>drawing selectors</b> (rectangle, circle, polygon) and <b>right-double click</b> to flag. To save the flagging, click <b>Save Edits</b> button in the flagging tab of Data Display Option window. <br />
<br />
<pre><br />
Flagging data interactively can be useful, but is discouraged, especially for large data volumes.<br />
</pre><br />
<br />
A more proper form of flagging would be to narrow your search on bad data points (Locate data as described above) and use a flagging task to remove them as showed in the next section below.<br />
<br />
== Identifying remaining bad data with plotms()/msview() ==<br />
<br />
Once the bulk of the RFI and corrupted data has been removed with the methods described earlier in this guide, it is worth having another look at the data with the visualisation tools, either {{plotms()}} or [https://casa.nrao.edu/casadocs/casa-5.4.0/global-task-list/task_msview/about msview()]. This step may be valuable to perform once your data are calibrated and ready for imaging, to double check all the bad data has been indeed removed.<br />
<br />
Let's again inspect amplitude vs uv-distance of our calibrated target data (Figure 21) with plotms(). Instead of flagging the data interactively (the Flag button), it is recommended to run the {{flagdata}} task. While locating the bad data points in the previous section (<b>Mark Region</b> and then click <b>Locate</b> button), CASA logger provided us with information on the data points. Notice that they all belong to scans 88 and 87. The scans also involve a baseline shared with ea16. To flag these data run the flagdata() task:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', scan='87,88', antenna='ea16', flagbackup=True)<br />
</source><br />
<br />
To see the effect of this flagging, either Reload the image (tick Reload and click Plot) if your plotms() session from the previous section is still open, or otherwise run the following command:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='50~100', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', coloraxis = 'spw')<br />
</source><br />
<br />
We can now see the image (Figure 22) looks better without scans 87 and 88 for ea16. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:Corr.Amp_vs_UVdist.png|400px|thumb|right|'''Figure 21''' <br />Amplitude vs. UV-Distance plot for scans 50-100. Outliers have been highlighted to show interactive flagging through plotms.]]<br />
| [[Image:Corr.Amp_vs_UVdist.flagged.png|400px|thumb|right|'''Figure 22''' <br /> Amplitude vs. UV-Distance plot for scans 50-100. Scans 87 and 88 have been flagged for ea16 via flagdata.]]<br />
|}<br />
<br />
If you would like to perform these steps with [https://casa.nrao.edu/casadocs/casa-5.4.0/global-task-list/task_msview/about msview()], instead of plotms(), plot <b>time vs. baseline length</b>. Here baseline length is equivalent to the uv-distance.<br />
<br />
<!-- Now would be also a good time to play with the plotms parameters and look for more data to flag interactively. As a reminder, at the start of this guide we highlighted and located some of the worst sections with RFI. When reducing/flagging your data, it will prove helpful to review this information and revisit the most RFI affected spectral windows and channels.--><br />
<br />
<!--<br />
=== Amplitude vs. Baseline ===<br />
<br />
We will now plot corrected amplitude vs. baseline. We will be looking to flag baselines with consistent high corrected amplitudes as we iterate through each scan. Iterate by clicking on the green arrow buttons on the toolbar within the plotms(). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='30,75,120,165,190,235,303', xaxis='baseline', yaxis='amp',<br />
ydatacolumn='corrected', iteraxis='scan', coloraxis = 'baseline')<br />
</source><br />
[[Image:corr.amp_vs_baseline_scan75.png|300px|thumb|right|'''Figure 17''' <br /> Amplitude vs. Baseline plot, which is used to try and identify baselines with high amplitudes for mutliple scans.]]<br />
<br />
It appears that there are several baselines which have a consistently higher-amplitude than the others, indicating that they're probably contaminated by RFI (Figure 17). <br />
<br />
Use the plotms tools to identify the baselines (see the section [[#Preliminary_Data_Evaluation |Preliminary Data Evaluation]] at the beginning of the guide). Iterate through to the different scans to verify that these higher amplitudes correspond to the same baselines. We will flag several of them: ea04&ea16, ea02&ea08, and ea02&ea27. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', antenna='ea04&ea16', spw='1', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', antenna='ea02&ea27', spw='1', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', antenna='ea02&ea08', spw='1', flagbackup=False)<br />
</source><br />
--><br />
<br />
<br />
== Flagging Summary & Report ==<br />
<br />
We will now run flagdata in summary mode to inspect how much total data has been flagged. We can also create a plot of the percentage of flagged data with respect to frequency by setting parameter ''display='report' ''(Figure 23). The report parameter will also create a plot of the antennas, with the circle size representing the percentage of flagged data per antenna (Figure 24). This can be viewed by clicking Next. One can also zoom in and out for a better view. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagInfo = flagdata(vis='SNR_G55_10s-hanning.ms', mode='summary', action='calculate', display='report', spwchan=True)<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
print("\n %2.1f%% of G55.7+3.4, %2.1f%% of 3C147, and %2.1f%% of J1925+2106 are flagged. \n" %<br />
(100.0 * flagInfo['field']['G55.7+3.4']['flagged'] / flagInfo['field']['G55.7+3.4']['total'],<br />
100.0 * flagInfo['field']['0542+498=3C147']['flagged'] / flagInfo['field']['0542+498=3C147']['total'],<br />
100.0 * flagInfo['field']['J1925+2106']['flagged'] / flagInfo['field']['J1925+2106']['total']))<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
print("Spectral windows are flagged as follows:")<br />
<br />
for spw in range(0,4):<br />
print("SPW %s: %2.1f%%" % (spw, 100.0 * flagInfo['spw'][str(spw)]['flagged'] / flagInfo['spw'][str(spw)]['total']))<br />
--<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:Percent.Flagged.Plot.png|400px|thumb|right|'''Figure 23''' <br /> Total percentage of flagged data vs. frequency, colorized by spectral window.]]<br />
| [[Image:Percent.Flagged.Ant.png|400px|thumb|right|'''Figure 24''' <br /> Total Percentage of data flagged from each antenna. A larger circle represents more flagged data for that antenna. ]]<br />
|}<br />
<br />
As a result of the flagging we have removed over 40% of the data for G55.7+3.4 and, as expected, spectral windows 0 and 2 have the most flagged data.<br />
<br />
In addition, we can inspect the CASA log, which will report the percentage of flagged data for: <br /><br />
1. per Spw/ per Channel <br /><br />
2. Each Antenna <br /><br />
3. Every Correlation (RR, RL, LL, LR) <br /><br />
4. Every Scan <br /><br />
5. Every spectral window <br /><br />
<br />
[[Main Page | &#8629; '''CASAguides''']]<br />
<br />
{{Checked 5.5.0}}<br />
</div><br />
<br />
<!--<br />
--Original: Miriam Hartman <br /><br />
--Modifications: Lorant Sjouwerman (4.4.0, 2015/07/07) <br /><br />
--Modifications: Juergen Ott (4.5.2, 2016/04/13) <br /><br />
--Topical Guide: Jose Salcido (4.5.2, 2016/04/15) <br /><br />
--Edits: Tony Perreault (4.5.2, 2016/05/25) <br /><br />
--Edits: Tony Perreault (4.7.0, 2016/10/14) <br /><br />
--Edits: Jose Salcido (4.7.0, 2016/10/26) <br /><br />
--Edits: Tony Perreault (4.7.0, 2016/10/26 <br /><br />
--Edits: Anna Kapinska (5.4.0, 2018/11/30 <br /><br />
--Edits: Angelica Vargas (5.5.0, 2019/06/24 <br /><br />
--></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Flagging-CASA5.5.0&diff=27874VLA CASA Flagging-CASA5.5.02019-07-05T20:49:42Z<p>Emomjian: </p>
<hr />
<div>* '''This CASA guide is designed for CASA 5.5.0''' <br />
* Printing Disclaimer: When printing, CASA commands in this guide may be cropped depending on your web browser text size and printer settings.<br />
<br />
<br />
== Overview ==<br />
<br />
<div style="text-align: justify;"><br />
This CASA guide covers online data flagging, shadowing, zero-clipping, and quacking. It also covers auto-flagging RFI (Radio Frequency Interference) via TFCrop (Time-Frequency Crop) and rflag.<br />
<br />
We will be utilizing data taken, with the Karl G. Jansky Very Large Array, of [http://simbad.u-strasbg.fr/simbad/sim-id?Ident=SNR+G055.7%2B03.4&NbIdent=1&Radius=2&Radius.unit=arcmin&submit=submit+id G055.7+3.4.], which is a supernova remnant. The data were taken on August 23, 2010 in the first D-configuration for which the new wideband capabilities of the WIDAR (Wideband Interferometric Digital ARchitecture) correlator were available. The 8-hour long session included all available 1 GHz of bandwidth in L-band, from 1&ndash;2 GHz in frequency., although in this guide we will use dataset composed of only 4 spectral windows (128 MHz bandwidth each). <br />
<br />
The guide will often reference the CASA documentation archives which are available [https://casa.nrao.edu/casadocs/casa-5.5.0 online].<br />
<br />
<br />
== Obtaining the Data ==<br />
<br />
A copy of the data can be downloaded from [http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.tar.gz http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.tar.gz] <font color=green>(5.0 GB)</font>. These data are already in the CASA Measurement Set (MS) format and were prepared specifically for this guide. <br />
<br />
The following explains how the data were processed to achieve the above MS file, as it has implications for later flagging steps. <br />
<br />
Note that commands with a grey background do not need to be run in CASA.<br />
<br />
The data were first loaded from the [http://archive.nrao.edu NRAO archive]. On the archive search form, we specified '''Archive File ID''' to be ''AB1345_sb1800808_1.55431.004049953706'' and selected '''SDM-BDF dataset (all files)''' as the data download option on the following page. Be aware that the full dataset is 170GB!<br />
<br />
Once the SDM-BDF was downloaded, we created an MS with the task {{importasdm}}:<br />
<br />
''importasdm(asdm='AB1345_sb1800808_1.55431.004049953706', vis='SNR_G55.ms', process_flags=True, tbuff=1.5, applyflags=False, <br />
ocorr_mode='co', savecmds=True, outfile='SNR_G55.ms.onlineflags.txt', flagbackup=False)''<br />
<br />
process_flags=True: This parameter controls the creation of online flags from the Flag.xml SDM table.<br />
It will create online flags in the FLAG_CMD sub-table within the MS (more on this later).<br /><br />
tbuff=1.5: This parameter adds a time buffer padding to the flags in both directions to deal with timing mismatches.<br />
This is important for VLA data taken before April 2011. This value should be set to 1.5x integration time.<br />
This particular observing session had 1 second correlator integration time.<br /><br />
applyflags=False: We will apply these flags later in the tutorial.<br /><br />
ocorr_mode ='co': Only choosing to import the cross-only (co) correlations.<br /><br />
savecmds=True: Save the online-flag commands to an ASCII file.<br /><br />
outfile='SNR_G55.ms.onlineflags.txt': This will create a text file with a list of online flags that can be applied to the data.<br />
The online flags within the file will include the time buffer requested in the ''tbuff'' parameter. <br />
Mainly created for demonstration purposes.<br />
<br />
The data were split from the original MS and time-averaged using the CASA task {{split}}. <br />
<br />
''split(vis='SNR_G55.ms', outputvis='SNR_G55_10s.ms', field='3~5', spw='4~5,7~8', timebin='10s', <br />
antenna='!ea06,ea17,ea20,ea26', datacolumn='data', keepflags=False)''<br />
<br />
field='3~5': We only include the complex gain calibrator, flux density/bandpass calibrator, and the target source at this stage.<br /><br />
spw='4~5,7~8': Several spectral windows have been removed which were heavily impacted with RFI, which neither auto-flagging nor<br />
hand flagging could remedy. We are keeping spectral windows 4,5,7,8.<br /><br />
timebin='10s': Time-averaged to 10-seconds, to reduce size and processing time when running tasks. <br /><br />
antenna='!ea06,ea17,ea20,ea26': Several antennas have been removed, which at the time of the observations, did not have an L-Band<br />
receiver installed (please see the [[#Identifying_Problematic_Antennas_from_the_Operator_Logs | Identifying Problematic Antennas from the Operator Logs]] section).<br />
The exclamation point (!) before ea06 is called a negation operator. It will exclude all the antennas<br />
provided here, as long as they are separated by commas. For more on CASA syntax, please see this page<br />
on [https://casa.nrao.edu/casadocs/casa-5.5.0/calibration-and-visibility-data/data-selection-in-a-measurementset].<br />
<br />
== Unpacking the Data == <br />
<br />
Once you've downloaded the MS file from the link above, unzip and untar the file from a terminal window (before starting CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf SNR_G55_10s.tar.gz<br />
</source><br />
<br />
This will create a directory called "SNR_G55_10s.ms" <font color=green>(5.5GB)</font>, which is the MS.<br />
<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing '''casa''' on a terminal command line. If you have not used CASA before, some helpful tips are available on the [[Getting Started in CASA]] page.<br />
<br />
This guide has been written for CASA version 5.5.0. Please confirm your version before proceeding by checking the message in the command line interface window or the CASA logger after startup.<br />
<br />
For this tutorial, we will be running tasks using the ''task (parameter = value)'' syntax. All parameters not explicitly called by this manner will use their default values.<br />
</div><br />
<br />
== Preliminary Data Evaluation == <br />
<br />
As a first step, use {{listobs}} to have a look at the MS:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='SNR_G55_10s.ms', listfile='SNR_G55_10s.listobs')<br />
</source><br />
<br />
*listfile: Creates a text document ''SNR_G55_10s.listobs'' with the summary of the data set. <br />
<br />
The ''SNR_G55_10s.listobs'' text document can be opened with your favorite text viewer.<br />
<pre style="white-space: pre-wrap;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/akapinsk/CASAguides-5.5.0/flagging/SNR_G55_10s.ms MS Version 2<br />
================================================================================<br />
Observer: Dr. Sanjay Sanjay Bhatnagar Project: uid://evla/pdb/1072564 <br />
Observation: EVLA<br />
Data records: 2732400 Total elapsed time = 26926 seconds<br />
Observed from 23-Aug-2010/00:56:36.0 to 23-Aug-2010/08:25:22.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
23-Aug-2010/00:56:36.0 - 00:58:06.0 14 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
00:58:06.0 - 00:59:36.0 15 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
00:59:36.0 - 01:01:05.0 16 0 J1925+2106 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:01:05.0 - 01:02:35.0 17 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:02:35.0 - 01:04:05.0 18 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:04:05.0 - 01:05:34.0 19 0 J1925+2106 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:05:34.0 - 01:07:04.0 20 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:07:04.0 - 01:08:34.0 21 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:08:34.0 - 01:10:04.0 22 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:10:04.0 - 01:11:34.0 23 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:11:34.0 - 01:13:03.0 24 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:13:03.0 - 01:14:33.0 25 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:14:33.0 - 01:16:03.0 26 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:16:03.0 - 01:17:33.0 27 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:17:33.0 - 01:19:02.0 28 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:19:02.0 - 01:20:32.0 29 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:20:32.0 - 01:22:02.0 30 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:22:02.0 - 01:23:32.0 31 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:23:32.0 - 01:25:01.0 32 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:25:01.0 - 01:26:31.0 33 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:26:31.0 - 01:28:01.0 34 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:28:01.0 - 01:29:31.0 35 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:29:31.0 - 01:31:00.0 36 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:31:00.0 - 01:32:30.0 37 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:32:30.0 - 01:34:00.0 38 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:34:00.0 - 01:35:30.0 39 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:35:30.0 - 01:36:59.0 40 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
01:36:59.0 - 01:38:29.0 41 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:38:29.0 - 01:39:59.0 42 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:39:59.0 - 01:41:29.0 43 0 J1925+2106 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_PHASE#UNSPECIFIED]<br />
01:41:29.0 - 01:42:58.0 44 1 G55.7+3.4 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [OBSERVE_TARGET#UNSPECIFIED]<br />
[....]<br />
08:11:54.0 - 08:13:24.0 305 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:13:24.0 - 08:14:54.0 306 1 G55.7+3.4 9108 [0,1,2,3] [10, 10, 10, 10] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:14:54.0 - 08:16:24.0 307 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:16:24.0 - 08:17:54.0 308 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:17:54.0 - 08:19:23.0 309 2 0542+498=3C147 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:19:23.0 - 08:20:53.0 310 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:20:53.0 - 08:22:23.0 311 2 0542+498=3C147 9108 [0,1,2,3] [10, 10, 10, 10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:22:23.0 - 08:23:52.0 312 2 0542+498=3C147 9108 [0,1,2,3] [9.89, 9.89, 9.89, 9.89] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
08:23:52.0 - 08:25:22.0 313 2 0542+498=3C147 9108 [0,1,2,3] [10,10,10,10] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS]<br />
(nRows = Total number of rows per scan) <br />
Fields: 3<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 D J1925+2106 19:25:59.605371 +21.06.26.16218 J2000 0 391644<br />
1 NONE G55.7+3.4 19:21:40.000000 +21.45.00.00000 J2000 1 2277000<br />
2 N 0542+498=3C147 05:42:36.137916 +49.51.07.23356 J2000 2 63756<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 Subband:0 64 TOPO 1256.000 2000.000 128000.0 1319.0000 4 RR RL LR LL<br />
1 Subband:2 64 TOPO 1384.000 2000.000 128000.0 1447.0000 4 RR RL LR LL<br />
2 Subband:1 64 TOPO 1648.000 2000.000 128000.0 1711.0000 8 RR RL LR LL<br />
3 Subband:0 64 TOPO 1776.000 2000.000 128000.0 1839.0000 8 RR RL LR LL<br />
Sources: 12<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1925+2106 0 - - <br />
0 J1925+2106 1 - - <br />
0 J1925+2106 2 - - <br />
0 J1925+2106 3 - - <br />
1 G55.7+3.4 0 - - <br />
1 G55.7+3.4 1 - - <br />
1 G55.7+3.4 2 - - <br />
1 G55.7+3.4 3 - - <br />
2 0542+498=3C147 0 - - <br />
2 0542+498=3C147 1 - - <br />
2 0542+498=3C147 2 - - <br />
2 0542+498=3C147 3 - - <br />
Antennas: 23:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 ea01 W09 25.0 m -107.37.25.2 +33.53.51.0 -521.9416 -332.7766 -1.2001 -1601710.017000 -5042006.925200 3554602.355600<br />
1 ea02 E02 25.0 m -107.37.04.4 +33.54.01.1 9.8240 -20.4293 -2.7806 -1601150.060300 -5042000.619800 3554860.729400<br />
2 ea03 E09 25.0 m -107.36.45.1 +33.53.53.6 506.0564 -251.8670 -3.5825 -1600715.950800 -5042273.187000 3554668.184500<br />
3 ea04 W01 25.0 m -107.37.05.9 +33.54.00.5 -27.3562 -41.3030 -2.7418 -1601189.030140 -5042000.493300 3554843.425700<br />
4 ea05 W08 25.0 m -107.37.21.6 +33.53.53.0 -432.1167 -272.1478 -1.5054 -1601614.091000 -5042001.652900 3554652.509300<br />
5 ea07 E05 25.0 m -107.36.58.4 +33.53.58.8 164.9788 -92.8032 -2.5268 -1601014.462000 -5042086.252000 3554800.799800<br />
6 ea08 N01 25.0 m -107.37.06.0 +33.54.01.8 -30.8810 -1.4664 -2.8597 -1601185.634945 -5041978.156586 3554876.424700<br />
7 ea09 E06 25.0 m -107.36.55.6 +33.53.57.7 236.9058 -126.3369 -2.4443 -1600951.588000 -5042125.911000 3554773.012300<br />
8 ea10 N03 25.0 m -107.37.06.3 +33.54.04.8 -39.0773 93.0192 -3.3330 -1601177.376760 -5041925.073200 3554954.584100<br />
9 ea11 E04 25.0 m -107.37.00.8 +33.53.59.7 102.8054 -63.7682 -2.6414 -1601068.790300 -5042051.910200 3554824.835300<br />
10 ea12 E08 25.0 m -107.36.48.9 +33.53.55.1 407.8285 -206.0065 -3.2272 -1600801.926000 -5042219.366500 3554706.448200<br />
11 ea13 N07 25.0 m -107.37.07.2 +33.54.12.9 -61.1037 344.2331 -4.6138 -1601155.635800 -5041783.843800 3555162.374100<br />
12 ea15 W06 25.0 m -107.37.15.6 +33.53.56.4 -275.8288 -166.7451 -2.0590 -1601447.198000 -5041992.502500 3554739.687600<br />
13 ea16 W02 25.0 m -107.37.07.5 +33.54.00.9 -67.9687 -26.5614 -2.7175 -1601225.255200 -5041980.383590 3554855.675000<br />
14 ea18 N09 25.0 m -107.37.07.8 +33.54.19.0 -77.4346 530.6273 -5.5859 -1601139.485100 -5041679.036800 3555316.533200<br />
15 ea19 W04 25.0 m -107.37.10.8 +33.53.59.1 -152.8599 -83.8054 -2.4614 -1601315.893000 -5041985.320170 3554808.304600<br />
16 ea21 E01 25.0 m -107.37.05.7 +33.53.59.2 -23.8638 -81.1510 -2.5851 -1601192.467800 -5042022.856800 3554810.438800<br />
17 ea22 N04 25.0 m -107.37.06.5 +33.54.06.1 -42.6239 132.8436 -3.5494 -1601173.979400 -5041902.657700 3554987.517500<br />
18 ea23 E07 25.0 m -107.36.52.4 +33.53.56.5 318.0509 -164.1850 -2.6957 -1600880.571400 -5042170.388000 3554741.457400<br />
19 ea24 W05 25.0 m -107.37.13.0 +33.53.57.8 -210.0959 -122.3887 -2.2577 -1601377.009500 -5041988.665500 3554776.393400<br />
20 ea25 N02 25.0 m -107.37.06.2 +33.54.03.5 -35.6245 53.1806 -3.1345 -1601180.861480 -5041947.453400 3554921.628700<br />
21 ea27 E03 25.0 m -107.37.02.8 +33.54.00.5 50.6641 -39.4835 -2.7273 -1601114.365500 -5042023.151800 3554844.944000<br />
22 ea28 N08 25.0 m -107.37.07.5 +33.54.15.8 -68.9057 433.1889 -5.0602 -1601147.940400 -5041733.837000 3555235.956000<br />
</pre><br />
<br />
<div style="text-align: justify;"><br />
* J1925+2106, field ID 0: Phase (complex gain (amplitude and phase)) calibrator;<br />
* G55.7+3.4, field ID 1: The supernova remnant;<br />
* 0542+498=3C147, field ID 2: Flux density/bandpass calibrator.<br />
[[Image:plotAnts.png|300px|thumb|right|'''Figure 1''' <br />Antenna plot showing the configuration during the observing session.]]<br />
<br />
We can also see that these sources have associated scan intents that indicate their function in the observations. Note that you can select sources based on their intents in certain CASA tasks. The various scan intents in this data set are:<br />
<br />
* CALIBRATE_PHASE indicates that this is a scan to be used for complex gain (amplitude and phase) calibration;<br />
* OBSERVE_TARGET indicates that this is the science target;<br />
* CALIBRATE_AMPLI indicates that this is to be used for flux density calibration, and; <br />
* CALIBRATE_BANDPASS indicates that these scans are to be used for bandpass calibration.<br />
<br />
Note that more recent data sets may have different scan intents assigned to calibration scans.<br />
<!-- Also note that 3C147 is used for both flux density and bandpass calibration.--><br />
<br />
It's important to be aware that the antennas have both a name and an ID associated with them. For example antenna ID 15 is named ea19 (the ''ea'' stemming from the Expanded VLA project). When specifying an antenna within a task parameter, we will primarily reference them by their ''ea'' name. <br />
<br />
We can see the antenna configuration for this observing session by using {{plotants}}. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis='SNR_G55_10s.ms', figfile='SNR_G55_10s.plotants.png')<br />
</source><br />
<br />
Figure 1 the antenna configuration for our observations; antennas ea01, ea03, and ea18 were on the extreme ends of the west, east, and north arms, respectively. <br />
<br />
<!-- This is really not for this guide, so commented out for the time being. ADK, Oct 2018--><br />
<!-- The antenna position diagram is particularly useful as a guide to help determine which antenna to use as the reference antenna later during calibration. Antennas on station 8 of each arm (N08, E08, W08) do not get moved during array re-configurations; they can, therefore, at times be good choices as reference antennas. In this case, we'll probably want to choose something closer to the center of the array with no shadowing.--><br />
<br />
<br />
=== Identifying RFI with Plotms() ===<br />
[[Image:CASA5.4.0-FLAG-Plotms-amp-time.png|300px|thumb|right|'''Figure 2''' <br />Plotms image of supernova remnant data, showing strong RFI through the observation (x axis) and affecting a number of spectral windows (data are colorised by spw).]]<br />
<br />
The data can be inspected with the {{plotms}} task. <br />
<br />
It is worth first having a look at the data, separately per each field, in <b>amplitude vs. frequency</b> (to identify narrow-band RFI) and <b>amplitude vs. time</b> (intermittent RFI). <br />
<br />
To start with, let's look at the supernova remnant observations (field='1') for all scans (default) and all antennas (default):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', field='1', xaxis='time', yaxis='amp', coloraxis='spw')<br />
</source><br />
<br />
We can immediately see that there is plenty of RFI throughout the observations, affecting a number of spectral windows (data are colorised by spectral window; Figure 2). The RFI is discerned by the large spikes in amplitude.<br />
<br />
To narrow down where the RFI occurs let's now have a look at the data in amplitude vs frequency, plotted separately per each scan:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', scan='30,75,120,165,190,235,303', antenna='ea24', xaxis='freq', <br />
yaxis='amp', coloraxis='spw', iteraxis='scan')<br />
</source><br />
[[Image:amp_v_freq_rawdata.png|300px|thumb|right|'''Figure 3''' <br />Plotms image of scan 190 showing strong RFI spikes in amplitude for several spectral windows.]]<br />
* coloraxis='spw': Parameter indicates that a different color will be assigned to each spectral window.<br />
* antenna='ea24' : Here we chose only information for antenna ea24.<br />
* iteraxis='scan': Parameter tells plotms to iterate over scans, that is to display a new plot for each scan. <br />
<br />
The command above will set the {{plotms}} for iteration, but if you want to set this option from within {{plotms}}, go onto the Plot tab (horizontal), and then Page tab (vertical). You will now find Iteration options. Set Axis to the parameter you want to iterate through. Iterate by clicking on the green arrow buttons on the toolbar within the {{plotms}}. <br />
<br />
Progressing through to scan 190 , we can see there is significant time and frequency variable RFI present in the observing session. To determine which spectral windows are affected, click on the <b>Mark Regions</b> tool at the bottom of the {{plotms}} GUI (the open box with a green plus sign) and use the mouse to select a few of the highest-amplitude points in each of the spectral windows (Figure 3). Click on the <b>Locate</b> button (magnifying glass on white background). Information about the selected areas should now display in the logger window. For example:<br />
<br /><br />
</div><br />
<pre><br />
INFO PlotMS Frequency in [1.30517 1.31368] or [1.32584 1.33313] or [1.68146 1.69544], Amp in [0.132051 0.223077] or [0.124359 0.188034] or [0.139316 0.237179]:<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea01@W09 & ea24@W05[0&19] Spw=0 Chan=27 Freq=1.31 Corr=LR X=1.31 Y=0.139029 (110/144/110)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea03@E09 & ea24@W05[2&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.145391 (623/144/623)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea04@W01 & ea24@W05[3&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.166033 (879/144/879)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea04@W01 & ea24@W05[3&19] Spw=0 Chan=37 Freq=1.33 Corr=LR X=1.33 Y=0.131179 (918/144/918)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea07@E05 & ea24@W05[5&19] Spw=0 Chan=27 Freq=1.31 Corr=RL X=1.31 Y=0.14596 (1389/144/1389)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea07@E05 & ea24@W05[5&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.142578 (1391/144/1391)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea09@E06 & ea24@W05[7&19] Spw=0 Chan=27 Freq=1.31 Corr=RR X=1.31 Y=0.141398 (1900/144/1900)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea12@E08 & ea24@W05[10&19] Spw=0 Chan=37 Freq=1.33 Corr=LR X=1.33 Y=0.133163 (2710/144/2710)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea13@N07 & ea24@W05[11&19] Spw=0 Chan=37 Freq=1.33 Corr=RR X=1.33 Y=0.129079 (2964/144/2964)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea15@W06 & ea24@W05[12&19] Spw=0 Chan=27 Freq=1.31 Corr=RR X=1.31 Y=0.135901 (3180/144/3180)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea16@W02 & ea24@W05[13&19] Spw=0 Chan=27 Freq=1.31 Corr=RR X=1.31 Y=0.17187 (3436/144/3436)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea16@W02 & ea24@W05[13&19] Spw=0 Chan=27 Freq=1.31 Corr=LR X=1.31 Y=0.155113 (3438/144/3438)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea16@W02 & ea24@W05[13&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.142598 (3439/144/3439)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea19@W04 & ea24@W05[15&19] Spw=0 Chan=27 Freq=1.31 Corr=LR X=1.31 Y=0.154241 (3950/144/3950)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea19@W04 & ea24@W05[15&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.178607 (3951/144/3951)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea19@W04 & ea24@W05[15&19] Spw=0 Chan=37 Freq=1.33 Corr=RR X=1.33 Y=0.126879 (3988/144/3988)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea21@E01 & ea24@W05[16&19] Spw=0 Chan=27 Freq=1.31 Corr=RL X=1.31 Y=0.144868 (4205/144/4205)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea21@E01 & ea24@W05[16&19] Spw=0 Chan=27 Freq=1.31 Corr=LL X=1.31 Y=0.158044 (4207/144/4207)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:19:58.0 BL=ea22@N04 & ea24@W05[17&19] Spw=0 Chan=37 Freq=1.33 Corr=RR X=1.33 Y=0.18105 (4500/144/4500)<br />
<snip><br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:08.0 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RR X=1.686 Y=0.195303(10060/169/4428)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:08.0 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RL X=1.686 Y=0.19185 (10061/169/4429)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:08.0 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.169598 (10068/169/4436)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea08@N01 & ea24@W05[6&19] Spw=2 Chan=19 Freq=1.686 Corr=LR X=1.686 Y=0.157531(7246/170/1614)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea10@N03 & ea24@W05[8&19] Spw=2 Chan=19 Freq=1.686 Corr=LL X=1.686 Y=0.15168 (7759/170/2127)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea15@W06 & ea24@W05[12&19] Spw=2 Chan=19 Freq=1.686 Corr=LR X=1.686 Y=0.202071(8782/170/3150)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea18@N09 & ea24@W05[14&19] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.159526 (9300/170/3668)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea19@W04 & ea24@W05[15&19] Spw=2 Chan=19 Freq=1.686 Corr=RR X=1.686 Y=0.192973(9548/170/3916)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea19@W04 & ea24@W05[15&19] Spw=2 Chan=19 Freq=1.686 Corr=LR X=1.686 Y=0.155768(9550/170/3918)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea19@W04 & ea24@W05[15&19] Spw=2 Chan=19 Freq=1.686 Corr=LL X=1.686 Y=0.147995(9551/170/3919)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RR X=1.686 Y=0.199225(10060/170/4428)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=19 Freq=1.686 Corr=RL X=1.686 Y=0.193489(10061/170/4429)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea22@N04 & ea24@W05[17&19] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.179036 (10068/170/4436)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea24@W05 & ea25@N02[19&20] Spw=2 Chan=19 Freq=1.686 Corr=RL X=1.686 Y=0.156535(10573/170/4941)<br />
INFO PlotMS Scan=190 Field=G55.7+3.4[1] Time=2010/08/23/05:21:17.5 BL=ea24@W05 & ea25@N02[19&20] Spw=2 Chan=21 Freq=1.69 Corr=RR X=1.69 Y=0.140293 (10580/170/4948)<br />
INFO PlotMS Found 287 points (287 unflagged) among 202752 in 0.01s.<br />
</pre><br />
<div style="text-align: justify;"><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:RFI_1244-1372.png|380px|thumb|right|'''Figure 4a''' <br />Amplitude vs. frequency plot of a portion of the L-band spectrum, showing RFI spikes from different sources.]]<br />
| [[Image:amp_v_freq.spw0_Scan190.png|330px|thumb|right|'''Figure 4b''' <br />Amplitude vs. frequency plot of L-Band, spectral window 0, for scan 190. Generated by running the command "plotms(vis='SNR_G55_10s.ms', spw='0', scan='190', xaxis='freq', yaxis='amp')" within CASA.]]<br />
|}<br />
<br />
Reviewing the log output, we see that spw 0 and 2 are affected the most by RFI. We also get the corresponding frequency where the RFI is present, which appears to be 1.31 and 1.33 GHz for spectral window 0, and 1.686 GHz for spectral window 2. This information is important, especially when flagging data interactively as we will be doing towards the end of this guide.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', spw='0', scan='190', xaxis='freq',yaxis='amp')<br />
</source><br />
<br />
The VLA has obtained RFI sweeps over its full frequency range. The results are available on the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/rfi Radio Frequency Interference website.] Most of the RFI features can be identified as radar, communications, satellites, airplanes, or birdies (i.e., RFI generated by the VLA electronics itself). For our observations, we inspect the L-band specific RFI plots [https://science.nrao.edu/facilities/vla/observing/RFI/L-Band website] and find that the 1310 and 1330 MHz features are due to FAA ASR radars, while the one at 1686 MHz is most likely due to a GOES weather satellite. We can compare the plots of the FAA ASR radar from the website (Figure 4a), with that of spw 0 (Figure 4b), and see that the spikes are practically identical and fall within the same frequencies.<br />
<br />
=== Identifying RFI with MSview() ===<br />
<br />
Similar to using plotms() for the ms file editing, one can also use the [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] task.<br />
<br />
MSview() displays data in a 3D fashion, where the intensity of the plotted data points represents amplitude, phase, or their derivatives, while x and y axes of the display can be set to any of the following: time, baseline, channel, correlation or spectral window. For the introduction to the [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] tool see its dedicated [https://casaguides.nrao.edu/index.php/Data_flagging_with_viewer guide] (note that msview() can also be invoked as viewer(), as it is done in the linked guide).<br />
[[Image:CASA5.4.0-flagging-msview-load-data.png|300px|thumb|right|'''Figure 5''' <br /> Loading data into msview(). The red box marks where data preselection can be made.]]<br />
<br />
<b>Note:</b> msview() will not handle large data volumes very well, and it may crash if your ms file is too large. If that happens try preselecting a fraction of your data for viewing. Also, sometimes restarting CASA may help. <br />
<br />
Since the task does not allow for much data selection in the command line, open [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] tool first, and load and select the data from its graphical interface:<br />
<br />
<source lang="python"><br />
# In CASA<br />
msview()<br />
</source><br />
<br />
Once the tool opens, it will prompt you for data upload. At the same time as selecting the ms file to open, choose which field, spw etc you would like to load from that ms file as shown in Figure 5, and hit Raster Image button. Only the preselected data will be available in the viewer, and further selection for viewing can be done via Data Display Options window (wrench button, within [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()]) under its ''display axes'' tab. Note that the ''ms and visibility selection'' tab will inform you about the overall shape of your data, even if you have not loaded the full ms file.<br />
<br />
To identify RFI in [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()], one can look at the data in multiple ways. For example, let's plot <b>time vs. channel</b> for a single spectral window. We can set these selections in the Data Display Options (click on the Wrench button) as shown in Figure 6. The selection window will appear (highlighted with red box in Figure 6). We plotted here spw=2, which corresponds to the orange-coloured spw from Figure 3. The blue coloured pixels are fairly well behaved data, while the green and red pixels are the data affected by the RFI. We can narrow down where that RFI occurs by hovering cursor over the RFI areas (white box/magenta circle in Figure 6) and details on affected baselines, scans, etc will be listed in the Cursors window (highlighted with magenta box in Figure 6). Note that [https://casa.nrao.edu/casadocs/casa-5.5.0/global-task-list/task_msview/about msview()] refers to the antennas via their ID numbers and not the ''ea'' names (compare with the listobs output and commentary in section [[#Preliminary_Data_Evaluation | Preliminary Data Evaluation]]). Also, note that here only RR polarisation is plotted, and you will need to check also LL (as well as RL and LR if you have and use them in further data processing).<br />
<br />
One can also have a quick look at which spectral windows most RFI occurs, and when. For example, let's plot <b>channel vs. spectral window</b> and iterate over time Figure 7). You can choose iteration by selecting animation axis under the ''display axes'' tab within the Data Display Options (marked with red box in Figure 7). To inspect how the RFI changes with time use the iteration buttons within the Animators panel (marked with yellow box in Figure 7). Again, the blue colour marks the well behaved data here; you can change the colour scheme in the Data Display Option window.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:CASA5.4.0-flagging-Msview-time-chan-spw2.png|400px|thumb|right|'''Figure 6''' <br /> Time vs channel plot visualised with msview(). The data selection window for viewing is highlighted with the red box. The magenta box gives information on pixels which we hovered over with a cursor (white box/magenta circle) and which are affected by RFI.]]<br />
| [[Image:CASA5.4.0-flagging-Msview-chan-spw-iterate-time.png|400px|thumb|right|'''Figure 7''' <br /> Channel vs spectral window plotted with msview(). The data selection window for viewing is highlighted with the red box. The yellow box indicates buttons to play a movie over the iteration axis, which here is set to time.]]<br />
|}<br />
<br />
== Identifying Problematic Antennas from the Operator Logs ==<br />
<br />
We first check the operator log for the observing session to see if there were any issues noted during the run that need to be addressed. The logs are available from the [http://www.vla.nrao.edu/cgi-bin/oplogs.cgi VLA operator log website]. Here is the link for the [http://www.vla.nrao.edu/operators/logs/2010/8/2010-08-23_0005_AB1345.pdf observing log] file for our observing session.<br />
<br />
The log has various pieces of information including the start/end times for the observing session, frequency bands used, weather, baseline information for recently moved antennas, and any outages or issues that may have been encountered. We see that antenna ea07 may need position corrections (see one of our calibration tutorials, [https://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.5.0 IRC+10216], on how to fix this), and several antennas (ea06, ea17, ea20, and ea26) are missing an L-Band receiver. Antenna 07 position corrections will need to be applied during data calibration and will not be covered in this tutorial. Additionally, the antennas with missing receivers were already removed from the dataset used here (as explained in the above section Obtaining the Data), so we can continue with online flagging.<br />
<br /><br />
<br /><br />
<br />
== Online Flags ==<br />
<br />
At the time of importing from the SDM-BDF raw data to a MS, we chose to process the online flags from the Flags.xml file to the FLAG_CMD sub-table within the MS. We also created the ''SNR_G55.ms.onlineflags.txt'' file, a plain text document that includes the list of online flags. Note that this step is not necessary if you choose to apply online flags at the time of the data download from the archive.<br />
<br />
The Flags.xml file holds information of flags created during the observing session, such as subreflector issues and antennas not being on source. <br />
We will now apply these online flags to the data by employing {{flagcmd}}, but first, let's create a plot of the flags to get an idea of what will be flagged.<br />
<br />
[[Image:flaggingreason_vs_time.png|300px|thumb|right|'''Figure 8''' Online Flags]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagcmd(vis='SNR_G55_10s.ms', inpmode='table', reason='any', action='plot', plotfile='flaggingreason_vs_time.png')<br />
</source><br />
<br />
Open the ''flaggingreason_vs_time.png'' with your favorite image viewing program. Figure 8 shows several instances of online flagging. Most notably, ea28 and ea08 had some subreflector issues throughout the observing session. Online flags are instances of possible missing data, including:<br />
<br />
* ANTENNA_NOT_ON_SOURCE <br />
The VLA antennas have slewing speeds of 40 degrees per minute in azimuth and 20 degrees per minute in elevation. Some antennas are slower than others and may take a few more seconds to reach the next source. The antennas can also take a few seconds to settle down due to small oscillations after having slewed. <br />
<br />
* SUBREFLECTOR_ERROR <br />
The Focus Rotation Mount (FRM), located at the apex of the antenna, is responsible for focusing the incoming radio signal to the corresponding receiver. At times there can be issues with the focus and/or rotation axes.<br />
<br />
Now that we've plotted the online flags, we will apply them to the MS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagcmd(vis='SNR_G55_10s.ms', inpmode='table', reason='any', action='apply', flagbackup=False)<br />
</source><br />
<br />
* inpmode='table': Will use the online-flags imported to the FLAG_CMD sub-table from the Flags.xml file. This was done during the importing of our data with {{importasdm}}. Note that the FLAG_CMD sub-table already includes a 1.5 second time buffer, as was requested during the importing of the data (parameter ''tbuff=1.5'').<br />
<br />
The CASA logger should report the progress as the task applies these flags in segments. Once finished, it will report the percentage of flagged data. The terminal window will display warnings about the four missing antennas; this is expected as we had previously removed the antennas during the split task.<br />
<br />
Alternatively, we could have applied the online flags from the text file (''SNR_G55.ms.onlineflags.txt'') created during the importing of the data. The one caveat when it comes to applying flags from a list is not being able to un-apply them (setting parameter ''action='unapply' ''within {{flagcmd}}. Transferring the flags to the FLAGS_CMD table ''before'' applying them ''does'' allow for the use of the un-apply feature. For convenience, we provide the command for applying online flags from a list (don't run the following command in CASA):<br />
<br />
<pre><br />
flagcmd(vis='SNR_G55_10s.ms', inpmode='list', inpfile='SNR_G55.ms.onlineflags.txt', reason='any', action='apply', flagbackup=False)<br />
</pre><br />
<br />
== Shadowed Antennas ==<br />
<br />
The VLA D-configuration is the most compact; there may be instances where one antenna blocks, or shadows another. For more on shadowing, please refer to the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/dynsched#section-1 antenna shadowing] section in the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide Guide to Observing with the VLA]. Generally, observing sources at 40 degrees elevation and higher will result in less shadowing. We will create a plot of elevation vs. time by using the {{plotms}} task and note the elevation of the radio sources we observed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', xaxis='time', yaxis='elevation', antenna='0&1;2&3', spw='*:31',<br />
coloraxis='field', title='Elevation vs. Time', plotfile='Elevation_vs_Time.png')<br />
</source><br />
<br />
[[Image:Elevation_vs_Time.png|300px|thumb|right|'''Figure 9''' Plot of elevation vs. time.]]<br />
<br />
Figure 9 shows that most of the sources were at or above 40 degrees in elevation; accordingly, we expect a minimal amount of shadowing. The flux density / bandpass calibrator 3C147 was observed towards the end of the session, which could result in some shadowing of antennas due to its low elevation of about 20&ndash;22 degrees. <br />
<br />
Task {{flagdata}} can determine and flag shadowed antennas by their location and observing direction:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s.ms', mode='shadow', tolerance=0.0, flagbackup=False)<br />
</source><br />
* ''mode='shadow''': has a subparameter ''tolerance=0.0'' that controls how many meters of shadowing overlap is allowed<br />
<br />
The logger will report on the percentage of flagged data due to shadowing. As expected in this observing session, there does not appear to be much data affected by shadowing (approx. 0.4%).<br />
<br />
== Zero-Amplitude Data ==<br />
<br />
In addition to shadowing, there may be times during which the correlator writes out pure zero-valued data. In order to remove this bad data, we run {{flagdata}} to remove any pure zeroes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s.ms', mode='clip', clipzeros=True, flagbackup=False) <br />
</source><br />
* ''mode='clip''': is used to flag all values below a given threshold. With ''clipzeros=True'' this mode will flag exact zero values that are sometimes being produced by the VLA correlator.<br />
<br />
Inspecting the logger output generated by {{flagdata}} shows that there is a small quantity of zero-valued data present in this MS.<br />
<br />
== Quacking ==<br />
<br />
Now we utilize the flagdata task one more time to run quacking mode.<br />
<br />
It's common for the array to settle down at the start of a scan. Quacking is used to remove data at scan boundaries, the same edit can be applied to all scans for all baselines. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s.ms', mode='quack', quackinterval=5.0, quackmode='beg', flagbackup=False)<br />
</source><br />
<br />
*quackmode='beg': Data from the start of each scan will be flagged.<br />
*quackinterval=5.0: Flag the first 5 seconds of every scan.<br />
<br />
<br />
== Backup Data - Flagmanager ==<br />
<br />
Flags can be backed up in a file ''ms.flagversions'' that is related to the MS. For our MS, the related flag backups are stored in ''SNR_G55_10s.ms.flagversions''. <br />
<br />
'''Note: flags that are applied to the data are contained in the MS;''' '''''ms.flagversions''''' '''only contains backup flags that can be restored if required.'''<br />
<br />
Most flagging tasks, like {{flagdata}}, offer a ''flagbackup'' parameter that controls whether or not the current flags are being backed up in ms.flagversions before new (additional) flags will be applied. Backup flags can also be manually saved to or restored from ms.flagversions using the {{flagmanager}} task. <br />
<br />
Now that we've applied online flags, clipped zero amplitude data, removed shadowed data, and quacked the data, we will create a backup of the flags by setting the parameter ''versionname='after_online_flagging' '': <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='SNR_G55_10s.ms', mode='save', versionname='after_online_flagging')<br />
</source><br />
<br />
The version we just created is saved inside the SNR_G55_10s.ms.flagversions directory. From here onward, if we make a mistake, we can always revert back to this flag version of the MS by setting parameter ''mode='restore' '' and providing the version name we want to revert back to, such as:<br />
<br />
<pre><br />
flagmanager(vis='SNR_G55_10s.ms', mode='restore', versionname='after_online_flagging')<br />
</pre><br />
<br />
The flagmanager() can also be run on calibration tables. In this instance just specify the table name as the ''vis'' parameter and run is as above.<br />
<br />
== Benefit of Hanning Smoothing ==<br />
<br />
Strong RFI sources can give rise to the Gibbs phenomenon. This is seen by ringing: a zig-zag pattern across the channels that neighbor the strong, usually narrow, RFI. To remedy this ringing across the frequency channels, we employ the Hanningsmooth algorithm via the {{hanningsmooth}} task (an implementation based on {{mstransform}}). Hanning-smoothing applies a triangle kernel across the pattern which diminishes the ringing, reducing the number of channels that may look bad and get flagged. Note that this smoothing procedure will also decrease the spectral resolution by a factor of two.<br />
<br />
The task requires the creation of a new MS, which we will call ''SNR_G55_10s-hanning.ms''. <br />
<br />
<b>Note: As Hanning-smoothing will remove amplitude spikes, it is not recommended for spectral analysis related science (e.g. strong narrow maser lines).</b><br />
<br />
Let's create before (Figure 10a) and after images (Figure 10b) with the {{plotms}} task to see the effect. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s.ms', scan='190', antenna='ea24', spw='0~2',<br />
xaxis='freq', yaxis='amp', coloraxis='spw', title='Before Hanning',<br />
correlation='RR,LL', plotrange=[1.2,1.8,-0.01,0.25],<br />
plotfile='amp_v_freq.beforeHanning.png')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
hanningsmooth(vis='SNR_G55_10s.ms', outputvis='SNR_G55_10s-hanning.ms', datacolumn='data')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190', antenna='ea24', spw='0~2',<br />
xaxis='freq', yaxis='amp', coloraxis='spw', title='After Hanning',<br />
correlation='RR,LL', plotrange=[1.2,1.8,-0.01,0.25],<br />
plotfile='amp_v_freq.afterHanning.png')<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:amp_v_freq.beforeHanning.png|350px|thumb|right|'''Figure 10a''' <br />Plot of amplitude vs. frequency before Hanning smoothing data.]]<br />
| [[Image:amp_v_freq.afterHanning.png|350px|thumb|right|'''Figure 10b''' <br />Plot of amplitude vs. frequency after Hanning smoothing data.]]<br />
|}<br />
<br />
Figure 10b shows the effect of applying Hanning-smoothing. Notice that single channel RFI has spread into three channels. This spreading of RFI to other channels will ultimately result in a little more data being flagged.<br />
<br />
== Automatic RFI Excision ==<br />
<br />
Now that we're done with online flagging, we can move on to removing some of the RFI present with auto-flagging algorithms used within {{flagdata}}, TFCrop and rflag. For further details on the two algorithm based tasks we will employ, please see the [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/EVLAWorkshop_RFI_2016_UR.pdf RFI presentation] given at the 5th VLA Data Reduction Workshop.<br />
<br />
=== TFCrop ===<br />
<br />
Task {{flagdata}}'s TFCrop is an algorithm that detects outliers in the 2D time-frequency plane and can operate on uncalibrated (non-bandpass corrected) data. TFCrop will iterate through segments of time and undergo several steps in order to find and excise different types of RFI:<br />
<br />
Step 1: Detect short-duration RFI spikes (narrow- and broad-band). <br /><br />
Step 2: Search for time-persistent RFI. <br /><br />
Step 3: Search for time-persistent, narrow-band RFI. <br /><br />
Step 4: Search for low-level wings of very strong RFI. <br /><br />
<br />
[[Image:amp_v_freq_before_tfcrop.png|350px|thumb|right|'''Figure 7''' <br />Amplitude vs. Frequency plot of Scan 190, before the TFCrop algorithm is applied.]]<br />
We will apply the auto-flagging TFCrop algorithm for each spectral window. With TFCrop, it's a good idea to first inspect a small portion of data and review what and how much will be flagged. Once you are comfortable with the results, you can apply the algorithm to the remaining data. We will walk through the first spw and then include the remaining spws with the same command. First plot the corrected data for scan 190 (Figure 11) with all spectral windows so we can compare the data before and after TFCrop.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190', antenna='ea24', xaxis='freq', iteraxis='scan', yaxis='amp',<br />
ydatacolumn='data', plotfile='amp_v_freq_before_tfcrop.png', title='Before TFCrop',<br />
coloraxis='spw', plotrange=[1.2,2,-0.01,0.25])<br />
</source><br />
<br />
The following are a set of {{flagdata}} commands which have been found to work reasonably well with these data. Take some time to play with the parameters, and the plotting capabilities, as this will be important when you are working with your own data later on. Because we set parameters ''display='both' ''and ''action='calculate', '' the flags are displayed but not actually written to the MS. This allows one to try different sets of parameters before actually applying the flags to the data.<br />
[[Image:Tfcrop.png|350px|thumb|right|'''Figure 12''' <br />TFCrop flagging results. Top row is before, and bottom row is after TFCrop has applied flagging.]]<br />
<br />
Some representative TFCrop plots are displayed (Figure 12). Each column displays an individual polarization product; all four polarizations are, from left to right, RR, RL, LR, and LL. The first row shows the data with current flags applied and the second includes the flags generated by flagdata. The x-axis is channel number (the spectral window ID is displayed in the top title) and the y-axis of the first two rows is all integrations included in a time segment, set by the parameter ''ntime''. These are the data considered by the TFCrop algorithm during its flagging process, and changes in ''ntime'' will have some (relatively small) effect on what data are flagged. <br />
<br />
Each plot page displays data for a single baseline and time segment, with colours representing the intensity of the visibilities (amplitude) in the heat colour scheme ranging from black for lowest amplitude points to white for highest amplitude data points. The flagged, or to be flagged, visibilities are masked in blue. The buttons at the bottom allow one to step through baseline (backward and forward), spw, scan, and field while Stop Display will continue the flagging operation without the GUI, and Quit aborts the run.<br />
<br />
''' spw 0 '''<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='tfcrop', spw='0',<br />
datacolumn='data', action='calculate',<br />
display='both', flagbackup=False)<br />
</source><br />
<br />
The first two scans (14,15) are fully flagged (hence displayed as completely blue). Click on Next Scan until you reach scan 17 where it will be easier to see the flagging the algorithm performs. The top row shows current flags already applied to the data. The bottom row illustrates the flagging that can be applied with TFCrop, represented by the additional blue areas. Iterate through several scans and baselines. Once you're done, click on the Quit button on the bottom right corner. Let's now apply these flags by changing the action parameter.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='tfcrop', spw='0',<br />
datacolumn='data', action='apply',<br />
display='', flagbackup=False)<br />
</source><br />
<br />
The logger will report the percentage of flagged data in the table selection. We can now apply the TFCrop algorithm to the remaining spectral windows. <br />
<br />
''' spw 1, 2, 3 '''<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='tfcrop', spw='1~3',<br />
datacolumn='data', action='apply',<br />
display='both', flagbackup=False)<br />
</source><br />
Iterate through several scans and baselines and, once you're ready to apply the flags, click on Stop Display. After TFCrop has gone through and flagged some of the worst RFI, we can inspect the log report and take note of how much has been flagged.<br />
<br />
We can also use {{plotms}} to review the effects of using TFCrop. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190', antenna='ea24', xaxis='freq', iteraxis='scan', yaxis='amp',<br />
ydatacolumn='data', plotfile='amp_v_freq_after_tfcrop.png', title='After TFCrop',<br />
correlation='RR,LL', coloraxis='spw', plotrange=[1.2,2,-0.01,0.25])<br />
</source><br />
<br />
Compare the plots before applying TFCrop (Figure 11) to the plot after applying TFCrop (Figure 13). Figure 13 shows great improvements, especially for spectral window 0 and 2, which had some of the worst RFI.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:amp_v_freq_before_tfcrop.png|350px|thumb|right|'''Figure 11''' <br />Amplitude vs. Frequency plot of Scan 190, before the TFCrop algorithm is applied.]]<br />
| [[Image:amp_v_freq_after_tfcrop_Scan190.png|350px|thumb|right|'''Figure 13''' <br />Plot of amplitude vs. frequency showing the effects of applying TFCrop to our data, for scan 190.]]<br />
|}<br />
<br />
''' Flagging Summary '''<br />
<br />
We now calculate the amount of data flagged so far in our Measurement Set by using the {{flagdata}} task with parameter ''mode='summary' ''and assign the returned Python dictionary to the variable flagInfo. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagInfo = flagdata(vis='SNR_G55_10s-hanning.ms', mode='summary')<br />
</source><br /><br />
<br />
Now we parse the dictionary to print some of the flagging statistics, first per source, then per spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
print("\n %2.1f%% of G55.7+3.4, %2.1f%% of 3C147, and %2.1f%% of J1925+2106 are flagged. \n" % <br />
(100.0 * flagInfo['field']['G55.7+3.4']['flagged'] / flagInfo['field']['G55.7+3.4']['total'], <br />
100.0 * flagInfo['field']['0542+498=3C147']['flagged'] / flagInfo['field']['0542+498=3C147']['total'], <br />
100.0 * flagInfo['field']['J1925+2106']['flagged'] / flagInfo['field']['J1925+2106']['total']))<br />
</source><br /><br />
<br />
Note: IPython currently prevents copy/pasting several lines of code. To remedy this, we will use the %cpaste command to allow multiple lines to be placed in the terminal. Make sure you also copy the "--" characters to close the %cpaste input.<br />
<br />
<source lang="python"><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
print("Spectral windows are flagged as follows:")<br />
<br />
for spw in range(0,4):<br />
print("SPW %s: %2.1f%%" % (spw, 100.0 * flagInfo['spw'][str(spw)]['flagged'] / flagInfo['spw'][str(spw)]['total']))<br />
--<br />
</source><br />
<br />
We can now move to the other auto-flagging algorithm, rflag.<br />
<br />
=== RFlag ===<br />
<br />
RFlag, like TFCrop, is an autoflag algorithm which uses a sliding window statistical filter. Data are iterated through in segments of time where statistics are accumulated and thresholds calculated. <br />
<br />
'''WARNING:''' It is important to not assume that rflag can run on your target. This is especially important for spectral lines, as it may mistake them for interference. <br />
<br />
'''IMPORTANT:''' Task {{rflag}} should be executed <u>on calibrated data only</U>. For more details, please see Emmanuel Momjian's presentation on [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/E_Momjian_Data_Reduction_Techniques_Calibration_2016.pdf VLA Data Reduction Techniques and Calibration] given during the [https://science.nrao.edu/science/meetings/2016/vla-data-reduction 5th VLA Data Reduction Workshop].<br />
<br />
<div style="background-color: #eeeeee; margin: 40px"><br />
<div style="background-color: #eeeeee; margin: 43px"><br />
<BR><br />
==== Bandpass calibration (to demonstrate RFlag) ====<br />
<br />
For the purpose of demonstrating the functionality of RFlag on our data, we need to perform bandpass calibration first. Here, we just briefly outline the bandpass calibration process. For detailed calibration tutorial see [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials#VLA_Data_Reduction_Tutorials VLA Data Reduction Tutorials].<br />
<br />
First, run {{gaincal}} to solve for an initial set of antenna-based phases over a narrow range of channels. To find channels for each spectral window that are relatively RFI-free over the course of the observing session, look at the data with {{plotms}}. Use only a narrow channel range.<br />
<!-- <br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
<br />
In order to get the best possible result from the automatic RFI excision with {{flagdata}}'s mode ''rflag'', we will first apply bandpass calibration to the MS. Since the RFI is time-variable, using the phase calibration source to make an average bandpass over the entire observing session will mitigate the amount of RFI present in the calculated bandpass. For the final calibration, we will use the designated bandpass source 3C147, which will give a much higher signal to noise in the bandpass. Since 3C147 was only observed in the last set of scans, it doesn't sample the time variability and would not provide a good average bandpass.<br />
<br />
Since there are likely to be gain variations over the course of the observing session, we will run {{gaincal}} to solve for an initial set of antenna-based phases over a narrow range of channels. Those solutions will be applied to the data when the bandpass solutions are determined. While amplitude variations will have little effect on the bandpass solutions, it is important to solve for these phase variations with sufficient time resolution to prevent decorrelation when vector averaging the data in computing the bandpass solutions.<br />
<br />
In order to choose a narrow range of channels for each spectral window that are relatively RFI-free over the course of the observing session, we can look at the data with {{plotms}}. Note that it's important to solve only for phase using a narrow channel range, since an antenna-specific delay will cause the phase to vary with respect to frequency over the spectral window, perhaps by a substantial amount.<br />
--><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='42,65,88,11,134,157', antenna='ea24', xaxis='channel', <br />
yaxis='amp', iteraxis='spw')<br />
</source><br />
<br />
Iterating over each spectral window plot, search for channel ranges with stable amplitudes. An example of a few appropriate channel ranges include: SPW 0:20~24, SPW 1:49~52, SPW 2:38~41, SPW 3:41~44.<br />
<br />
Using these ranges, run {{gaincal}} to calculate phase-only solutions that will be used as input during the initial bandpass calibration. Remember: the calibration tables we are creating now are so that we can use an automatic RFI flagging algorithm. The final calibration tables will be generated later, after automated flagging. Here are the inputs for the initial pre-bandpass phase calibration:<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='SNR_G55_10s-hanning.ms', caltable='SNR_G55_10s-hanning.initPh', field='J1925+2106', <br />
solint=' int ', spw='0:20~24,1:49~52,2:38~41,3:41~44', refant='ea24', minblperant=3, <br />
minsnr=3.0, calmode='p')<br />
</source><br />
<!--<br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
<br />
* caltable='SNR_G55_10s-hanning.initPh': this is the output calibration table that will be written.<br />
* field='J1925+2106': this is the phase calibrator we will use to calibrate the phases.<br />
* solint=' int ': we request a solution for each 10-second integration.<br />
* spw='0:20~24,1:49~52,2:38~41,3:41~44': note the syntax of this selection: a ''''':''''' is used to separate the SPW from channel selection and '''''~''''' is used to indicate an inclusive range. <br />
* refant='ea24': we have chosen ea24 as the reference antenna after inspecting the antenna position diagram (see above). It is relatively close to the center of the array. Since the antennas in the very center have a high probability of being shadowed by nearby antennas, choose one that is not directly in the center. This is most important for the more compact (D, C) array configurations.<br />
[[Image:gain.phase_v_time.AllAnt.plotcal.png|300px|thumb|right|'''Figure 10b''' <br /> Gain Phase vs. Time for spw0, for all antennas, with plotsymbol='-', in order to connect the data points and inspect for phase jumps.]]<br />
* minblperant=3: the minimum number of baselines which must be present to attempt a phase solution.<br />
* minsnr=3.0: the minimum signal-to-noise a solution must have to be considered acceptable. Note that solutions which fail this test will cause these data to be flagged downstream of this calibration step.<br />
* calmode='p': perform phase-only solutions.<br />
<br />
Note that a number of solutions do not pass the requirements of the minimum 3 baselines (generating the terminal message''' "Insufficient unflagged antennas to proceed with this solve."''') or minimum signal-to-noise ratio (outputting''' "n of x solutions rejected due to SNR being less than 3 ..."'''). The logger output indicates 320-324 solutions succeeded out of 387 attempted. <br />
[[Image:gain.phase_v_time.plotcal.png|300px|thumb|right|'''Figure 10a''' <br /> Gain Phase vs. Time for spw0, for four antennas.]]<br />
--><br />
[[Image:CASA5.4.0-Plotms-pahse-time.png|300px|thumb|right|'''Figure 14''' <br /> Gain Phase vs. Time for spw0, for four antennas.]]<br />
<br />
It is always a good idea to inspect the resulting calibration table. We can do this with {{plotms}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.initPh', xaxis='time', yaxis='phase', <br />
iteraxis='spw', antenna='ea01,ea05,ea10,ea24', coloraxis='Antenna1')<br />
</source><br />
<br />
<br />
Figure 14 shows the phases do not change much over the course of the observing session. <br />
<!-- <br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
We deliberately chose to only plot four antennas in our last call to plotcal, as plotting all of them together will make the plot look convoluted which, in turn, makes it difficult to determine any phase jumps that may be present. To remedy this, we will employ plotcal again with the plotsymbol parameter set to connect the data points.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='SNR_G55_10s-hanning.initPh', xaxis='time', yaxis='phase', <br />
iteration='spw', antenna='', plotsymbol = '-', plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
Figure 10b may look to have phase jumps, but the V-shapes present are the program attempting to wrap the data points. As a user exercise, plot the phase from -180 to 0, and 0 to +180 to get the stable lines.<br />
--><br />
<br />
Since the plots look fairly reasonable, we will now create a time-averaged bandpass solution for the phase calibration source using the {{bandpass}} task.<br />
<br />
<source lang="python"><br />
# In CASA<br />
bandpass(vis='SNR_G55_10s-hanning.ms', caltable='SNR_G55_10s-hanning.initBP', <br />
field='J1925+2106', solint=' inf ', combine='scan', refant='ea24', <br />
minblperant=3, minsnr=10.0, gaintable='SNR_G55_10s-hanning.initPh', <br />
interp='nearest', solnorm=False)<br />
</source><br />
<!--<br />
THIS IS NOT NEEDED FOR THIS FLAGGING TUTORIAL, REF TO CALIBRATION TUTORIALS GIVEN<br />
<br />
* solint=' inf ', combine='scan': the solution interval of infinite, up to the boundaries controlled by the combine parameter; this requests that the solution intervals be combined over scans, so that we will get one solution per antenna.<br />
* gaintable= 'SNR_G55_10s-hanning.initPh': we will pre-apply the initial phase solutions.<br />
* interp='nearest': by default, {{bandpass}} will use linear interpolation for pre-applied calibration. However, we want the <i>nearest</i> phase solution to be used for a given time.<br />
<br />
Again, we can see that a number of solutions have been rejected by our choice of <tt>minsnr</tt>.<br />
--><br />
<br />
Now, inspect the resulting bandpass plots with plotms (Figures 15a,b,c). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.initBP',xaxis='freq', yaxis='amp', iteraxis='antenna', <br />
coloraxis='corr',gridrows=2)<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:CASA5.4.0-Plotms-amp-freq-ea01-02.png|300px|thumb|right|'''Figure 15a''' <br /> Bandpass for antennas ea01 and ea02.]]<br />
| [[Image:CASA5.4.0-Plotms-amp-freq-ea03-04.png|300px|thumb|right|'''Figure 15b''' <br /> Bandpass for antennas ea03 and ea04.]]<br />
| [[Image:CASA5.4.0-Plotms-amp-freq-ea05-07.png|300px|thumb|right|'''Figure 15c''' <br /> Bandpass for antennas ea05 - ea07 (note ea06 is flagged).]]<br />
|}<br />
<br />
<br><br />
</div><br />
</div><br />
<br />
==== Interactive flagging of bad solutions (calibration tables) ====<br />
<br />
[[Image:CASA5.4.0-Plotms-amp-freq-ea01-05-flagging.png|300px|thumb|right|'''Figure 16''' <br /> Interactive plotms flagging of the offset points for ea01 and ea05 in the calibration table.]]<br />
We notice that antennas ea01 and ea05 have a point that is offset from the rest (Figures 15a,c). Let's plot just these two antennas, locate the point on the plot, and flag it interactively through {{plotms}}. The interactive flagging within plotms will not create a backup, therefore it may be wise to use {{flagmanager}} before doing so (see Section [[#Backup_Data_-_Flagmanager |Flagmanager]] earlier in this tutorial). <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='SNR_G55_10s-hanning.initBP', mode='save', versionname='before_flagging')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.initBP',xaxis='freq', yaxis='amp', iteraxis='antenna', <br />
antenna='ea01,ea05', coloraxis='corr', gridrows=2)<br />
</source><br />
<br />
We will highlight the points by clicking on the <b>Mark Region</b> button, drawing boxes over the points, and clicking on the <b>Flag</b> button. Before doing this, one could also get information on the points by clicking on the Locate button, which will display details about the highlighted regions (Figure 16). After having flagged the offset points, your plots should update.<br />
<br />
<div style="background-color: #eeeeee; margin: 40px"><br />
<div style="background-color: #eeeeee; margin: 43px"><br />
<br><br />
<br />
==== Applying calibration ====<br />
<br />
We will now apply the bandpass calibration table using {{applycal}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='SNR_G55_10s-hanning.ms', gaintable='SNR_G55_10s-hanning.initBP', calwt=False)<br />
</source><br />
<br />
This operation will flag data that correspond to flagged solutions, so {{applycal}} makes a backup version of the flags prior to operating on the data. Running this task might take a little while.<br />
<br />
<br><br />
</div><br />
</div><br />
<br />
==== Running RFlag on bandpass calibrated data ====<br />
[[Image:amp_v_freq.beforeRFlag.png|320px|thumb|right|'''Figure 17''' <br />Amplitude vs. Frequency plot of scan 190, before RFlag is applied.]]<br />
<br />
Now that we have bandpass-corrected data, we will run {{flagdata}} in rflag mode.<br />
<br />
We will use flagdata with parameters ''mode='rflag','' and ''action='calculate' ''to first review the amount of data to be flagged. We will also change the parameter ''datacolumn='corrected' ''since we've applied the bandpass corrections to the MS and, in the process, created the ''corrected_data'' column in the MS table. <br />
<br />
''' spw 0 '''<br />
<br />
First, let's create a plot of our corrected data before we apply RFlag (Figure 17).<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190' , antenna='ea24', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', coloraxis='spw', title='Before RFlag', <br />
plotfile='amp_v_freq.beforeRFlag.png') <br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='0', datacolumn='corrected', <br />
action='calculate', display='both', flagbackup=False)<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:rflag_default_values.png|400px|thumb|right|'''Figure 18''' <br />RFlag with Default Parameters for scan 22 (use the next scan, next baseline, and next field buttons to see this specific screen).]]<br />
| [[Image:rflag_freq2.5_time3.5.png|400px|thumb|right|'''Figure 19''' <br />RFlag with freqdevscale=2.5, and timedevscale=3.5 for scan 22. We can see more data are being flagged with these more stringent values.]]<br />
|}<br />
<br />
After reviewing the calculated flags, we can see that a lot of RFI is being missed (Figure 18). We will need to modify the default parameters for this spectral window. Click on Quit on the lower right hand side of the window. Let's review the parameters for {{flagdata}} and modify them:<br />
<br />
<source lang="python"><br />
# In CASA, get the last called parameter values for flagdata<br />
tget flagdata<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, inspect the inputs<br />
inp<br />
</source><br />
<br />
The default ''timedevscale=5.0'' and ''freqdevscale=5.0'' parameters are not flagging enough bad data. We can provide more stringent values to the time/freq deviation scales parameters. A value smaller than the default (5.0) will flag more data in either the time or frequency axes, as can be seen on the plots in the displayed window. A value larger than the default will result in less flagged data. <br />
<br />
Since we know spw 0 has lots of RFI, we will change parameters ''freqdevscale=2.5'' and ''timedevscale=3.5''. We will also change parameter ''action='apply'.'' Note that we can always decide that this still isn't good enough and click on <b>Quit</b> to change our parameter values (will not perform flagging). Once the window opens, proceed to scan 22 (Figure 19), review what will be flagged, and click on <b>Stop Display</b> (as we did during TFCrop) to allow the task to continue with the flagging.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='0', datacolumn='corrected', <br />
freqdevscale=2.5, timedevscale=3.5, action='apply', display='both', flagbackup=False)<br />
</source><br />
<br />
Although RFlag has done a pretty good job of finding the bad data, some still remains. One way to excise the remaining bad data is to use the parameter ''mode='extend' ''feature of flagdata, which can extend flags along a chosen axis and removes islands of small data patches in the midst of flagged data. We first extend the flags across polarization so if any one polarization is flagged, all data for that time / channel will be flagged:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='0', extendpols=True, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
Now we extend the flags in time and frequency using the parameters ''growtime'' and ''growfreq''. For this data, the rflag algorithm seems most likely to miss RFI that should be flagged along more of the time axis. We will try with parameter ''growtime=50.0'', which will flag all data for a given channel if more than 50% of that channel's time is already flagged, and parameter ''growfreq=90.0'', which will flag the entire spectrum for an integration if more than 90% of the channels in that integration are already flagged. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='0', growtime=50.0, growfreq=90.0, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
''' spw 1, 2, and 3 '''<br />
<br />
Now, let's work on SPW's 1,2, and 3. We've chosen parameter ''display='both' ''in order to first review the flags to decide if too much or too little is being flagged. We can always click on Quit and decide to change parameter values for ''freqdevscale'' and ''timedevscale'' as we did with spectral window 0. Note that spw 2 has more stringent parameter values compared to spw 1 and 3, as it contains more RFI. To allow the flagging to continue, click on <b>Stop Display</b>.<br />
<br />
<source lang="python"><br />
# In CASA, run RFlag on spectral windows 1 & 3<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='1,3', datacolumn='corrected', <br />
freqdevscale=4.0, timedevscale=4.0, action='apply', display='both', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in polarization for spectral windows 1 & 3<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='1,3', extendpols=True, action='apply', <br />
display='', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in time and frequency for spectral windows 1 & 3<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='1,3', growtime=50.0, growfreq=90.0, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, run RFlag on spectral window 2 with more RFI, hence smaller frequency and time deviation scale values<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='rflag', spw='2', datacolumn='corrected', <br />
freqdevscale=2.5, timedevscale=2.5, action='apply', display='both', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in polarization for spectral window 2<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='2', extendpols=True, action='apply',<br />
display='', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA, extend flags in time and frequency for spectral window 2<br />
flagdata(vis='SNR_G55_10s-hanning.ms', mode='extend', spw='2', growtime=50.0, growfreq=90.0, <br />
action='apply', display='', flagbackup=False)<br />
</source><br />
<br />
Let's create an after RFlag plot to see the improvements (Figure 20). Compare the resulting plot to the before TFCrop plot (Figure 17).<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='190' , antenna='ea24', xaxis='freq', yaxis='amp', <br />
ydatacolumn='corrected', coloraxis='spw', title='After RFlag', plotfile='amp_v_freq.afterRFlag.png',<br />
plotrange=[1.2,2,0,2.5])<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:amp_v_freq.beforeRFlag.png|350px|thumb|right|'''Figure 17''' <br />Amplitude vs. Frequency plot of scan 190, before RFlag is applied.]]<br />
| [[Image:amp_v_freq.before_after_RFlag.png|350px|thumb|right|'''Figure 20''' <br /> Plot of amplitude vs. frequency of scan 190, after having run RFlag.]]<br />
|}<br />
<br />
<b>Summary of flagdata modes tfcrop and rflag</b><br><br />
Tfcrop algorithim iterates through the data in sections of time, creates a 2D time-frequency (x-y) plane with amplitude in the z axis, one for each baseline and correlation-expression to flag outliers. <br />
RFlag algorithm also iterates through the data in sections of time, but compares local rms versus global statistics of the data to remove outliers. For more info on TFCrop and RFlag, please see the [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/EVLAWorkshop_RFI_2016_UR.pdf RFI presentation] given at the 5th VLA Data Reduction Workshop.<br />
<br />
{| class='wikitable'<br />
|-<br />
!Case<br />
!Tfcrop<br />
!RFlag<br />
|-<br />
| Strong peaks in RFI ||works well.||works well, though continuous RFI (time or frequency) needs tuning.<br />
|-<br />
|Noisy RFI ||Not good for noisy, low level RFI.||works well.<br />
|-<br />
|Broadband RFI ||Not robust, but possible by tuning the polynomial fit (maxnpieces).|| Good if RFI looks noisy. Continuous RFI needs work.<br />
|-<br />
|Non-calibrated data ||Yes, because it can fit for the bandpass.||No, it needs a flat base. Calibration is essential.<br />
|-<br />
|Extended emission|| Yes, solved for each baseline.||high flux on short baselines biases the statistics (use uvrange, or operate on residual data).<br />
|}<br />
<br />
== Interactive Flagging ==<br />
<br />
To flag data interactively (manually), one can use {{plotms}}. We will be scanning for data points that look odd. Please note that flagging data through plotms will not create a backup, so it's important to use the flagmanager() before deciding to mark your regions for flagging purposes. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='SNR_G55_10s-hanning.ms', mode='save',<br />
versionname='before_interactive_flagging')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='50~100', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', coloraxis = 'spw')<br />
</source><br />
<br />
Figure 21 (next section) shows some points in green that have higher amplitudes than the rest. Use the <b>Mark Regions</b> button (square with green plus sign) to highlight the region, and the <b>Locate</b> button (magnifying glass with white page) to obtain information on the data points. We can now decide to flag all of the highlighted area by clicking on the <b>Flag</b> button (red flag). The region is now flagged.<br />
<br />
<br />
Similarly, one can also perform interactive flagging with {{msview}}. Again, the flagging is performed directly on the data, and so it is beneficial to execute flagmanager as above before embarking on interactive flagging with msview(). <br />
<br />
Open msview() and load the data as described in section [[#Identifying_RFI_with_MSview | Identifying RFI with MSview]]:<br />
<br />
<source lang="python"><br />
# In CASA<br />
msview()<br />
</source><br />
<br />
Select what you would like the axes to display (by default msview will plot time vs baseline), and using zoom tool (magnifying glass button with plus sign) zoom in on the data chunks you would like to inspect and potentially flag. Now, draw a region around the bad data using the <b>drawing selectors</b> (rectangle, circle, polygon) and <b>right-double click</b> to flag. To save the flagging, click <b>Save Edits</b> button in the flagging tab of Data Display Option window. <br />
<br />
<pre><br />
Flagging data interactively can be useful, but is discouraged, especially for large data volumes.<br />
</pre><br />
<br />
A more proper form of flagging would be to narrow your search on bad data points (Locate data as described above) and use a flagging task to remove them as showed in the next section below.<br />
<br />
== Identifying remaining bad data with plotms()/msview() ==<br />
<br />
Once the bulk of the RFI and corrupted data has been removed with the methods described earlier in this guide, it is worth having another look at the data with the visualisation tools, either {{plotms()}} or [https://casa.nrao.edu/casadocs/casa-5.4.0/global-task-list/task_msview/about msview()]. This step may be valuable to perform once your data are calibrated and ready for imaging, to double check all the bad data has been indeed removed.<br />
<br />
Let's again inspect amplitude vs uv-distance of our calibrated target data (Figure 21) with plotms(). Instead of flagging the data interactively (the Flag button), it is recommended to run the {{flagdata}} task. While locating the bad data points in the previous section (<b>Mark Region</b> and then click <b>Locate</b> button), CASA logger provided us with information on the data points. Notice that they all belong to scans 88 and 87. The scans also involve a baseline shared with ea16. To flag these data run the flagdata() task:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', scan='87,88', antenna='ea16', flagbackup=True)<br />
</source><br />
<br />
To see the effect of this flagging, either Reload the image (tick Reload and click Plot) if your plotms() session from the previous section is still open, or otherwise run the following command:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='50~100', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', coloraxis = 'spw')<br />
</source><br />
<br />
We can now see the image (Figure 22) looks better without scans 87 and 88 for ea16. <br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:Corr.Amp_vs_UVdist.png|400px|thumb|right|'''Figure 21''' <br />Amplitude vs. UV-Distance plot for scans 50-100. Outliers have been highlighted to show interactive flagging through plotms.]]<br />
| [[Image:Corr.Amp_vs_UVdist.flagged.png|400px|thumb|right|'''Figure 22''' <br /> Amplitude vs. UV-Distance plot for scans 50-100. Scans 87 and 88 have been flagged for ea16 via flagdata.]]<br />
|}<br />
<br />
If you would like to perform these steps with [https://casa.nrao.edu/casadocs/casa-5.4.0/global-task-list/task_msview/about msview()], instead of plotms(), plot <b>time vs. baseline length</b>. Here baseline length is equivalent to the uv-distance.<br />
<br />
<!-- Now would be also a good time to play with the plotms parameters and look for more data to flag interactively. As a reminder, at the start of this guide we highlighted and located some of the worst sections with RFI. When reducing/flagging your data, it will prove helpful to review this information and revisit the most RFI affected spectral windows and channels.--><br />
<br />
<!--<br />
=== Amplitude vs. Baseline ===<br />
<br />
We will now plot corrected amplitude vs. baseline. We will be looking to flag baselines with consistent high corrected amplitudes as we iterate through each scan. Iterate by clicking on the green arrow buttons on the toolbar within the plotms(). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='SNR_G55_10s-hanning.ms', scan='30,75,120,165,190,235,303', xaxis='baseline', yaxis='amp',<br />
ydatacolumn='corrected', iteraxis='scan', coloraxis = 'baseline')<br />
</source><br />
[[Image:corr.amp_vs_baseline_scan75.png|300px|thumb|right|'''Figure 17''' <br /> Amplitude vs. Baseline plot, which is used to try and identify baselines with high amplitudes for mutliple scans.]]<br />
<br />
It appears that there are several baselines which have a consistently higher-amplitude than the others, indicating that they're probably contaminated by RFI (Figure 17). <br />
<br />
Use the plotms tools to identify the baselines (see the section [[#Preliminary_Data_Evaluation |Preliminary Data Evaluation]] at the beginning of the guide). Iterate through to the different scans to verify that these higher amplitudes correspond to the same baselines. We will flag several of them: ea04&ea16, ea02&ea08, and ea02&ea27. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', antenna='ea04&ea16', spw='1', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', antenna='ea02&ea27', spw='1', flagbackup=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='SNR_G55_10s-hanning.ms', antenna='ea02&ea08', spw='1', flagbackup=False)<br />
</source><br />
--><br />
<br />
<br />
== Flagging Summary & Report ==<br />
<br />
We will now run flagdata in summary mode to inspect how much total data has been flagged. We can also create a plot of the percentage of flagged data with respect to frequency by setting parameter ''display='report' ''(Figure 23). The report parameter will also create a plot of the antennas, with the circle size representing the percentage of flagged data per antenna (Figure 24). This can be viewed by clicking Next. One can also zoom in and out for a better view. <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagInfo = flagdata(vis='SNR_G55_10s-hanning.ms', mode='summary', action='calculate', display='report', spwchan=True)<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
print("\n %2.1f%% of G55.7+3.4, %2.1f%% of 3C147, and %2.1f%% of J1925+2106 are flagged. \n" %<br />
(100.0 * flagInfo['field']['G55.7+3.4']['flagged'] / flagInfo['field']['G55.7+3.4']['total'],<br />
100.0 * flagInfo['field']['0542+498=3C147']['flagged'] / flagInfo['field']['0542+498=3C147']['total'],<br />
100.0 * flagInfo['field']['J1925+2106']['flagged'] / flagInfo['field']['J1925+2106']['total']))<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
print("Spectral windows are flagged as follows:")<br />
<br />
for spw in range(0,4):<br />
print("SPW %s: %2.1f%%" % (spw, 100.0 * flagInfo['spw'][str(spw)]['flagged'] / flagInfo['spw'][str(spw)]['total']))<br />
--<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:Percent.Flagged.Plot.png|400px|thumb|right|'''Figure 23''' <br /> Total percentage of flagged data vs. frequency, colorized by spectral window.]]<br />
| [[Image:Percent.Flagged.Ant.png|400px|thumb|right|'''Figure 24''' <br /> Total Percentage of data flagged from each antenna. A larger circle represents more flagged data for that antenna. ]]<br />
|}<br />
<br />
As a result of the flagging we have removed over 40% of the data for G55.7+3.4 and, as expected, spectral windows 0 and 2 have the most flagged data.<br />
<br />
In addition, we can inspect the CASA log, which will report the percentage of flagged data for: <br /><br />
1. per Spw/ per Channel <br /><br />
2. Each Antenna <br /><br />
3. Every Correlation (RR, RL, LL, LR) <br /><br />
4. Every Scan <br /><br />
5. Every spectral window <br /><br />
<br />
[[Main Page | &#8629; '''CASAguides''']]<br />
<br />
{{Checked 5.5.0}}<br />
</div><br />
<br />
<!--<br />
--Original: Miriam Hartman <br /><br />
--Modifications: Lorant Sjouwerman (4.4.0, 2015/07/07) <br /><br />
--Modifications: Juergen Ott (4.5.2, 2016/04/13) <br /><br />
--Topical Guide: Jose Salcido (4.5.2, 2016/04/15) <br /><br />
--Edits: Tony Perreault (4.5.2, 2016/05/25) <br /><br />
--Edits: Tony Perreault (4.7.0, 2016/10/14) <br /><br />
--Edits: Jose Salcido (4.7.0, 2016/10/26) <br /><br />
--Edits: Tony Perreault (4.7.0, 2016/10/26 <br /><br />
--Edits: Anna Kapinska (5.4.0, 2018/11/30 <br /><br />
--Edits: Angelica Vargas (5.5.0, 2019/06/24 <br /><br />
--></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=AIPS-Simple-Self-Cal&diff=27498AIPS-Simple-Self-Cal2019-05-31T21:35:23Z<p>Emomjian: /* Apply Calibration and SPLIT the Data */</p>
<hr />
<div><div style="text-align: justify;"><br />
== Overview ==<br />
<br />
A good resource for reducing basic VLBI data in AIPS can be found in [ftp://ftp.aoc.nrao.edu/pub/software/aips/TEXT/PUBL/COOKC.PS Appendix C] of the [http://www.aips.nrao.edu/cook.html AIPS Cookbook]. [ftp://ftp.aoc.nrao.edu/pub/software/aips/TEXT/PUBL/COOK9.PS Chapter 9:Reducing VLBI Data in AIPS] of the AIPS Cookbook contains much more detail, but is best used as a reference. So with this tutorial you will:<br />
<br />
*Calibrate the regular data all the way through imaging.<br />
*Perform self-calibration to improve the imaging <br />
<br />
This guide only gives the important inputs to each task. The other inputs should be default. An easy way to set all the inputs to the default in a task is to type "default ''taskname''".<br />
<br />
== AIPS ==<br />
<br />
For a comprehensive overview of AIPS see the [http://www.aips.nrao.edu/CookHTML/CookBook.html AIPS COOKBOOK], especially [http://www.aips.nrao.edu/CookHTML/CookBookch3.html#x20-240003 Chapter 3:Basic AIPS Utilities] and [http://www.aips.nrao.edu/CookHTML/CookBookch12.html#x107-31300012 Chapter 12:AIPS for the More Sophisticated User]for the basics. [http://www.aips.nrao.edu/CookHTML/CookBookch9.html#x87-2010009 Chapter 9:Reducing VLBI Data in AIPS] is the chapter explaining VLBI data reduction, but it is rather overwhelming so [http://www.aips.nrao.edu/CookHTML/CookBookap3.html#x171-401000C Appendix C:A Step-by-Step Recipe for VLBA Data Calibration in AIPS] is recommended as a first step. However, below I summarize some useful commands (in parenthesis are the short form of the command):<br />
<br />
*getname (getn) ''catalog #'': get the name of catalog # and put in INNAME, INCLASS and INSEQ. Other similar: geton (get outname); get2n (get in2name); get3n (get in3name) etc..<br />
*input (inp) ''taskname'': show the inputs of taskname.<br />
*tget ''taskname'': get a task and fill in the inputs of the last time the task was run, one of the most useful commands in AIPS.<br />
*ucat (uc): list uvdata catalog.<br />
*mcat (mc): list map catalog.<br />
*tvlod (tvlo): load a map onto the TV.<br />
*tvinit (tvin): initialize and clear TV.<br />
*imhead (imh): Print the file header (this can be used on both uv data and images despite the name). This will list informative things like the date of observation and the number and types of tables attached to the data.<br />
<br />
== Getting the Data and Other Useful Files ==<br />
To obtain these files, right click on the desired file below and select "Save Link As...":<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/bl193bg_3sour_avg.fits bl193bg_3sour_avg.fits] (543 MB): Data we will be reducing today.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/1828_1.box 1828_1.box]: Box file for imaging.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/1828_2.box 1828_2.box]: Box file for imaging.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/1828_3.box 1828_3.box]: Box file for imaging.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/1828_4.box 1828_4.box]: Box file for imaging.<br />
<br />
Files you might want to get in case something goes wrong with the primary calibration or if you just want to try the imaging and self-calibration:<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/0851+202.SPLIT.1 0851+202.SPLIT.1] (6.3 MB): Calibrated data for source 0851+202.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/1828+487.SPLIT.1 1828+487.SPLIT.1] (7.8 MB): Calibrated data for source 1828+487.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/2200+420.SPLIT.1 2200+420.SPLIT.1] (6.8 MB): Calibrated data for source 2200+420.<br />
<br />
If that doesn't work you can also ftp the data using anonymous ftp:<br />
*ftp ftp.aoc.nrao.edu<br />
*login as anonymous<br />
*cd pub/SIW<br />
*get bl193bg_3sour_avg.fits<br />
*mget *.box<br />
and optionally<br />
*mget *.SPLIT.1<br />
<br />
The best place to put these files is in your $FITS area. This is an AIPS data area that should already have an environmental variable set up for it.<br />
<br />
== About the Data ==<br />
<br />
This is part of an observation from the [http://www.physics.purdue.edu/MOJAVE/index.html MOJAVE] survey, which is a large project that uses the VLBA to monitor Active Galactic Nuclei at 15 GHz in total and polarized flux. This project has been monitoring AGN for over a decade. The observation we are using took place on 16 June 2016 and contained 22 sources. However, in order to make the data a manageable size for a data reduction tutorial, three sources have been copied out of the dataset and the frequency channels and sample times have both been averaged by a factor of two. The sources in the the bl193bg_3sour_avg.fits dataset are as follows:<br />
<br />
*0851+202 - BL Lac object, bright source with not a lot of structure, used as bandpass calibrator, aka OJ287. [http://www.physics.purdue.edu/astro/MOJAVE/sourcepages/0851+202.shtml MOJAVE source page] <br />
*1828+487 - Quasar, bright source with a very extended defuse jet, aka 3C380. [http://www.physics.purdue.edu/astro/MOJAVE/sourcepages/1828+487.shtml MOJAVE source page]<br />
*2200+420 - The original BL Lac object, bright source with extended curved jet, aka BL Lac. [http://www.physics.purdue.edu/astro/MOJAVE/sourcepages/2200+420.shtml MOJAVE source page]<br />
<br />
We thank the MOJAVE team for maintaining the MOJAVE database and Matt Lister (Purdue) in particular for advice on the best data to select [http://adsabs.harvard.edu/abs/2009AJ....138.1874L (Lister et al., 2009, AJ, 137, 3718)].<br />
<br />
== Starting AIPS and Loading the Data ==<br />
<br />
=== Starting AIPS ===<br />
<br />
Start AIPS by typing '''aips''' or possibly '''aips tv=local:0.0''' in a terminal window. Depending on your AIPS installation, you might be prompted to pick a printer, then you will be asked for an AIPS number, if you are not sharing disks with anyone pick any number other than 1. Remember that number because the data you load and work on will be under that number. ''It would be best for this tutorial for you to use a clean AIPS number, in other words, an AIPS number that you have not used for anything else and that has no data loaded into it.''<br />
<br />
After AIPS has started type:<br />
* dowait=1<br />
<br />
This means that AIPS will not allow you to run more than one task at a time. I usually run with dowait=-1; but for this tutorial and for beginners it is better to set dowait=1.<br />
<br />
=== Loading Data Into AIPS ===<br />
<br />
Load the data with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?FITLD '''FITLD''']. Note that I am forcing an outname, outclass and outseq, we are doing it here so it will be easy to talk about specific files.<br />
<br />
First load the target and calibrators<br />
# default fitld<br />
# clint 0.25 ➜ set CL table interval to 15 seconds.<br />
# datain 'MYAIP:bl193bg_3sour_avg.fits ➜ Two things to note: 1) this assumes that the data is in the $MYAIP directory, you can use a full directory path (e.g., datain '/home/mydata/bl193bg_3sour_avg.fits) or another environmental variable if the data is elsewhere; 2) The lack of a close ' is not a typo, you must leave off the closing ' or AIPS will capitalize everything within the ' ' and you will get an error because FITLD will not be able to find the data.<br />
# outname 'bl193bg'<br />
# outclass 'uvdata'<br />
# outseq 1<br />
# outdisk 1<br />
# inp ➜ check the inputs<br />
# go<br />
<br />
Lets look at our uv data catalog<br />
* uc<br />
You should get something that looks similar to (with a different Usid and Last access):<br />
<pre><br />
AIPS 1: Catalog on disk 1<br />
AIPS 1: Cat Usid Mapname Class Seq Pt Last access Stat<br />
AIPS 1: 1 194 BL193BG .UVDATA. 1 UV 29-JUN-17 19:39:03<br />
</pre><br />
<br />
Note the data file, BL193BG.UVDATA.1, has a catalog number (Cat) which in this case 1. Catalog numbers can be, and usually are, used to fill in IN and OUTNAMES etc., using getn, geton etc..<br />
<br />
== Reducing the data ==<br />
<br />
=== Preliminary Data Evaluation === <br />
<br />
Look at the structure of the data with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?LISTR'''LISTR''']. This will give you a listing of the scans as well as the sources and frequency structure in the observation. Note that even though this dataset contains only three sources, the SU (source) table still contains all the sources in the original dataset, so LISTR will list all the sources and show 0 visibilities for the sources with no data.<br />
<br />
# default listr<br />
# getn '''BL193BG.UVDATA.1 file'''; in other words ''getn 1''<br />
# optype 'scan'<br />
# docrt 1<br />
# inp<br />
# go<br />
<br />
This will give you a listing of the scans and frequencies in the observation.<br />
<br />
=== Data Reduction Procedure === <br />
<br />
Load in the VLBA data reduction procedures: see [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBAUTIL EXPLAIN VLBAUTIL] for a full description of the numerous procedures. <br />
<br />
Procedures are run by just typing their name, rather than using "go".<br />
<br />
# run vlbautil<br />
<br />
If you get a BLEW CORE or other similar error message, you have filled your procedure memory (VLBAUTIL is very large and loading it three times will do this). To fix it type "restore 0" then reload VLBAUTIL <br />
<br />
=== Calibrate Ionospheric Delay and Fix Earth Orientation Parameters ===<br />
<br />
Data observed at 8 GHz or lower frequencies may be effected by the dispersive delay caused by the ionosphere. This will distort images and is especially bad at very low frequencies. An ionospheric correction based on maps of the electron content of the ionosphere can be performed with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBATECR '''VLBATECR''']. ''BL193BG is observed at 15 GHz so no ionospheric correction is needed.''<br />
<br />
Data using phase referencing can be affected by imprecise Earth Orientation Parameters (EOPs) used in correlation. Therefore all phase referenced data should have its EOPs corrected. This can be done with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBAEOPS '''VLBAEOPS''']. ''BL193BG does not use phase referencing so no EOP correction is needed.''<br />
<br />
=== Apply Digital Sampling Correction to VLBA data ===<br />
<br />
Correct sampler threshold errors from the correlator by running [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBACCOR '''VLBACCOR'''].<br />
<br />
# default vlbaccor<br />
# getn 1<br />
# inp<br />
# vlbaccor<br />
<br />
VLBACCOR runs [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?ACCOR '''ACCOR'''] which creates a solution (SN) table, then smooths the table with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?SNSMO '''SNSMO'''], and then runs [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?CLCAL '''CLCAL'''] to apply the solutions to the calibration (CL) table. When VLBACCOR is finished you should get messages like those below that tell you what SN table and CL table are created. It is a good idea to save these messages.<br />
<br />
<pre><br />
AIPS 1: 'SN #1 CONTAINS SAMPLER CORRECTIONS'<br />
AIPS 1: 'CL #2 ADDS SAMPLER CORRECTIONS'<br />
</pre><br />
<br />
=== Plot Cross Power Spectrum ===<br />
<br />
[[Image:0851_possm1.png|250px|thumb|right|'''Figure 1''' <br /> Plot of amplitudes (bottom) and phases (top) for source 0851+202.]]<br />
<br />
Now lets take a look at the fringe finders by running [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBACRPL '''VLBACRPL''']. VLBACRPL runs [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?POSSM '''POSSM'''] and displays the spectrum of each baseline (to Fort Davis which is antenna 2), with the amplitude on the bottom and the phases on the top.<br />
<br />
# default vlbacrpl<br />
# getn 1<br />
# stokes 'half'<br />
# refant 2<br />
# gainuse 0<br />
# solint -1<br />
# dotv 1<br />
# source '0851+202'<br />
# inp<br />
# vlbacrpl <br />
<br />
As you can see from Figure 1, 0851+202 has very strong fringes. So we will use it as the calibrator to set the instrumental delays. Also look at your terminal for instructions on how to control POSSM:<br />
<br />
<pre><br />
POSSM1: Waiting 30 seconds:<br />
POSSM1: Hit TV button A to pause indefinitely. Hit button<br />
POSSM1: B or C to continue sooner, button D to stop plotting<br />
</pre><br />
So hit A if you want to look at a plot longer than the 30 seconds, B or C to go to the next plot, D to exit.<br />
<br />
=== Determine Delay Corrections ===<br />
<br />
Find and remove instrumental delay by running [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBAPCOR '''VLBAPCOR'''].<br />
<br />
[[Image:0851_possm2.png|250px|thumb|right|'''Figure 2''' <br /> Plots for source 0851+202 which show the amplitude (bottom panel) and corrected phases (top panel), for each IF in both polarizations.]]<br />
<br />
#default vlbapcor<br />
#getn 1<br />
#calsour '0851+202' '<br />
#timer 0 19 26 33 0 19 36 38 ➜ scan with good fringes on all telescopes we found in POSSM plots.<br />
#refant 2 ➜ choose reference antenna from in the middle of array, antenna 2 or FD (Fort Davis, TX) is a good choice.<br />
#gainu 0 ➜ apply highest CL table.<br />
#inp<br />
#vlbapcor<br />
<br />
(Note that the second quote sign ' after the source name causes all other values of CALSOUR to be blank.) At this point you should check on the calibration:<br />
<br />
*VLBAPCOR runs PCCOR which used the pulse cals to correct the intrumental phases and then runs CLCAL to apply the solutions to the CL (calbration table). The output from '''VLBAPCOR''' will look something like this:<br />
<pre><br />
AIPS 1: 'RUN PCCOR'<br />
PCCOR1: Task PCCOR (release of 31DEC17) begins<br />
PCCOR1: Writing to SN table 2<br />
PCCOR1: Appears to have ended successfully<br />
PCCOR1: localhos 31DEC17 TST: Cpu= 0.9 Real= 2 IO= 93<br />
AIPS 1: Resumes<br />
AIPS 1: 'RUN CLCAL'<br />
CLCAL1: Task CLCAL (release of 31DEC17) begins<br />
CLCAL1: Using interpolation mode SELF<br />
CLCAL1: Copied CL file from vol/cno/vers 1 183 2 to 1 183 3<br />
CLCAL1: CALSEL: Making room for new CL data by<br />
CLCAL1: Deleting CL table no. 3<br />
CLCAL1: Processing SN table 2<br />
CLCAL1: SNMRG: Merging SN table<br />
CLCAL1: SNMRG: Write 2974 merged records from 2974 input records<br />
CLCAL1: SN2CL: Applying SN tables to CL table 2, writing CL table 3<br />
CLCAL1: Appears to have ended successfully<br />
CLCAL1: localhos 31DEC17 TST: Cpu= 0.3 Real= 1 IO= 138<br />
AIPS 1: Resumes<br />
AIPS 1: 'SN #2 CONTAINS PCAL INSTR. PHASE CORRECTIONS'<br />
AIPS 1: 'CL #3 ADDS INSTR. PHASE CORRECTIONS'<br />
</pre><br />
Again, it is a good idea to make note of the last two lines that tell you which tables contain which corrections. Most VLBAUTIL calibration scripts produce such messages.<br />
<br />
*check solutions in POSSM, the jumps in phase between the IFs should be gone. The phases may also be flattened.<br />
*# tget possm ➜ to "get" all the inputs from the last run (it was run when we ran the procedure '''VLBACRPL''').<br />
*# gainu 3<br />
*# inp<br />
*# go<br />
<br />
As you can see from Figure 2, the phases for 0851+202 have been flattened and the phase jumps between the IFs are gone. For other sources farther away in time the phases may be different from 0 but there will still be no phase jumps.<br />
<br />
=== Create BP Table and Calibrate Bandpass Shape ===<br />
<br />
[[Image:BPASS.png|250px|thumb|right|'''Figure 3''' <br /> Plot of bandpass for source '0851+202' .]]<br />
<br />
Calibrate bandpass shape with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBABPSS '''VLBABPSS'''].<br />
<br />
# default vlbabpss<br />
# getn 1<br />
# calsour '0851+202' ' ➜ use a strong source as the bandpass calibrator.<br />
# refant 2<br />
# inp<br />
# vlbabpss<br />
<br />
VLBABPSS will create a bandpass (BP) table:<br />
<pre><br />
AIPS 1: 'BP #1 BANDPASS TABLE'<br />
</pre><br />
<br />
Now check the bandpass solutions with '''POSSM'''. <br />
<br />
# tget possm <br />
# baseline=0<br />
# aparm=0<br />
# aparm(8)=2 <br />
# aparm(9)=1 <br />
# bpver=1; doband=1 <br />
# go<br />
<br />
Figure 3 shows the plots generated by POSSM. The solutions should look like a reasonable fit of the bandpass shape.<br />
<br />
=== Apply Amplitude Corrections ===<br />
<br />
Perform amplitude calibration by running [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBAAMP '''VLBAAMP'''].<br />
<br />
# default vlbaamp<br />
# getn 1<br />
# inp<br />
# vlbaamp<br />
<br />
Again, the procedure should finish with the following lines:<br />
<pre><br />
AIPS 1: 'SN #3 CONTAINS SAMPLER CORRECTIONS'<br />
AIPS 1: 'SN #4 CONTAINS GAIN CORRECTIONS'<br />
AIPS 1: 'CL #4 ADDS SAMPLER CORRECTIONS'<br />
AIPS 1: 'CL #5 ADDS GAIN CORRECTIONS'<br />
</pre><br />
which should be noted.<br />
<br />
=== Examine Amplitude Calibration ===<br />
<br />
[[Image:SNPLT.png|250px|thumb|right|'''Figure 4''' <br /> Plot of amplitude gains vs. time with the different colors representing the different IF polarizations.]]<br />
<br />
Examine amplitude calibration by running [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?SNPLT '''SNPLT'''].<br />
<br />
# default snplt<br />
# getn 1<br />
# dotv 1<br />
# inext 'cl'<br />
# invers 0<br />
# opty 'amp' ➜ Plot amplitudes.<br />
# nplots 10<br />
# opco 'alsi' ➜ Plot all IFs and Stokes.<br />
# do3col 1 ➜ Use 3-color symbols. <br />
# inp<br />
# tvin ➜ tvinitialize clears the tv before plotting. Otherwise you will get a mess as the table is plotted over the bandpass plot from before.<br />
# go<br />
<br />
Figure 4 shows that the amplitude gains change over time as the sources rise and set, with higher gains at lower elevations. The different IFs and polarizations (RR and LL in this case) are shown as different colors.<br />
<br />
=== Correct Phases for Parallactic Angles ===<br />
Correct phases for parallactic angle effects with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBAPANG '''VLBAPANG'''].<br />
<br />
[[Image:EDITR2.png|250px|thumb|right|'''Figure 5''' <br /> EDITR display showing all the baselines to antenna 10 for 2200+420. You can see for all the baselines the amplitude is low for the first scan and for baseline 01-10 the phases for this scan look like noise. We want to flag this data.]]<br />
<br />
# default vlbapang<br />
# getn 1<br />
# inp<br />
# vlbapang<br />
<br />
<pre><br />
AIPS 1: 'CL #6 ADDS PARALLACTIC ANGLE CORRECTIONS'<br />
</pre><br />
<br />
=== Manually Edit Data ===<br />
<br />
<br />
[[Image:EDITR3.png|250px|thumb|right|'''Figure 6''' <br /> To flag this data we click FLAG TIME RANGE and hit B, and a blue box appears in the 01-10 box on the bottom.]]<br />
<br />
[[Image:EDITR4.png|250px|thumb|right|'''Figure 7''' <br /> After moving the sides of the box to encompass the data that need to be flagged, hit C and the data are flagged and turn red.]]<br />
<br />
<br />
Edit the data using [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?EDITR '''EDITR'''] on each source separately.<br />
<br />
# default editr<br />
# getn 1<br />
# docal 1; gainuse 0 ➜ calibrate data with highest CL table.<br />
# crowded 1 ➜ to plot all polarization and IFs on top of each other.<br />
# do3col 1<br />
# doband 1<br />
# bpver 1<br />
# antuse 1 2 3 4 5 6 7 8 9 10 ➜ to plot all the baselines to one antenna at the same time.<br />
<br />
Run on one source at a time:<br />
*0851+202:<br />
*#source <nowiki>'0851+202''</nowiki><br />
*#inp<br />
*#go<br />
<br />
Again look in you terminal window for instructions:<br />
<pre> <br />
EDITR1: Press buttons A, B, or C to choose an operation<br />
EDITR1: Press button D for on-line help<br />
</pre><br />
<br />
EDITR plots both amplitudes and phases verses time, so this is a good way to just look at your data. For the inputs we chose, all the polarizations and IFs are plotted on top of each other. The two boxes on the bottom plot these as different colors and show the amplitude and phase for one baseline. The other boxes, where the data are shown in green, are the other baselines to that single antenna. It is especially useful to look at the data this way because problems tend to be antenna based so all the baselines to an antenna will be effected. It you see potentially problematic data (especially high or low amplitudes or with bad phases (note that weak sources will have incoherent phases)), then you should wait to flag until you can see all the baselines to the other antenna in the baseline pair with the bad data. That way you can flag once, telling EDITR to flag all the data for one antenna (this is the default in EDITR, but can be changed). The data for this source looks great. Click "NEXT ANTENNA" found on the menu to the right and press "B" until you have looked at all the baselines on all the antennas for this source. At this point the data has been amplitude calibrated but the source amplitude changes from scan to scan, do you know the reason for this?<br />
<br />
Now plot the other sources and inspect.<br />
<br />
*1828+487:<br />
*#source <nowiki>'1828+487''</nowiki><br />
*#go<br />
<br />
1828+487 also looks pretty good, although there are some ratty amplitudes and phase on the last scan on baselines to antenna 10. We will just leave these for now because the final fringe fit and self-calibration might fix it.<br />
<br />
*2200+420:<br />
*#source <nowiki>'2200+420''</nowiki><br />
*#go<br />
<br />
As you can see in Figure 5 this source the first scan on the baselines to antenna 10 looks bad. The amplitude is very low and the phases are noise. To flag this data click FLAG TIME RANGE and hit B (Figure 6), and a blue box appears in the 01-10 box on the bottom. By hitting A and moving the cursor move the sides of the box to encompass the data which needs to be flagged; then hit C. The data will be flagged on all the baselines to the antenna and will turn white in the lower colored parts and red in the upper comparison parts of the display (Figure 7).<br />
<br />
=== Fringe Fit the Data ===<br />
<br />
Perform a global fringe fit with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?FRING '''FRING''']. The following steps through imaging are not strictly necessary since we will have to redo them once the geodetic calibration is done, but they are an excellent way to check that the calibration up to this point is correct.<br />
<br />
# default fring<br />
# getn 1<br />
# calsour '0851+202', '1828+487', '2200+420' ➜ do a global fringe fit on all sources.<br />
# refant 2<br />
# search 2 9 5 4 1 3 7 8 ➜ list of antennas to search if a solution is hard to find.<br />
# solint 0.25<br />
# doband 1<br />
# bpver 1<br />
# aparm(6) 2 ➜ print some information on solutions. <br />
# aparm(9) 1 ➜ do exhaustive baseline search.<br />
# docal 1; gainu 0<br />
# inp<br />
# go<br />
<br />
You should look at the number of good and failed solutions:<br />
<pre><br />
FRING1: Found 97646 good solutions<br />
FRING1: Failed on 1042 solutions<br />
FRING1: Adjusting solutions to a common reference antenna<br />
</pre><br />
You should get something similar, but not necessarily exactly the same numbers of good and failed solutions. A small percentage of failed solutions are fine at this point. FRING will have created SN table number 5.<br />
<br />
Since we did not use a source model, FRING assumes that the calibration source is a point source at the phase center. None of these sources are point sources. Therefore the corrections FRING finds will not correct the data perfectly and, once we apply all the calibration, it will be necessary to do self-calibration with source models to get the best images. Also note that I said "at the phase center" so, no matter what the the true position of your target, this fringe fit will move the brightest spot of the source to the phase center (the position of the target in the correlation).<br />
<br />
=== Apply FRING solutions ===<br />
<br />
Note that the previous step and this step can be done in the single procedure [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBAFRNG '''VLBAFRNG''']. It is broken up here just to show both the steps explicitly, but also because running the tasks individually allows for more control. Here the global fringe fit which was done in the last step is interpolated and applied to the all the calibration that has gone before with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?CLCAL '''CLCAL'''].<br />
<br />
# default clcal<br />
# getn 1<br />
# gainv 6 ➜ CL table with all the calibration.<br />
# gainu 7 ➜ CL table to write next step of calibration in.<br />
# snver 5 ➜ global fringe fit solutions from FRING<br />
# interpol 'SELF' ➜ use 2PT interpolation.<br />
# refant 2<br />
# sour '0851+202', '1828+487', '2200+420' ➜ sources to which to apply calibration.<br />
# calsour sour ➜ set calibrator sources to the same sources in parameter sources<br />
# inp<br />
# go<br />
<br />
So SN table #5 will be applied to CL table #6 and create CL table #7 which now includes all the calibration.<br />
<br />
=== Backing up Your Calibration ===<br />
<br />
We will want to save the calibration up to this point with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?TASAV '''TASAV''']. TASAV makes a new file with all your calibration tables. This is very useful because it contains everything you need to recalibrate your data in a very compact and portable form. It is also a good idea to do this before you do anything you are unsure about or will change important tables (AN, SU) in ways that are hard to change back. <br />
<br />
# default tasav<br />
# getn 1<br />
# outdisk 1<br />
# inp<br />
# go<br />
<br />
Lets look at our catalog again:<br />
*uc<br />
You should get something that look similar to:<br />
<pre><br />
AIPS 1: Catalog on disk 1<br />
AIPS 1: Cat Usid Mapname Class Seq Pt Last access Stat<br />
AIPS 1: 1 193 BL193BG .UVAVG . 1 UV 01-JUL-17 14:28:24<br />
AIPS 1: 2 193 BL193BG .TASAV . 1 UV 27-JUN-17 11:52:14<br />
</pre><br />
Note that you now have a second catalog entry which TASAV automatically named BL193BG.TASAV.1. This uv data file has 1 visibility and all the tables attached to BM272HC.UVDATA.1. Lets look at the TASAVed file.:<br />
#getn 2<br />
#imh<br />
You should get something that looks like:<br />
<pre><br />
AIPS 1: Image=MULTI (UV) Filename=BL193BG .TASAV . 1<br />
AIPS 1: Telescope=VLBA Receiver=VLBA<br />
AIPS 1: Observer=BL193BG User #= 193<br />
AIPS 1: Observ. date=16-JUN-2016 Map date=27-JUN-2017<br />
AIPS 1: # visibilities 1 Sort order TB<br />
AIPS 1: Rand axes: UU-L-SIN VV-L-SIN WW-L-SIN TIME1 SUBARRAY<br />
AIPS 1: SOURCE REMOVED INTTIM GATEID CORR-ID ANTENNA1<br />
AIPS 1: ANTENNA2<br />
AIPS 1: ----------------------------------------------------------------<br />
AIPS 1: Type Pixels Coord value at Pixel Coord incr Rotat<br />
AIPS 1: COMPLEX 3 1.0000000E+00 1.00 1.0000000E+00 0.00<br />
AIPS 1: STOKES 2 -1.0000000E+00 1.00 -1.0000000E+00 0.00<br />
AIPS 1: FREQ 32 1.5240375E+10 17.00 1.0000000E+06 0.00<br />
AIPS 1: IF 8 1.0000000E+00 1.00 1.0000000E+00 0.00<br />
AIPS 1: RA 1 00 00 00.000 1.00 3600.000 0.00<br />
AIPS 1: DEC 1 00 00 00.000 1.00 3600.000 0.00<br />
AIPS 1: ----------------------------------------------------------------<br />
AIPS 1: Coordinate equinox 2000.00<br />
AIPS 1: Maximum version number of extension files of type HI is 1<br />
AIPS 1: Maximum version number of extension files of type AT is 1<br />
AIPS 1: Maximum version number of extension files of type CT is 1<br />
AIPS 1: Maximum version number of extension files of type FQ is 1<br />
AIPS 1: Maximum version number of extension files of type AN is 1<br />
AIPS 1: Maximum version number of extension files of type CL is 7<br />
AIPS 1: Maximum version number of extension files of type CQ is 1<br />
AIPS 1: Maximum version number of extension files of type GC is 1<br />
AIPS 1: Maximum version number of extension files of type IM is 1<br />
AIPS 1: Maximum version number of extension files of type MC is 1<br />
AIPS 1: Maximum version number of extension files of type PC is 1<br />
AIPS 1: Maximum version number of extension files of type SU is 1<br />
AIPS 1: Maximum version number of extension files of type TY is 1<br />
AIPS 1: Maximum version number of extension files of type WX is 1<br />
AIPS 1: Maximum version number of extension files of type SN is 5<br />
AIPS 1: Maximum version number of extension files of type BP is 1<br />
AIPS 1: Maximum version number of extension files of type FG is 1<br />
</pre><br />
Note that there is 1 visibility but 7 calibration (CL) tables and 5 solution (SN) tables. As well as numerous other tables with information about you data (history (HI), frequency (FQ), antenna (AN), system temperature (TY), bandpass (BP), flag (FG)...).<br />
<br />
=== Apply Calibration and SPLIT the Data ===<br />
<br />
Apply calibration and make single source data sets with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?SPLIT '''SPLIT''']. I like to work with single source files, it's less confusing, especially when self-caling and imaging.<br />
<br />
# default split<br />
# getn 1<br />
# freqid 0<br />
# docal 1; gainu 7 ➜ apply calibration; from CL#7.<br />
# sour '0851+202', '1828+487', '2200+420'<br />
# doband 1; bpver 1<br />
# aparm 2 0 0 0 1 1 ➜ aparm(1)=2 tells SPLIT to average all the channels in each IF, reducing the size of the data significantly.<br />
# inp<br />
# go<br />
<br />
This will produce three files named sourcename.SPLIT.1, again type '''uc''' to look at the catalog.<br />
<pre><br />
AIPS 1: Catalog on disk 1<br />
AIPS 1: Cat Usid Mapname Class Seq Pt Last access Stat<br />
AIPS 1: 1 193 BL193BG .UVAVG . 1 UV 01-JUL-17 14:28:24<br />
AIPS 1: 2 193 BL193BG .TASAV . 1 UV 03-JUL-17 15:01:55<br />
AIPS 1: 3 193 2200+420 .SPLIT . 1 UV 03-JUL-17 15:12:12<br />
AIPS 1: 4 193 0851+202 .SPLIT . 1 UV 03-JUL-17 15:14:41<br />
AIPS 1: 5 193 1828+487 .SPLIT . 1 UV 03-JUL-17 15:15:29<br />
</pre><br />
<br />
=== OPTIONAL: Getting the Calibrated Data ===<br />
<br />
'''''THIS STEP IS OPTIONAL: If everything went well in the above steps there is no need to get the calibrated data.'''''<br />
<br />
However If something went wrong with the calibration or if you want to start with the imaging and self-calibration then download the following files and put them in your $FITS area:<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/0851+202.SPLIT.1 0851+202.SPLIT.1] (6.3 MB): Calibrated data for source 0851+202.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/1828+487.SPLIT.1 1828+487.SPLIT.1] (7.8 MB): Calibrated data for source 1828+487.<br />
*[ftp://ftp.aoc.nrao.edu/pub/SIW/2200+420.SPLIT.1 2200+420.SPLIT.1] (6.8 MB): Calibrated data for source 2200+420.<br />
<br />
Then load the data with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?FITLD '''FITLD''']:<br />
# default fitld<br />
# datain 'MYAIP:0851+202.SPLIT.1<br />
# outdisk 1<br />
# inp<br />
# go<br />
<br />
and repeat for 1828+487.SPLIT.1 and 2200+420.SPLIT.1.<br />
<br />
'''Note that these data do not include self-calibration but does include flagging.'''<br />
<br />
=== Flagging 1828+487 ===<br />
[[Image:WIPER1.png|250px|thumb|right|'''Figure 8''' <br /> WIPER plot of amplitude vs UV distance.]]<br />
[[Image:WIPER2.png|250px|thumb|right|'''Figure 9''' <br /> WIPER plot with high amp points flagged (in blue).]]<br />
[[Image:WIPER3.png|250px|thumb|right|'''Figure 10''' <br /> WIPER plot with low amp points flagged (in blue).]]<br />
<br />
Let's do some more flagging before we get into imaging. A nice graphical flagging task that is especially useful after SPLITing is [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?WIPER '''WIPER''']. WIPER defaults to plot amplitude vs uv distance:<br />
<br />
# default wiper<br />
# getn 5<br />
# imsi 512<br />
# inp<br />
# go<br />
<br />
Ick! Look at those high points (Figure 8). Flag those by clicking on FLAG AREA, setting a box hitting the "A" key then hitting the "C" key when done, the flaged points will turn blue as in Figure 9. Now click EXIT and hit "A". Immediately type "go" to run WIPER again. This is more like it (Figure 10), but there are some low points, flag those with FLAG AREA but don't be too overzealous.<br />
<br />
Now you have 2 flag (FG) tables attached to the data, one for each run of WIPER. FG table 2 has all the flags in it.<br />
<br />
=== Imaging and Self-Cal of 1828+487 ===<br />
<br />
[[Image:IMAGR1.png|250px|thumb|right|'''Figure 11''' <br /> Dirty map of 1828+487.]]<br />
[[Image:IMAGR2.png|250px|thumb|right|'''Figure 12''' <br /> Residual map of 1828+487.]]<br />
[[Image:IMAGR3.png|250px|thumb|right|'''Figure 13''' <br /> Image of 1828+487 after first run of IMAGR.]]<br />
<br />
Lets start by imaging 1828+487 with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?IMAGR '''IMAGR''']. Either use the clean box files provided (make sure they are in the $MYAIP directory) or do interactive boxing. Using the supplied box file is easier. Also lets make sure that dowait=1 by typing:<br />
* dowait=1<br />
<br />
Note that I came up with the parameters below especially the IMSIZE, ROBUST, RASH and DECSH by imaging and experimenting.<br />
Task [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?SETFC '''SETFC'''] can be used to recommend CELLSIZE and IMSIZE if you specify a desired field of view. The shifts were chosen, for this source only, to center the object including its jet within the image.<br />
It is recommended that you use the supplied box files. They reflect both the very obvious emission in the field and additional information from the MOJAVE project.<br />
<br />
# default imagr<br />
# getn 5<br />
# cell 0.0001; imsi 1024 512 ➜ cell size of 0.1 mas; image size of 1024x512.<br />
# rash -0.0125 0<br />
# decsh 0.0125 0<br />
# dotv 1<br />
# robust 5 ➜ Do natural weighting. This source has a big diffuse jet, therefore sensitivity and more weight on the short spacings is more important than high resolution.<br />
# niter 5000 ➜ do interactive clean; with 5000 iterations, you will probably "stop cleaning" before hitting 5000 on the first iteration of imaging.<br />
# boxfile 'MYAIP:1828_1.box<br />
# inp<br />
# go<br />
# recat<br />
<br />
<br />
Hit TVPSEUDO then "B" immediately to get color rather than grey scale. Figure 11 shows the dirty map. If you want to clean interactively, then click CONTINUE CLEAN (and "A"); after a few rounds of cleaning (about 300 clean components which the messages from IMAGR tell you) the residuals will look like Figure 12. You can see in Figure 12 there is not much else to clean inside the clean boxes although there is some flux outside the clean boxes. This isn't significant enough to box at this point, so click STOP CLEANING ("A"). IMAGR will return the cleaned map, Figure 13, as you can see there are artifacts in the map.<br />
<br />
Once IMAGR is done you run [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?RECAT '''RECAT'''], which compresses the catalog so there are no empty spaces. This is not strictly necessary, but to follow this tutorial and have the GETN work, we have to be sure that the catalog numbers are consistent. Now do a [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?MCAT '''mcat'''] (mc for short) to look at the maps in your catalog:<br />
*mc<br />
<pre><br />
AIPS 1: Catalog on disk 1<br />
AIPS 1: Cat Usid Mapname Class Seq Pt Last access Stat<br />
AIPS 1: 6 195 1828+487 .IBM001. 1 MA 04-JUL-17 10:33:14<br />
AIPS 1: 7 195 1828+487 .ICL001. 1 MA 04-JUL-17 10:33:14<br />
</pre><br />
Note that you have made two files, a "dirty beam" (the IBM001 file) and a cleaned map (ICL001).<br />
<br />
To look at the image use [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?TVLOD '''TVLOD'''] (getn 6; tvin; tvlo). Use [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?TVPSEUDO '''TVPSEUDO'''] to put it into color (tvps). You can use your mouse to manipulate the colors; again look at your terminal for instructions. Hit "D" once it looks the way you want. Figure 11 shows the image with the stretch set so the noise level is obvious. <br />
<br />
Lets measure the rms in the cleaned image with [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?IMEAN '''IMEAN''']:<br />
<br />
# default imean<br />
# getn 7<br />
# go<br />
<br />
The rms for this image using IMEAN's histogram fitting in units of Janskys/beam is:<br />
<pre><br />
IMEAN1: Mean=-1.0708E-04 Rms= 2.2822E-03 **** from histogram<br />
</pre><br />
IMEAN also gives you other statistics about the image such as the maximum and total flux.<br />
<br />
So the image isn't the best, the noise is much higher than desirable, and there are hints of lower level structure but they are not much higher than the noise. So it looks like self-calibration would be beneficial. At some level self-cal is an art rather than a science, and there are various different philosophies concerning how to best do self-cal. I tend to be conservative, partially because VLBA data has fewer baselines and therefore it is easier to go off in a bad direction with the self-cal. However, these are fairly strong sources so it will be harder to make big mistakes in the self-calibration. Self-cal is done by the task [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?CALIB '''CALIB''']. Generally phase-only self-cal should be done first, then amplitude and phase. Also we will be using clean component models (i.e., the image we just made) for the self-calibration.<br />
<br />
# default calib<br />
# getn 5<br />
# get2n 7 ➜ use clean components from this map as a model for the self-cal<br />
# refant 2<br />
# solint 1 ➜ 1 minute solution interval<br />
# solty 'l1r' ➜ type of method for solution (see EXPLAIN CALIB for more details)<br />
# solmo 'P' ➜ solve for phases only<br />
# inp<br />
# go<br />
# recat<br />
<br />
You will get messages that look like this:<br />
<pre><br />
CALIB1: Task CALIB (release of 31DEC17) begins<br />
CALIB1: Using 1828+487 . SPLIT . 1 Disk= 1 Usid= 195<br />
CALIB1: L1 Solution type<br />
CALIB1: UVGET: Using flag table version 2 to edit data<br />
CALIB1: Create 1828+487 .CALIB . 1 (UV) on disk 1 cno 8<br />
CALIB1: Selecting and editing the data<br />
CALIB1: Using Clean Component self-cal source model<br />
CALIB1: FACSET: 1.375524 Jy found from 209 components<br />
CALIB1: Divide data by model - first compute model by summing<br />
CALIB1: QINIT: did a GET of 5120 Kwords, OFF 5241849<br />
CALIB1: ALGMEM field 1 Ipol gridded model subtraction chns 1- 8<br />
CALIB1: ALGMEM: at visibility number 1<br />
CALIB1: ALGMEM: 1024 - 0 cells, with 23349 Pts<br />
CALIB1: Field 1 used 209 CCs<br />
CALIB1: Determining solutions using full vector averaging<br />
CALIB1: Writing SN table 1<br />
CALIB1: Found 7380 good solutions<br />
CALIB1: 7 solutions had no data<br />
CALIB1: Average closure rms = 0.03470 +- 0.00913<br />
CALIB1: Fraction of times having data > 2.5 rms from solution<br />
CALIB1: 0.24661 of the times had 2 - 4 percent outside 2.5 times rms<br />
CALIB1: 0.09842 of the times had 4 - 6 percent outside 2.5 times rms<br />
CALIB1: 0.06674 of the times had 6 - 8 percent outside 2.5 times rms<br />
CALIB1: 0.08258 of the times had 8 - 10 percent outside 2.5 times rms<br />
CALIB1: 0.07579 of the times had 10 - 12 percent outside 2.5 times rms<br />
CALIB1: 0.06900 of the times had 12 - 14 percent outside 2.5 times rms<br />
CALIB1: 0.04186 of the times had 14 - 16 percent outside 2.5 times rms<br />
CALIB1: 0.02602 of the times had 16 - 18 percent outside 2.5 times rms<br />
CALIB1: 0.01810 of the times had 18 - 20 percent outside 2.5 times rms<br />
CALIB1: 0.01131 of the times had 22 - 24 percent outside 2.5 times rms<br />
CALIB1: 0.00113 of the times had 26 - 28 percent outside 2.5 times rms<br />
CALIB1: Applying solutions to data<br />
CALIB1: Previously flagged Flagged by gain Kept<br />
CALIB1: Partially 341 0 341<br />
CALIB1: Fully 0 0 23008<br />
CALIB1: Copied WX file from vol/cno/vers 1 5 1 to 1 8 1<br />
CALIB1: Copied AN file from vol/cno/vers 1 5 1 to 1 8 1<br />
CALIB1: Copied FG file from vol/cno/vers 1 5 1 to 1 8 1<br />
CALIB1: Copied FG file from vol/cno/vers 1 5 2 to 1 8 2<br />
CALIB1: Appears to have ended successfully<br />
</pre><br />
<br />
There are several things you should note in these messages:<br />
* CALIB creates a new UV file called 1828+487.CALIB.1 at catalog number 8<br />
* CALIB also writes an SN table attached to the input data file<br />
* CALIB found 7380 good solutions, 7 had no data and did not fail on any<br />
* CALIB didn't flag any new data. Since this is such a strong source CALIB probably will not flag any of the data. However for weaker sources this is a concern, a bad model and a weak source could mean that some portion of your input data is flagged by CALIB, meaning you can take your imaging to strange places if you are not careful.<br />
<br />
Now we image the corrected data. We can either use the file that CALIB created ''or'' the original file applying the SN table that CALIB attached to it. Just for brevity we will use the file CALIB created:<br />
<br />
# tget imagr<br />
# getn 8<br />
# boxfi 'MYAIP:1828_2.box<br />
# inp<br />
# go; recat<br />
<br />
After about 1500 clean components the map looks sufficiently clean. While doing the interactive clean the residuals did look better than in the first imaging but lets do IMEAN again (tget imean; getn 10; go) to confirm the rms is getting lower. The noise is now ~1.3E-03 Jy/beam, so a significant drop, but we can probably still do better. So more self-cal, with a shorter solution interval:<br />
<br />
# tget calib<br />
# get2n 10<br />
# solint 0.5<br />
# go; recat<br />
<br />
Now we have:<br />
<pre><br />
CALIB1: Writing SN table 2<br />
CALIB1: Found 14749 good solutions<br />
CALIB1: 12 solutions had no data<br />
</pre><br />
<br />
Image again:<br />
# tget imagr<br />
# getn 11<br />
# go; recat<br />
<br />
IMEAN again:<br />
# tget imean<br />
# getn 13<br />
# go<br />
<br />
Now the rms is about 1.1E-3 Jy/beam, so a little lower than last time but a much smaller drop. Lets try a much smaller solution interval.<br />
<br />
# tget calib<br />
# get2n 13<br />
# solint 0.1<br />
# go; recat<br />
<br />
Now we have:<br />
<pre><br />
CALIB1: Writing SN table 3<br />
CALIB1: Found 53090 good solutions<br />
CALIB1: Failed on 72 solutions<br />
CALIB1: 92 solutions had no data<br />
</pre><br />
So CALIB failed on a tiny fraction of solutions and flagged a tiny fraction of the data. This is OK, it just means that with this solution interval we are on the edge of not having enough sensitivity to get good solutions.<br />
<br />
Image again:<br />
# tget imagr<br />
# getn 14<br />
# boxfi 'MYAIP:1828_3.box<br />
# go; recat<br />
<br />
IMEAN again:<br />
# tget imean<br />
# getn 16<br />
# go<br />
<br />
Now the rms is about 8.8E-4 Jy/beam, another significant drop, either the low solint or the extra box is making things better, probably both. So lets do one more phase only self cal with this solution interval before going to amplitude and phase.<br />
<br />
# tget calib<br />
# get2n 16<br />
# go; recat<br />
<br />
Now we have:<br />
<pre><br />
CALIB1: Writing SN table 4<br />
CALIB1: Found 53093 good solutions<br />
CALIB1: Failed on 69 solutions<br />
CALIB1: 92 solutions had no data<br />
</pre><br />
Note that it failed on fewer solutions that last time, another indication that the additional box added to the cleaning allowed for model that fit the data better.<br />
<br />
Image again:<br />
# tget imagr<br />
# getn 17<br />
# go; recat<br />
<br />
IMEAN again:<br />
# tget imean<br />
# getn 19<br />
# go<br />
<br />
Now the rms is about 8.5E-4 Jy/beam, so another small drop.<br />
<br />
Lets move to amplitude self cal. You might have noticed that for the runs of CALIB we went back to the original data rather than the self-caled data. This is one of those philosophies of self-cal I mentioned earlier, you could have used the the output file from CALIB and done smaller increments of self-cal, I prefer to go back to the original data and just improve the model. I do this because here is less chance of getting an incorrect model stuck in the data with this method. However once we move to amplitude self-cal we want to use the data with the best calibrated phases so we will start with the last file out of CALIB.<br />
<br />
# tget calib<br />
# getn 17<br />
# get2n 19<br />
# solint 1<br />
# solmo 'a&p'<br />
# go; recat<br />
<br />
Now we have:<br />
<pre><br />
CALIB1: Writing SN table 1<br />
CALIB1: RPOL, IF= 1 The average gain over these antennas is 9.897E-01<br />
CALIB1: RPOL, IF= 2 The average gain over these antennas is 1.001E+00<br />
CALIB1: RPOL, IF= 3 The average gain over these antennas is 1.007E+00<br />
CALIB1: RPOL, IF= 4 The average gain over these antennas is 1.008E+00<br />
CALIB1: RPOL, IF= 5 The average gain over these antennas is 1.007E+00<br />
CALIB1: RPOL, IF= 6 The average gain over these antennas is 1.003E+00<br />
CALIB1: RPOL, IF= 7 The average gain over these antennas is 1.012E+00<br />
CALIB1: RPOL, IF= 8 The average gain over these antennas is 1.003E+00<br />
CALIB1: LPOL, IF= 1 The average gain over these antennas is 1.002E+00<br />
CALIB1: LPOL, IF= 2 The average gain over these antennas is 1.003E+00<br />
CALIB1: LPOL, IF= 3 The average gain over these antennas is 1.012E+00<br />
CALIB1: LPOL, IF= 4 The average gain over these antennas is 1.009E+00<br />
CALIB1: LPOL, IF= 5 The average gain over these antennas is 1.000E+00<br />
CALIB1: LPOL, IF= 6 The average gain over these antennas is 1.010E+00<br />
CALIB1: LPOL, IF= 7 The average gain over these antennas is 1.020E+00<br />
CALIB1: LPOL, IF= 8 The average gain over these antennas is 1.016E+00<br />
CALIB1: Found 7380 good solutions<br />
CALIB1: 7 solutions had no data<br />
</pre><br />
Similar messages to the phase only except it now tells you how it is adjusting the amplitudes in each IF/polarization.<br />
<br />
Image again:<br />
# tget imagr<br />
# getn 20<br />
# boxfi 'MYAIP:1828_4.box<br />
# go; recat<br />
<br />
You may have noticed that I am avoiding boxed the bright stuff around the core, because it is around the brightest thing in the map I am worried it is just sidelobes that have not been cleaned properly, at this point I am contemplating boxing it. Also next time we run IMAGR I will increase the number of iterations, 5000 seems like barely enough.<br />
<br />
IMEAN again:<br />
# tget imean<br />
# getn 22<br />
# go<br />
<br />
Now the rms is about 2.8E-4 Jy/beam, a big drop.<br />
<br />
Since we have a lot more structures in the model we should do more self-cal.<br />
<br />
# tget calib<br />
# get2n 22<br />
# go; recat<br />
<br />
Now we have:<br />
<pre><br />
CALIB1: Writing SN table 2<br />
CALIB1: RPOL, IF= 1 The average gain over these antennas is 9.909E-01<br />
CALIB1: RPOL, IF= 2 The average gain over these antennas is 1.003E+00<br />
CALIB1: RPOL, IF= 3 The average gain over these antennas is 1.008E+00<br />
CALIB1: RPOL, IF= 4 The average gain over these antennas is 1.009E+00<br />
CALIB1: RPOL, IF= 5 The average gain over these antennas is 1.008E+00<br />
CALIB1: RPOL, IF= 6 The average gain over these antennas is 1.004E+00<br />
CALIB1: RPOL, IF= 7 The average gain over these antennas is 1.013E+00<br />
CALIB1: RPOL, IF= 8 The average gain over these antennas is 1.004E+00<br />
CALIB1: LPOL, IF= 1 The average gain over these antennas is 1.003E+00<br />
CALIB1: LPOL, IF= 2 The average gain over these antennas is 1.004E+00<br />
CALIB1: LPOL, IF= 3 The average gain over these antennas is 1.013E+00<br />
CALIB1: LPOL, IF= 4 The average gain over these antennas is 1.010E+00<br />
CALIB1: LPOL, IF= 5 The average gain over these antennas is 1.001E+00<br />
CALIB1: LPOL, IF= 6 The average gain over these antennas is 1.011E+00<br />
CALIB1: LPOL, IF= 7 The average gain over these antennas is 1.021E+00<br />
CALIB1: LPOL, IF= 8 The average gain over these antennas is 1.017E+00<br />
CALIB1: Found 7380 good solutions<br />
CALIB1: 7 solutions had no data<br />
</pre><br />
<br />
Image again:<br />
# tget imagr<br />
# getn 23<br />
# niter 10000<br />
# go; recat<br />
<br />
IMEAN again:<br />
# tget imean<br />
# getn 25<br />
# go<br />
<br />
Now the rms is about 1.7E-4 Jy/beam, a big drop, CALIB liked the new model with all that extra part of the jet boxed, which is a good sign. Also in the residuals the emission around the core is much fainter, so it looks like it was good we did not box it. More self-cal can be one, but a majority of the improvement has been accomplished.<br />
<br />
=== Making a Contour Map === <br />
[[Image:KNTR1.png|250px|thumb|right|'''Figure 14''' <br /> Contour map of 1828+487 after self calibration.]]<br />
[[Image:KNTR2.png|250px|thumb|right|'''Figure 15''' <br /> Countour map of 1828+487 before self calibration. ]]<br />
<br />
What is the best way to look at the images, particularly with all this low level structure? One of the best ways is with a contour map. We will use [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?KNTR '''KNTR'''] to make such a map. I came up with the following parameters after some experimentation.<br />
<br />
# default kntr<br />
# getn 25<br />
# dovect -1<br />
# dotv 1<br />
# ofmfile 'ryphlame'<br />
# clev 5e-4<br />
# blc 295 35<br />
# trc 721 419<br />
# pixra -0.0002 0.05<br />
# functy 'l2'<br />
# levs(1)=-1<br />
# for i=2 to 30; levs(i)=2**((i-2)/2.); end<br />
# inp<br />
# tvin; go<br />
# recat<br />
<br />
Note that I used verb [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?TVWINDOW '''TVWIN'''] to set blc and trc (bottom left corner and top right corner of the image to plot). Also I am using a for loop to set the contour levels in a logarithmic fashion. I have set the contour level multiplier (clev) and the pixel range (pixra) so that we can see some of the noise. I am also using an ofm file for the background. To see all the ofm files that come with AIPS type "OFMDIR"; ofmfile 'rainbow' is closest to tvpseudo. Figure 14 shows what the figure looks like on the TV. <br />
<br />
Just for fun run KNTR with the same inputs on the image of 1828+487 before all the self-calibration. Figure 15 shows what you get, the vast improvement made by the self calibration is obvious.<br />
<br />
To make a hard copy of this figure set DOTV to -1 and run KNTR, KNTR will produce a PL (plot) table. Then run [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?LWPLA '''LWPLA'''] to make a postscript of that PL table.<br />
<br />
# default lwpla<br />
# getn 25; outfi 'MYAIP:1828+487.ps<br />
# go<br />
<br />
This produces a postscript file in the $MYAIP directory.<br />
<br />
=== Create More Images ===<br />
[[Image:OJ287.png|250px|thumb|right|'''Figure 16''' <br /> Contour map of 0851+202 (OJ287) after self calibration. This is a version of the image that uses LWPLA to make a hardcopy after KNTR.]]<br />
[[Image:BLLAC.png|250px|thumb|right|'''Figure 17''' <br /> Contour map of 2200+420 (BL Lac) after self calibration. This is a version of the image that uses LWPLA to make a hardcopy after KNTR.]]<br />
<br />
Image 2200+420 and/or 0851+202 with '''IMAGR'''.<br />
<br />
# tget imagr<br />
# getn 3 or 4<br />
# rash 0<br />
# decsh 0<br />
# imsi 512<br />
# inp<br />
# go<br />
<br />
You can follow similar steps with WIPER and self-calibration on these sources.<br />
<br />
== Doing All the Calibration (Except the Self-Calibration) With a Pipeline ==<br />
<br />
Inside AIPS there is a procedure [http://www.aips.nrao.edu/cgi-bin/ZXHLP2.PL?VLBARUN '''VLBARUN'''] which will do all the calibration steps above except for the flagging and self-calibration. VLBARUN uses the VLBAUTIL procedures to calibrate the data with little intervention.<br />
<br />
Just to be safe type RESTORE 0, which will remove all the procedures from the AIPS "memory". It is also best if VLBARUN is run in a pristine environment, to do this restart AIPS (just type "restart" from inside AIPS) and use a different AIPS number. Then load the two sets of procedures:<br />
<br />
# run vlbautil<br />
# run vlbarun<br />
<br />
Next set up the VLBARUN inputs, here I assume you want VLBARUN load the data, but VLBARUN can also be be run on data that is already loaded :<br />
# default vlbarun<br />
# datain 'FITS:bl193bg_3sour_avg.fits<br />
# outname 'bl195bg'<br />
# outdisk 1<br />
# indisk outdisk<br />
# opty 'cont'<br />
# clint 0.25<br />
# calsour '0851+202' '1828+487' '2200+420'<br />
# solint 0.25<br />
# factor 2<br />
# doplot 1<br />
# outfi ''some directory to put plots and .html file for examination''<br />
# outtext ''your@email.edu''<br />
# vlbarun<br />
<br />
Since I set doplot=1 and outfile the procedure will produce diagnostic plots put them in the directory set in outfile and write an html page to allow you to examine the plots easily. For an example of the output for this data see [http://www.aoc.nrao.edu/~amiodusz/VLBARUN/vlbarun.html vlbarun.html], the main difference is that VLBARUN ran VLBAEOPS, which we skipped so there is one more CL table.<br />
<br />
<br />
<!--<br />
--Original: Mioduszewski, Amy <br /><br />
--Wiki: Salcido, Jose <br /> <br />
--></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Imaging-CASA5.5.0&diff=27310VLA CASA Imaging-CASA5.5.02019-05-15T00:47:01Z<p>Emomjian: /* Imaging Spectral Cubes */</p>
<hr />
<div><br />
== Imaging ==<br />
<br />
'''This guide has been prepared for CASA 5.4.0'''<br />
<br />
<div style="text-align: justify;"><br />
This tutorial provides guidance on imaging procedures in CASA. The tutorial covers basic continuum cleaning and the influence of image weights, as well as wide-band and wide-field imaging techniques, multi-scale clean, an outlier field setup, and primary beam correction. Spectral line imaging procedures are explained but not covered in detail. For a more thorough example of spectral line imaging, refer to the [https://casaguides.nrao.edu/index.php/VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 VLA high frequency Spectral Line tutorial - IRC+10216]. This imaging tutorial concludes with basic image header calls and the conversion of Jy/beam to K surface brightness units through image arithmetic and header manipulation.<br />
<br />
We will be utilizing data taken with the Karl G. Jansky Very Large Array, of a supernova remnant G055.7+3.4. The data were taken on August 23, 2010, in the first D-configuration for which the new wide-band capabilities of the WIDAR (Wideband Interferometric Digital ARchitecture) correlator were available. The 8-hour-long observation includes all available 1 GHz of bandwidth in L-band, from 1 - 2 GHz in frequency. <br />
<br />
We will skip the calibration process in this guide, as examples of calibration can be found in several other guides, including the <br />
[https://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 VLA Continuum Tutorial 3C391] and <br />
[https://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 VLA high frequency Spectral Line tutorial - IRC+10216] guides.<br />
<br />
A copy of the calibrated data <font color=green>(1.2GB)</font> can be downloaded from [http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.calib.tar.gz http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.calib.tar.gz] <br />
<br />
Your first step will be to unzip and untar the file in a terminal (before you start CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf SNR_G55_10s.calib.tar.gz<br />
</source><br />
<br />
Then start CASA as usual via the '''casa''' command, which will bring up the ipython interface and launches the logger.<br />
</div><br />
== The CLEAN Algorithm ==<br />
<div style="text-align: justify;"><br />
[[Image:CLEAN_Cycle.png|500px|thumb|right|'''Figure 1''' <br /> The CLEAN major and minor cycles, indicating the steps undertaken during gridding, projection algorithms, and creation of images.]]<br />
<br />
The CLEAN algorithm, developed by J. Högbom (1974) enabled the synthesis of complex objects, even if they have relatively poor Fourier uv-plane coverage. Poor coverage occurs with partial earth rotation synthesis, or with arrays composed of few antennas. The "dirty" image is formed by a simple Fourier inversion of the sampled visibility data, with each point on the sky being represented by a suitably scaled and centered PSF (Point Spread Function), or dirty beam, which itself is the Fourier inversion of the visibility (u,v) coverage.<br />
<br />
The convolution with the dirty beam creates artifacts in the image and limits the dynamic range. The CLEAN algorithm attempts to remove the dirty beam pattern from the image via deconvolution. This implies that it interpolates from the measured (u,v) points across gaps in the (u,v) coverage. In short, CLEAN provides solutions to the convolution equation by representing radio sources by a number of point sources in an empty field. The brightest points are found by performing a cross-correlation between the dirty image, and the PSF. The brightest parts (actually their PSFs) are subtracted, and the process is repeated again for the next brighter sources. Variants of CLEAN, such as multi-scale CLEAN, take into account extended kernels which may be better suited for extended objects.<br />
<br />
For single pointings, CASA uses the Hogbom cleaning algorithm by default in the task {{tclean}} (''deconvolver='hogbom'''), which breaks the process into major and minor cycles (see Figure 1). To start with, the visibilities are gridded, weighted, and Fourier transformed to create a (dirty) image. The minor cycles then operate in the image domain to find the clean components that are added to the clean model: repeatedly performing a PSF+image correlation to find the next bright point, then subtracting its PSF from the image. The model image is Fourier transformed back to the visibility domain, degridded, and subtracted from the visibilities. This creates a new residual that is then gridded, weighted, and FFT'ed again to the image domain for the next iteration. The gridding, FFT, degridding, and subtraction processes form a major cycle. <br />
<br />
This iterative process is continued until a stopping criterion is reached, such as a maximum number of clean components, or a flux threshold in the residual image.<br />
<br />
In CASA {{tclean}}, two versions of the PSF can be used: setting ''deconvolver='hogbom' '' uses the full-sized PSF for subtraction. This is a thorough but slow method. All other options use a smaller beam patch, which increases the speed. The patch size and length of the minor cycle are internally chosen such that clean converges well without giving up the speed improvement. <br />
<br />
<!--<br />
''' 1. Högbom Algorithm '''<br /><br />
This algorithm will initially find the strength and position of a peak in a dirty image, subtract it from the dirty image, record this position and maginitude, and repeat for further peaks. The remainder of the dirty image is known as the residuals. <br />
<br />
The accumulated point sources, now residing in a model, is convolved with an idealized CLEAN beam (usually a Gaussian fitted to the central lobe of the dirty beam), creating a CLEAN image. As the final step, the residuals of the dirty image are then added to the CLEAN image. <br />
<br />
''' 2. Clark Algorithm '''<br /><br />
Clark (1980), developed a FFT-based CLEAN algorithm, which more efficiently shifts and scales the dirty beam by approximating the position and strength of components using a small patch of the dirty beam. This algorithm is the default within the {{clean}} task, which involves major and minor cycles. <br />
<br />
The algorithm will first select a beam patch, which will include the highest exterior sidelobes. Points are then selected from the dirty image, which are up to a fraction of the image peak, and are greater than the highest exterior sidelobe of the beam. It will then conduct a list-based Högbom CLEAN, creating a model and convolution with an idealized CLEAN beam. This process is the minor cycle. <br />
<br />
The major cycle involves transforming the point source model via a FFT (Fast-Fourier Transform), mutiplying this by the weight sampling function (more on this below), and transformed back. This is then subtracted from the dirty image, creating your CLEAN image. The process is then repeated with subsequent minor cycles. <br />
<br />
''' 3. Cotton-Schwab Algorithm '''<br /><br />
This is the default imager mode (''csclean''), and is a variant of the Clark algorith in which the major cycle involves the subtraction of CLEAN components of ungridded visibility data. This allows the removal of gridding errors, as well as noise. One advantage is its ability to image and clean many seperate fields simultaneously. Fields are cleaned independently in the minor cycle, and components from all fields cleaned together in the major cycles.<br />
<br />
This algorithm is faster than the Clark algorithm, except when dealing with a large number of visibility samples, due to the re-gridding process it undergoes. It is most useful in cleaning sensitive high-resolution images at lower frequencies where a number of confusing sources are within the primary beam. <br />
--> <br />
In a final step, {{tclean}} derives a Gaussian fit to the inner part of the dirty beam, which defines the clean beam. The clean model is then convolved with the clean beam and added to the last residual image to create the final image. <br />
<br />
For more details on imaging and deconvolution, we refer to the Astronomical Society of the Pacific Conference Series book entitled [http://www.aspbooks.org/a/volumes/table_of_contents/?book_id=292 Synthesis Imaging in Radio Astronomy II]. The chapter on [http://www.aspbooks.org/a/volumes/article_details/?paper_id=17942 Deconvolution] may prove helpful. In addition, imaging presentations are available on the [http://www.cvent.com/events/15th-synthesis-imaging-workshop/agenda-e34ab08b2a854a48bae723c8fe86d2d4.aspx Synthesis Imaging Workshop] and [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/program VLA Data Reduction Workshop] webpages. The [https://casa.nrao.edu/casadocs/casa-5.4.0 CASA Documentation] chapter on [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-imaging/imaging-overview Synthesis Imaging] provides a wealth of information on the CASA implementation of {{tclean}} and related tasks.<br />
<br />
Finally, we refer users to the [https://science.nrao.edu/facilities/vla/docs/manuals/oss VLA Observational Status Summary] and the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide Guide to Observing with the VLA] for information on the VLA capabilities and observing strategies.<br />
</div><br />
<br />
== Weights and Tapering ==<br />
<div style="text-align: justify;"><br />
[[Image:SNR-G55-uvcoverage-CASA4.5.2.png|400px|thumb|right|'''Figure 2''' <br /> u,v coverage for the 8-hour observation of the supernova remnant G055.7+3.4]]<br />
<br />
'Weighting' amounts to giving more or less weight to certain visibilities in your data set, based on their location in the uv-plane. Emphasizing long-baseline visibilities improves the resolution of your image, whereas emphasizing shorter baselines improves the surface brightness sensitivity. There are three main weighting schemes that are used in interferometry:<br />
<br />
1) '''Natural''' weighting: uv cells are weighted based on their rms. Data visibility weights are gridded onto a uv-cell and summed. More visibilities in a cell will thus increase the cell's weight, which will usually emphasize the shorter baselines. Natural weighting therefore results in a better surface brightness sensitivity, but also a larger PSF and therefore degraded resolution. <br />
<br />
2) '''Uniform''' weighting: The weights are first gridded as in natural weighting but then each cell is corrected such that the weights are independent of the number of visibilities inside. The 'uniform' weighting of the baselines is a better representation of the uv-coverage and sidelobes are more suppressed. Compared to natural weighting, uniform weighting usually emphasizes the longer baselines. Consequently the PSF is smaller, resulting in a better spatial resolution of the image. At the same time, however, the surface brightness sensitivity is reduced compared to natural weighting.<br />
<br />
3) '''Briggs''' weighting: This scheme provides a compromise between natural and uniform weighting. The transition can be controlled with the ''robust'' parameter where ''robust=-2'' is close to uniform and ''robust=2'' is close to natural weighting. <br />
Briggs weighting therefore offers a compromise for between spatial resolution and surface brightness sensitivity. Typically, ''robust'' values near zero are used.<br />
<br />
Details on the weighting schemes are given in [http://www.aoc.nrao.edu/dissertations/dbriggs/ Daniel Brigg's dissertation (Chapter 3)].<br />
<br />
For a visual comparison between these three weighting schemes, please see the section on "CLEAN with Weights" in this guide.<br />
<!--<br />
For a brief intro to the different clean algorithms, as well as other deconvolution and imaging information, please see the website kept by Urvashi R.V. [http://www.aoc.nrao.edu/~rurvashi/ImagingAlgorithmsInCasa/node2.html#SECTION00223000000000000000 here].<br />
<br />
<br />
There are different options for <math> W(u,v) </math> (parameter ''weighting'' in {{tclean}}):<br />
<br />
'''1. Natural''': For the CASA parameter ''weighting=’natural’'', visibilities are weighted only by the data weights, which are calculated during filling and calibration and should be equal to the inverse noise variance on that visibility. Imaging weight <math>W_i</math> of sample <math>i</math> is given by <math>W_i = \omega_i = 1/\sigma_i^2</math><br />
<br />
where the data weight <math>\omega_i</math> is determined from <math>\sigma_i</math>, the rms noise on visibility <math>i</math>. When data is gridded into the same uv-cell for imaging, the weights are summed, and thus a higher uv density results in higher imaging weights.<br />
<br />
The weight function can be described as <math> W(u,v) = 1/ \sigma^2 </math>, where <math> \sigma^2 </math> is the noise variance. Natural weighting will maximize point source sensitivity, and provide the lowest rms noise within an image, as well as the highest signal-to-noise. It will also generally give more weight to short baselines, thus the synthesized beam will be large. This form of weighting is the default within the clean task. <br />
<br />
'''2. Uniform''': For ''weighting = ’uniform’'', the data weights are calculated as in ’natural’ weighting. The data is then gridded to a number of cells in the uv-plane, and after all data is gridded the uv-cells are re-weighted to have 'uniform' imaging weights. This pumps up the influence on the image of data with low weights (they are multiplied up to be the same as for the highest weighted data), which sharpens resolution and reduces the sidelobe level in the field-of-view, but increases the rms image noise. <br />
<br />
For uniform weighting, we first grid the inverse variance <math>\omega_i</math> for all selected data onto a grid with uv cell-size given by <math>2/FOV</math> where <math>FOV</math> is the specified field of view (defaults to the image field of view). This forms the gridded weights <math>W_k</math>. The weight of the <math>i</math>-th sample is then:<math>W_i=\omega_i/W_k</math><br />
<br />
The weight function can be described as <math> W(u,v) = W(u,v) / W_k </math>, where <math> W_k </math> represents the local density of (u,v) points, otherwise known as the gridded weights. This form of weighting will increase the influence of data with lower weight, filling the (u,v) plane more uniformly, thereby reducing sidelobe levels in the field-of-view, but increasing the rms image noise. More weight is given to long baselines, therefore increasing angular resolution. Point source sensitivity is degraded due to the downweighting of some data. <br />
<br />
'''3. Briggs''': A flexible weighting scheme, that is a variant of uniform, and avoids giving too much weight to (u,v) points with a low natural weight. Weight function can be described as <br />
<math> W(u,v) = 1/ \sqrt{1+S_N^2/S_{thresh}^2} </math>, where <math> S_N </math> is the natural weight of the cell, <math> S_{thresh} </math> is a <br />
threshold. <br />
A high threshold will go to a natural weight, where as a low threshold will go to a uniform weight. This form of weighting also has adjustable parameters. The ''robust'' parameter will give variation between resolution and maximum point source sensitivity. It's value can range from -2.0 (close to uniform weight) to 2.0 (close to natural weight). By default, the parameter is set to 0.0, which gives a good trade-off. <br />
<br /> <br />
--><br />
<br />
[[Image:Weight_Tapering_table.png|450px|thumb|right|'''Table 1''' <br /> Table summarizing the effects of using weights and tapering.]]<br />
'''Tapering''': In conjunction with the above weighting schemes, one can specify the ''uvtaper'' parameter within {{tclean}}, which will control the radial weighting of visibilities in the uv-plane. Figure 2 illustrates the uv-coverage during the observing session used in this guide. The taper in {{tclean}} is an elliptical Gaussian function which effectively removes long baselines and degrades the resolution. For extended structure this may be desirable when the long baselines contribute a large fraction of the noise. <br />
<!--<br />
The tapering will apodize, or filter/change the shape of the weight function (which is itself a Gaussian), which can be expressed as: <br /><br />
<math> W(u,v) = e^{-(u^2+v^2)/t^2} </math>, where t is the adjustable tapering parameter. <br />
--><br />
Tapering can therefore increase the surface brightness sensitivity of the data, but will decrease the point source sensitivity. Too aggressive tapering, however, may also take its toll on the surface brightness sensitivity.<br />
<!--<br />
Due to the downweight of some data, point source sensitivity can be degraded, as some antennas and baselines will suppressed and, in the extreme case, effectively removed from the data averaging. If your observation was sampled by short baselines, tapering may improve sensitivity to extended structures. <br />
--><br />
<br />
Table 1 summarizes the main effects of the different weighting schemes. We refer to the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-imaging/imaging-overview Synthesis Imaging] section of the CASA Documentation for the details of the weighting implementation in CASA's {{tclean}}.<br />
</div><br />
<br />
== Primary and Synthesized Beam ==<br />
<div style="text-align: justify;"><br />
The primary beam of a single antenna defines the sensitivity across the field of view. For the VLA antennas, the main part of the primary beam can be approximated by a Gaussian with a FWHM equal to <math>42/ \nu_{GHz}</math> arcminutes (for frequencies in the range 1 - 50 GHz). But note that there are sidelobes beyond the Gaussian kernel that are sensitive to bright sources (see below). Taking our observed frequency to be the middle of the L-band, 1.5 GHz, our primary beam (FWHM) will be about 28 arcmin in diameter.<br />
<br />
Note: New beam measurements were made recently and are described in [https://library.nrao.edu/public/memos/evla/EVLAM_195.pdf EVLA memo #195]. These newer beam corrections are the default in CASA 5.4.0. <br />
<br />
If your science goal is to image a source or field of view that is significantly larger than the FWHM of the VLA primary beam, then creating a mosaic from a number of telescope pointings is usually the preferred method. For a tutorial on mosaicking, see the [https://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 tutorial]. In this guide, we discuss methods for imaging large areas from single-pointing data. <br />
<br />
As these data were taken in the D-configuration, we can check the [https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/resolution Observational Status Summary]'s section on VLA resolution to find that the synthesized beam for uniform weighting will be approximately 46 arcsec. Variations in flagging, weighting scheme, and effective frequency may result in deviations from this value. The synthesized beam is effectively the angular resolution of the image. As we will see later, the synthesized beam of our data hovers around 29 arcsec or, for the extreme of uniform weighting, around 26"x25". <!--Oversampling the minor axis by a factor of ~3, we will use a cell (pixel) size of 8 arcsec.--><br />
<br />
== Cell (Pixel) Size and Image Size ==<br />
<br />
For the most effective cleaning, we recommend using a pixel size such that there are 3 - 5 pixels across the synthesized beam. Based on the assumed synthesized beam size of 46 arcsec, we will use a cell (pixel) size of 8 arcsec. <br />
<br />
In the {{tclean}} task, the image size is defined by the number of pixels in the RA and dec directions. The execution time of {{tclean}} depends on the image size. Large images generally take more computing time. There are some particular image sizes (by number of pixels) that are computationally inadvisable. For inputs corresponding to such image sizes, the logger will show a recommendation for an appropriate larger, but computationally faster, image size. As a general guideline, we recommend image sizes <math>2^n*10</math> pixels, e.g. 160, 1280 pixels, etc. for improved processing speeds.<br />
<br />
Our target field contains bright point sources that are well outside the primary beam. The VLA, in particular when using multi-frequency synthesis (see below), will have significant sensitivity outside the main lobe of the primary beam. Particularly at the lower VLA frequencies, sources that are located outside the primary beam may still be bright enough to be detected in the primary beam sidelobes, causing artifacts that interfere with the targeted field in the main part of the primary beam. Such sources need to be cleaned to remove the dirty beam response of these interfering sources from the entire image. This can be achieved either by creating a very large image that encompasses these interfering sources, or by using 'outlier' fields centered on the strongest sources (see section on outlier fields below). A large image has the added advantage of increasing the field of view for science (albeit at lower sensitivity beyond the primary beam). But other effects will start to become significant, like the non-coplanarity of the sky. Large image sizes will also slow down the deconvolution process. Details are provided in Sanjay Bhatnagar's presentation: [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/copy_of_WF_WB.pdf "Advanced Imaging: Imaging in the Wide-band Wide-field era"] given at the [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/program 2016 VLA data reduction workshop].<br />
<br />
The calls to the {{tclean}} task within this guide will create images that are approximately 170 arcminutes on a side, or almost 6x the size of the primary beam, encompassing its first and second sidelobes. This is ideal for showcasing both the problems inherent in wide-band/wide-field imaging, as well as some of the solutions currently available in CASA to deal with these issues. We therefore set the image size to 1280 pixels on each side, for efficient processing speed. (1280 pixels x 8 arcsec/pixel = 170.67 arcmin)<br />
</div><br />
<br />
== Clean Output Images ==<br />
<div style="text-align: justify;"><br />
As a result of the CLEAN algorithm, {{tclean}} will create a number of output images. For a parameter setting of ''imagename='<imagename>''', these image names will be:<br />
<br />
'''<imagename>.residual''': the residual after subtracting the clean model (unit: Jy/beam, where beam refers to the dirty beam)<br><br />
'''<imagename>.model''': the clean model, not convolved (unit: Jy/pixel)<br><br />
'''<imagename>.psf''': the point-spread function (dirty beam)<br><br />
'''<imagename>.pb''': the normalized sensitivity map, which corresponds to the primary beam in the case of a single-pointing image<br><br />
'''<imagename>.image''': the residual + the model convolved with the clean beam; this is the final image (unit: Jy/beam, where beam refers to the clean beam)<br><br />
'''<imagename>.sumwt''': a single pixel image containing sum-of-weights (for natural weighting, the sensitivity = 1/sqrt(sumwt).<br />
Additional images will be created for specific algorithms like multi-term frequency synthesis or mosaicking.<br />
<br />
<font color=red>'''Important:'''</font> If an image file is present in the working directory and the same name is provided in ''imagename'', {{tclean}} will use that image (in particular the residual and model image) as a starting point for further cleaning. If you want a fresh run of {{tclean}}, first remove all images of that name using 'rmtables()':<br />
<br />
<font color=red>'''Important:'''</font> By default, {{tclean}} sets ''savemodel'' to a value of 'none', meaning no model image is saved. Be sure to set this parameter to 'modelcolumn' for any model you wish to save. This is especially important for self-calibration. <br />
<br />
<source lang="python"><br />
# In CASA<br />
rmtables('<imagename>.*')<br />
</source><br />
<br />
This method is preferable over 'rm -rf' as it also clears the cache.<br />
<br />
<font color=red>'''Note that interrupting {{tclean}} by Ctrl+C may corrupt your visibilities - you may be better off choosing to let {{tclean}} finish. We are currently implementing a command that will nicely exit to prevent this from happening, but for the moment try to avoid Ctrl+C.'''</font><br />
<br />
== Dirty Image ==<br />
<br />
First, we will create a dirty image (Figure 3a) to see the improvements as we step through several cleaning algorithms and parameters. The dirty image is the true image on the sky, convolved with the dirty beam, also known as the Point Spread Function (PSF). We create a dirty image by running {{tclean}} with niter=0, which will run the task without performing any CLEAN iterations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.dirty', <br />
imsize=1280, cell='8arcsec', pblimit=-0.01, niter=0, <br />
stokes='I', savemodel='modelcolumn')<br />
</source><br />
* ''imagename='SNR_G55_10s.dirty''': the root filename used for the various {{tclean}} outputs. <br />
<br />
* ''imsize=1280'': the image size in number of pixels. A single value will result in a square image.<br />
<br />
* ''cell='8arcsec''': the size of one pixel; again, entering a single value will result in a square pixel size.<br />
<br />
* ''niter=0'': this controls the number of iterations done in the minor cycle.<br />
<br />
* ''interactive=False'': For a tutorial that covers more of an interactive clean, please see the [https://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 VLA high frequency Spectral Line tutorial - IRC+10216.]<br />
<br />
* ''savemodel='modelcolumn'': controls writing the model visibilities to the model data column. For self-calibration we currently recommend setting savemodel='modelcolumn'. The default value is "none": so, this must be changed for the model to be saved. The use of the virtual model with MTMFS in {{tclean}} is still under commissioning, so we recommend currently to set ''savemodel='modelcolumn'.''<br />
<br />
* ''stokes='I''': since we have not done any polarization calibration, we only create a total-intensity image. For using CLEAN while including various Stoke's parameters, please see the [https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 CASA guide.]<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.dirty.image')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.dirty.psf')<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:SNR_G55_10s.dirty.image.png|400px|thumb|'''Figure 3a''' <br /> A dirty image of the supernova remnant G55.7+3.4 in greyscale, with apparent sidelobes.]]<br />
| [[Image:SNR_G55_10s.dirty.psf.png|400px|thumb|'''Figure 3b''' <br /> The Point Spread Function (PSF) in greyscale. This is also known as the dirty beam.]]<br />
|}<br />
<br />
The images may be easier to see in grey scale. To change the color scheme, click on "Data Display Options" (wrench icon, upper left corner) within the viewer, and change the color map to "Greyscale 1". You may also wish to change the scaling power options to your liking (e.g., -1.5). To change the brightness and contrast, assign a mouse button to this type of editing by clicking on the "Colormap fiddling" icon (black/white circle), and click/drag the mouse over the image to change the brightness (left-right mouse movement) and contrast (up-down mouse movement). <br />
<br />
Note that the clean beam is only defined after some clean iterations. The dirty image has therefore no beam size specified in the header, and the PSF image (Figure 3b) is the representation of the response of the array to a point source. Even though it is empty because we set ''niter=0'', {{tclean}} will still produce a model file. Thus we could progress into actual cleaning by simply restarting {{tclean}} with the same image root name (and ''niter''>0).<br />
<br />
== Regular CLEAN & RMS Noise ==<br />
Now, we will create a regular clean image using mostly default values to see how deconvolution improves the image quality. The first run of {{tclean}} will use a fixed number of minor cycle iterations of ''niter=1000'' (default is 0); the second will use ''niter=10000''. Note that you may have to play with the image color map and brightness/contrast to get a better view of the image details. <br />
<br />
<source lang="python"><br />
# In CASA. Create default clean image.<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.Reg.Clean.niter1K', <br />
imsize=1280, cell='8arcsec', pblimit=-0.01, niter=1000, savemodel='modelcolumn')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA.<br />
viewer('SNR_G55_10s.Reg.Clean.niter1K.image')<br />
</source><br />
<br />
The logger indicates that the image was obtained in two major cycles, and some improvements over the dirty image are visible. But clearly we have not cleaned deep enough yet; the image still has many sidelobes, and an inspection of the residual image shows that it still contains source flux and structure. So let's increase the ''niter'' value to 10,000 and compare the images.<br />
<br />
<source lang="python"><br />
# In CASA. Create default clean image with niter = 10000<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.Reg.Clean.niter10K', <br />
imsize=1280, cell='8arcsec', pblimit=-0.01, niter=10000, savemodel='modelcolumn')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA.<br />
viewer('SNR_G55_10s.Reg.Clean.niter10K.image')<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:SNR_G55_10s.niter1K.png|400px|thumb|'''Figure 4a''' <br /> Regular run of TCLEAN, with niter=1000.]]<br />
| [[Image:SNR_G55_10s.niter10K.png|400px|thumb|'''Figure 4b''' <br /> Regular run of TCLEAN, with niter=10000.]]<br />
|}<br />
<br />
[[Image:SNR_G55_10s.rms.screen.png|500px|thumb|right|'''Figure 5''' <br /> Attempting to find the lowest rms value within the CLEAN'ed image using niter=1000, in order to calculate our threshold.]]<br />
<br />
As we can see from the resulting images, increasing the niter values (minor cycles) improves our image by reducing prominent sidelobes significantly (Figure 4b). One could now further increase the ''niter'' parameter until the residuals are down to an acceptable level. To determine the number of iterations, one needs to keep in mind that {{tclean}} will fail once it starts cleaning too deeply into the noise. At that point, the cleaned flux and the peak residual flux values will start to oscillate as the number of iterations increase. This effect can be monitored on the CASA logger output. To avoid cleaning too deeply, we will set a ''threshold'' parameter that will stop minor cycle clean iterations once a peak residual value is being reached. <br />
<br />
First, we will utilize the ''SNR_G55_10s.Reg.Clean.niter1K.image'' (Figure 4a) image to give us an idea of the rms noise (your sigma value).<br />
With the image open within the viewer, click on the 'Rectangle Drawing' button (Rectangle with R) and draw a square on the image at a position with little source or sidelobe contamination. Doing this should open up a "Regions" dock, which holds information about the selected region, including the pixel statistics in the aptly named "Statistics" tab. Take notice of the rms values as you click/drag the box around empty image locations, or by drawing additional boxes at suitable positions. If the "Regions" dock is not displayed, click on "View" in the menu bar at the top of the viewer and click on the "Regions" check-box.<br />
<br />
The lowest rms value we found was in the order of about <math>5\cdot10^{-5}</math> Jy/beam, which we will use to calculate our threshold. There really is no set standard, but fairly good threshold values can vary anywhere between 3.0 - 5.0*sigma; using clean boxes (see the section on interactive cleaning) allows one to go to lower thresholds. For our purposes, we will choose a threshold of 3*sigma. Doing the math results in a value of <math> 15\cdot10^{-5}</math> or equivalently 0.15mJy/beam. Therefore, for future calls to the {{tclean}} task, we will set ''threshold='0.15mJy'''. The clean cycle will be stopped when the residual peak flux density is equals to or less than the ''threshold'' value, or when the maximum number of iterations ''niter'' is reached. To ensure that the stopping criterion is indeed ''threshold'', ''niter'' should be set to a very high number. '''In the following, we nevertheless will use ''niter=1000'' to keep the execution times of {{tclean}} on the low end as we focus on explaining different imaging methods.'''<br />
<br />
An alternative method to determine the approximate rms of an image is to use the [https://science.nrao.edu/facilities/vla/docs/manuals/propvla/determining VLA Exposure Calculator] and to enter the observing conditions. Make sure that the chosen bandwidth reflects the data after RFI excision.<br />
</div><br />
<br />
== CLEAN with Weights ==<br />
<div style="text-align: justify;"><br />
To see the effects of using different weighting schemes to the image, let's change the ''weighting'' parameter within {{tclean}} and inspect the resulting images. We will be using the ''Natural, Uniform, and Briggs'' weighting algorithms. Here, we have chosen a smaller image size to mainly focus on our science target. <br />
<br />
<source lang="python"><br />
# In CASA. Natural weighting<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.natural', weighting='natural',<br />
imsize=540, cell='8arcsec', pblimit=-0.01, niter=1000, interactive=False, threshold='0.15mJy', savemodel='modelcolumn')<br />
</source><br />
<br />
* ''weighting'': specification of the weighting scheme. For Briggs weighting, the robust parameter will be used.<br />
<br />
* ''threshold='0.15mJy''': threshold at which the cleaning process will halt.<br />
<br />
<source lang="python"><br />
# In CASA. Uniform weighting<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.uniform',weighting='uniform',<br />
imsize=540, cell='8arcsec', pblimit=-0.01, niter=1000, interactive=False, threshold='0.15mJy', savemodel='modelcolumn')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA. Briggs weighting, with robust set to 0.0<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.briggs', weighting='briggs', robust=0.0,<br />
imsize=540, cell='8arcsec', pblimit=-0.01, niter=1000, interactive=False, threshold='0.15mJy', savemodel='modelcolumn')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA. Open the viewer and select the created images.<br />
viewer()<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:SNR_G55_10s.natural.image.png|375px|thumb|'''Figure 6a''' <br /> Natural weighting.]]<br />
| [[File:SNR_G55_10s.uniform.image.png|375px|thumb|'''Figure 6b''' <br /> Uniform weighting.]]<br />
| [[File:SNR_G55_10s.briggs.image.png|375px|thumb|'''Figure 6c''' <br /> Briggs weighting.]]<br />
|}<br />
<br />
<br />
Figure 6a shows that the natural weighted image is most sensitive to extended emission (beam size of 46"x41"). The negative values around the extended emission (often referred to as a negative 'bowl') are a typical signature of unsampled visibilities near the origin of the uv-plane. That is, the flux density present at shorter baselines (or larger angular scales) than measured in this observation is not represented well. Uniform weighted data (Figure 6b) shows the highest resolution (26"x25") and Briggs (Figure 6c) 'robust=0' (default value is 0.5) is a compromise with a beam of 29"x29". To image more of the extended emission, the 'robust' parameter could be tweaked further toward more positive values.<br />
<br />
== Multi-Scale CLEAN ==<br />
[[Image:SN_G55.MultiScale.image.png|300px|thumb|right|'''Figure 7''' <br /> G55.7+3.4 Multi-Scale Clean]]<br />
[[Image:SN_G55_MultiScale.artifacts.png|300px|thumb|right|'''Figure 8''' <br /> Artifacts around point sources]]<br />
<br />
Since G55.7+3.4 is an extended source with many angular scales, a more advanced form of imaging involves the use of multiple scales. MS-CLEAN is an extension of the classical CLEAN algorithm for handling extended sources. Multi-Scale CLEAN works by assuming the sky is composed of emission at different angular scales and works on them simultaneously, thereby creating a linear combination of images at different angular scales. For a more detailed description of Multi Scale CLEAN, see T.J. Cornwell's paper [http://arxiv.org/abs/0806.2228 Multi-Scale CLEAN deconvolution of radio synthesis images].<br />
<br />
We will use a set of scales (expressed in units of the requested pixel or cell size) which are representative of the scales that are present in the data, including a zero-scale for point sources. <br />
<br />
<source lang="python"><br />
# In CASA<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.MultiScale', deconvolver='multiscale', scales=[0,6,10,30,60],<br />
smallscalebias=0.9, imsize=1280, cell='8arcsec', pblimit=-0.01, niter=1000,weighting='briggs', stokes='I',<br />
robust=0.0, interactive=False, threshold='0.12mJy', savemodel='modelcolumn')<br />
</source><br />
<br />
* ''multiscale=[0,6,10,30,60]'': a set of scales on which to clean. A general guideline when using multiscale is 0, 2xbeam, 5xbeam (where beam is the synthesized beam), and larger scales up to about half the minor axis maximum scale of the mapped structure. Since these are in units of the pixel size, our chosen values will be multiplied by the requested cell size. Thus, we are requesting scales of 0 (a point source), 48, 80, 240, and 480 arcseconds (8 arcminutes). Note that 16 arcminutes (960 arcseconds) roughly corresponds to the size of G55.7+3.4.<br />
<br />
* ''smallscalebias=0.9'': This parameter is known as the '''small scale bias''', and helps with faint extended structure by balancing the weight given to smaller structures, which tend to have higher surface brightness but lower integrated flux density. Increasing this value gives more weight to smaller scales. A value of 1.0 weights the largest scale to zero, and a value of less than 0.2 weights all scales nearly equally. The default value is 0.6.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.MultiScale.image')<br />
</source><br />
<br />
The logger will show how much cleaning is performed on the individual scales.<br />
<br />
This is the fastest of the advanced imaging techniques described here, but it's easy to see that there are artifacts in the resulting image (Figure 7). We can use the {{viewer}} to explore the point sources near the edge of the field by zooming in on them (Figure 8). Click on the "Zooming" button on the upper left corner and highlight an area by making a square around the portion where you would like to zoom-in. Double-click within the square to zoom in on the selected area. The square can be resized by clicking/dragging the corners, or removed by pressing the "Esc" key. After zooming in on the area, we can see some radio sources that have prominent arcs, as well as spots with a six-pointed pattern surrounding them.<br />
<br />
Next we will further explore advanced imaging techniques to mitigate the artifacts seen toward the edges of the image.<br />
<br />
== Multi-Scale, Wide-Field CLEAN (w-projection) ==<br />
<br />
[[Image:Faceting.png|325px|thumb|right|'''Figure 9''' <br /> Faceting when using wide-field gridder, which can be used in conjunction with w-projection.]]<br />
<br />
The next {{tclean}} algorithm we will employ is w-projection, which is a wide-field imaging technique that takes into account the non-coplanarity of the baselines as a function of distance from the phase center (Figure 9). For wide-field imaging, the sky curvature and non-coplanar baselines result in a non-zero w-term. The w-term, introduced by the sky and array non-coplanarity, results in a phase term that will limit the dynamic range of the final image. Applying 2-D imaging to such data will result in artifacts around sources away from the phase center, as we saw in running MS-CLEAN. Note that this affects mostly the lower frequency bands where the field of view is large.<br />
<br />
The w-term can be corrected by faceting (which projects the sky curvature onto many smaller planes) in either the image or uv-plane. The latter is known as w-projection. A combination of the two can also be employed within {{tclean}} by setting the parameter ''gridder='wproject''' and''deconvolver='multiscale''' . If w-projection is employed, it will be done for each facet. Note that w-projection is an order of magnitude faster than the image-plane based faceting algorithm, but requires more memory. <br />
<br />
For more details on w-projection, as well as the algorithm itself, see [http://adsabs.harvard.edu/abs/2008ISTSP...2..647C "The Noncoplanar Baselines Effect in Radio Interferometry: The W-Projection Algorithm"]. Also, the chapter on [http://www.aspbooks.org/a/volumes/article_details/?paper_id=17953 Imaging with Non-Coplanar Arrays] may be helpful. <br />
<br />
<source lang="python"><br />
# In CASA<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.wProj',<br />
gridder='wproject', wprojplanes=-1, deconvolver='multiscale', scales=[0,6,10,30,60], <br />
smallscalebias=0.9, imsize=1280, cell='8arcsec', pblimit=-0.01, niter=1000, weighting='briggs',<br />
stokes='I', robust=0.0, interactive=False, threshold='0.15mJy', savemodel='modelcolumn')<br />
</source><br />
<br />
* ''gridder='wproject''': Use the w-projection algorithm.<br />
<br />
* ''wprojplanes=-1'': The number of w-projection planes to use for deconvolution. Setting to -1 allows CLEAN to automatically choose an acceptable number of planes for the given data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.wProj.image')<br />
</source><br />
<br />
This will take slightly longer than the previous imaging round; however, the resulting image which employs Multi-Scale and w-projection (Figure 10b) has noticeably fewer artifacts than the image which just employed Multi-Scale (Figure 10a). In particular, compare the same outlier source in the Multi-Scale w-projected image with the Multi-Scale-only image: note that the swept-back arcs have disappeared. There are still some obvious imaging artifacts remaining, though.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[Image:SNR_G55_10s.multiscale.png|450px|thumb|'''Figure 10a''' <br /> Multi-Scale image which shows arcs around point sources far from the phase center.]]<br />
| [[Image:SNR_G55_10s.ms.wProj.png|450px|thumb|'''Figure 10b''' <br /> Multi-Scale and w-projection image shows improvements in the image by removing prominent artifacts.]]<br />
|}<br />
<br />
== Multi-Scale, Multi-Term Multi-Frequency Synthesis ==<br />
[[Image:MultiFrequency_Synthesis_snapshot.png|350px|thumb|right|'''Figure 11''' <br /> ''Left:'' (u,v) coverage snapshot at a single frequency. ''Right:'' Multi-Frequency Synthesis snapshot of (u,v) coverage. Using this algorithm can greatly improve (u,v) coverage, thereby improving image fidelity.]]<br />
[[Image:SNR_G55_10s.ms.MFS.colwedge.image.png|300px|thumb|right|'''Figure 12''' <br /> Multi-Scale imaged with MT-MFS and nterms=2. Artifacts around point sources close to phase center are reduced while point sources away from the phase center still show artifacts.]]<br />
[[Image:SN_G55_MS.MFS.alpha.png|300px|thumb|right|'''Figure 13''' <br /> Spectral Index image]]<br />
<br />
A consequence of simultaneously imaging all the frequencies across the wide fractional bandwidth of the VLA is that the primary and synthesized beams have substantial frequency-dependent variation over the observed band. If this variation is not accounted for, it will lead to imaging artifacts and will compromise the achievable image rms.<br />
<br />
The coordinates of the (u,v) plane are measured in wavelengths. Observing at several frequencies results means that a single baseline will sample several ellipses in the (u,v) plane, each with different sizes. We can therefore fill in the gaps in the single frequency (u,v) coverage (Figure 11) to achieve a much better image fidelity. This method is called Multi-Frequency Synthesis (MFS), which is the default cleaning mode in CLEAN. For a more detailed explanation of the MS-MFS deconvolution algorithm, please see the paper [http://arxiv.org/abs/1106.2745 A multi-scale multi-frequency deconvolution algorithm for synthesis imaging in radio interferometry].<br />
<br />
<!--When observing in low-frequencies, it may prove beneficial to observe in small time-chunks which are spread out in time. This will allow the coverage of more spatial-frequencies, allowing us to employ this algorithm more efficiently.<br />
<br />
--><br />
The Multi-Scale, Multi-Term Multi-Frequency Synthesis (MS-MT-MFS) algorithm provides the ability to simultaneously image and fit for the intrinsic source spectrum in each pixel. The spectrum is approximated using a polynomial Taylor-term expansion in frequency, with the degree of the polynomial as a user-controlled parameter. A least-squares approach is used, along with the standard clean-type iterations. <br />
<br />
<source lang="python"><br />
# In CASA<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MTMFS',<br />
gridder='standard', imsize=1280, cell='8arcsec', specmode='mfs',<br />
deconvolver='mtmfs', nterms=2, scales=[0,6,10,30,60], smallscalebias=0.9, <br />
interactive=False, niter=1000, weighting='briggs', pblimit=-0.01,<br />
stokes='I', threshold='0.15mJy', savemodel='modelcolumn')<br />
</source><br />
<br />
* ''nterms=2'': the number of Taylor-terms to be used to model the frequency dependence of the sky emission. Note that the speed of the algorithm will depend on the value used here (more terms will be slower). nterms=2 will fit a spectral index, while nterms=3 will fit a spectral index and spectral curvature.<br />
<br />
For Multi-Term MFS, {{tclean}} produces images with extension tt*, where the number in place of the asterisk indicates the Taylor term: <imagename>.image.tt0 is the total intensity image (Figure 12) and <imagename>.image.alpha will contain an image of the spectral index in regions where there is sufficient signal-to-noise (Figure 13). For Figure 12, we have included a color wedge (use the data display option icon to have it displayed) to give an idea of the flux within the image. The spectral index image can help convey information about emission mechanisms as well as optical depth of the source.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MTMFS.image.tt0')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MTMFS.alpha')<br />
</source><br />
<br />
Note: To replicate Figure 13, open the image within the viewer, click on "Panel Display Options" (wrench with a small P), change "background color" to white, and adjust your margins under the "Margins" button. To view the color wedge, click on the "Data Display Options" (wrench without the P, next to the folder icon), click on the "Color Wedge" button, click on "Yes" under "display color wedge", and adjust the various parameters to your liking.<br />
<br />
For more information on the multi-term, multi-frequency synthesis mode and its outputs, see the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-imaging/deconvolution-algorithms#cit5 Deconvolution Algorithyms section] in the CASA documentation. <br />
<br />
Inspect the brighter point sources in the field near the supernova remnant. You will notice that some of the artifacts, which had been symmetric around the sources, are gone; however, as we did not use w-projection, there are still strong features related to the non-coplanar baseline effects still apparent for sources further away.<br />
<br />
At this point, {{tclean}} takes into account the frequency variation of the synthesized beam but not the frequency variation of the primary beam. For low frequencies and large bandwidths, this can be substantial; e.g., 1 - 2 GHz L-band observations result in a variation of a factor of 2. One effect of imaging with MFS across such a large fractional bandwidth is that primary beam nulls will be blurred; the interferometer is sensitive everywhere in the field of view. However, if there is no correction made to account for the variation in primary beam with frequency, sources away from the phase center appear to have a steeply false spectral slope. A correction for this effect should be made with the task {{widebandpbcor}} or set ''pbcor=True'' (see section on Primary Beam Correction below).<br />
<br />
== Multi-Scale, Multi-Term Multi-Frequency Synthesis with W-Projection ==<br />
<br />
We will now combine the w-projection and MS-MT-MFS algorithms. Be forewarned -- these imaging runs will take a while, and it's best to start them running and then move on to other things.<br />
<br />
Using the same parameters for the individual-algorithm images above, but combined into a single {{tclean}} run, we have:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MTMFS.wProj',<br />
gridder='wproject',wprojplanes=-1, pblimit=-1, imsize=1280, cell='8arcsec', specmode='mfs',<br />
deconvolver='mtmfs', nterms=2, scales=[0,6,10,30,60], smallscalebias=0.9,<br />
interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.15mJy', savemodel='modelcolumn')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MTMFS.wProj.image.tt0')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MTMFS.wProj.alpha')<br />
</source><br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:SNR_G55_10s.multiscale.group.png|283px|thumb|'''Figure 14a''' <br /> Multi-Scale.]]<br />
| [[File:SNR_G55_10s.ms.MTFS.group.png|283px|thumb|'''Figure 14b''' <br /> Multi-Scale, MTMFS.]]<br />
| [[File:SNR_G55_10s.ms.wProj.group.png|283px|thumb|'''Figure 14c''' <br /> Multi-Scale, w-projection.]]<br />
| [[File:SNR_G55_10s.ms.MTFS.wProj.group.png|283px|thumb|'''Figure 14d''' <br /> Multi-Scale, MTMFS, w-projection.]]<br />
|}<br />
<br />
Again, looking at the same outlier source, we can see that the major sources of error have been removed, although there are still some residual artifacts. One possible source of error is the time-dependent variation of the primary beam; another is the fact that we have only used ''nterms=2'', which may not be sufficient to model the spectra of some of the point sources. Some weak RFI may also show up that may need additional flagging.<br />
<br />
== Imaging Outlier Fields ==<br />
<br />
When strong sources are far from the main target but still strong enough to produce sidelobes in the main image, they should be cleaned. Sometimes, however, it is not practical to image very large fields for this purpose. An alternative is to add outlier fields. This mode will allow the user to specify a few locations that are then cleaned along with the main image. The outlier fields should be centered on strong sources that, e.g., are known from sky catalogs or are identified by other means. <br />
<br />
In order to find outlying sources, it will help to first image a very large field with lower resolution, and to identify bright outliers using the CASA viewer. By moving the mouse cursor over the sources, we can grab their positions and use the information within an outlier file. In this example, we've called our file 'outliers.txt'. It contains the names, sizes, and positions of the outliers we want to image (see section [https://casa.nrao.edu/docs/cookbook/casa_cookbook006.html#sec356 5.3.18.1] of the CASA cookbook). <br />
<br />
Open your favorite text editor and input the following: <br />
<br />
<pre><br />
#content of outliers.txt<br />
#<br />
#outlier field1<br />
imagename=Outlier1<br />
imsize=[320,320]<br />
phasecenter = J2000 19:23:27.693 22.37.37.180<br />
#<br />
#outlier field2<br />
imagename=Outlier2<br />
imsize=[320,320]<br />
phasecenter = J2000 19:25:46.888 21.22.03.365<br />
<br />
</pre><br />
<br />
{{tclean}} will then be executed as follows:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tclean(vis='SNR_G55_10s.calib.ms', imagename='SNR.MS.MFS-Main', outlierfile='outliers.txt',<br />
imsize=[640,640], cell='8arcsec',deconvolver='multiscale', scales=[0,6,10,30,60], smallscalebias=0.9,<br />
interactive=False, niter=1000, weighting='briggs', robust=0, stokes='I', threshold='0.15mJy', <br />
savemodel='modelcolumn',pblimit=-1)<br />
</source><br />
<br />
* ''outlierfile='outliers.txt''': the name of the outlier file.<br />
<br />
We can view the images with {{viewer}} one at a time. Due to the difference in sky coordinates, they cannot be viewed on the same window display.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR.MS.MFS-Main.image')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Outlier1.image')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Outlier2.image')<br />
</source><br />
<br />
In the resulting images, we have changed the color map to "Hot Metal 2" in order to show the different colors that can be used within the viewer. Feel free to play with other color maps that may be better suited for your screen.<br />
<br />
{|style="margin: 0 auto;"<br />
| [[File:Imaging-outlier-main.png|400px|thumb'''Figure 15a''' <br /> Main field of the {{tclean}}.]]<br />
| [[File:Imaging-outlier-out1.png|400px|thumb'''Figure 15b''' <br /> First outlier field.]]<br />
| [[File:Imaging-outlier-out2.png|400px|thumb'''Figure 15c''' <br /> Second outlier field.]]<br />
|}<br />
<br />
== Primary Beam Correction ==<br />
<br />
[[Image:SNR_G55_10s.MS.MFS.wProj.pbcor.colwedge.image.png|300px|thumb|right|'''Figure 16''' <br /> Primary beam corrected image using the widebandpbcor task for the MS.MTMFS.wProj image created during the CLEAN process.]]<br />
[[Image:SNR_G55_10s.MS.MFS.wProj.pbcor.image.alpha-CASA4.6.png|300px|thumb|right|'''Figure 17''' <br /> Primary beam corrected spectral index image using the widebandpbcor task.]]<br />
<br />
In interferometry, the images formed via deconvolution are representations of the sky, multiplied by the primary beam response of the antenna. The primary beam can be described as Gaussian with a size depending on the observing frequency. Images produced via {{tclean}} are by default not corrected for the primary beam pattern (important for mosaics), and therefore do not have the correct flux density away from the phase center. <br />
<br />
Correcting for the primary beam, however, can be done during {{tclean}} by using the ''pbcor'' parameter. Alternatively, it can be done after imaging using the task [https://casa.nrao.edu/docs/TaskRef/impbcor-task.html impbcor] for regular data sets, and {{widebandpbcorr}} for those that use Taylor-term expansion (<i>nterms > 1</i>). A third alternative is utilizing the task {{immath}} to manually divide the ''<imagename>.image'' by the ''<imagename>.flux'' image (''<imagename>.flux.pbcoverage'' for mosaics). <br />
<br />
Flux corrected images usually don't look pretty due to the noise at the edges being increased. Flux densities, however, should only be calculated from primary beam corrected images. Let's run the {{impbcor}} task to correct our multiscale image.<br />
<br />
<source lang="python"><br />
# In CASA<br />
impbcor(imagename='SNR_G55_10s.MultiScale.image', pbimage='SNR_G55_10s.MultiScale.pb', <br />
outfile='SNR_G55_10s.MS.pbcorr.image')<br />
</source><br />
<br />
* ''imagename'': the image to be corrected<br />
<br />
* ''pbimage'': the <imagename>.pb image as a representation of the primary beam<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.MS.pbcorr.image')<br />
</source><br />
<br />
Let us now use the {{widebandpbcor}} task for wide-band (''nterms''>1) images. Note that for this task, we will be supplying the image name that is the prefix for the Taylor expansion images, tt0 and tt1, which must be on disk. Such files were created during the last Multi-Scale, Multi-Term, Widefield run of CLEAN. {{widebandpbcor}} will generate a set of images with a "pbcor.image.*" extension.<br />
<br />
<source lang="python"><br />
# In CASA<br />
widebandpbcor(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MTMFS.wProj', <br />
nterms=2, action='pbcor', pbmin=0.2, spwlist=[0,1,2,3],<br />
weightlist=[0.5,1.0,0,1.0], chanlist=[25,25,25,25], threshold='0.6mJy')<br />
</source><br />
<br />
* ''spwlist=[0,1,2,3]'': We want to apply this correction to all spectral windows in our calibrated measurement set. <br />
<br />
* ''weightlist=[0.5,1.0,0,1.0]'': Since we did not specify reference frequencies during {{tclean}}, the {{widebandpbcor}} task will pick them from the provided image. Running the task, the logger reports the multiple frequencies used for the primary beam, which are 1.305, 1.479, 1.652, and 1.825 GHz. We selected weights using information external to this guide: The first chosen frequency lies within spectral window 0, which had lots of data flagged due to RFI. The ''weightlist'' parameter allows us to give this chosen frequency less weight. The primary beam at 1.6 GHz lies in an area with no data, therefore we give a weight value of zero for this frequency. The remaining frequencies 1.479 and 1.825 GHz lie within spectral windows which contained less RFI, therefore we provide a larger weight percentage. (Exercise for the reader: use an amplitude vs. frequency plot in {{plotms}} to verify these claims.)<br />
<br />
* ''pbmin=0.2'': Gain level below which not to compute Taylor-coefficients or apply a primary beam correction.<br />
<br />
* ''chanlist=[25,25,25,25]'': Make primary beams at frequencies corresponding to channel 25 in each spectral window.<br />
<br />
* ''threshold='0.6mJy''': Threshold in the intensity map, below which not to recalculate the spectral index. We adjusted the threshold to get good results, as too high of a threshold resulted in errors and no spectral index image, and too low of a threshold resulted in bogus values for the spectral index image.<br />
<br />
'''Note''': CASA may produce a warning stating that ''Images are not contiguous along the concatenation axis.'' This is usually an indication of missing channels due to flagging (or unevenly spaced basebands) and is not a cause for worry.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MTMFS.wProj.pbcor.image.tt0')<br />
</source><br /><br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MTMFS.wProj.pbcor.image.alpha')<br />
</source><br />
<br />
It's important to note that the image will cut off at about 20% of the HPBW, as we are confident of the accuracy within this percentage. Anything outside becomes less accurate, thus there is a mask associated with the creation of the corrected primary beam image (Figure 16). Spectral indices may still be unreliable below approximately 70% of the primary beam HPBW (Figure 17). <br />
<br />
It would be a good exercise to use the {{viewer}} to plot both the primary beam corrected image, and the original cleaned image and compare the intensity (Jy/beam) values, which should differ slightly.<br />
<br />
== Imaging Spectral Cubes ==<br />
<br />
For spectral line imaging using CASA {{tclean}}, set ''specmode='cube', and the ''width'' parameter accepts either frequency or velocity values. Both options will create a spectral axis in frequency, but entering a velocity width will add an additional velocity label. <br />
<br />
The following keywords are important for spectral modes (velocity in this example):<br />
<br />
<source lang="python"><br />
# In CASA<br />
specmode = 'cube' # Spectral definition type (mfs, cube, cubedata, cubesource)<br />
nchan = -1 # Number of channels (planes) in output image; -1 = all data specified by spw, start, and width parameters. <br />
start = '' # Velocity of first channel: e.g '0.0km/s'(''=first channel in first SpW of MS)<br />
width = '' # Channel width e.g '-1.0km/s' (''=width of first channel in first SpW of MS)<br />
outframe = '' # spectral reference frame of output image; '' =input<br />
veltype = 'radio' # Velocity definition of output image<br />
restfreq = '' # Rest frequency to assign to image (see help)<br />
</source><br />
<br />
The spectral dimension of the output cube will be defined by these parameters and {{tclean}} will regrid the visibilities to it. Note that invoking {{cvel}} before imaging is in most cases not necessary, even when two or more measurement sets are being provided at the same time. <br />
<br />
The cube is specified by a ''start'' velocity in km/s or MHz, the ''nchan'' number of channels and a channel ''width'' (where the latter can also be negative for decreasing velocity cubes). <br />
<br />
To correct for Doppler effects, {{tclean}} also requires a ''outframe'' velocity frame, where the default ''LSRK'' and ''BARY'' are the most popular Local Standard of Rest (kinematic) and sun-earth Barycenter references. ''veltype'' defines whether the data will be gridded via the optical or radio velocity definition. A description of the available options and definitions can be found in the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/line VLA observing guide] and the [https://casa.nrao.edu/docs/cookbook/casa_cookbook014.html CASA cookbook]. By default, {{tclean}} will produce a cube using the LRSK and radio definitions. <br />
<br />
An example of spectral line imaging procedures is provided in the [https://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 VLA high frequency Spectral Line tutorial - IRC+10216].<br />
<br />
== Beam per Plane ==<br />
For spectral cubes spanning relatively wide ranges in frequency, the synthesized beam can vary substantially across the channels. To account for this, CASA will calculate separate beams for each channel when the difference in beams is more than half a pixel across the cube. All CASA image analysis tasks are capable of handling such cubes. <br />
<br />
If it is desired to have a cube with a single synthesized beam, two options are available. It is possible for {{tclean}} to do the smoothing, by setting the parameter ''smonothfactor''. The second option is to use the task {{imsmooth}}, after cleaning, with ''kernel='commonbeam'.'' This task will clean up all header variables such that only a single beam appears in the data. It may be wise to go with this second option, which will allow you to first verify the clean results before smoothing the image.<br />
<br />
== Image Header ==<br />
<br />
The image header holds meta data associated with your CASA image. The task {{imhead}} will display this data within the casalog, and also output the result as a Python dictionary in the CASA terminal. We will first run {{imhead}} with ''mode='summary' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MTMFS.wProj.image.tt0', mode='summary')<br />
</source><br />
<br />
* ''mode='summary' '': gives general information about the image, including the object name, sky coordinates, image units, the telescope the data was taken with, and more. <br />
<br />
For further information, run ''mode='list' ''. We will assign a variable 'header' to also capture the output dictionary:<br />
<br />
<source lang="python"><br />
# In CASA<br />
header=imhead(imagename='SNR_G55_10s.ms.MTMFS.wProj.image.tt0', mode='list')<br />
</source><br />
<br />
* ''mode='list' '': gives more detailed information, including beam major/minor axes, beam primary angle, the location of the max/min intensity, and lots more. Essentially this mode displays the FITS header variables.<br />
<br />
== Image Conversion ==<br />
<br />
Finally, as an example, we convert our image from intensity to main beam brightness temperature.<br />
<br />
We will use the standard equation <br />
<br />
<math>T=1.222\times 10^{6} \frac{S}{\nu^{2} \theta_{maj} \theta_{min}} </math><br />
<br />
where the main beam brightness temperature <math>T</math> is given in units of Kelvin, the intensity <math>S</math> in Jy/beam, the reference frequency <math>\nu</math> in GHz, and the major and minor beam sizes <math>\theta</math> in arcseconds. <br />
<br />
For a beam of 29.30"x29.03" and a reference frequency of 1.579 GHz (as taken from the previous {{imhead}} run) we calculate the brightness temperature using {{immath}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
immath(imagename='SNR_G55_10s.ms.MTMFS.wProj.image.tt0', mode='evalexpr', <br />
expr='1.222e6*IM0/1.579^2/(29.30*29.03)', <br />
outfile='SNR_G55_10s.ms.MTMFS.wProj.image.tt0-Tb')<br />
</source><br />
* ''mode='evalexpr' '': {{immath}} is used to calcuate with images<br />
<br />
* ''expr'': the mathematical expression to be evaluated. The images are abbreviated as IM0, IM1, ... in the sequence of their appearance in the ''imagename'' parameter.<br />
<br />
Since {{immath}} only changes the values but not the unit of the image, we will now change the new image header 'bunit' key to 'K'. <br />
To do so, we will run {{imhead}} with ''mode='put' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MTMFS.wProj.image.tt0-Tb', mode='put', hdkey='bunit', hdvalue='K')<br />
</source><br />
* ''hdkey'': the header keyword that will be changed<br />
<br />
* ''hdvalue'': the new value for header keyword<br />
<br />
Launching the viewer will now show our image in brightness temperature units: <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MTMFS.wProj.image.tt0-Tb')<br />
</source><br />
<br />
<br />
<br />
A few particular issues to look out for:<br />
* The default value of ''robust'' is now 0.5; this produces the same behavior as ''robust=0.0'' in AIPS.<br />
* Parameter ''pblimit'' is used to ensure that the image is only produced out to a distance from the phasecenter where the sensitivity of the Primary Beam is equal to ''pblimit''. The default value of ''pblimit'' is 0.2. For this example, where we are making an image that is much, much larger than the Primary Beam, it is necessary to set ''pblimit=-1'' so that no limit is applied.<br />
* Parameter ''savemodel'' is used rather than {{clean}}'s ''usescratch'' to indicate whether the model should be saved in a new column in the ms, as a virtual model, or not at all. The default value is "none": so, this must be changed for the image to be saved. The use of the virtual model with MTMFS in {{tclean}} is still under commissioning, so we recommend currently to set ''savemodel='modelcolumn'.''<br />
* Some parameter default values in {{tclean}} are different than the equivalent default values in {{clean}}. <br />
<br />
{{Checked 5.4.0}}<br />
<br />
[[Main Page | &#8629; '''CASAguides''']]<br />
</div></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=MG0414%2B0534_P-band_Spectral_Line_Tutorial_-_CASA_5.4.0&diff=27162MG0414+0534 P-band Spectral Line Tutorial - CASA 5.4.02019-04-25T14:46:44Z<p>Emomjian: </p>
<hr />
<div><!--By Bjorn Emonts--><br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b> <br />
<br />
{| class="wikitable"<br />
|-<br />
| style="background: red; color: white;" | '''DISCLAIMER'''<br />
|-<br />
| The following guide is for users who are experts in data reduction using CASA. If you are a beginning or novice user, please review other CASA guides found at [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials VLA Tutorials].<br />
|}<br />
<br />
== Overview ==<br />
<br />
This tutorial describes how to use CASA 5.4.0 to reduce spectral-line data in the low-frequency P-band of the VLA (230&ndash;470 MHz). The goal is to make an image cube containing HI 21cm absorption against the strong radio continuum of gravitationally lensed radio galaxy MG0414+0534. As a results of the high redshift of z=2.6365, the HI absorption signal in MG0414+0534 is redshifted to an observed frequency of 390.597 MHz. The HI absorption in MG0414+0534 was previously imaged with the VLA by Moore, Carilli & Menten 1999 ([http://iopscience.iop.org/article/10.1086/311818/meta ApJ, 510, 87]), before the upgrade to the WIDAR system.<br />
<br />
=== Observing strategy ===<br />
<br />
To perform P-band spectrsocopy, there are three important considerations for planning the observations:<br />
<br />
* Use a bandpass calibrator that is strong enough to accurately calibrate the frequency-dependent gain variations. This is particularly important for most HI 21cm absorption projects, which are typically performed against strong radio continuum sources. As a rule-of-thumb, use ''t''<sub>cal</sub> > ''t''<sub>obj</sub> × (''S''<sub>obj</sub> / ''S''<sub>cal</sub>)<sup>2</sup>, with ''t'' the exposure time and ''S'' the source flux density. This rule-of-thumb serves to avoid introducing excessive additional noise in the spectrum.<br />
<br />
* Additionally, if very high spectral dynamic range is needed (i.e., when expecting a ratio between the detection limit and the radio continuum of about 1:10,000 or more), consider observing a bandpass calibrator several times during your run to be able to correct for time-varying bandpass changes, which scale with the continuum emission in the target field. See the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/calibration Calibration section] of the Guide to Observing with the VLA for more information.<br />
<br />
* Use a bandwidth that is wide enough to perform accurate self calibration. Using a wide bandwidth for self-calibration is important for fields with relatively weak continuum sources. For strong continuum sources, a narrower bandwidth can be used to avoid excessive RFI. <br />
<br />
The P-band test data on MG0414+0534 that we use in this tutorial were obtained using a large bandwidth. This is to ensure that good bandpass solutions can be procured, and that self-calibration can accurately be performed. However, due to the strong radio continuum of MG0414+0534 (3.3 Jy at 390 MHz), and the large amounts of RFI across the entire band, we only use a small fraction of the total band for data reduction and analysis in this tutorial.<br />
<br />
== Obtaining the data ==<br />
<br />
<font color=red>'''This 15 GB data set can be downloaded by clicking [https://casa.nrao.edu/Data/EVLA/MG0414/MG0414_d1_data.ms.tgz here].'''</font> The filename should be MG0414_d1_data.ms.tgz (add the ".tgz" if this is missing). We are providing this starting data set, rather than the original data set for two reasons. First, many of these initial processing steps can be rather time consuming (> 1 hr). Second, while necessary, many of these steps are not fundamental to the calibration and imaging process, which is the focus of this tutorial.<br />
<br />
We will use test data that was taken in a hybrid configuration when the VLA array was moved from B-config to A-config. The data set can be downloaded from the [https://archive.nrao.edu/archive/advquery.jsp NRAO archive] by searching for the following Project Code: <br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
TSUB0001<br />
</pre><br />
</div><br />
<br />
This returns a long list with test-data that are publicly available. Our observations were performed on 14 Sept 2016:<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
TSUB0001.sb32720781.eb32763188.57645.263958564814<br />
</pre><br />
</div><br />
<br />
Note that this observation was duplicated on 15 September 2016. <!-- This second day of observations can be found under filename:<br />
TSUB0001.sb32720781.eb32773507.57646.24443702547 -->For the purpose of this tutorial, we only reduce and image the first data set. The 15 GB data set from the download has already had online flagging and Hanning smoothing applied, and the smaller data set split out from the larger data set. If you wish, you can download the SDM from the NRAO archive in order to apply the online flags and run Hanning smoothing on the large data set before splitting out the smaller data set to work on.<br />
<br />
<!-- The second data set can be reduced in an identical way (pending the flagging, given that the RFI conditions most likely changed). During the imaging stage you can then combine both reduced data sets. --><br />
<br />
<!-- To download the data, fill in your email, select either the "SDM-BDF (all files)" or “MS” option, and check the box next to the data sets that you want. The practical difference between selecting the MeasurementSet (MS) and Science Data Model (SDM) is that for MS the first step in the data reduction described below, namely reading in the SDM data to save it as a MeasurementSet, is already done by the archival engine. Note that you can also opt to download the data as a tar-file by tickking the appropriate box.<br />
<br />
Optionally, you can request to discard data marked as ‘bad’ by ticking the box “Apply Telescope Flags”, which gets rid of data taken during times of instrument calibration, shadowing, slewing, etc. However, it also applies all flags on the science data that were automatically created during the observations. Therefore, a safer option is to not apply any flags before downloading the data, but first inspect and subsequently apply the flags generated during the observations using <i>flagcmd</i>, as part of the data reduction plan.<br />
<br />
Click "Get My Data" will forward you to the next page, where you should choose the delivery method (either downloading over the internet, or sending home a hard-drive with the data). If you opt to retrieve the data over the internet, wait until you get an email confirming that the data is ready for download. --><br />
<br />
== Examining the Data ==<br />
<br />
=== Loading data into CASA ===<br />
<br />
We will have to untar the 15 GB data set that you've downloaded.<br />
<br />
<pre style="background-color:lightgrey;"><br />
# in a terminal, outside of CASA:<br />
tar -xzf MG0414_d1_data.ms.tgz<br />
</pre><br />
<br />
Please use CASA 5.4.0 for this tutorial (typing ''casa -ls'' in a linux window shows the available versions and the current version; to explicitly change the version type, e.g., ''casa -r 5.4.0-68'')<br />
<br />
<source lang='bash'><br />
casa<br />
</source><br />
on the command line. This should start a CASA interactive python (iPython) session, and open a separate log window. To guarantee that the below mentioned procedure for data reduction and imaging works, make sure you are using CASA version 5.4.0. While older versions may work as well for the purpose of this tutorial, it is good to read instructions on how to [https://casa.nrao.edu/casa_obtaining.shtml download and install] the latest version of CASA.<br />
<!-- <br />
We will begin by importing our data from the binary format (SDM-BDF) into the MeasurementSet format, which is the standard for CASA data. For this, we use <i>importevla</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
importevla(asdm='TSUB0001.sb32720781.eb32763188.57645.263958564814', vis='MG0414_d1.ms', flagpol=False, applyflags=False, savecmds=True, outfile='flagfile.txt')<br />
</source><br />
<br />
<!-- In this case, we do not apply the flags that were created as part of the observations, but we will write them out to a <i>flagfile.txt</i> file. We will inspect and apply the flags as follows:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagcmd(vis='MG0414_d1.ms', action='plot')<br />
flagcmd(vis='MG0414_d1.ms', action='apply')<br />
</source><br />
<br />
As an alternative to <i>flagcmd</i>, given that we saved the flags to <i>flagfile.txt</i>, we can also use <br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1.ms', mode='list', inpfile='flagfile.txt', action='apply')<br />
</source><br />
<br />
to apply the flags.<br />
<br />
One of the flag commands is to <i>clip</i> all data points that have a 0-value, as these most likely have not recorded any science data. This is normally done with the settings <i>correlation=`ABS_RR’</i> and <i>correlation=`ABS_LL’</i>. Because the VLA P-band and 4-band systems use linear XX and YY polarisation, rather than the circular RR and LL polarisation of the other VLA bands, you will encounter an error message when running <i>flagcmd</i>. We can get around this issue by manually clipping the 0-data for the XX and LL polarization with <i>flagdata</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1.ms', mode='clip', correlation='ABS_XX,ABS_YY', clipzeros=True, action='apply')<br />
</source><br />
--><br />
<br />
It is a good practice to read the Operator Log that was created by the duty operator during the observations. This log can provide additional information on data that should be flagged manually during the data reduction stage. The Operator logs can be found at [http://www.vla.nrao.edu/cgi-bin/oplogs.cgi Ops Logs form entry], and this is the [http://www.vla.nrao.edu/operators/logs/2016/9/2016-09-14_0620_TSUB0001.pdf link for the log of the observation] (in PDF format).<br />
<br />
=== Inspecting the observation info ===<br />
<br />
We will inspect the content of our data set using CASA task {{listobs}}.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='MG0414_d1_data.ms', listfile='listobs.txt')<br />
</source><br />
<br />
This normally plots an overview of the data in the CASA logger, but with the parameter <i>listfile=`listobs.txt’</i>, this information is written out to the file <i>listobs.txt</i> and is included below:<br />
<br />
<small style="font-size:62%;"><pre><br />
Observer: . Frazer Owen Project: uid://evla/pdb/1695465 <br />
Observation: EVLA<br />
Data records: 991814 Total elapsed time = 9390 seconds<br />
Observed from 14-Sep-2016/06:43:48.0 to 14-Sep-2016/09:20:18.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
14-Sep-2016/06:43:48.0 - 06:48:42.0 7 0 3C48 33861 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
06:56:21.0 - 07:08:39.0 8 1 MG0414+0534 82702 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:08:42.0 - 07:28:36.0 9 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:28:39.0 - 07:48:33.0 10 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:48:36.0 - 08:08:30.0 11 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:10:30.0 - 08:14:30.0 12 0 3C48 26549 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
08:16:30.0 - 08:34:27.0 13 1 MG0414+0534 124918 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:34:30.0 - 08:54:21.0 14 1 MG0414+0534 139347 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:54:24.0 - 09:14:18.0 15 1 MG0414+0534 139048 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
09:16:21.0 - 09:20:18.0 16 0 3C48 26295 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
(nRows = Total number of rows per scan) <br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C48 01:37:41.299431 +33.09.35.13299 J2000 0 86705<br />
1 NONE MG0414+0534 04:14:37.800000 +05.34.41.99999 J2000 1 905109<br />
Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 EVLA_P#A0C0#17 466 TOPO 386.312 15.625 7281.2 389.9453 12 XX XY YX YY<br />
Sources: 2<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 3C48 0 - - <br />
1 MG0414+0534 0 - - <br />
Antennas: 27:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 ea01 W24 25.0 m -107.38.49.0 +33.53.04.0 -2673.3457 -1784.6009 10.4844 -1604008.744400 -5042135.825100 3553403.710800<br />
1 ea02 W56 25.0 m -107.44.26.7 +33.49.54.6 -11333.1991 -7637.6832 15.3636 -1613255.391400 -5042613.097800 3548545.906000<br />
2 ea03 W40 25.0 m -107.41.13.5 +33.51.43.1 -6377.9680 -4286.7773 8.2312 -1607962.451800 -5042338.218100 3551324.962400<br />
3 ea04 E04 25.0 m -107.37.00.8 +33.53.59.7 102.7973 -63.7800 -2.6177 -1601068.806000 -5042051.932700 3554824.838800<br />
4 ea05 E36 25.0 m -107.33.20.2 +33.52.34.3 5761.3733 -2704.6731 33.0133 -1596127.730800 -5045193.742100 3552652.419700<br />
5 ea06 N24 25.0 m -107.37.16.1 +33.55.37.7 -290.3584 2961.8653 -12.2425 -1600930.069900 -5040316.395500 3557330.390000<br />
6 ea07 N16 25.0 m -107.37.10.9 +33.54.48.0 -155.8512 1426.6377 -9.3934 -1601061.954400 -5041175.875300 3556058.026700<br />
7 ea08 W64 25.0 m -107.46.20.1 +33.48.50.9 -14240.7524 -9606.2900 17.0885 -1616361.575500 -5042770.516600 3546911.419900<br />
8 ea09 N40 25.0 m -107.37.29.5 +33.57.44.4 -633.6056 6878.5897 -20.7984 -1600592.749000 -5038121.341300 3560574.826900<br />
9 ea10 E20 25.0 m -107.35.43.6 +33.53.29.9 2082.1759 -987.0087 9.4361 -1599340.800100 -5043150.963000 3554065.231500<br />
10 ea12 N32 25.0 m -107.37.22.0 +33.56.33.6 -441.7248 4689.9674 -16.9328 -1600781.044500 -5039347.439100 3558761.527100<br />
11 ea13 E28 25.0 m -107.34.39.3 +33.53.04.9 3732.7776 -1757.3308 21.4271 -1597899.895900 -5044068.684700 3553432.450200<br />
12 ea14 E08 25.0 m -107.36.48.9 +33.53.55.1 407.8283 -206.0315 -3.2236 -1600801.931400 -5042219.382600 3554706.429400<br />
13 ea15 N64 25.0 m -107.37.58.7 +34.02.20.5 -1382.3871 15410.1326 -40.6450 -1599855.687000 -5033332.368600 3567636.606800<br />
14 ea16 W72 25.0 m -107.48.24.0 +33.47.41.2 -17419.4641 -11760.2694 14.9442 -1619757.299900 -5042937.656400 3545120.392300<br />
15 ea17 N56 25.0 m -107.37.47.9 +34.00.38.4 -1105.2042 12254.2800 -34.2710 -1600128.378100 -5035104.139200 3565024.633900<br />
16 ea18 W48 25.0 m -107.42.44.3 +33.50.52.1 -8707.9181 -5861.7682 15.5302 -1610451.901900 -5042471.123800 3550021.073100<br />
17 ea19 W16 25.0 m -107.37.57.4 +33.53.33.0 -1348.7121 -890.6209 1.2959 -1602592.853500 -5042054.992400 3554140.702800<br />
18 ea20 E16 25.0 m -107.36.09.8 +33.53.40.0 1410.0403 -673.4656 -0.7790 -1599926.104100 -5042772.977200 3554319.801100<br />
19 ea21 E24 25.0 m -107.35.13.4 +33.53.18.1 2858.1906 -1349.1352 13.7186 -1598663.082000 -5043581.391200 3553767.014100<br />
20 ea22 W32 25.0 m -107.39.54.8 +33.52.27.2 -4359.4399 -2923.1314 11.7614 -1605808.634100 -5042230.084000 3552459.197800<br />
21 ea23 E12 25.0 m -107.36.31.7 +33.53.48.5 848.6994 -411.6226 -2.7575 -1600416.518000 -5042462.430500 3554536.041700<br />
22 ea24 W08 25.0 m -107.37.21.6 +33.53.53.0 -432.1080 -272.1514 -1.5061 -1601614.083200 -5042001.656900 3554652.505900<br />
23 ea25 N08 25.0 m -107.37.07.5 +33.54.15.8 -68.9088 433.1829 -5.0674 -1601147.942500 -5041733.833600 3555235.947000<br />
24 ea26 E32 25.0 m -107.34.01.5 +33.52.50.3 4701.6503 -2209.7119 25.1976 -1597053.124400 -5044604.675000 3553058.992700<br />
25 ea27 N48 25.0 m -107.37.38.1 +33.59.06.2 -855.2719 9405.9407 -25.9485 -1600374.881000 -5036704.201700 3562667.858900<br />
26 ea28 N72 25.0 m -107.38.10.5 +34.04.12.2 -1685.6797 18861.8306 -43.5015 -1599557.928700 -5031396.353400 3570494.736800<br />
</pre></small><br />
<br />
<!-- === Hanning smoothing the data ===<br />
<br />
When plotting the data in plotms (plotting amplitude against frequency), it is immediately clear that our observing band contains lots of Radio Frequency Interference (RFI), which is unavoidable at these low frequencies. The situation actually looks worse than it is, because ‘Gibbs ringing’ causes the blending of strong RFI signal into adjacent channels (Fig.1). To reduce the effect of Gibbs ringing, we first Hanning smooth the data (please note that this can take a while to run on the entire 142 Gb data set):<br />
<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='MG0414_d1_data.ms', datacolumn='data', outputvis='MG0414_d1_hanning.ms')<br />
</source><br />
<br />
[[File:Pband_hanning.png|400px|thumb|right|Fig.1 Reducing the effects of Gibbs ringing by Hanning smoothing the data. Shown is only a small fraction of the band, both before and after Hanning smooth. The amplitude scale is the same in both plots.]]<br />
<br />
=== Splitting the data ===<br />
<br />
Upon further inspection in “plotms” (using ea01, field 0, corr xx,yy, and coloraxis baseline), it is clear that there is a lot of RFI across most of the 64 MHz (or 43,000 km/s) band that we used for the observations. Luckily, the region around the expected HI absorption line at 390.6 MHz is relatively clean of RFI. Because our target is strong enough in the continuum to perform a good self-calibration, we select this relatively clean part of the band centered around the expected HI line. Note that there were two IFs with different spectral resolution included in the observations. We need only one of the two IFs, and choose the one with the highest spectral resolution. Moreover, we only need the sources 3C48 (which we will use for bandpass, gain and flux calibration) and our target MG0414+0534, hence we select field=’3,4’. This approach has the additional advantage that we reduce our data volume a lot, bringing it down to about 15 Gb.<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_hanning.ms', outputvis='MG0414_d1_data.ms', datacolumn='data', field='3,4', spw='17:20~485', keepflags=False)<br />
</source><br />
<br />
<font color=red>'''This 15 Gb data set can be downloaded by clicking [https://casa.nrao.edu/Data/EVLA/MG0414/MG0414_d1_data.ms.tgz here].'''</font> The filename should be MG0414_d1_data.ms.tgz (add the ".tgz" if this is missing), and the file can be unpacked by typing <i>tar xzf MG0414_d1_data.ms.tgz</i><br />
--><br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white;"<br />
| style="border: 1px solid darkgray;" | [[File:MG0414_plotants_5.4.0.png|200px|thumb]] || style="border: 1px solid darkgray" | [[File:Ant_logplot_5.4.0.png |200px|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 1a. Plotants plot for <br />MG0414. || Figure 1b. Plotants plot <br />for MG0414 with a log scale.<br />
|}<br />
<br />
Next we look at a graphical plot of the antenna locations and save a hard copy<br />
in case you want it later using CASA task {{plotants}} (see Figures 1a and 1b). This will be useful for selecting a reference antenna. Typically a good choice is an antenna close to the center of the array (see Figure 1b). Unless it shows problems after inspection of the data, we provisionally choose ea04. (Note: clicking on any image will open up a larger version.)<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis='MG0414_d1_data.ms',figfile='ant_locations.png')<br />
<br />
# In CASA<br />
# If you would like to see another view of the antennas you may plot them on a log plot<br />
plotants(vis='MG0414_d1_data.ms',logpos=True,figfile='ant_logplot.png')<br />
<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
<!-- NOTE: when revising for CASA release 5.3.0, EM wants to revisit how flagging is approached in the guide. Some of the flagging tasks, such as running rflag and flagging out antennas ea09, 14, and 17 should be introduced earlier - perhaps in this section - so that major flagging is done once and then the user can proceed to the self-calibration and imaging section. Tony --><br />
<br />
[[File:Pband_flagchan.png|400px|thumb|right|Figure 2. Flagging of calibrator 3C48. Shown are the XX and YY polarization on antenna ea01, both before (left) and after (right) flagging the bad channels. The colors represent the different baselines with ea01.]]<br />
<br />
We will now perform a more in-depth flagging of the bad data. Our strategy is to first focus on the calibrator source (in this case only 3C48). If the calibrator data are flagged well, we should be able to perform an accurate calibration of the target data. Later in this document we will then flag the calibrated target data using an automated flagger. <br />
<br />
When plotting frequency against amplitude in CASA task {{plotms}}, we see that there are still two ranges of channels filled with RFI throughout the observing run (see Figure 2). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',antenna='ea01',xaxis='chan',yaxis='amp',correlation='xx,yy',coloraxis='baseline', scan='7',averagedata=True, avgtime='1e9')<br />
</source><br />
<br />
We will flag this data with the CASA task {{flagdata}}. Because we are flagging channels on all scans of the bandpass calibrator 3C48, these channels cannot be calibrated accurately, so we flag these channels also in the data of the target.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', spw='0:148~155', action='apply')<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', spw='0:51~59', action='apply')<br />
</source><br />
<br />
The data on our bandpass+gain+flux calibrator 3C48 now look clean, with no dead or misbehaving antennas (plotted using the same {{plotms}} call as above). We now start with the calibration.<br />
<br />
== Calibration ==<br />
<br />
Many of the steps below are based on the CASA tutorial on [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials#3C129_P-band_continuum_imaging_data_reduction_tutorial reducing VLA P-band continuum data].<br />
<br />
=== <i>A priori</i> Antenna Position Corrections ===<br />
<br />
We start the calibration by obtaining the latest set of antenna-position corrections compared to the stored values that were derived at the start of the observing period. For this you need to run CASA task {{gencal}} with parameter <i>caltype='antpos'</i>. (Be sure to have internet connection in order to grab the antenna position corrections.)<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='antpos', caltable='antpos.cal')<br />
</source><br />
<br />
The output is a list of antenna position corrections that are written to the calibration table <i>antpos.cal</i>:<br />
<br />
<small style="font-size:62%;"><pre><br />
2017-07-25 19:54:51 INFO gencal Determine antenna position offests from the baseline correction database<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea01 : 0.00200 0.00000 0.00200<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea02 : 0.00210 0.01800 0.00430<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea03 : -0.00700 0.01600 -0.02000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea06 : 0.00200 -0.00200 0.00500<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea08 : -0.01290 0.01280 0.02920<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea09 : -0.01200 -0.01300 0.02400<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea15 : -0.00500 -0.01000 0.01100<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea16 : 0.01100 -0.01500 -0.01200<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea17 : -0.01300 0.00000 0.05000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea18 : -0.00600 0.02200 -0.02000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea22 : 0.00600 0.00000 0.00000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea25 : -0.00140 0.00140 -0.00140<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea27 : -0.00400 -0.00500 0.03100<br />
2017-07-25 19:54:52 INFO calibrater Beginning specifycal-----------------------<br />
2017-07-25 19:54:52 INFO Creating KAntPos Jones table from specified parameters.<br />
2017-07-25 19:54:52 WARN NB: This EVLA dataset appears to fall within the period<br />
2017-07-25 19:54:52 WARN + of semester 16B during which the online tropospheric<br />
2017-07-25 19:54:52 WARN + delay model was mis-applied.<br />
2017-07-25 19:54:52 WARN A correction for the online tropospheric delay model error WILL BE APPLIED!<br />
2017-07-25 19:54:52 WARN Marking antpos caltable to turn ON the trop delay correction.<br />
</pre></small><br />
<br />
Note that these observations were taken during a period in which the atmospheric delay terms were calculated incorrectly. In CASA versions 4.7.1 and up a [https://science.nrao.edu/facilities/vla/data-processing/vla-atmospheric-delay-problem correction] for this term is taken into account automatically when running CASA task {{gencal}} with parameter <i>caltype='antpos'</i>.<br />
<br />
=== Ionospheric calibration ===<br />
<br />
At frequencies below 5 GHz, ionospheric effects may need to be corrected. These corrections become important for frequencies < 1 GHz. CASA’s strategy is to obtain information on the total electron content (TEC) for the date of observations, which is based on data from the [https://science.nrao.edu/facilities/vlba/publications/memos/sci/gps_ion/node3 global navigation satellite system (GNSS)]. A series of CASA images in the form of a 24 hour movie of the TEC, as function of longitude and latitude, is then generated and stored as ''filename.IGS_TEC.im'', with a corresponding TEC error movie named ''filename.IGS_RMS_TEC.im'' (these movies can be viewed in CASA’s {{viewer}}. In order to download the data, you need to be connected to the internet:<br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='MG0414_d1_data.ms', doplot=True)<br />
</source><br />
(If, for any reason, the file cannot be retrieved from the server automatically, you can also download this specific TEC file for the tutorial [http://casa.nrao.edu/Data/EVLA/MG0414/igsg2580.16i here]).<br />
<br />
<br />
The information regarding the TEC is sparse and either active or direction-dependent ionospheric conditions may not be corrected very well. Also, the online TEC information improves with time, hence the quality of these data is best about two weeks after the observations.<br />
<br />
[[File:Pband_TECIM.png|200px|thumb|right|Figure 3. Total Electron Content (TEC) as functions of time for the day and location of the observations. The time of the observing run is shown in red.]]<br />
<br />
An ionosphere correction table is subsequently generated using CASA task {{gencal}}, in which the projected line-of-sight TEC (which depends on the zenith angle) is sampled for all times in the observation and stored in a standard CASA caltable. Figure 3 shows a plot that is generated for this caltable.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='tecim', caltable='tecim.cal',infile=tec_image)<br />
</source><br />
<br />
See the CASA Cookbook for a [https://casa.nrao.edu/casa_cookbook.pdf#subsection.4.3.9 detailed description] on ionosphere corrections in CASA (thanks are due to Jason Kooi, University of Iowa, for his contributions).<br />
<br />
=== Calibration of requantizer gains ===<br />
<br />
Next we will calibrate the requantizer gain levels, which are the visibility amplitudes that were set as the input of the WIDAR correlator. Requantizer scans are added to a P-band scheduling block (SB) to optimize the digital power in each spectral window which, in turn, maximizes the signal-to-noise of each window. Because there is a significant variation in power across the 240 MHz of P-band, and because some spectral windows may experience a high power due to strong RFI, setting the requantizer levels during the observations may improve the quality of the data, including the shape of the bandpass across multiple spectral windows. As part of the test observations of MG0414+0534, the very strong source Cygnus-A was observed at the start of the run. Re-setting the requantizer levels both before observing Cygnus-A and before targeting the other (much weaker) sources was essential to optimize the digital power and avoid correlation errors. Although we do not use Cygnus-A as part of this tutorial, and we use only a single spectral window, performing calibration of the requantizer gains is still good practice for P-band spectral line reduction using CASA task {{gencal}}.<br />
:<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='rq', caltable='rq.cal')<br />
</source><br />
<br />
=== Initial absolute flux density scale calibration ===<br />
<br />
<br />
The next step is to set the absolute flux levels of our flux calibrator 3C48 (field='0' from our {{listobs}} output), for which a well-known model exists using CASA task {{setjy}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
setjy(vis='MG0414_d1_data.ms', field='0', scalebychan=True, standard='Perley-Butler 2017', listmodels=False, usescratch=False)<br />
</source><br />
<br />
It is crucial for spectral line work to set the parameter ''scalebychan=True'' to ensure that the absolute flux level is calculated per channel and correctly interpolated across the observing band. If ''scalebychan=False'', then only a single value per spectral window is calculated, resulting in a step function in flux between spectral windows.<br />
<br />
=== Delay calibration ===<br />
<br />
The delay of each antenna, for each polarization and each spectral window, is now determined. Doing this on a short scan of the primary calibrator is generally sufficient and will correct for both internal (e.g., electronics, cables) and external (e.g., ionosphere) effects. We select ea04 as reference antenna as it produced good data, has baselines to all other used antennas in the array, and is located in the center of the array. We also need to apply, "on-the-fly," the calibration tables for the antenna positions, the total electron content, and the requantizer levels using the CASA task {{gaincal}}. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='delays.cal', field='0', selectdata=True, timerange='06:43:48~06:48:42', solint='inf', refant='ea04', gaintype='K', gaintable=['antpos.cal','tecim.cal','rq.cal'])<br />
</source><br />
<br />
[[File:Pband_delaycal_5.4.0.png|200px|thumb|right|Figure 4. Delays for each antenna and each polarization. Plot made with CASA task {{plotms}}.]]<br />
The calibration solutions can be plotted using CASA task {{plotms}} (see Figure 4):<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='delays.cal',xaxis='antenna1',yaxis='delay',field='0',coloraxis='corr')<br />
</source><br />
<br />
<br />
Note that any delays deviating by more than 30 nsec should be treated with caution and, if needed, flagged.<br />
<br />
=== Bandpass calibration ===<br />
<br />
Next step is to calibrate the bandpass, which corrects for the frequency dependent gain variations. Before determining the frequency dependent gains, we first calibrate the phases in time, deriving a solution for each integration, to make sure that we can derive accurate bandpass solutions when integrating over a full scan on the bandpass calibrator 3C48. This is done using CASA task {{gaincal}}:<br />
<br />
[[File:396px-Pband BPcal.png|200px|thumb|right|Figure 5. Effects of bandpass calibration on the frequency dependent amplitudes (top) and phases (bottom) across the observing band. (For corrections applied using the CASA task {{applycal}} see Figure 8a below.) The various colors show the different baselines with reference antenna ea04. The plots were made in CASA task {{plotms}} and in the ''Axes'' tab choosing the option ''Data Column: data''.]]<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='bpphase.gcal', field='0', spw='0:250~300', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintype='G', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal'])<br />
</source><br />
<br />
We use these phase solutions to perform the actual bandpass calibration. In this case, we have scans of our bandpass calibrator 3C48 taken at three different times during the observation. By interpolating the bandpass solutions in time, we try to minimize any possible time-varying bandpass effects and improve our spectral dynamic range. Bandpass calibration is performed in CASA by the task {{bandpass}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
bandpass(vis='MG0414_d1_data.ms', caltable='bandpass.cal', field='0', spw='0', solint='inf', combine='', refant='ea04', solnorm=False, bandtype='B', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bpphase.gcal'])<br />
</source><br />
<br />
To interpolate between bandpass scans, it is essential to specify parameter <i>combine=' '</i>. The reason being that the default of <i>combine='scan'</i>, when coupled with the parameter <i>solint='inf'</i>, would cross the scan boundaries to form one single solution for the bandpass. Only when combining <i>solint='inf'</i> with <i>combine=' '</i> is a bandpass solution obtained for each individual scan on 3C48. The bandpass plots can be viewed using CASA task {{plotms}} (see Figure 5).<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='amp',scan='7',correlation='xx,yy', coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='phase',scan='7',correlation='xx,yy', coloraxis='baseline')<br />
</source><br />
<br />
=== Gain calibration ===<br />
<br />
After the bandpass calibration, we perform a phase calibration of our data. Because we apply the bandpass solutions "on-the-fly," we can use the full bandwidth to determine the phase solutions. We will create two different calibration tables to correct for the time-varying phases. First we calibrate the phases on each interval with the CASA task {{gaincal}} parameter of ''solint'' set to ''int'': <br />
<br />
<source lang='python'><br />
# In CASA <br />
gaincal(vis='MG0414_d1_data.ms', caltable='intphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
</source><br />
<br />
[[File:NewFig6_plot.png|200px|thumb|right|Figure 6. Effects of phase calibration. (For corrections applied using the CASA task {{applycal}} see Figure 8b below.) The various colors show the different baselines with reference antenna ea04.]]<br />
<br />
The sole purpose of generating phase solutions on each 10 second interval is to be able to accurately correct for the time-varying amplitudes as we will see below. This method, however, is not ideal for interpolating the phase solutions in time. For that we do a second phase calibration, this time averaging the signal across the phase calibrator scans with the CASA task {{gaincal}} parameter of <i>solint='inf'</i>:<br />
<br />
<source lang='python'><br />
# In CASA <br />
gaincal(vis='MG0414_d1_data.ms', caltable='scanphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='inf', combine='', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
</source><br />
<br />
Note that an alternative approach to deriving phase solutions per scan is to use the CASA task {{smoothcal}} to average the phase corrections obtained every 10s with <i>solin='int'</i> over the full duration of a single scan. Generate a phase plot using CASA task {{plotms}} with the parameters below (see Figure 6):<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='phase',coloraxis='baseline')<br />
</source><br />
<br />
=== Scaling the Amplitude Gains ===<br />
<br />
[[File:NewFig7_plot.png|200px|thumb|right|Figure 7. Effects of amplitude and flux density scale calibration. The green points are the YY products on an offending antenna ea17, which we will flag in our target data. (For corrections applied using the CASA task {{applycal}} see Figure 9 below.)]]<br />
<br />
<br />
We now perform a calibration of the amplitude variations in time. For this, we will apply the phase solution obtained in each 10 second interval via CASA task {{gaincal}}: <br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='amp.gcal', field='0', spw='0', selectdata=False, solint='inf', combine='', refant='ea04', calmode='ap', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','intphase.gcal'])<br />
</source><br />
<br />
Normally this procedure will only calibrate the relative amplitudes, and the CASA task {{fluxscale}} is required to place the data on an absolute flux scale. However, because our flux-calibrator is also our bandpass and gain calibrator, the absolute flux scale will already be ok, so we can skip this step. Generate an amplitude gain plot using CASA task {{plotms}} (see Figure 7) with the parameters below:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='amp',coloraxis='baseline')<br />
</source><br />
<br />
=== Applying the calibration ===<br />
<br />
We have now finished the calibration of the data. We will apply the calibration tables that were derived from 3C48 to both the calibrator 3C48 itself and our target MG0414+0354. The CASA task {{applycal}} creates a '''corrected''' data column where the calibrated data are stored.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_data.ms', field='0', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], parang=True, calwt=False, applymode='calflagstrict', flagbackup=True)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_data.ms', field='1', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], calwt=False, applymode='calflagstrict', flagbackup=True)<br />
</source><br />
<br />
Sometimes after calibrating the data it is easier to see any offending antennas or baselines. Replotting with <i>ydatacolumn='corrected'</i> will produce the corrected calibration plots (Figures 8a, 8b). <br />
<br />
<source lang='python'><br />
# In CASA for corrected amplitude and phase vs frequency figure 8a<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='amp',scan='7',ydatacolumn='corrected',correlation='xx,yy', coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='phase',scan='7',ydatacolumn='corrected',correlation='xx,yy', coloraxis='baseline')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA for corrected amplitude and phase vs time figure 8b<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy', xaxis='time',yaxis='amp',ydatacolumn='corrected',coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='phase',ydatacolumn='corrected',coloraxis='baseline')<br />
</source><br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white; vertical-align: top;"<br />
| style="border: 1px solid darkgray;" | [[File:Pband_BPcal_corrected.png|thumb]] || style="border: 1px solid darkgray" | [[File:Pband_phase_cal_corrected.png|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 8a. Effects of bandpass <br />calibration on the frequency <br />dependent amplitudes (top) and <br />phases (bottom) across the <br /> observing band. The plots were <br />made in CASA task [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/using-plotms-to-plot-and-edit-visibilities-and-calibration-tables plotms] and <br />in the ''Axes'' tab choosing option <br />''Data Column: corrected''. || Figure 8b. Effects of phase calibration.<br /> The various colors show the different <br />baselines with reference antenna <br />ea04. The plot was made in CASA task<br /> [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/using-plotms-to-plot-and-edit-visibilities-and-calibration-tables plotms] and in the ''Axes'' tab choosing <br />option ''Data Column: corrected''.<br />
|}<br />
<br />
The calibrated data of the primary calibrator 3C48 show that the YY amplitudes of ea17 are off during the second part of the run. Because calibration solutions are antenna based, to get the best calibration, flag any poor data and re-run the calibration process. A currently known issue is that CASA task {{gaincal}} will not find good solutions if any correlation, including cross-correlation, in the data is completely flagged. In this case, there is no difference between flagging only the YY polarization of ea17 or the entire antenna.<br />
Antenna ea17 will be flagged using CASA task {{flagdata}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea17', action='apply')<br />
</source><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
'''Then run the calibration process again starting with the section on <i>Delay calibration</i>.''' Once you've re-run the calibration, plot the corrected time vs amplitudes with ea17 flagged (see Figure 9):<br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white; vertical-align: top;"<br />
| style="border: 1px solid darkgray;" | [[File:Pband_ampcal_corrected.png|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 9. Effects of amplitude and <br />flux density scale calibration. The green points <br />(the YY products of antenna ea17) <br />have been flagged out. The plot was <br />made in CASA task [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/using-plotms-to-plot-and-edit-visibilities-and-calibration-tables plotms] and in <br />the ''Axes'' tab choosing option <br />''Data Column: corrected''.<br />
|}<br />
<br />
<source lang='python'><br />
# In CASA for corrected amp vs time with ea17 flagged out figure 9<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='amp',ydatacolumn='corrected',coloraxis='baseline')<br />
</source><br />
<br />
Having re-run the calibration steps with ea17 flagged, we now split the calibrated data of our target into a new measurement set, including only the XX and YY polarization products. Using CASA task {{split}} the calibrated target data are placed into the '''data''' column of the new measurement set:<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_data.ms', outputvis='MG0414_d1_calibrated.ms', datacolumn='corrected', field='1', correlation='xx,yy', keepflags=False)<br />
</source><br />
<br />
== Imaging and Self-calibration ==<br />
<br />
We now have a calibrated data set that has all calibration applied except for possible self-calibration. Self-calibration relies on the target itself to better calibrate antenna-based gains (phases and amplitudes) as a function of time. It can be applied to targets that are strong in the continuum or have a strong enough spectral line to derive good phase and, if desired, also amplitude solutions on short (order of minutes) timescales. The procedure relies on an initial model of the target, which you then use iteratively to improve the gains and hence the quality of the image itself (see Figure 10). Here we show the various steps in self-calibrating our data based on the strong continuum emission of MG0414+0354. We refer to a [https://casaguides.nrao.edu/index.php?title=First_Look_at_Self_Calibration dedicated CASA guide] on self-calibration for further details.<br />
<br />
[[File:Pband_sc1.png|400px|thumb|right|Figure 10. Continuum image of MG0414+0354 before (left) and after (right) the first iteration of self-calibration, where we only calibrated the phases. Subsequent iterations may further improve the phases and amplitudes. This image was made in the [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/2-d-visualization-of-visibility-data-msview viewer], and scaling is the same in both plots.]]<br />
<br />
If strong RFI is present in the target data, it may be necessary to flag your data before proceeding with self-calibration. This is not the case with MG0414+0354.<br />
<br />
We create an initial model of our target field using CASA task {{tclean}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_calibrated.ms', datacolumn='data', imagename='MG0414_d1_cont_R03', spw='0', specmode='mfs', niter=100, threshold='1.5Jy', imsize=[256,256], weighting='briggs', robust=0.3, savemodel='modelcolumn')<br />
</source><br />
<br />
The information in the MODEL column consist of visibilities that present a model of the target field. CASA task {{tclean}} produces a ''.model'' image of the model that you can visualize with {{viewer}} or CASA task {{plotms}}. It is essential that only components that you know for sure represent the continuum in the target field are taken into account in self-calibration. If this is not the case, you probably cleaned too deep and included noise features or artifacts (e.g., from the beam-pattern) in the model. Self-calibration would permanently lock these as features in your data, so this should be avoided. On the other hand, it is equally crucial to ensure that the model is not empty, as this will result in your target being shifted to the phase center.<br />
<br />
Next, we derive phase corrections to this model. Because our target is strong enough, we should get enough S/N per baseline on a 60 sec interval, using CASA task {{gaincal}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_calibrated.ms', caltable='sc1.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='60s', combine='', minsnr=3.0)<br />
</source><br />
<br />
This creates a new phase-calibration table that we call ''sc1.gcal''. We now apply this calibration table to the calibrated data in order to improve the phase-only solutions using CASA task {{applycal}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_calibrated.ms', field='0', gaintable=['sc1.gcal'], interp='linear')<br />
</source><br />
<br />
Because self-calibration is an iterative process, we want to repeat this step again in order to further improve the gain calibration. For that, we first split the calibrated data with the self-calibration solution applied to the DATA column of a new measurement set, again using CASA task {{split}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_calibrated.ms', outputvis='MG0414_d1_sc1.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
Now we can use this newly created measurement set to make a better continuum image and subsequently perform the next round of self-calibration:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc1.ms', datacolumn='data', imagename='MG0414_d1_cont_sc1.R03', spw='0', specmode='mfs', niter=500, threshold='500mJy', imsize=[256,256],weighting='briggs',robust=0.3,savemodel='modelcolumn')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_sc1.ms', caltable='sc2.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='30s', combine='', minsnr=3.0)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_sc1.ms', field='0', gaintable=['sc2.gcal'], interp='linear')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_sc1.ms', outputvis='MG0414_d1_sc2.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc2.ms', datacolumn='data', imagename='MG0414_d1_cont_sc2_R03', spw='0', specmode='mfs', niter=1000, threshold='100mJy', imsize=[256,256],weighting='briggs',robust=0.3,savemodel='modelcolumn')<br />
</source><br />
<br />
Now that we have a good model of the continuum in the field of MG0414+0354, we will attempt a final self-calibration using both the phases and amplitudes via CASA task {{gaincal}} :<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_sc2.ms', caltable='sc3ap.gcal', field='0', spw='0', refant='ea04', calmode='ap', solint='30s', combine='', minsnr=3.0)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_sc2.ms', field='0', gaintable=['sc3ap.gcal'], interp='linear')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_sc2.ms', outputvis='MG0414_d1_sc3ap.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc3ap.ms', datacolumn='data', imagename='MG0414_d1_cont_sc3ap_R03', spw='0', specmode='mfs', niter=1000, threshold='100mJy', imsize=[256,256],weighting='briggs',robust=0.3,savemodel='modelcolumn')<br />
</source><br />
<br />
The output continuum image from the last iteration of CASA task {{tclean}} continues to be an improvement over previous imaging, but also shows that we are not likely to gain further improvement with additional rounds of self-calibration. This indicates that we now have a calibrated data-set of MG0414+0354 that is almost ready for final imaging. <br />
<br />
However, before imaging, we need to flag the target data to remove poor-quality data. First, we run an automated flagging routine using CASA task {{flagdata}} with the parameter <i>mode='rflag'</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_sc3ap.ms', mode='rflag', datacolumn='data')<br />
</source><br />
<br />
After having ran this automated flagging, we inspect the data via the CASA tasks {{viewer}} or {{plotms}} by displaying ''time'' vs. ''channel'' for the different baselines, or you may display ''time'' vs. ''baseline'' for the different channels. Keep in mind that when the data are properly calibrated, it is easier to recognize low-level imperfections which may have to be flagged to improve the image quality. In this case, {{viewer}} or {{plotms}} show that antennas ea09 and ea14 have significantly higher noise than the other antennas, something which was not picked up by the automated flagging routine. Additionally, for a number of baselines, the data quality looks relatively poor. We flag these antennas and baselines before imaging the data via the CASA {{flagdata}} task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea09', datacolumn='data')<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea14', datacolumn='data')<br />
</source><br />
<br />
To flag offending individual baselines, in CASA task {{flagdata}} you may use <i>antenna='antenna&antenna(&antenna...)'</i>. <br />
<br />
[[File:Screenshot-Spectral_Profile_-_MG0414_d1_line_sc3ap_vel_R03.image-raster.png|200px|thumb|right|Figure 11. Spectral line profile with ea09, ea14, and ea17 flagged out but re-calibration has not been run.]]<br />
<br />
Flagging data can be an iterative process. Often, the better the data are calibrated, the easier it is to recognize low-level imperfections in the data. To optimize the data reduction, you can script the calibration process, and opt to flag the data also in the original data set (i.e., before calibration), after which you run the entire calibration again.<br />
<br />
When you flag these antennas you must go back and rerun the calibration before the final imaging, starting at the '''<i>Delay calibration</i>''' section. If you rerun the calibration you will either have to rename your ms's created from split during self calibration or delete them in order to redo self calibration as split will not overwrite a ms name. If you do not re-run the calibration sequence and proceed ahead with the final {{tclean}} run then inspect the line-data cube (see below), you will get a plot that should look similar to Figure 11.<br />
<br />
Finally, performing the calibration sequence again after flagging antennas ea09 and ea14, we image the data using the CASA task {{tclean}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc3ap.ms', datacolumn='data', imagename='MG0414_d1_line_sc3ap_vel_R03', spw='0', specmode='cube', outframe='bary', veltype='optical', restfreq='390.600GHz', niter=100, threshold='1Jy', imsize=[256,256], pbcor=True, weighting='briggs', robust=0.3, savemodel='none')<br />
</source><br />
<br />
[[File:spec_Pband_plot.png|200px|thumb|right|Figure 12. Test-data of the redshifted HI 21cm absorption line on top of the radio continuum in MG 0414+0354; see More, Carilli & Mention 1999 (Apj, 510, 87) for the original detection.]]<br />
<br />
We can inspect the resulting line-data cube in the {{viewer}}. The data cube contains both continuum and line emission, which can optionally be separated using the CASA task {{uvcontsub}}. The HI absorption feature is clearly present (see Figure 12).<br />
<br />
To see your spectral line, open the last image of the source. Go to a channel that shows the source, then zoom in on the source. Click the ''Point marking'' icon, find the brightest pixel in the source and click on it. Now click on the ''spectral profile tool'' button to get the single point profile of the spectrum. Here you will see a spectrum with 2 dips that drop to zero; these dips are a part of the flagging we have done. You will need to zoom in in order to notice any spectral lines due to the auto scaling; zoom in by clicking on the graph and creating a yellow box then release.<br />
<br />
<br />
<!-- --[[User:bemonts|Bjorn Emonts]] <br />
-- 5.0 corrections: [[User:kradford| Karlee Radford]] (2018-03-21)<br />
-- Edits [[User:tperreau| Tony Perreault]] (26 Apr 2018)<br />
-- Edits [[User:Hmedlin | Heidi Medlin]] (27 April 2018)<br />
-- Yet more edits [[User:tperreau| Tony Perreault]] (2 May 2018)<br />
-- And yet more edits [[User:tperreau| Tony Perreault]] (May the 4th be with you 2018)<br />
-- 5.4.0 updates: [User:kradford| Karlee Radford]] (2018-10-11)<br />
--><br />
<br />
{{Checked 5.4.0}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=MG0414%2B0534_P-band_Spectral_Line_Tutorial_-_CASA_5.4.0&diff=27161MG0414+0534 P-band Spectral Line Tutorial - CASA 5.4.02019-04-25T14:46:00Z<p>Emomjian: </p>
<hr />
<div><!--By Bjorn Emonts--><br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b> <br />
<br />
{| class="wikitable"<br />
|-<br />
| style="background: red; color: white;" | '''DISCLAIMER'''<br />
|-<br />
| The following guide is for users who are experts in data reduction using CASA. If you are a beginning or novice user, please review other CASA guides found at [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials VLA Tutorials].<br />
|}<br />
<br />
== Overview ==<br />
<br />
This tutorial describes how to use CASA 5.4.0 to reduce spectral-line data in the low-frequency P-band of the VLA (230&ndash;470 MHz). The goal is to make an image cube containing HI 21cm absorption against the strong radio continuum of gravitationally lensed radio galaxy MG0414+0534. As a results of the high redshift of z=2.6365, the HI absorption signal in MG0414+0534 is redshifted to an observed frequency of 390.597 MHz. The HI absorption in MG0414+0534 was previously imaged with the VLA by Moore, Carilli & Menten 1999 ([http://iopscience.iop.org/article/10.1086/311818/meta ApJ, 510, 87]), before the upgrade to the WIDAR system.<br />
<br />
=== Observing strategy ===<br />
<br />
To perform P-band spectrsocopy, there are three important considerations for planning the observations:<br />
<br />
* Use a bandpass calibrator that is strong enough to accurately calibrate the frequency-dependent gain variations. This is particularly important for most HI 21cm absorption projects, which are typically performed against strong radio continuum sources. As a rule-of-thumb, use ''t''<sub>cal</sub> > ''t''<sub>obj</sub> × (''S''<sub>obj</sub> / ''S''<sub>cal</sub>)<sup>2</sup>, with ''t'' the exposure time and ''S'' the source flux density. This rule-of-thumb serves to avoid introducing excessive additional noise in the spectrum.<br />
<br />
* Additionally, if very high spectral dynamic range is needed (i.e., when expecting a ratio between the detection limit and the radio continuum of about 1:10,000 or more), consider observing a bandpass calibrator several times during your run to be able to correct for time-varying bandpass changes, which scale with the continuum emission in the target field. See the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/calibration Calibration section] of the Guide to Observing with the VLA for more information.<br />
<br />
* Use a bandwidth that is wide enough to perform accurate self calibration. Using a wide bandwidth for self-calibration is important for fields with relatively weak continuum sources. For strong continuum sources, a narrower bandwidth can be used to avoid excessive RFI. <br />
<br />
The P-band test data on MG0414+0534 that we use in this tutorial were obtained using a large bandwidth. This is to ensure that good bandpass solutions can be procured, and that self-calibration can accurately be performed. However, due to the strong radio continuum of MG0414+0534 (3.3 Jy at 390 MHz), and the large amounts of RFI across the entire band, we only use a small fraction of the total band for data reduction and analysis in this tutorial.<br />
<br />
== Obtaining the data ==<br />
<br />
<font color=red>'''This 15 GB data set can be downloaded by clicking [https://casa.nrao.edu/Data/EVLA/MG0414/MG0414_d1_data.ms.tgz here].'''</font> The filename should be MG0414_d1_data.ms.tgz (add the ".tgz" if this is missing). We are providing this starting data set, rather than the original data set for two reasons. First, many of these initial processing steps can be rather time consuming (> 1 hr). Second, while necessary, many of these steps are not fundamental to the calibration and imaging process, which is the focus of this tutorial.<br />
<br />
We will use test data that was taken in a hybrid configuration when the VLA array was moved from B-config to A-config. The data set can be downloaded from the [https://archive.nrao.edu/archive/advquery.jsp NRAO archive] by searching for the following Project Code: <br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
TSUB0001<br />
</pre><br />
</div><br />
<br />
This returns a long list with test-data that are publicly available. Our observations were performed on 14 Sept 2016:<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
TSUB0001.sb32720781.eb32763188.57645.263958564814<br />
</pre><br />
</div><br />
<br />
Note that this observation was duplicated on 15 September 2016. <!-- This second day of observations can be found under filename:<br />
TSUB0001.sb32720781.eb32773507.57646.24443702547 -->For the purpose of this tutorial, we only reduce and image the first data set. The 15 GB data set from the download has already had online flagging and Hanning smoothing applied, and the smaller data set split out from the larger data set. If you wish, you can download the SDM from the NRAO archive in order to apply the online flags and run Hanning smoothing on the large data set before splitting out the smaller data set to work on.<br />
<br />
<!-- The second data set can be reduced in an identical way (pending the flagging, given that the RFI conditions most likely changed). During the imaging stage you can then combine both reduced data sets. --><br />
<br />
<!-- To download the data, fill in your email, select either the "SDM-BDF (all files)" or “MS” option, and check the box next to the data sets that you want. The practical difference between selecting the MeasurementSet (MS) and Science Data Model (SDM) is that for MS the first step in the data reduction described below, namely reading in the SDM data to save it as a MeasurementSet, is already done by the archival engine. Note that you can also opt to download the data as a tar-file by tickking the appropriate box.<br />
<br />
Optionally, you can request to discard data marked as ‘bad’ by ticking the box “Apply Telescope Flags”, which gets rid of data taken during times of instrument calibration, shadowing, slewing, etc. However, it also applies all flags on the science data that were automatically created during the observations. Therefore, a safer option is to not apply any flags before downloading the data, but first inspect and subsequently apply the flags generated during the observations using <i>flagcmd</i>, as part of the data reduction plan.<br />
<br />
Click "Get My Data" will forward you to the next page, where you should choose the delivery method (either downloading over the internet, or sending home a hard-drive with the data). If you opt to retrieve the data over the internet, wait until you get an email confirming that the data is ready for download. --><br />
<br />
== Examining the Data ==<br />
<br />
=== Loading data into CASA ===<br />
<br />
We will have to untar the 15 GB data set that you've downloaded.<br />
<br />
<pre style="background-color:lightgrey;"><br />
# in a terminal, outside of CASA:<br />
tar -xzf MG0414_d1_data.ms.tgz<br />
</pre><br />
<br />
Please use CASA 5.4.0 for this tutorial (typing ''casa -ls'' in a linux window shows the available versions and the current version; to explicitly change the version type, e.g., ''casa -r 5.4.0-68'')<br />
<br />
<source lang='bash'><br />
casa<br />
</source><br />
on the command line. This should start a CASA interactive python (iPython) session, and open a separate log window. To guarantee that the below mentioned procedure for data reduction and imaging works, make sure you are using CASA version 5.4.0. While older versions may work as well for the purpose of this tutorial, it is good to read instructions on how to [https://casa.nrao.edu/casa_obtaining.shtml download and install] the latest version of CASA.<br />
<!-- <br />
We will begin by importing our data from the binary format (SDM-BDF) into the MeasurementSet format, which is the standard for CASA data. For this, we use <i>importevla</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
importevla(asdm='TSUB0001.sb32720781.eb32763188.57645.263958564814', vis='MG0414_d1.ms', flagpol=False, applyflags=False, savecmds=True, outfile='flagfile.txt')<br />
</source><br />
<br />
<!-- In this case, we do not apply the flags that were created as part of the observations, but we will write them out to a <i>flagfile.txt</i> file. We will inspect and apply the flags as follows:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagcmd(vis='MG0414_d1.ms', action='plot')<br />
flagcmd(vis='MG0414_d1.ms', action='apply')<br />
</source><br />
<br />
As an alternative to <i>flagcmd</i>, given that we saved the flags to <i>flagfile.txt</i>, we can also use <br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1.ms', mode='list', inpfile='flagfile.txt', action='apply')<br />
</source><br />
<br />
to apply the flags.<br />
<br />
One of the flag commands is to <i>clip</i> all data points that have a 0-value, as these most likely have not recorded any science data. This is normally done with the settings <i>correlation=`ABS_RR’</i> and <i>correlation=`ABS_LL’</i>. Because the VLA P-band and 4-band systems use linear XX and YY polarisation, rather than the circular RR and LL polarisation of the other VLA bands, you will encounter an error message when running <i>flagcmd</i>. We can get around this issue by manually clipping the 0-data for the XX and LL polarization with <i>flagdata</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1.ms', mode='clip', correlation='ABS_XX,ABS_YY', clipzeros=True, action='apply')<br />
</source><br />
--><br />
<br />
It is a good practice to read the Operator Log that was created by the duty operator during the observations. This log can provide additional information on data that should be flagged manually during the data reduction stage. The Operator logs can be found at [http://www.vla.nrao.edu/cgi-bin/oplogs.cgi Ops Logs form entry], and this is the [http://www.vla.nrao.edu/operators/logs/2016/9/2016-09-14_0620_TSUB0001.pdf link for the log of the observation] (in PDF format).<br />
<br />
=== Inspecting the observation info ===<br />
<br />
We will inspect the content of our data set using CASA task {{listobs}}.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='MG0414_d1_data.ms', listfile='listobs.txt')<br />
</source><br />
<br />
This normally plots an overview of the data in the CASA logger, but with the parameter <i>listfile=`listobs.txt’</i>, this information is written out to the file <i>listobs.txt</i> and is included below:<br />
<br />
<small style="font-size:62%;"><pre><br />
Observer: . Frazer Owen Project: uid://evla/pdb/1695465 <br />
Observation: EVLA<br />
Data records: 991814 Total elapsed time = 9390 seconds<br />
Observed from 14-Sep-2016/06:43:48.0 to 14-Sep-2016/09:20:18.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
14-Sep-2016/06:43:48.0 - 06:48:42.0 7 0 3C48 33861 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
06:56:21.0 - 07:08:39.0 8 1 MG0414+0534 82702 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:08:42.0 - 07:28:36.0 9 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:28:39.0 - 07:48:33.0 10 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
07:48:36.0 - 08:08:30.0 11 1 MG0414+0534 139698 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:10:30.0 - 08:14:30.0 12 0 3C48 26549 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
08:16:30.0 - 08:34:27.0 13 1 MG0414+0534 124918 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:34:30.0 - 08:54:21.0 14 1 MG0414+0534 139347 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
08:54:24.0 - 09:14:18.0 15 1 MG0414+0534 139048 [0] [3] [OBSERVE_TARGET#UNSPECIFIED]<br />
09:16:21.0 - 09:20:18.0 16 0 3C48 26295 [0] [3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]<br />
(nRows = Total number of rows per scan) <br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C48 01:37:41.299431 +33.09.35.13299 J2000 0 86705<br />
1 NONE MG0414+0534 04:14:37.800000 +05.34.41.99999 J2000 1 905109<br />
Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 EVLA_P#A0C0#17 466 TOPO 386.312 15.625 7281.2 389.9453 12 XX XY YX YY<br />
Sources: 2<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 3C48 0 - - <br />
1 MG0414+0534 0 - - <br />
Antennas: 27:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 ea01 W24 25.0 m -107.38.49.0 +33.53.04.0 -2673.3457 -1784.6009 10.4844 -1604008.744400 -5042135.825100 3553403.710800<br />
1 ea02 W56 25.0 m -107.44.26.7 +33.49.54.6 -11333.1991 -7637.6832 15.3636 -1613255.391400 -5042613.097800 3548545.906000<br />
2 ea03 W40 25.0 m -107.41.13.5 +33.51.43.1 -6377.9680 -4286.7773 8.2312 -1607962.451800 -5042338.218100 3551324.962400<br />
3 ea04 E04 25.0 m -107.37.00.8 +33.53.59.7 102.7973 -63.7800 -2.6177 -1601068.806000 -5042051.932700 3554824.838800<br />
4 ea05 E36 25.0 m -107.33.20.2 +33.52.34.3 5761.3733 -2704.6731 33.0133 -1596127.730800 -5045193.742100 3552652.419700<br />
5 ea06 N24 25.0 m -107.37.16.1 +33.55.37.7 -290.3584 2961.8653 -12.2425 -1600930.069900 -5040316.395500 3557330.390000<br />
6 ea07 N16 25.0 m -107.37.10.9 +33.54.48.0 -155.8512 1426.6377 -9.3934 -1601061.954400 -5041175.875300 3556058.026700<br />
7 ea08 W64 25.0 m -107.46.20.1 +33.48.50.9 -14240.7524 -9606.2900 17.0885 -1616361.575500 -5042770.516600 3546911.419900<br />
8 ea09 N40 25.0 m -107.37.29.5 +33.57.44.4 -633.6056 6878.5897 -20.7984 -1600592.749000 -5038121.341300 3560574.826900<br />
9 ea10 E20 25.0 m -107.35.43.6 +33.53.29.9 2082.1759 -987.0087 9.4361 -1599340.800100 -5043150.963000 3554065.231500<br />
10 ea12 N32 25.0 m -107.37.22.0 +33.56.33.6 -441.7248 4689.9674 -16.9328 -1600781.044500 -5039347.439100 3558761.527100<br />
11 ea13 E28 25.0 m -107.34.39.3 +33.53.04.9 3732.7776 -1757.3308 21.4271 -1597899.895900 -5044068.684700 3553432.450200<br />
12 ea14 E08 25.0 m -107.36.48.9 +33.53.55.1 407.8283 -206.0315 -3.2236 -1600801.931400 -5042219.382600 3554706.429400<br />
13 ea15 N64 25.0 m -107.37.58.7 +34.02.20.5 -1382.3871 15410.1326 -40.6450 -1599855.687000 -5033332.368600 3567636.606800<br />
14 ea16 W72 25.0 m -107.48.24.0 +33.47.41.2 -17419.4641 -11760.2694 14.9442 -1619757.299900 -5042937.656400 3545120.392300<br />
15 ea17 N56 25.0 m -107.37.47.9 +34.00.38.4 -1105.2042 12254.2800 -34.2710 -1600128.378100 -5035104.139200 3565024.633900<br />
16 ea18 W48 25.0 m -107.42.44.3 +33.50.52.1 -8707.9181 -5861.7682 15.5302 -1610451.901900 -5042471.123800 3550021.073100<br />
17 ea19 W16 25.0 m -107.37.57.4 +33.53.33.0 -1348.7121 -890.6209 1.2959 -1602592.853500 -5042054.992400 3554140.702800<br />
18 ea20 E16 25.0 m -107.36.09.8 +33.53.40.0 1410.0403 -673.4656 -0.7790 -1599926.104100 -5042772.977200 3554319.801100<br />
19 ea21 E24 25.0 m -107.35.13.4 +33.53.18.1 2858.1906 -1349.1352 13.7186 -1598663.082000 -5043581.391200 3553767.014100<br />
20 ea22 W32 25.0 m -107.39.54.8 +33.52.27.2 -4359.4399 -2923.1314 11.7614 -1605808.634100 -5042230.084000 3552459.197800<br />
21 ea23 E12 25.0 m -107.36.31.7 +33.53.48.5 848.6994 -411.6226 -2.7575 -1600416.518000 -5042462.430500 3554536.041700<br />
22 ea24 W08 25.0 m -107.37.21.6 +33.53.53.0 -432.1080 -272.1514 -1.5061 -1601614.083200 -5042001.656900 3554652.505900<br />
23 ea25 N08 25.0 m -107.37.07.5 +33.54.15.8 -68.9088 433.1829 -5.0674 -1601147.942500 -5041733.833600 3555235.947000<br />
24 ea26 E32 25.0 m -107.34.01.5 +33.52.50.3 4701.6503 -2209.7119 25.1976 -1597053.124400 -5044604.675000 3553058.992700<br />
25 ea27 N48 25.0 m -107.37.38.1 +33.59.06.2 -855.2719 9405.9407 -25.9485 -1600374.881000 -5036704.201700 3562667.858900<br />
26 ea28 N72 25.0 m -107.38.10.5 +34.04.12.2 -1685.6797 18861.8306 -43.5015 -1599557.928700 -5031396.353400 3570494.736800<br />
</pre></small><br />
<br />
<!-- === Hanning smoothing the data ===<br />
<br />
When plotting the data in plotms (plotting amplitude against frequency), it is immediately clear that our observing band contains lots of Radio Frequency Interference (RFI), which is unavoidable at these low frequencies. The situation actually looks worse than it is, because ‘Gibbs ringing’ causes the blending of strong RFI signal into adjacent channels (Fig.1). To reduce the effect of Gibbs ringing, we first Hanning smooth the data (please note that this can take a while to run on the entire 142 Gb data set):<br />
<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='MG0414_d1_data.ms', datacolumn='data', outputvis='MG0414_d1_hanning.ms')<br />
</source><br />
<br />
[[File:Pband_hanning.png|400px|thumb|right|Fig.1 Reducing the effects of Gibbs ringing by Hanning smoothing the data. Shown is only a small fraction of the band, both before and after Hanning smooth. The amplitude scale is the same in both plots.]]<br />
<br />
=== Splitting the data ===<br />
<br />
Upon further inspection in “plotms” (using ea01, field 0, corr xx,yy, and coloraxis baseline), it is clear that there is a lot of RFI across most of the 64 MHz (or 43,000 km/s) band that we used for the observations. Luckily, the region around the expected HI absorption line at 390.6 MHz is relatively clean of RFI. Because our target is strong enough in the continuum to perform a good self-calibration, we select this relatively clean part of the band centered around the expected HI line. Note that there were two IFs with different spectral resolution included in the observations. We need only one of the two IFs, and choose the one with the highest spectral resolution. Moreover, we only need the sources 3C48 (which we will use for bandpass, gain and flux calibration) and our target MG0414+0534, hence we select field=’3,4’. This approach has the additional advantage that we reduce our data volume a lot, bringing it down to about 15 Gb.<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_hanning.ms', outputvis='MG0414_d1_data.ms', datacolumn='data', field='3,4', spw='17:20~485', keepflags=False)<br />
</source><br />
<br />
<font color=red>'''This 15 Gb data set can be downloaded by clicking [https://casa.nrao.edu/Data/EVLA/MG0414/MG0414_d1_data.ms.tgz here].'''</font> The filename should be MG0414_d1_data.ms.tgz (add the ".tgz" if this is missing), and the file can be unpacked by typing <i>tar xzf MG0414_d1_data.ms.tgz</i><br />
--><br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white;"<br />
| style="border: 1px solid darkgray;" | [[File:MG0414_plotants_5.4.0.png|200px|thumb]] || style="border: 1px solid darkgray" | [[File:Ant_logplot_5.4.0.png |200px|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 1a. Plotants plot for <br />MG0414. || Figure 1b. Plotants plot <br />for MG0414 with a log scale.<br />
|}<br />
<br />
Next we look at a graphical plot of the antenna locations and save a hard copy<br />
in case you want it later using CASA task {{plotants}} (see Figures 1a and 1b). This will be useful for selecting a reference antenna. Typically a good choice is an antenna close to the center of the array (see Figure 1b). Unless it shows problems after inspection of the data, we provisionally choose ea04. (Note: clicking on any image will open up a larger version.)<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis='MG0414_d1_data.ms',figfile='ant_locations.png')<br />
<br />
# In CASA<br />
# If you would like to see another view of the antennas you may plot them on a log plot<br />
plotants(vis='MG0414_d1_data.ms',logpos=True,figfile='ant_logplot.png')<br />
<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
<!-- NOTE: when revising for CASA release 5.3.0, EM wants to revisit how flagging is approached in the guide. Some of the flagging tasks, such as running rflag and flagging out antennas ea09, 14, and 17 should be introduced earlier - perhaps in this section - so that major flagging is done once and then the user can proceed to the self-calibration and imaging section. Tony --><br />
<br />
[[File:Pband_flagchan.png|400px|thumb|right|Figure 2. Flagging of calibrator 3C48. Shown are the XX and YY polarization on antenna ea01, both before (left) and after (right) flagging the bad channels. The colors represent the different baselines with ea01.]]<br />
<br />
We will now perform a more in-depth flagging of the bad data. Our strategy is to first focus on the calibrator source (in this case only 3C48). If the calibrator data are flagged well, we should be able to perform an accurate calibration of the target data. Later in this document we will then flag the calibrated target data using an automated flagger. <br />
<br />
When plotting frequency against amplitude in CASA task {{plotms}}, we see that there are still two ranges of channels filled with RFI throughout the observing run (see Figure 2). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',antenna='ea01',xaxis='chan',yaxis='amp',correlation='xx,yy',coloraxis='baseline', scan='7',averagedata=True, avgtime='1e9')<br />
</source><br />
<br />
We will flag this data with the CASA task {{flagdata}}. Because we are flagging channels on all scans of the bandpass calibrator 3C48, these channels cannot be calibrated accurately, so we flag these channels also in the data of the target.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', spw='0:148~155', action='apply')<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', spw='0:51~59', action='apply')<br />
</source><br />
<br />
The data on our bandpass+gain+flux calibrator 3C48 now look clean, with no dead or misbehaving antennas (plotted using the same {{plotms}} call as above). We now start with the calibration.<br />
<br />
== Calibration ==<br />
<br />
Many of the steps below are based on the CASA tutorial on [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials#3C129_P-band_continuum_imaging_data_reduction_tutorial reducing VLA P-band continuum data].<br />
<br />
=== <i>A priori</i> Antenna Position Corrections ===<br />
<br />
We start the calibration by obtaining the latest set of antenna-position corrections compared to the stored values that were derived at the start of the observing period. For this you need to run CASA task {{gencal}} with parameter <i>caltype='antpos'</i>. (Be sure to have internet connection in order to grab the antenna position corrections.)<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='antpos', caltable='antpos.cal')<br />
</source><br />
<br />
The output is a list of antenna position corrections that are written to the calibration table <i>antpos.cal</i>:<br />
<br />
<small style="font-size:62%;"><pre><br />
2017-07-25 19:54:51 INFO gencal Determine antenna position offests from the baseline correction database<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea01 : 0.00200 0.00000 0.00200<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea02 : 0.00210 0.01800 0.00430<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea03 : -0.00700 0.01600 -0.02000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea06 : 0.00200 -0.00200 0.00500<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea08 : -0.01290 0.01280 0.02920<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea09 : -0.01200 -0.01300 0.02400<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea15 : -0.00500 -0.01000 0.01100<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea16 : 0.01100 -0.01500 -0.01200<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea17 : -0.01300 0.00000 0.05000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea18 : -0.00600 0.02200 -0.02000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea22 : 0.00600 0.00000 0.00000<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea25 : -0.00140 0.00140 -0.00140<br />
2017-07-25 19:54:52 INFO gencal offsets for antenna ea27 : -0.00400 -0.00500 0.03100<br />
2017-07-25 19:54:52 INFO calibrater Beginning specifycal-----------------------<br />
2017-07-25 19:54:52 INFO Creating KAntPos Jones table from specified parameters.<br />
2017-07-25 19:54:52 WARN NB: This EVLA dataset appears to fall within the period<br />
2017-07-25 19:54:52 WARN + of semester 16B during which the online tropospheric<br />
2017-07-25 19:54:52 WARN + delay model was mis-applied.<br />
2017-07-25 19:54:52 WARN A correction for the online tropospheric delay model error WILL BE APPLIED!<br />
2017-07-25 19:54:52 WARN Marking antpos caltable to turn ON the trop delay correction.<br />
</pre></small><br />
<br />
Note that these observations were taken during a period in which the atmospheric delay terms were calculated incorrectly. In CASA versions 4.7.1 and up a [https://science.nrao.edu/facilities/vla/data-processing/vla-atmospheric-delay-problem correction] for this term is taken into account automatically when running CASA task {{gencal}} with parameter <i>caltype='antpos'</i>.<br />
<br />
=== Ionospheric calibration ===<br />
<br />
At frequencies below 5 GHz, ionospheric effects may need to be corrected. This corrections becomes important for frequencies < 1 GHz. CASA’s strategy is to obtain information on the total electron content (TEC) for the date of observations, which is based on data from the [https://science.nrao.edu/facilities/vlba/publications/memos/sci/gps_ion/node3 global navigation satellite system (GNSS)]. A series of CASA images in the form of a 24 hour movie of the TEC, as function of longitude and latitude, is then generated and stored as ''filename.IGS_TEC.im'', with a corresponding TEC error movie named ''filename.IGS_RMS_TEC.im'' (these movies can be viewed in CASA’s {{viewer}}. In order to download the data, you need to be connected to the internet:<br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='MG0414_d1_data.ms', doplot=True)<br />
</source><br />
(If, for any reason, the file cannot be retrieved from the server automatically, you can also download this specific TEC file for the tutorial [http://casa.nrao.edu/Data/EVLA/MG0414/igsg2580.16i here]).<br />
<br />
<br />
The information regarding the TEC is sparse and either active or direction-dependent ionospheric conditions may not be corrected very well. Also, the online TEC information improves with time, hence the quality of these data is best about two weeks after the observations.<br />
<br />
[[File:Pband_TECIM.png|200px|thumb|right|Figure 3. Total Electron Content (TEC) as functions of time for the day and location of the observations. The time of the observing run is shown in red.]]<br />
<br />
An ionosphere correction table is subsequently generated using CASA task {{gencal}}, in which the projected line-of-sight TEC (which depends on the zenith angle) is sampled for all times in the observation and stored in a standard CASA caltable. Figure 3 shows a plot that is generated for this caltable.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='tecim', caltable='tecim.cal',infile=tec_image)<br />
</source><br />
<br />
See the CASA Cookbook for a [https://casa.nrao.edu/casa_cookbook.pdf#subsection.4.3.9 detailed description] on ionosphere corrections in CASA (thanks are due to Jason Kooi, University of Iowa, for his contributions).<br />
<br />
=== Calibration of requantizer gains ===<br />
<br />
Next we will calibrate the requantizer gain levels, which are the visibility amplitudes that were set as the input of the WIDAR correlator. Requantizer scans are added to a P-band scheduling block (SB) to optimize the digital power in each spectral window which, in turn, maximizes the signal-to-noise of each window. Because there is a significant variation in power across the 240 MHz of P-band, and because some spectral windows may experience a high power due to strong RFI, setting the requantizer levels during the observations may improve the quality of the data, including the shape of the bandpass across multiple spectral windows. As part of the test observations of MG0414+0534, the very strong source Cygnus-A was observed at the start of the run. Re-setting the requantizer levels both before observing Cygnus-A and before targeting the other (much weaker) sources was essential to optimize the digital power and avoid correlation errors. Although we do not use Cygnus-A as part of this tutorial, and we use only a single spectral window, performing calibration of the requantizer gains is still good practice for P-band spectral line reduction using CASA task {{gencal}}.<br />
:<br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='MG0414_d1_data.ms', caltype='rq', caltable='rq.cal')<br />
</source><br />
<br />
=== Initial absolute flux density scale calibration ===<br />
<br />
<br />
The next step is to set the absolute flux levels of our flux calibrator 3C48 (field='0' from our {{listobs}} output), for which a well-known model exists using CASA task {{setjy}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
setjy(vis='MG0414_d1_data.ms', field='0', scalebychan=True, standard='Perley-Butler 2017', listmodels=False, usescratch=False)<br />
</source><br />
<br />
It is crucial for spectral line work to set the parameter ''scalebychan=True'' to ensure that the absolute flux level is calculated per channel and correctly interpolated across the observing band. If ''scalebychan=False'', then only a single value per spectral window is calculated, resulting in a step function in flux between spectral windows.<br />
<br />
=== Delay calibration ===<br />
<br />
The delay of each antenna, for each polarization and each spectral window, is now determined. Doing this on a short scan of the primary calibrator is generally sufficient and will correct for both internal (e.g., electronics, cables) and external (e.g., ionosphere) effects. We select ea04 as reference antenna as it produced good data, has baselines to all other used antennas in the array, and is located in the center of the array. We also need to apply, "on-the-fly," the calibration tables for the antenna positions, the total electron content, and the requantizer levels using the CASA task {{gaincal}}. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='delays.cal', field='0', selectdata=True, timerange='06:43:48~06:48:42', solint='inf', refant='ea04', gaintype='K', gaintable=['antpos.cal','tecim.cal','rq.cal'])<br />
</source><br />
<br />
[[File:Pband_delaycal_5.4.0.png|200px|thumb|right|Figure 4. Delays for each antenna and each polarization. Plot made with CASA task {{plotms}}.]]<br />
The calibration solutions can be plotted using CASA task {{plotms}} (see Figure 4):<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='delays.cal',xaxis='antenna1',yaxis='delay',field='0',coloraxis='corr')<br />
</source><br />
<br />
<br />
Note that any delays deviating by more than 30 nsec should be treated with caution and, if needed, flagged.<br />
<br />
=== Bandpass calibration ===<br />
<br />
Next step is to calibrate the bandpass, which corrects for the frequency dependent gain variations. Before determining the frequency dependent gains, we first calibrate the phases in time, deriving a solution for each integration, to make sure that we can derive accurate bandpass solutions when integrating over a full scan on the bandpass calibrator 3C48. This is done using CASA task {{gaincal}}:<br />
<br />
[[File:396px-Pband BPcal.png|200px|thumb|right|Figure 5. Effects of bandpass calibration on the frequency dependent amplitudes (top) and phases (bottom) across the observing band. (For corrections applied using the CASA task {{applycal}} see Figure 8a below.) The various colors show the different baselines with reference antenna ea04. The plots were made in CASA task {{plotms}} and in the ''Axes'' tab choosing the option ''Data Column: data''.]]<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='bpphase.gcal', field='0', spw='0:250~300', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintype='G', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal'])<br />
</source><br />
<br />
We use these phase solutions to perform the actual bandpass calibration. In this case, we have scans of our bandpass calibrator 3C48 taken at three different times during the observation. By interpolating the bandpass solutions in time, we try to minimize any possible time-varying bandpass effects and improve our spectral dynamic range. Bandpass calibration is performed in CASA by the task {{bandpass}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
bandpass(vis='MG0414_d1_data.ms', caltable='bandpass.cal', field='0', spw='0', solint='inf', combine='', refant='ea04', solnorm=False, bandtype='B', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bpphase.gcal'])<br />
</source><br />
<br />
To interpolate between bandpass scans, it is essential to specify parameter <i>combine=' '</i>. The reason being that the default of <i>combine='scan'</i>, when coupled with the parameter <i>solint='inf'</i>, would cross the scan boundaries to form one single solution for the bandpass. Only when combining <i>solint='inf'</i> with <i>combine=' '</i> is a bandpass solution obtained for each individual scan on 3C48. The bandpass plots can be viewed using CASA task {{plotms}} (see Figure 5).<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='amp',scan='7',correlation='xx,yy', coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='phase',scan='7',correlation='xx,yy', coloraxis='baseline')<br />
</source><br />
<br />
=== Gain calibration ===<br />
<br />
After the bandpass calibration, we perform a phase calibration of our data. Because we apply the bandpass solutions "on-the-fly," we can use the full bandwidth to determine the phase solutions. We will create two different calibration tables to correct for the time-varying phases. First we calibrate the phases on each interval with the CASA task {{gaincal}} parameter of ''solint'' set to ''int'': <br />
<br />
<source lang='python'><br />
# In CASA <br />
gaincal(vis='MG0414_d1_data.ms', caltable='intphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='int', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
</source><br />
<br />
[[File:NewFig6_plot.png|200px|thumb|right|Figure 6. Effects of phase calibration. (For corrections applied using the CASA task {{applycal}} see Figure 8b below.) The various colors show the different baselines with reference antenna ea04.]]<br />
<br />
The sole purpose of generating phase solutions on each 10 second interval is to be able to accurately correct for the time-varying amplitudes as we will see below. This method, however, is not ideal for interpolating the phase solutions in time. For that we do a second phase calibration, this time averaging the signal across the phase calibrator scans with the CASA task {{gaincal}} parameter of <i>solint='inf'</i>:<br />
<br />
<source lang='python'><br />
# In CASA <br />
gaincal(vis='MG0414_d1_data.ms', caltable='scanphase.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='inf', combine='', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal'])<br />
</source><br />
<br />
Note that an alternative approach to deriving phase solutions per scan is to use the CASA task {{smoothcal}} to average the phase corrections obtained every 10s with <i>solin='int'</i> over the full duration of a single scan. Generate a phase plot using CASA task {{plotms}} with the parameters below (see Figure 6):<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='phase',coloraxis='baseline')<br />
</source><br />
<br />
=== Scaling the Amplitude Gains ===<br />
<br />
[[File:NewFig7_plot.png|200px|thumb|right|Figure 7. Effects of amplitude and flux density scale calibration. The green points are the YY products on an offending antenna ea17, which we will flag in our target data. (For corrections applied using the CASA task {{applycal}} see Figure 9 below.)]]<br />
<br />
<br />
We now perform a calibration of the amplitude variations in time. For this, we will apply the phase solution obtained in each 10 second interval via CASA task {{gaincal}}: <br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_data.ms', caltable='amp.gcal', field='0', spw='0', selectdata=False, solint='inf', combine='', refant='ea04', calmode='ap', minsnr=3.0, gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','intphase.gcal'])<br />
</source><br />
<br />
Normally this procedure will only calibrate the relative amplitudes, and the CASA task {{fluxscale}} is required to place the data on an absolute flux scale. However, because our flux-calibrator is also our bandpass and gain calibrator, the absolute flux scale will already be ok, so we can skip this step. Generate an amplitude gain plot using CASA task {{plotms}} (see Figure 7) with the parameters below:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='amp',coloraxis='baseline')<br />
</source><br />
<br />
=== Applying the calibration ===<br />
<br />
We have now finished the calibration of the data. We will apply the calibration tables that were derived from 3C48 to both the calibrator 3C48 itself and our target MG0414+0354. The CASA task {{applycal}} creates a '''corrected''' data column where the calibrated data are stored.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_data.ms', field='0', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], parang=True, calwt=False, applymode='calflagstrict', flagbackup=True)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_data.ms', field='1', gaintable=['antpos.cal','tecim.cal','rq.cal','delays.cal','bandpass.cal','scanphase.gcal','amp.gcal'], gainfield=['','','','0','0','0','0'], calwt=False, applymode='calflagstrict', flagbackup=True)<br />
</source><br />
<br />
Sometimes after calibrating the data it is easier to see any offending antennas or baselines. Replotting with <i>ydatacolumn='corrected'</i> will produce the corrected calibration plots (Figures 8a, 8b). <br />
<br />
<source lang='python'><br />
# In CASA for corrected amplitude and phase vs frequency figure 8a<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='amp',scan='7',ydatacolumn='corrected',correlation='xx,yy', coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', xaxis='frequency',yaxis='phase',scan='7',ydatacolumn='corrected',correlation='xx,yy', coloraxis='baseline')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA for corrected amplitude and phase vs time figure 8b<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy', xaxis='time',yaxis='amp',ydatacolumn='corrected',coloraxis='baseline')<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='phase',ydatacolumn='corrected',coloraxis='baseline')<br />
</source><br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white; vertical-align: top;"<br />
| style="border: 1px solid darkgray;" | [[File:Pband_BPcal_corrected.png|thumb]] || style="border: 1px solid darkgray" | [[File:Pband_phase_cal_corrected.png|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 8a. Effects of bandpass <br />calibration on the frequency <br />dependent amplitudes (top) and <br />phases (bottom) across the <br /> observing band. The plots were <br />made in CASA task [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/using-plotms-to-plot-and-edit-visibilities-and-calibration-tables plotms] and <br />in the ''Axes'' tab choosing option <br />''Data Column: corrected''. || Figure 8b. Effects of phase calibration.<br /> The various colors show the different <br />baselines with reference antenna <br />ea04. The plot was made in CASA task<br /> [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/using-plotms-to-plot-and-edit-visibilities-and-calibration-tables plotms] and in the ''Axes'' tab choosing <br />option ''Data Column: corrected''.<br />
|}<br />
<br />
The calibrated data of the primary calibrator 3C48 show that the YY amplitudes of ea17 are off during the second part of the run. Because calibration solutions are antenna based, to get the best calibration, flag any poor data and re-run the calibration process. A currently known issue is that CASA task {{gaincal}} will not find good solutions if any correlation, including cross-correlation, in the data is completely flagged. In this case, there is no difference between flagging only the YY polarization of ea17 or the entire antenna.<br />
Antenna ea17 will be flagged using CASA task {{flagdata}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea17', action='apply')<br />
</source><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
'''Then run the calibration process again starting with the section on <i>Delay calibration</i>.''' Once you've re-run the calibration, plot the corrected time vs amplitudes with ea17 flagged (see Figure 9):<br />
<br />
{| style="border-spacing: 2px; border: 1px solid darkgray; float:right;"<br />
|- style="background: white; vertical-align: top;"<br />
| style="border: 1px solid darkgray;" | [[File:Pband_ampcal_corrected.png|thumb]]<br />
|- style="vertical-align: top; background: white; border: 1px solid darkgray;"<br />
| Figure 9. Effects of amplitude and <br />flux density scale calibration. The green points <br />(the YY products of antenna ea17) <br />have been flagged out. The plot was <br />made in CASA task [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/using-plotms-to-plot-and-edit-visibilities-and-calibration-tables plotms] and in <br />the ''Axes'' tab choosing option <br />''Data Column: corrected''.<br />
|}<br />
<br />
<source lang='python'><br />
# In CASA for corrected amp vs time with ea17 flagged out figure 9<br />
plotms(vis='MG0414_d1_data.ms',field='0',antenna='ea04', correlation='xx,yy',avgchannel='1e8', xaxis='time',yaxis='amp',ydatacolumn='corrected',coloraxis='baseline')<br />
</source><br />
<br />
Having re-run the calibration steps with ea17 flagged, we now split the calibrated data of our target into a new measurement set, including only the XX and YY polarization products. Using CASA task {{split}} the calibrated target data are placed into the '''data''' column of the new measurement set:<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_data.ms', outputvis='MG0414_d1_calibrated.ms', datacolumn='corrected', field='1', correlation='xx,yy', keepflags=False)<br />
</source><br />
<br />
== Imaging and Self-calibration ==<br />
<br />
We now have a calibrated data set that has all calibration applied except for possible self-calibration. Self-calibration relies on the target itself to better calibrate antenna-based gains (phases and amplitudes) as a function of time. It can be applied to targets that are strong in the continuum or have a strong enough spectral line to derive good phase and, if desired, also amplitude solutions on short (order of minutes) timescales. The procedure relies on an initial model of the target, which you then use iteratively to improve the gains and hence the quality of the image itself (see Figure 10). Here we show the various steps in self-calibrating our data based on the strong continuum emission of MG0414+0354. We refer to a [https://casaguides.nrao.edu/index.php?title=First_Look_at_Self_Calibration dedicated CASA guide] on self-calibration for further details.<br />
<br />
[[File:Pband_sc1.png|400px|thumb|right|Figure 10. Continuum image of MG0414+0354 before (left) and after (right) the first iteration of self-calibration, where we only calibrated the phases. Subsequent iterations may further improve the phases and amplitudes. This image was made in the [https://casa.nrao.edu/casadocs/casa-5.4.0/data-examination-and-editing/2-d-visualization-of-visibility-data-msview viewer], and scaling is the same in both plots.]]<br />
<br />
If strong RFI is present in the target data, it may be necessary to flag your data before proceeding with self-calibration. This is not the case with MG0414+0354.<br />
<br />
We create an initial model of our target field using CASA task {{tclean}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_calibrated.ms', datacolumn='data', imagename='MG0414_d1_cont_R03', spw='0', specmode='mfs', niter=100, threshold='1.5Jy', imsize=[256,256], weighting='briggs', robust=0.3, savemodel='modelcolumn')<br />
</source><br />
<br />
The information in the MODEL column consist of visibilities that present a model of the target field. CASA task {{tclean}} produces a ''.model'' image of the model that you can visualize with {{viewer}} or CASA task {{plotms}}. It is essential that only components that you know for sure represent the continuum in the target field are taken into account in self-calibration. If this is not the case, you probably cleaned too deep and included noise features or artifacts (e.g., from the beam-pattern) in the model. Self-calibration would permanently lock these as features in your data, so this should be avoided. On the other hand, it is equally crucial to ensure that the model is not empty, as this will result in your target being shifted to the phase center.<br />
<br />
Next, we derive phase corrections to this model. Because our target is strong enough, we should get enough S/N per baseline on a 60 sec interval, using CASA task {{gaincal}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_calibrated.ms', caltable='sc1.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='60s', combine='', minsnr=3.0)<br />
</source><br />
<br />
This creates a new phase-calibration table that we call ''sc1.gcal''. We now apply this calibration table to the calibrated data in order to improve the phase-only solutions using CASA task {{applycal}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_calibrated.ms', field='0', gaintable=['sc1.gcal'], interp='linear')<br />
</source><br />
<br />
Because self-calibration is an iterative process, we want to repeat this step again in order to further improve the gain calibration. For that, we first split the calibrated data with the self-calibration solution applied to the DATA column of a new measurement set, again using CASA task {{split}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_calibrated.ms', outputvis='MG0414_d1_sc1.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
Now we can use this newly created measurement set to make a better continuum image and subsequently perform the next round of self-calibration:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc1.ms', datacolumn='data', imagename='MG0414_d1_cont_sc1.R03', spw='0', specmode='mfs', niter=500, threshold='500mJy', imsize=[256,256],weighting='briggs',robust=0.3,savemodel='modelcolumn')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_sc1.ms', caltable='sc2.gcal', field='0', spw='0', refant='ea04', calmode='p', solint='30s', combine='', minsnr=3.0)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_sc1.ms', field='0', gaintable=['sc2.gcal'], interp='linear')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_sc1.ms', outputvis='MG0414_d1_sc2.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc2.ms', datacolumn='data', imagename='MG0414_d1_cont_sc2_R03', spw='0', specmode='mfs', niter=1000, threshold='100mJy', imsize=[256,256],weighting='briggs',robust=0.3,savemodel='modelcolumn')<br />
</source><br />
<br />
Now that we have a good model of the continuum in the field of MG0414+0354, we will attempt a final self-calibration using both the phases and amplitudes via CASA task {{gaincal}} :<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='MG0414_d1_sc2.ms', caltable='sc3ap.gcal', field='0', spw='0', refant='ea04', calmode='ap', solint='30s', combine='', minsnr=3.0)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='MG0414_d1_sc2.ms', field='0', gaintable=['sc3ap.gcal'], interp='linear')<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='MG0414_d1_sc2.ms', outputvis='MG0414_d1_sc3ap.ms', datacolumn='corrected', field='0', keepflags=False)<br />
</source><br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc3ap.ms', datacolumn='data', imagename='MG0414_d1_cont_sc3ap_R03', spw='0', specmode='mfs', niter=1000, threshold='100mJy', imsize=[256,256],weighting='briggs',robust=0.3,savemodel='modelcolumn')<br />
</source><br />
<br />
The output continuum image from the last iteration of CASA task {{tclean}} continues to be an improvement over previous imaging, but also shows that we are not likely to gain further improvement with additional rounds of self-calibration. This indicates that we now have a calibrated data-set of MG0414+0354 that is almost ready for final imaging. <br />
<br />
However, before imaging, we need to flag the target data to remove poor-quality data. First, we run an automated flagging routine using CASA task {{flagdata}} with the parameter <i>mode='rflag'</i>:<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_sc3ap.ms', mode='rflag', datacolumn='data')<br />
</source><br />
<br />
After having ran this automated flagging, we inspect the data via the CASA tasks {{viewer}} or {{plotms}} by displaying ''time'' vs. ''channel'' for the different baselines, or you may display ''time'' vs. ''baseline'' for the different channels. Keep in mind that when the data are properly calibrated, it is easier to recognize low-level imperfections which may have to be flagged to improve the image quality. In this case, {{viewer}} or {{plotms}} show that antennas ea09 and ea14 have significantly higher noise than the other antennas, something which was not picked up by the automated flagging routine. Additionally, for a number of baselines, the data quality looks relatively poor. We flag these antennas and baselines before imaging the data via the CASA {{flagdata}} task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea09', datacolumn='data')<br />
flagdata(vis='MG0414_d1_data.ms', mode='manual', antenna='ea14', datacolumn='data')<br />
</source><br />
<br />
To flag offending individual baselines, in CASA task {{flagdata}} you may use <i>antenna='antenna&antenna(&antenna...)'</i>. <br />
<br />
[[File:Screenshot-Spectral_Profile_-_MG0414_d1_line_sc3ap_vel_R03.image-raster.png|200px|thumb|right|Figure 11. Spectral line profile with ea09, ea14, and ea17 flagged out but re-calibration has not been run.]]<br />
<br />
Flagging data can be an iterative process. Often, the better the data are calibrated, the easier it is to recognize low-level imperfections in the data. To optimize the data reduction, you can script the calibration process, and opt to flag the data also in the original data set (i.e., before calibration), after which you run the entire calibration again.<br />
<br />
When you flag these antennas you must go back and rerun the calibration before the final imaging, starting at the '''<i>Delay calibration</i>''' section. If you rerun the calibration you will either have to rename your ms's created from split during self calibration or delete them in order to redo self calibration as split will not overwrite a ms name. If you do not re-run the calibration sequence and proceed ahead with the final {{tclean}} run then inspect the line-data cube (see below), you will get a plot that should look similar to Figure 11.<br />
<br />
Finally, performing the calibration sequence again after flagging antennas ea09 and ea14, we image the data using the CASA task {{tclean}}:<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='MG0414_d1_sc3ap.ms', datacolumn='data', imagename='MG0414_d1_line_sc3ap_vel_R03', spw='0', specmode='cube', outframe='bary', veltype='optical', restfreq='390.600GHz', niter=100, threshold='1Jy', imsize=[256,256], pbcor=True, weighting='briggs', robust=0.3, savemodel='none')<br />
</source><br />
<br />
[[File:spec_Pband_plot.png|200px|thumb|right|Figure 12. Test-data of the redshifted HI 21cm absorption line on top of the radio continuum in MG 0414+0354; see More, Carilli & Mention 1999 (Apj, 510, 87) for the original detection.]]<br />
<br />
We can inspect the resulting line-data cube in the {{viewer}}. The data cube contains both continuum and line emission, which can optionally be separated using the CASA task {{uvcontsub}}. The HI absorption feature is clearly present (see Figure 12).<br />
<br />
To see your spectral line, open the last image of the source. Go to a channel that shows the source, then zoom in on the source. Click the ''Point marking'' icon, find the brightest pixel in the source and click on it. Now click on the ''spectral profile tool'' button to get the single point profile of the spectrum. Here you will see a spectrum with 2 dips that drop to zero; these dips are a part of the flagging we have done. You will need to zoom in in order to notice any spectral lines due to the auto scaling; zoom in by clicking on the graph and creating a yellow box then release.<br />
<br />
<br />
<!-- --[[User:bemonts|Bjorn Emonts]] <br />
-- 5.0 corrections: [[User:kradford| Karlee Radford]] (2018-03-21)<br />
-- Edits [[User:tperreau| Tony Perreault]] (26 Apr 2018)<br />
-- Edits [[User:Hmedlin | Heidi Medlin]] (27 April 2018)<br />
-- Yet more edits [[User:tperreau| Tony Perreault]] (2 May 2018)<br />
-- And yet more edits [[User:tperreau| Tony Perreault]] (May the 4th be with you 2018)<br />
-- 5.4.0 updates: [User:kradford| Karlee Radford]] (2018-10-11)<br />
--><br />
<br />
{{Checked 5.4.0}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Jupiter:_continuum_polarization_calibration_5.5.0&diff=26277Jupiter: continuum polarization calibration 5.5.02018-12-12T17:25:36Z<p>Emomjian: </p>
<hr />
<div>{| class="wikitable"<br />
|-<br />
| style="background: red; color: white;" | '''DISCLAIMER'''<br />
|-<br />
| This is an advanced reference guide to calibration and imaging of pre-upgrade VLA polarimetric data with CASA 5.3.0. If you are a beginning or novice user, please consult relevant [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials CASA guides] first.<br />
|}<br />
<br />
<br />
<br />
== Data Import ==<br />
<br />
The data set for this tutorial lives [https://casa.nrao.edu/Data/VLA/Planets6cm/planets_6cm.fits here: planets_6cm.fits] (129MB). This is a VLA 6cm dataset that was observed back in 1999 to set the flux scale for calibration of the VLA. Included in the program were observations of the planets, including Jupiter which we will focus on in this tutorial. These are D-configuration data, with resolution of around 14".<br />
<br />
Import the data into CASA. Task <tt>importuvfits()</tt> will read the original AIPS friendly UVFITS format and create a CASA native MS.<br />
<br />
<source lang="python"><br />
importuvfits(fitsfile='planets_6cm.fits', vis='jupiter6cm.demo.ms', antnamescheme='old')<br />
</source><br />
<br />
Check the observation set up and print verbose summary of the observations to the CASA logger<br />
<br />
<source lang="python"><br />
listobs(vis='jupiter6cm.demo.ms')<br />
</source><br />
<br />
The output of this task is fairly long, the abridged version is shown below since we may need some of this information later on during our calibration.<br />
<br />
<pre style="background-color: #fffacd;"><br />
##########################################<br />
##### Begin Task: listobs #####<br />
Observer: FLUX99 Project: <br />
Observation: VLA<br />
Computing scan and subscan properties...<br />
Data records: 2021424 Total elapsed time = 85136.5 seconds<br />
Observed from 15-Apr-1999/23:15:25.0 to 16-Apr-1999/22:54:21.6 (TAI)<br />
<br />
[...]<br />
<br />
Fields: 13<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 A 0137+331 01:37:41.299500 +33.09.35.13400 J2000 0 72946<br />
1 T 0813+482 08:13:36.051800 +48.13.02.26200 J2000 1 227524<br />
2 A 0542+498 05:42:36.137900 +49.51.07.23400 J2000 2 227058<br />
3 T 0437+296 04:37:04.174700 +29.40.15.13600 J2000 3 113424<br />
4 VENUS 04:06:54.109428 +22.30.35.90609 J2000 4 121098<br />
5 A 0521+166 05:21:09.886000 +16.38.22.05200 J2000 5 81258<br />
6 T 1411+522 14:11:20.647700 +52.12.09.14100 J2000 6 243608<br />
7 A 1331+305 13:31:08.288100 +30.30.32.96000 J2000 7 307958<br />
8 MARS 14:21:41.365747 -12.21.49.45444 J2000 8 118570<br />
9 NGC7027 21:07:01.593000 +42.14.10.18600 J2000 9 136082<br />
10 NEPTUNE 20:26:01.136316 -18.54.54.21127 J2000 10 157736<br />
11 URANUS 21:15:42.828572 -16.35.05.59272 J2000 11 99412<br />
12 JUPITER 00:55:34.043951 +04.45.44.70633 J2000 12 114750<br />
<br />
Spectral Windows: (2 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) Corrs <br />
0 none 1 TOPO 4885.100 50000.000 50000.0 4885.1000 RR RL LR LL<br />
1 none 1 TOPO 4835.100 50000.000 50000.0 4835.1000 RR RL LR LL<br />
<br />
Sources: 26<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 0137+331 0 0 0 <br />
0 0137+331 1 0 0 <br />
1 0813+482 0 0 0 <br />
1 0813+482 1 0 0 <br />
2 0542+498 0 0 0 <br />
2 0542+498 1 0 0 <br />
3 0437+296 0 0 0 <br />
3 0437+296 1 0 0 <br />
4 VENUS 0 0 0 <br />
4 VENUS 1 0 0 <br />
5 0521+166 0 0 0 <br />
5 0521+166 1 0 0 <br />
6 1411+522 0 0 0 <br />
6 1411+522 1 0 0 <br />
7 1331+305 0 0 0 <br />
7 1331+305 1 0 0 <br />
8 MARS 0 0 0 <br />
8 MARS 1 0 0 <br />
9 NGC7027 0 0 0 <br />
9 NGC7027 1 0 0 <br />
10 NEPTUNE 0 0 0 <br />
10 NEPTUNE 1 0 0 <br />
11 URANUS 0 0 0 <br />
11 URANUS 1 0 0 <br />
12 JUPITER 0 0 0 <br />
12 JUPITER 1 0 0 <br />
<br />
Antennas 27<br />
[....]<br />
<br />
##### End Task: listobs #####<br />
##########################################<br />
</pre><br />
<br />
<br />
Check also the array configuration and pick a reference antenna. Here, we will use antenna '11' as a reference since it's located in the middle of the array.<br />
<br />
<source lang="python"><br />
plotants(vis='jupiter6cm.demo.ms', figfile='jupiter6cm.demo.ant.png')<br />
</source><br />
<br />
<p style="color: #ff0000;"> <b>Note</b> (for uvfits sourced directly from archive only): CASA 5.4 and earlier versions will not plot the antennas properly. This is to be resolved in future CASA versions.</p><br />
<br />
== Data Inspection and Editing ==<br />
<br />
To inspect and interactively flag bad data use task <tt>plotms()</tt>. <b>Note</b>: it is not recommended to use <tt>plotms()</tt> to interactively flag the new post-upgrade VLA data due to their large volume and large bandwidths. However, the old VLA data are much smaller and <tt>plotms()</tt> is capable of managing it interactively.<br />
<br />
First, inspect the primary flux and polarisation angle calibrator, 1331+305 (3C286). Inspect RR,LL and RL,LR correlations separately due to large amplitude differences between them (when all correlations are plotted together, the RL,LR will seem like bad data due to their much lower amplitudes as compared to RR,LL). Use <tt>Mark Regions</tt> within the <tt>plotms()</tt> GUI to draw boxes around points to flag, and hit <tt>Flag</tt> to flag. <br />
<br />
<source lang="python"><br />
plotms(vis='jupiter6cm.demo.ms', selectdata=True, field='1331+305', correlation='RR,LL', xaxis='uvdist', yaxis='amp')<br />
</source><br />
<br />
Now, inspect the other two sources we are interested in, the polarisation leakage calibrator (0137+331) and our target source (JUPITER).<br />
<br />
<br />
Flagging can also be done non-interactively as per currently recommended CASA practise. Use task <tt>flagdata()</tt> to do so. For example, from inspecting our data in <tt>plotms()</tt> you should have noticed that Antenna 9 (ID=8) in spw 1 is often bad. The bad data on Antenna 9 are in the last 4 scans in spw='1' for the 0137+331 calibrator and for our target source (JUPITER), as well as in the last scan for all antennas in both spws on the target source. We can flag these points with the following <tt>flagdata()</tt> task executions.<br />
<br />
<source lang="python"><br />
flagdata(vis='jupiter6cm.demo.ms', mode='manual', field='0137+331', spw='1', antenna='9', timerange='42:00:00~48:00:00')<br />
flagdata(vis='jupiter6cm.demo.ms', mode='manual', field='JUPITER', spw='1', antenna='9', timerange='16:26:00~22:20:00')<br />
flagdata(vis='jupiter6cm.demo.ms', mode='manual', field='JUPITER', spw='', timerange='21:40:00~22:20:00')<br />
</source><br />
<br />
<br />
There are more data points to flag -- keep flagging until you are happy with the results. The following commands will get rid of most bad data, but depending on how clean the data you want to proceed with, you may still want to inspect them in <tt>plotms()</tt> and flag interactively the reminder of bad datapoints.<br />
<br />
<source lang="python"><br />
flagdata(vis='jupiter6cm.demo.ms',mode='clip',field='1331+305',correlation='RR,LL',clipoutside=False,clipminmax=[0,0.75])<br />
flagdata(vis='jupiter6cm.demo.ms',mode='clip',field='1331+305',correlation='RL,LR',clipoutside=False,clipminmax=[0,0.04])<br />
flagdata(vis='jupiter6cm.demo.ms',mode='clip',field='0137+331',correlation='RR,LL',clipoutside=False,clipminmax=[0,0.55])<br />
flagdata(vis='jupiter6cm.demo.ms',mode='clip',field='0137+331',correlation='RL,LR',clipoutside=False,clipminmax=[0,0.01])<br />
</source><br />
If you are unfamiliar with flagging in CASA, consult detailed topical guide [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Flagging VLA Data].<br />
<br />
<br />
<br />
== Calibration ==<br />
<br />
=== Flux Density Scale ===<br />
<br />
Set the absolute flux density scale, but only for Stokes I at the moment (total flux density model). Our primary flux calibrator here is 1331+305 (3C286). The default model for CASA 5.3+ is 'Perley-Butler 2017'. <br />
<br />
<source lang="python"><br />
setjy(vis='jupiter6cm.demo.ms', field='1331+305', model='3C286_C.im')<br />
</source><br />
<br />
<br />
=== Initial gain calibration ===<br />
<br />
At this stage the data have an overall flux density scaling determined, but full gain solutions aren't there yet. The relevant task is <tt>gaincal()</tt> (analogous to the AIPS task CALIB). <tt>Gaincal()</tt> will produce a separate table with solutions, and we will use appropriate extensions to keep track of what is what.<br />
<br />
<b>NOTE</b>: Since we have only two single-channel continuum spw, we do not want to do <i>bandpass calibration</i> nor solve for <i>delays</i> within spws as is currently done with wide bandwidth post-upgrade VLA data. <br />
<br />
Firstly, generate an antenna zenith-angle dependent VLA gain curve calibration table<br />
<br />
<source lang="python"><br />
gencal(vis='jupiter6cm.demo.ms', caltable='jupiter6cm.demo.gc', caltype='gc')<br />
</source><br />
<br />
Now, solve for antenna gains on 1331+305 and 0137+331, using the generated gain curve table (.gc).<br />
<br />
<source lang="python"><br />
gaincal(vis='jupiter6cm.demo.ms', caltable='jupiter6cm.demo.G', field='1331+305,0137+331', spw='', gaintype='G', calmode='ap', solint='inf', combine='', refant='11', minsnr=3, gaintable=['jupiter6cm.demo.gc'], parang=False)<br />
<br />
# And check the solutions<br />
plotcal(caltable='jupiter6cm.demo.G', xaxis='time', yaxis='amp', subplot=333, iteration='antenna')<br />
plotcal(caltable='jupiter6cm.demo.G', xaxis='time', yaxis='phase', plotrange=[-1,-1,-200,200], subplot=333, iteration='antenna')<br />
</source><br />
<br />
If all looks good, bootstrap the flux density scale of the flux calibrator onto the phase calibrators (CASA's <tt>fluxscale()</tt> task is equivalent to GETJY in AIPS). When executing <tt>fluxscale()</tt>, the calibration table with the extension .G is modified and stored as a new table with the extension .Gflx.<br />
<br />
<source lang="python"><br />
myFluxscale = fluxscale(vis='jupiter6cm.demo.ms', caltable='jupiter6cm.demo.G', fluxtable='jupiter6cm.demo.Gflx', reference='1331+305', transfer='0137+331', incremental=True, append=False, display=False)<br />
</source><br />
<br />
The output is displayed in the logger as well as stored in the myFluxscale python dictionary.<br />
<pre style="background-color: #fffacd;"><br />
Beginning fluxscale--(MSSelection version)-------<br />
Found reference field(s): 1331+305<br />
Found transfer field(s): 0137+331<br />
Flux density for 0137+331 in SpW=0 (freq=4.8851e+09 Hz) is: 5.29665 +/- 0.00449217 (SNR = 1179.09, N = 54)<br />
Flux density for 0137+331 in SpW=1 (freq=4.8351e+09 Hz) is: 5.34890 +/- 0.00176819 (SNR = 3025.07, N = 54)<br />
Fitted spectrum for 0137+331 with fitorder=1: Flux density = 5.32271 +/- 1.70229e-07 (freq=4.86004 GHz) spidx=-0.954124 (degenerate)<br />
Storing result in jupiter6cm.demo.Gflx<br />
Writing solutions to table: jupiter6cm.demo.Gflx<br />
</pre><br />
<br />
<br />
Before proceeding, inspect the flux density calibration and save results to a file.<br />
<br />
<source lang="python"><br />
plotcal(caltable='jupiter6cm.demo.Gflx', xaxis='time', yaxis='amp', showgui=True, figfile='jupiter6cm.demo.Gflx.amp.png')<br />
<br />
plotcal(caltable='jupiter6cm.demo.Gflx', xaxis='time', yaxis='phase', plotrange=[-1,-1,-200,200], showgui=True, figfile='jupiter6cm.demo.Gflx.phase.png')<br />
</source><br />
<br />
<br />
<br />
=== Polarisation calibration ===<br />
<br />
Just as in the step of the initial gain calibration, since our old VLA data have single-channel continuum spws, we do not want to solve for KCROSS delays in our polarisation calibration. Instead, we directly solve for D and X terms.<br />
<br />
<br />
==== Set the polarisation model ====<br />
<br />
First, set the polarisation model for the polarised position-angle calibrator (here 1331+305=3C286 which is also our primary flux calibrator). For polarisation properties of your primary polarisation calibrator see [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol#section-4 VLA Polarimetry Guide].<br />
<br />
<source lang="python"><br />
i0=7.3109 # Stokes I value for spw 0 ch 0<br />
f0=4.8851 # Frequency for spw0 ch0 (note that in our data the 'lower' spw is actually higher frequency)<br />
alpha=log(i0/7.35974932)/log(4.8351/f0) # Values from our setjy() run on Stokes I earlier<br />
c0=0.114 # Fractional polarisation 11.4% for 5GHz<br />
d0=33*pi/180 # Polarisation angle 33deg in radians<br />
<br />
myPolSetjy = setjy(vis='jupiter6cm.demo.ms', field='1331+305', standard='manual', spw='', fluxdensity=[i0,0,0,0], spix=[alpha,0], reffreq=str(f0)+'GHz', polindex=[c0,0], polangle=[d0,0], scalebychan=True, usescratch=False)<br />
</source><br />
<br />
The results are displayed in the CASA logger as well as saved in the myPolSetjy python dictionary<br />
<br />
<pre style="background-color: #fffacd;"><br />
#In CASA<br />
CASA <49>: myPolSetjy<br />
Out[49]: <br />
{'7': {'0': {'fluxd': array([ 7.3109 , 0.33899165, 0.7613877 , 0. ])},<br />
'1': {'fluxd': array([ 7.26237491, 0.33674164, 0.7563341 , 0. ])},<br />
'fieldName': '1331+305'},<br />
'format': "{field Id: {spw Id: {fluxd: [I,Q,U,V] in Jy}, 'fieldName':field name }}"}<br />
</pre><br />
<br />
<br />
==== Solve for the leakage terms (D terms) ====<br />
<br />
Solve for polarization leakage on 0137+331. In this step we solve for the instrumental polarization. We will assume here our calibrator has unknown polarisation (poltype='D+QU' if good parallactic coverage, 'D' otherwise, consult [https://casa.nrao.edu/docs/taskref/polcal-task.html <tt>polcal()</tt> help] for more information on options). <br />
<br />
<source lang="python"><br />
polcal(vis='jupiter6cm.demo.ms', caltable='jupiter6cm.demo.D', field='0137+331', spw='', refant='11', poltype='D+QU', solint='inf', combine='scan',minsnr=3, gaintable=['jupiter6cm.demo.gc', 'jupiter6cm.demo.G'], gainfield=['','0137+331',''])<br />
</source><br />
<br />
Check the solutions<br />
<br />
<source lang="python"><br />
plotcal(caltable='jupiter6cm.demo.D', xaxis='antenna', yaxis='amp', showgui=True, figfile='jupiter6cm.demo.D.amp.png')<br />
<br />
plotcal(caltable='jupiter6cm.demo.D', xaxis='antenna', yaxis='phase', plotrange=[-1,-1,-200,200], showgui=True, figfile='jupiter6cm.demo.D.phase.png')<br />
<br />
plotcal(caltable='jupiter6cm.demo.D', xaxis='antenna', yaxis='snr', showgui=True, figfile='jupiter6cm.demo.D.snr.png')<br />
</source><br />
<br />
<br />
<br />
<br />
==== Solve for the R-L polarisation angle (X term) ====<br />
<br />
The total polarisation is now correct (since we just calibrated the instrumental polarisation, i.e. D terms), but now the R-L phase needs to be calibrated to obtain an accurate polarisation position angle. <br />
<br />
<source lang="python"><br />
polcal(vis='jupiter6cm.demo.ms', caltable='jupiter6cm.demo.X', field='1331+305', spw='', refant='11', poltype='Xf', solint='inf', combine='scan',minsnr=3, gaintable=['jupiter6cm.demo.gc', 'jupiter6cm.demo.G', 'jupiter6cm.demo.D'], gainfield=['','1331+305','',''])<br />
</source><br />
<br />
Check the solutions in CASA logger window<br />
<br />
<pre style="background-color: #fffacd;"><br />
The following calibration term is arranged for solve:<br />
. Xf Jones: table=jupiter6cm.demo.X append=false solint=inf,none refantmode='flex' refant='none' minsnr=3 apmode=AP solnorm=false<br />
For solint = inf, found 2 solution intervals.<br />
Mean position angle offset solution for 1331+305 (spw = 0) = 80.7525 deg.<br />
Mean position angle offset solution for 1331+305 (spw = 1) = 48.9853 deg.<br />
Found good Xf Jones solutions in 2 solution intervals.<br />
</pre><br />
<br />
<!--<p style="color: #ff0000;"><b> CASA 5.3 bug (present from CASA 5.0+; reported as JIRA CAS-11762). </b> For the time being please use CASA <=4.7, split the data set into single SPW (IF) MS files, and calibrate them separately, or just calibrate each spw separately by specifying, in each task, spw='x' where x is whichever spw you want.</p>--><br />
<br />
<b>NOTE</b>: If you are using CASA 4.7 or earlier, you may want to use <tt>poltype='X'</tt> in X-term calculations (Mueller matrices). For CASA 5.0 and later this option will not work (the relevant CASA documentation is being updated), and hence you need to use <tt>poltype='Xf'</tt> (Jones matrices). Although 'Xf' should be used predominantly for large bandwidths, which clearly is not the case here, it can also be used for the single channel old VLA data as long as you make sure the <tt>interp</tt> parameter is set to default (which should be <tt>nearest</tt>).<br />
<br />
=== Apply the calibration ===<br />
<br />
Now that we have derived all the calibration solutions and saved them in the tables of multiple extensions, we need to apply them to the actual data. The following tasks will apply the solution tables to the DATA and write a new column CORRECTED_DATA as it is standard for CASA. Important: make sure you set <tt>parang=True</tt>.<br />
<br />
<br />
<source lang="python"><br />
applycal(vis='jupiter6cm.demo.ms', field='1331+305', spw='', selectdata=False, gaintable=['jupiter6cm.demo.gc', 'jupiter6cm.demo.G', 'jupiter6cm.demo.D','jupiter6cm.demo.X'], gainfield=['','1331+305'], calwt=[False], parang=True)<br />
<br />
applycal(vis='jupiter6cm.demo.ms', field='0137+331', spw='', selectdata=False, gaintable=['jupiter6cm.demo.gc', 'jupiter6cm.demo.G', 'jupiter6cm.demo.Gflx','jupiter6cm.demo.D','jupiter6cm.demo.X'], gainfield=['','0137+331','0137+331'], calwt=[False], parang=True)<br />
<br />
applycal(vis='jupiter6cm.demo.ms', field='JUPITER', spw='', selectdata=False, gaintable=['jupiter6cm.demo.gc', 'jupiter6cm.demo.G', 'jupiter6cm.demo.Gflx','jupiter6cm.demo.D','jupiter6cm.demo.X'], gainfield=['','0137+331','0137+331'], calwt=[False], parang=True)<br />
</source><br />
<br />
<br />
It's a good idea to split the Jupiter target data at this point. We will write the corrected data of our target source to a new single-source MS.<br />
<br />
<source lang="python"><br />
split(vis='jupiter6cm.demo.ms', outputvis='jupiter6cm.demo.JUPITER.split.ms', field='JUPITER', datacolumn='corrected')<br />
</source><br />
<br />
You can use <tt>plotms()</tt> task to look at the split calibrated data.<br />
<br />
<br />
<br />
== Initial imaging (Stokes I) ==<br />
<br />
We will now image and clean the calibrated Jupiter data. In the next step we will perform self-calibration, so here in the initial imaging we will not clean too deeply and we will also save the model in the MS data (with the use of parameter <tt>usescratch=True</tt> in <tt>clean()</tt> or <tt>savemodel</tt> in <tt>tclean()</tt>).<br />
<br />
<b>NOTE</b>: As of CASA 5.3 the recommended task for deconvolution is <tt>tclean()</tt>. However, we found there may be a problem with how <tt>tclean()</tt> handles old, pre-upgrade VLA data. This is currently being investigated, and once resolved, this guide will be updated to demonstrate the <tt>tclean()</tt> use in this data reduction. Meanwhile, please use the standard <tt>clean()</tt> for deconvolution and imaging.<br />
<br />
<!--<br />
<source lang="python"><br />
tclean(vis='jupiter6cm.demo.JUPITER.split.ms', stokes='I', field='', spw='', datacolumn='data', imagename='jupiter6cm.demo.JUPITER.I.clean1', imsize=[288], cell=['3arcsec'], specmode='mfs', gridder='standard', deconvolver='hogbom', weighting='briggs', threshold=1e-04, mask='', niter=500, interactive=True, cycleniter=100, savemodel='modelcolumn')<br />
</source><br />
<br />
The <tt>tclean()</tt> task is new for CASA 5.3. However, if you are performing this data reduction in earlier versions of CASA, or just want to use the standard <tt>clean()</tt> here is how you would run it. Note though, that <tt>clean()</tt> will disappear in the future versions of CASA with <tt>tclean()</tt> becoming the only task to perform this step. Both <tt>tclean()</tt> and <tt>clean()</tt> should give you similar results here. <br />
--><br />
<br />
Since we set mask="" in our cleaning tasks, the interactive GUI will not allow you to do the deconvolution before you draw the mask on at least one plane. When drawing a mask, make sure to extend it to all channels and all polarisations.<br />
<br />
<source lang="python"><br />
clean(vis='jupiter6cm.demo.JUPITER.split.ms', stokes='I', field='', spw='', imagename='jupiter6cm.demo.JUPITER.I.clean1', imsize=[288], cell=['3arcsec'], mode='mfs', psfmode='clark', imagermode='csclean', weighting='briggs', threshold='0.1mJy', mask='', niter=500, interactive=True, npercycle=100, usescratch=True)<br />
</source><br />
<br />
<br />
Inspect the quality of the cleaned images with <tt>imstat()</tt>.<br />
<br />
<source lang="python"><br />
imstat(imagename='jupiter6cm.demo.JUPITER.I.clean1.image', stokes='I', box='216,1,287,72')<br />
</source><br />
<br />
In this initial imaging the deconvolution will stop with the iteration limit since we set it fairly low. We 'cleaned' approximately 4.3 Jy in this initial imaging step, and reached rms of about 1.34 mJy/bm.<br />
<br />
<!-- Either of the tasks 'cleaned' approximately 4.3 Jy in this initial imaging step, and we get rms of about 1.34 mJy/bm (using <tt>clean()</tt>).<!-- and 3.9 mJy/bm (using <tt>tclean()</tt>).--><br />
<br />
== Self-calibration ==<br />
<br />
In the self calibration step we run <tt>gaincal()</tt> that will make use of the newly created MODEL column in our MS file.<br />
<br />
<source lang="python"><br />
gaincal(vis='jupiter6cm.demo.JUPITER.split.ms', caltable='jupiter6cm.demo.JUPITER.split.selfcal1', field='', spw='', gaintype='G', calmode='p', solint='30s', combine='', refant='11', parang=False, append=False, selectdata=False)<br />
</source><br />
<br />
Inspect the solutions<br />
<br />
<source lang="python"><br />
plotcal(caltable='jupiter6cm.demo.JUPITER.split.selfcal1',xaxis='time',yaxis='amp',subplot=231,iteration='antenna')<br />
plotcal(caltable='jupiter6cm.demo.JUPITER.split.selfcal1',xaxis='time',yaxis='phase',subplot=231,iteration='antenna',plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
And apply the calibration to the DATA<br />
<br />
<source lang="python"><br />
applycal(vis='jupiter6cm.demo.JUPITER.split.ms', field='', spw='', selectdata=False, gaintable=['jupiter6cm.demo.JUPITER.split.selfcal1'], gainfield=[''], interp=['nearest'],calwt=[False], applymode='calflag')<br />
</source><br />
<br />
<br />
Now, image the self-calibrated data<br />
<br />
<source lang="python"><br />
clean(vis='jupiter6cm.demo.JUPITER.split.ms', stokes='I', field='', spw='', imagename='jupiter6cm.demo.JUPITER.I.clean2', imsize=[288], cell=['3arcsec'], mode='mfs', psfmode='clark', imagermode='csclean', weighting='briggs', threshold='0.05mJy', mask='', niter=10000, interactive=True, npercycle=100)<br />
</source><br />
<br />
<br />
Check the image statistics<br />
<br />
<source lang="python"><br />
imstat(imagename='jupiter6cm.demo.JUPITER.I.clean2.image', stokes='I', box='216,1,287,72')<br />
</source><br />
<br />
<br />
After one cycle of self-calibration with <tt>clean()</tt> we reached rms=1.10 mJy/bm. But bear in mind that the rms values may differ for you since they will depend on how deeply you cleaned and how good a mask you applied.<br />
<br />
<br />
<br />
== Full Stokes Imaging ==<br />
<br />
At this stage we will perform full Stokes imaging to create image of each Stokes parameter that we will later use to create polarised intensity and angle images.<br />
<br />
<!--<br />
Run <br />
<br />
<source lang="python"><br />
tclean(vis='jupiter6cm.demo.JUPITER.split.ms', stokes='IQUV', field='', spw='', datacolumn='data', imagename='jupiter6cm.demo.JUPITER.IQUV.clean', imsize=[288], cell=['3arcsec'], specmode='mfs', gridder='standard', deconvolver='clarkstokes', weighting='briggs', threshold=5e-05, mask='', niter=10000, interactive=True, cycleniter=100)<br />
</source><br />
<br />
or <br />
--><br />
<source lang="python"><br />
clean(vis='jupiter6cm.demo.JUPITER.split.ms', stokes='IQUV', field='', spw='', imagename='jupiter6cm.demo.JUPITER.IQUV.clean', imsize=[288], cell=['3arcsec'], mode='mfs', psfmode='clarkstokes', imagermode='csclean', weighting='briggs', threshold='0.05mJy', mask='', niter=10000, interactive=True, npercycle=100)<br />
</source><br />
<br />
Inspect the quality of the cleaned images with <tt>imstat()</tt> (do it for each Stokes separately, as otherwise imstat() will return an average of all Stokes). You should be reaching values of about 1.1 mJy/bm (I), 0.62 mJy/bm (Q), 0.90 mJy/bm (U), 0.16 mJy/bm (V).<br />
<br />
<br />
<br />
== Polarisation Intensity and Position Angle Images ==<br />
<br />
[[Image:Jupiter-poli.png|thumb|Polarised intensity (POLI) image of Jupiter with overlaid polarised angle vectors (POLA). ]]<br />
[[Image:Jupiter-oldVLA-stokesI.png|thumb|Total continuum intensity (Stokes I) image of Jupiter with overlaid contours of its polarised intensity (POLI).]]<br />
<br />
We now want to save each Stokes plane as separate images. We can use either the <tt>imsubimage()</tt> or <tt>immath()</tt> tasks to do so. <br />
<br />
Run <br />
<br />
<source lang="python"><br />
imsubimage(imagename='jupiter6cm.demo.JUPITER.IQUV.clean.image',outfile='JUPITER.I.image',stokes='I')<br />
imsubimage(imagename='jupiter6cm.demo.JUPITER.IQUV.clean.image',outfile='JUPITER.Q.image',stokes='Q')<br />
imsubimage(imagename='jupiter6cm.demo.JUPITER.IQUV.clean.image',outfile='JUPITER.U.image',stokes='U')<br />
</source><br />
<br />
or <br />
<br />
<source lang="python"><br />
immath(imagename='jupiter6cm.demo.JUPITER.IQUV.clean.image', mode='evalexpr', stokes='I', expr='IM0', outfile='JUPITER.I.image')<br />
immath(imagename='jupiter6cm.demo.JUPITER.IQUV.clean.image', mode='evalexpr', stokes='Q', expr='IM0', outfile='JUPITER.Q.image')<br />
immath(imagename='jupiter6cm.demo.JUPITER.IQUV.clean.image', mode='evalexpr', stokes='U', expr='IM0', outfile='JUPITER.U.image')<br />
</source><br />
<br />
<br />
<br />
Finally, we want to create the polarization intensity (POLI) and position angle (POLA) images. <br />
<br />
<source lang="python"><br />
immath(imagename=['JUPITER.Q.image','JUPITER.U.image'], mode='poli', outfile='JUPITER.POLI.image')<br />
</source><br />
<br />
<br />
We will now use the <tt>polithresh</tt> parameter when creating the polarisation angle image to get rid of the noise. First check the statistic in the POLI image, and then use it as a threshold while creating the POLA image (in the example below equal to 4 sigma).<br />
<br />
<source lang="python"><br />
imstat(imagename='JUPITER.POLI.image', box='216,1,287,72')<br />
</source><br />
<br />
<source lang="python"><br />
immath(imagename=['JUPITER.Q.image','JUPITER.U.image'], mode='pola', outfile='JUPITER.POLA.image',polithresh='4.5mJy/beam')<br />
</source><br />
<br />
Again, check the statistics of these final images with <tt>imstat()</tt> and/or <tt>viewer()</tt>.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
And that's it. For the tutorial on how to perform analysis, display and manipulation of the polarisation images now, consult the appropriate [https://casaguides.nrao.edu/index.php/VLA_Continuum_Tutorial_3C391-CASA5.0.0 VLA CASA guides].<br />
<br />
<br />
{{Checked 5.3.0}}<br />
<br />
[[Pre-upgrade VLA Tutorials | &#8629; '''Pre-upgrade VLA Tutorials''']]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=26276Karl G. Jansky VLA Tutorials2018-12-12T17:21:07Z<p>Emomjian: </p>
<hr />
<div><br />
Disclaimer: These tutorials provide guidelines to help users become familiar with VLA data reduction in [https://casa.nrao.edu/ CASA]. They offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results demonstrated here are specific to the given data sets. The VLA is an extremely flexible instrument, and the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
''For data reduction of pre-EVLA upgrade VLA data, we provide separate, [https://casaguides.nrao.edu/index.php/Pre-upgrade_VLA_Tutorials pre-upgrade VLA data reduction tutorials]''<br />
<br />
<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== High frequency (36GHz), spectral line data reduction: Carbon Star IRC+10216 ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.4.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 6cm Polarimetry and Continuum Imaging, Mosaicking: Supernova Remnant 3C391 ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.4.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in Stokes I. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== P-band continuum imaging data reduction tutorial: 3C129 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.4.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.4.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.4.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging (plotms, msview)<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.4.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.4.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.4.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Simulation Guide ==<br />
<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/Protoplanetary_Disk_Simulation_-_VLA VLA Protoplanetary Disk Simulation Guide] (CASA 5.4.0)<br />
* This tutorial does not require any download. <br />
<br />
The VLA Protoplanetary Disk Simulation Guide shows examples on how to simulate VLA data based on model images. The tutorial will explain the main simulation tools, apply different parameters to simulations, and compares the results to the VLA exposure calculator. <br />
</blockquote><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 5.1.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.4.0 / 5.4.1 versions ===<br />
* [https://casaguides.nrao.edu/index.php/VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.4.0 IRC+10216 Tutorial (CASA 5.4.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Continuum_Tutorial_3C391-CASA5.4.0 3C391 Continuum Tutorial (CASA 5.4.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0 3C129 P-band Tutorial (CASA 5.4.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414%2B0534_P-band_Spectral_Line_Tutorial_-_CASA_5.4.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.4.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.4.0 Topical Guide: Flagging VLA Data (CASA 5.4.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.4.0 Topical Guide: Imaging VLA Data (CASA 5.4.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.4.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.4.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.4.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.4.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/Protoplanetary_Disk_Simulation_-_VLA-CASA5.4.0 Protoplanetary Disk Simulation - VLA (CASA 5.4.0)]<br />
<br />
<!--<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA5.4.1 VLA CASA Pipeline Guide (CASA 5.4.1)]<br />
--><br />
<br />
<br />
<!-- === CASA 5.3 versions ===<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.3.0 C129 P-band Tutorial (CASA 5.3.0)] <br />
--><br />
<br />
=== CASA 5.0 / 5.1.2 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA5.1.2 VLA CASA Pipeline Guide (CASA 5.1.2)]<br />
<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0&diff=26250VLA Radio galaxy 3C 129: P-band continuum tutorial-CASA5.4.02018-12-03T16:21:58Z<p>Emomjian: </p>
<hr />
<div><!--By [http://www.aoc.nrao.edu/~pjaganna/Site/Home.html Preshanth Jagannathan] & [https://safe.nrao.edu/wiki/bin/view/Main/HuibIntemaWebHome Huib Intema] & Frank Schinzel--><br />
<br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b><p></p> <br />
<b> Note: This guide may take up to 2-days to complete </b><p></p> <br />
If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
<br />
This webpage provides a basic description on how to use CASA to reduce data from the upper part of the new VLA low-band system, also known as P-band, covering roughly 220-480 MHz. The goal is to make a wide-field continuum stokes I image of a typical blank field using the full effective bandwidth. The dataset that will be downloaded in this guide has a size of 26.46 GB and will require a total of ~140 GB of disk space if all generated files are kept. <br />
<br />
''Note: Starting with CASA 5.0 IPython 5.1.0 is used which for interactive CASA sessions. This does not allow to readily paste and execute multiple lines of code like in previous versions of the interpreter. If this is needed at any point in this tutorial, we use "%cpaste" to allow the copy/paste of loops, conditions, etc.. Press "Enter" (or "Return") after "--" at the end, this stops (or exits) the copy/paste prompt and will return you to the CASA prompt.''<br />
<br />
== Obtaining the raw data ==<br />
<br />
For this guide, we'll use some test data that was taken in B-configuration. To get a copy of the raw data, download [https://casa.nrao.edu/Data/EVLA/Pband/P_band_3C129.tgz P_band_3C129.tgz] onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
<br />
Alternatively, the raw data can also be obtained from the NRAO archive, go to the [https://archive.nrao.edu/archive/advquery.jsp NRAO advance archive query page], and enter the following search parameter in the field <i>Archive File ID</i> (leave the rest on default):<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
imag-test-copy.57080.956837025464<br />
</pre><br />
</div><br />
<br />
The search should return a single row for Archive File: imag-test-copy.57080.956837025464, taken on Feb. 27, 2015 (or "15-Feb-27 22:57:52"). Check the box next to it. Next, fill in your email and select the "SDM-BDF dataset (all files)" option. Note, you may find it easier to download a tar file, in which case, you'll want to check the box for Create MS or SDM tar file. Click "Get My Data" and follow the on-screen prompts to reach the next page, where you should click "Retrieve over internet". The next page should report that the data staging is in progress. Wait until you receive an e-mail reporting that your archive data is copied, which should take a few minutes. Download the data onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing <br />
<source lang='bash'><br />
casa<br />
</source><br />
on the (linux) command line. This should start a CASA interactive python (iPython) session, and open a separate log window. The CASA version is reported at startup, both in the python session and the log window.<br />
<br />
== Importing the raw data into CASA ==<br />
<br />
We will begin by importing our data into the measurement set format (CASA standard) from the binary format (SDM-BDF) as downloaded from the archive. We do this by means of the importasdm task. It is important to note that the importevla task has been deprecated.<br />
<br />
<source lang='python'><br />
# In CASA<br />
importasdm(asdm='imag-test-copy.57080.956837025464', vis ='3C129.ms', savecmds=True, outfile='importflags.txt')<br />
<br />
</source><br />
<br />
This task calls the external tool <i>asdm2MS</i> to perform the conversion. Together with the visibility data, also the flag commands from the VLA online system are imported. These flags are not directly applied, but stored in the <i>importflags.txt</i> file and applied later.<br />
<br />
== Preliminary data inspection ==<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms', verbose = True, listfile = '3C129.listobs')<br />
</source><br />
<br />
Alternatively, you can also output to the logger by running the command without the listfile argument. The sample listobs output shown has flag <i>verbose=False</i> for the sake of brevity.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/fschinze/P_band_tutorial/3C129.ms MS Version 2<br />
================================================================================<br />
Observer: Project: uid://evla/sdm/X2<br />
Observation: EVLA(27 antennas)<br />
Data records: 4714983 Total elapsed time = 1422 seconds<br />
Observed from 27-Feb-2015/22:57:54.0 to 27-Feb-2015/23:21:36.0 (UTC)<br />
<br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C147 05:42:36.127300 +49.51.07.17800 J2000 0 1307124<br />
1 NONE 3C129 04:48:58.200000 +45.02.01.00000 J2000 1 3407859<br />
Spectral Windows: (19 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 EVLA_P#A0C0#0 128 TOPO 224.000 125.000 16000.0 231.9375 12 XX XY YX YY<br />
1 EVLA_P#A0C0#1 128 TOPO 240.000 125.000 16000.0 247.9375 12 XX XY YX YY<br />
2 EVLA_P#A0C0#2 128 TOPO 256.000 125.000 16000.0 263.9375 12 XX XY YX YY<br />
3 EVLA_P#A0C0#3 128 TOPO 272.000 125.000 16000.0 279.9375 12 XX XY YX YY<br />
4 EVLA_P#A0C0#4 128 TOPO 288.000 125.000 16000.0 295.9375 12 XX XY YX YY<br />
5 EVLA_P#A0C0#5 128 TOPO 304.000 125.000 16000.0 311.9375 12 XX XY YX YY<br />
6 EVLA_P#A0C0#6 128 TOPO 320.000 125.000 16000.0 327.9375 12 XX XY YX YY<br />
7 EVLA_P#A0C0#7 128 TOPO 336.000 125.000 16000.0 343.9375 12 XX XY YX YY<br />
8 EVLA_P#A0C0#8 128 TOPO 352.000 125.000 16000.0 359.9375 12 XX XY YX YY<br />
9 EVLA_P#A0C0#9 128 TOPO 368.000 125.000 16000.0 375.9375 12 XX XY YX YY<br />
10 EVLA_P#A0C0#10 128 TOPO 384.000 125.000 16000.0 391.9375 12 XX XY YX YY<br />
11 EVLA_P#A0C0#11 128 TOPO 400.000 125.000 16000.0 407.9375 12 XX XY YX YY<br />
12 EVLA_P#A0C0#12 128 TOPO 416.000 125.000 16000.0 423.9375 12 XX XY YX YY<br />
13 EVLA_P#A0C0#13 128 TOPO 432.000 125.000 16000.0 439.9375 12 XX XY YX YY<br />
14 EVLA_P#A0C0#14 128 TOPO 448.000 125.000 16000.0 455.9375 12 XX XY YX YY<br />
15 EVLA_P#A0C0#15 128 TOPO 464.000 125.000 16000.0 471.9375 12 XX XY YX YY<br />
16 EVLA_P#B0D0#16 64 TOPO 310.000 500.000 32000.0 325.7500 15 XX XY YX YY<br />
17 EVLA_P#B0D0#17 1024 TOPO 310.000 62.500 64000.0 341.9688 15 XX XY YX YY<br />
18 EVLA_P#B0D0#18 64 TOPO 342.000 500.000 32000.0 357.7500 15 XX XY YX YY<br />
Antennas: 27 'name'='station'<br />
ID= 0-3: 'ea01'='N12', 'ea02'='N16', 'ea03'='W28', 'ea05'='E20',<br />
ID= 4-7: 'ea06'='N20', 'ea07'='N32', 'ea08'='E28', 'ea09'='W08',<br />
ID= 8-11: 'ea10'='E36', 'ea11'='W12', 'ea12'='N36', 'ea13'='W04',<br />
ID= 12-15: 'ea14'='E08', 'ea15'='E24', 'ea16'='W24', 'ea17'='E04',<br />
ID= 16-19: 'ea18'='W36', 'ea19'='MAS', 'ea20'='N04', 'ea21'='E32',<br />
ID= 20-23: 'ea22'='N24', 'ea23'='E16', 'ea24'='W32', 'ea25'='W20',<br />
ID= 24-26: 'ea26'='E12', 'ea27'='N08', 'ea28'='N28'<br />
<br />
</pre><br />
<br />
You can inspect the output text file <i>3C129.listobs</i> in your favorite text editor (e.g., <i>gedit</i>). When taking some time to familiarize yourself with the output format, you will see that:<br />
* The observation consists of 5 scans, namely:<br />
# 3C147 (hardware setup)<br />
# 3C147 (setting requantizers)<br />
# 3C147 (primary calibrator)<br />
# 3C129 (target)<br />
# 3C129 (target)<br />
* The frequency coverage is 222 - 478 MHz, divided into 16 x 16 MHz spectral windows, each having 128 x 0.125 MHz channels. There are also two spectral windows with 64x0.5 MHz and one spectral window with 1024x0.0625 MHz. <br />
* Visibilities are recorded every 2 seconds in full polarization (4 polarization products labelled XX,XY,YX,YY).<br />
* Of the 28 VLA antennas labelled ea01 - ea028, antenna ea04 is not participating in this observation.<br />
* Source, antenna and spectral window IDs start at zero, but scan IDs start at one.<br />
<br />
<br />
[[File:3C129_pband_plotants.png|200px|thumb|right|Figure 1: plotants figure]] <br />
<br />
The array configuration can be inspected using:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotants(vis='3C129.ms', figfile='3C129_pband_plotants.png')<br />
</source><br />
<br />
== Initial processing steps - Antenna Position, Requantizer Gain, Ionospheric Correction ==<br />
<br />
When importing the data, we saved the flags from the online system to an ASCII text file. This gives us the opportunity to review the flag commands before applying them. For our data set, the flag file is called <i>importflags.txt</i>. Please load this file into your favorite text editor. The bulk of the flag commands refer to times when the VLA is slewing (<i>ANTENNA_NOT_ON_SOURCE</i>) or when the movable secondary reflector of the VLA's Cassegrain system is not in place (<i>SUBREFLECTOR_ERROR</i>). The latter can cause antenna gain variations (amplitude and phase), so it is safest to apply all the flags. The final two rows in the flag file will remove visibilities that are pure zero, and flag antennas that are partly blocked by other antennas (shadowing) which occurs mostly in compact configurations when observing along a VLA arm).<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms', mode='list', inpfile='importflags.txt', action='apply', reason='any', flagbackup=True)<br />
</source><br />
This call, before applying the new flags to the data, created a new flagbackup file called flagdata_1 which can be used by the task <i>flagmanager</i> to restore the state of the measurement set prior applying the flags from <i>importflags.txt</i>.<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
Now that we have applied the import flags, we will correct for the antenna position offsets. Antenna positional errors translates to an error in the measured visibilities and need to be accounted for before we proceed with any of the other calibration steps. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltable='3C129_pband.antpos',caltype='antpos')<br />
</source><br />
<br />
The logger output shows that there are 13 antennas with positional offsets: <br />
<pre><br />
<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea03 : 0.00290 0.00000 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea08 : 0.00000 0.00390 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea10 : -0.00170 0.00000 0.00400<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea11 : 0.00000 0.00160 -0.00170<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea13 : 0.00000 0.00050 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea16 : -0.00180 0.00680 0.00230<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea18 : 0.00470 0.00350 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea21 : -0.00160 0.00000 0.00210<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea23 : 0.00140 0.00200 -0.00150<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea24 : 0.00380 0.00000 -0.00340<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea25 : 0.00330 0.00000 -0.00110<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea26 : 0.00000 0.00190 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea27 : -0.00120 0.00070 0.00000<br />
<br />
<br />
</pre><br />
<br />
=== Ionospheric TEC Corrections ===<br />
[[File:Pband_vla_tecu_new.png|200px|thumb|right|Figure 2: TEC value vs time generated by <i>tec_maps.create()</i>]] <br />
<br />
Low frequency observations are affected by the ionosphere. A delay in the signal path is introduced between the two polarization of light that varies both as a function of time and line of sight (direction dependent). The delay is proportional to the Total Electron Content (TEC) along the line of sight and is inversely proportional to the square of the frequency. GPS measurements at two different frequencies provides us with an estimate of the TEC per square meter. This correction has been implemented in CASA which we shall apply as a calibration table by means of the <i>gencal</i> task. The task requires a TEC map that we will generate utilizing CASA recipes. For more information, see the section on [https://casa.nrao.edu/casadocs/casa-5.0.0/synthesis-calibration/preparing-for-calibration/#title5 ionospheric corrections] in the CASA documentation archives. For some versions of CASA, the first plot to the right (Figure 2) may not be generated if you have already run plotants: to generate this plot, you will need to quit CASA and start it again. If the IONEX data file was not downloaded you might be missing the curl package in the environment you are running CASA in. In this case you can provide the required file manually by downloading: ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/2015/058/igsg0580.15i.Z, (for the tutorial it is also also available in an unzipped version [http://casa.nrao.edu/Data/EVLA/Pband/igsg0580.15i here]) unziping it, and placing it in the directory from which you started CASA. The current implementation in CASA is in the development stage, however since the CASA 5.0 release both the Faraday rotation correction and dispersive delay corrections for the ionosphere are enabled. In previous releases only Faraday rotation corrections are enabled. In this example we will generate the calibration table and apply it throughout calibration. <br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
#<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='3C129.ms',doplot=True)<br />
#[NOTE: If you are using CASA 5.0.0, or earlier this should be: tec_image, tec_rms_image = tec_maps.create(vis='3C129.ms', doplot=True)]<br />
#<br />
gencal(vis='3C129.ms',caltable='3C129_pband.tecim',caltype='tecim',infile=tec_image)<br />
# Spatial plots like shown in Figure 3 can be looked at using the CASA viewer for the TEC values<br />
viewer('3C129.ms.IGS_TEC.im')<br />
# and for TEC rms<br />
viewer('3C129.ms.IGS_RMS_TEC.im')<br />
</source><br />
<br />
[[File:Pband_tec_new.png|200px|thumb|right|Figure 3: TEC map movie generated by the TEC tasks.]] <br />
<br />
A word of caution regarding the TEC map generation. The IGS website updates measurements only two weeks after the date of observation.<br />
<br />
=== Requantizer Gains ===<br />
<br />
The next step will correct the visibility amplitudes for the signal leveling (requantizer gains) that occurs at the inputs of the WIDAR correlator. These levels (per antenna, per polarization, per spectral window) are stored with the measurement set (in the <i>SYSPOWER</i> sub-table). This step is currently not essential, since the levels get set only once at the start of an observation and bandpass calibration will correct for this. But it will make your bandpass plots look better if you have multiple spectral windows. And, more importantly, in general it is possible to trigger the setting of requantizers throughout an observation. This is the case for new observations, when a P-band scan is preceded by a P-band setup scan, which itself is preceded by a scan with a different telescope configuration. <br />
<br />
The correction is done by means of the <i>gencal</i> task, where information in the <i>SYSPOWER</i> sub-table gets translated into a gain table. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltype='rq',caltable='3C129_pband.rq')<br />
</source><br />
<br />
== Dead antennas ==<br />
<br />
Now it is time to have a first visual look at the uncalibrated visibility data. It is important to identify dead antennas / polarizations early on, so we can exclude them from further data processing (which is the most efficient data reduction). A convenient way of doing this is through the <i>plotms</i> task on bright calibrator(s) (see the <i>listobs()</i> output), in our case 3C147 (see Fig. 4a).[[File:3C129_pband_3C147_prebp.png|200px|thumb|right|Figure 4a: <i>plotms</i> output before RFI flagging]]<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY', field='3C147', <br />
plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',iteraxis='baseline', <br />
plotfile='3C129_pband_3C147_prebp.png')<br />
</source><br />
<br />
This will load the <i>plotms</i> window that has queued up all the baselines of antenna ea01 and is currently displaying the first amplitude vs frequency plot with the colors representing the different spectral windows. Note that we are only plotting the XX and YY correlation of the antennas. This is because we expect most of the power in the linear cross correlation products to be in these two correlations. Notice that some spectral windows in particular are badly affected due to radio frequency interference (RFI). This is the issue that we will deal with after identifying dead antennas. If you scroll through the baselines using the green forward button in the <i>plotms</i> viewer, you will notice that the amplitude is particularly low when you encounter ea19. If we make a quick plot of the baselines made by ea19 (put <i>ea19</i> in the antenna field) we can see that there is no power in any of the polarizations indicating this antenna was dead and can be flagged out. It is also good to check for high power levels in cross-hand correlations in comparison to the parallel-hand correlations, which would uncover swapped polarization labels and severely mis-aligned dipoles. At the end of this tutorial some hints on solving for swapped or mislabeled dipole polarizations are given. In addition to the dead antenna, we also make sure that the first two scans are flagged, which were not labelled as setup scans but were used to set the attenuators and requantizers.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms',mode='manual',antenna='ea19')<br />
#<br />
flagdata(vis='3C129.ms',mode='manual',scan='1~2')<br />
</source><br />
<br />
We also noticed sharp RFI peaks during our data examination. To prevent Gibbs ringing, it is best to hanning smooth the data at this juncture before we proceed further with automatic flagging. Now is a good time to get a cup of coffee while hanning smoothing the data. [[File:3C129_pband_3C147_prebp_hanning.png|200px|thumb|right|Figure 4b: <i>plotms</i> output after Hanning smooth and first flagging.]]<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='3C129.ms',outputvis='3C129_pband.ms',datacolumn='data')<br />
</source><br />
<br />
We should replot the primary calibrator to see the effect of hanning smoothing on the data and the RFI (Fig. 4b). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning.png')<br />
<br />
</source><br />
<br />
== Automatic flagging ==<br />
<br />
<br />
It is worthwhile noting at this juncture that the standard P-band setup will ensure that the worst of the RFI is restricted to four of the spectral windows. This particular dataset is an example of higher than average levels of RFI across the band. Before throwing away bad spectral windows by hand, we'll let the automated flaggers in CASA have a go at it. The main task for this is <i>flagdata()</i>. While working on your data, <i>flagdata()</i> produces an abundance of output, not all of which is easy to understand. To help with this, we can get the flag status of our data before and after auto-flagging by running <i>flagdata()</i> in the <i>summary</i> mode. This first call provides the flag statistics of the flagging up to now, before any auto-flagging:<br />
<source lang='python'><br />
summary_1 = flagdata(vis='3C129_pband.ms', mode='summary')<br />
</source><br />
<br />
This returns a python dictionary with flagged versus total visibilities along various axes (antenna, scan, spw, field, correlation, etc.). For example, if we want to know the percentage flagged per scan, run the following (note that the scans may not appear in sorted order):<br />
<source lang='python'><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for id, stats in summary_1[ axis ].iteritems():<br />
print '%s %s: %5.1f percent flagged' % ( axis, id, 100. * stats[ 'flagged' ] / stats[ 'total' ] )<br />
--<br />
</source><br />
<br />
For this example, you will notice that scans 1 and 2 (the dummy scans on 3C147) are 100 percent flagged, which is what we did during the initial flagging.<br />
<br />
We will run <i>flagdata()</i> in the <i>tfcrop</i> mode, which will (per scan, per baseline, per spectral window, per polarization) look for visibility amplitude outliers. It uses a 5-piece polynomial in an attempt to remove any intrinsic bandpass (amplitude) structure (we did not calibrate for bandpass yet). We run the task twice to allow for slightly deeper flagging and separate treatment of the parallel and cross-hand correlations. For the first run, we tell CASA to make a backup of our visibility flag status up to here, giving us an option to restore them if we choose over-aggresive flagging parameters and consequently over-flag our data. The flag backup file name is reported in the log window, and can be found in the <i>3C129_pband.ms.flagversions</i> directory.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=4., freqcutoff=3., maxnpieces=5,<br />
action='apply', display='report', flagbackup=True, combinescans=True, ntime='3600s', correlation='ABS_XY,ABS_YX')<br />
#<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=3., freqcutoff=3., maxnpieces=2,<br />
action='apply', display='report', flagbackup=False, combinescans=True, ntime='3600s', correlation='ABS_XX,ABS_YY')<br />
#<br />
flagdata(vis='3C129_pband.ms', mode='extend')<br />
</source><br />
<br />
As a note of caution: When executing flagdata with <i>mode='extend'</i>, all flags that were ever applied to the dataset will get extended. Repeat execution of this command will incrementally flag more and more around already flagged data.<br />
<br />
Let's get another flag summary to see how much extra data was flagged: <br />
<source lang='python'><br />
# In CASA<br />
summary_2 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
# <br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_2[ axis ].iteritems():<br />
old_stats = summary_1[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning_automatic.png')<br />
<br />
</source><br />
[[File:3C129_pband_3C147_prbp_tfcropped.png|200px|thumb|right|Figure 4c: <i>plotms</i> output after RFI flagging with tfcrop]]<br />
There is quite a bit of RFI that gets missed in RFI-rich spectral windows as can be seen in Fig. 4c. The solution to this is to provide the flagging routines with more contrast between healthy and affected data. For this, we will perform a preliminary bandpass calibration to take out the bandpass shape. We will do some coarse preliminary calibration and apply it to the calibrator before flagging for RFI once more.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.G0', gaintype='G', calmode='p', solint='int', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim'])<br />
#<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K0', gaintype='K', solint='inf', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0'])<br />
#<br />
bandpass(vis='3C129_pband.ms', caltable='3C129_pband.B0', solint='inf', field='3C147',refant='ea09', minsnr=2.0,<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0'])<br />
#<br />
applycal(vis='3C129_pband.ms', field='3C147', applymode='calflagstrict',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0','3C129_pband.B0'] )<br />
<br />
</source><br />
<br />
On the command line you should see a notification stating: "FJones (iononsphere) apply now provisionally includes disp. delay." which indicates that both Faraday rotation as well as dispersive delay corrections for the ionosphere are applied.<br />
<br />
[[File:3c147_postbandpass_rflag.png|200px|thumb|right|Figure 4d: <i>plotms</i> output after RFI flagging with <i>mode='rflag'</i>]]<br />
<br />
We will now flag the corrected data column that contains the coarsely calibrated visibilities which provide better contrast to the flagging algorithms to remove the RFI present. We will begin by running the <i>RFLAG</i> algorithm in the task <i>flagdata</i>. We will again use summary mode to see how much more of the observed data was flagged.<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
summary_3 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
#<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_3[ axis ].iteritems():<br />
old_stats = summary_2[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C147_postbandpass_rflag.png', ydatacolumn='corrected')<br />
</source><br />
<br />
Taking another visual look at the data using <i>plotms</i>, we examine the effect of the automated flagging on the calibrator (Fig. 4d). We find that most spectral windows are RFI free and that about 60% of the data was flagged. We typically expect between 40-60% of flagging due to interference in a significant fraction of the band. In this example, a more careful RFI flagging approach is likely to recover more usable spectrum.<br />
<br />
==Flux Density Calibration==<br />
The initial flagging has cleaned up most of the stray RFI across the band. This allows us to proceed with the actual calibration of the data. Before we get to the calibration tables, it's essential to do the flux density calibration of our calibrator 3C147. This is done by the <i>setjy</i> task. Before we run the task we first clear the preliminary calibration that was carried out to enable better flagging. We do that by running the <i>clearcal</i> task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
clearcal(vis='3C129_pband.ms')<br />
#<br />
setjy(vis='3C129_pband.ms', standard='Scaife-Heald 2012', field='3C147')<br />
</source><br />
<br />
== Delay and bandpass calibration ==<br />
<br />
First, we use the full bandwidth on a single scan on the primary calibrator to determine a single delay per antenna, per polarization. This will determine a single (approximate) phase slope across frequency, mainly caused by propagation effects in the (time-variable) ionosphere, cable length differences, and electronics in the signal paths from antenna feeds to correlator. Note that for a single short (5-10 minutes) scan on the calibrator, we can get away with solving for a time-invariant delay per antenna, per polarization. We will use scan 3 on 3C147.<br />
[[File:3C147_init_delay.png|200px|thumb|right|Figure 5: Initial per antenna delays derived by <i>gaincal</i>.]]<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K1', field='3C147', solint='inf', refant='ea09', gaintype='K',<br />
gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq'],parang=True)<br />
#<br />
plotcal(caltable='3C129_pband.K1', xaxis='antenna', yaxis='delay', markersize=2.0, <br />
plotrange=[0,30,-50.,50.],figfile='3C147_delay.png')<br />
</source><br />
<br />
Since we picked ea09 as our reference antenna, the delays for this antenna ID (=0) are arbitrarily set to zero. All other delays should be within 30 nanosec or so (which they should be for this data set). Larger values should be treated with suspicion, possibly indicating a problem with the antenna or polarization. We will use the bandpass calibration to verify this in the next step.<br />
<br />
To determine the bandpass calibration, we use the same source and scan and apply the delay calibration before solving for the bandpass. Note that we request a minimum SNR of 3, which will make the solve fail for the worst, but not all, channels.<br />
<source lang='python'><br />
# In CASA<br />
bandpass( vis='3C129_pband.ms', caltable='3C129_pband.B1', field='3C147', solint='inf', refant='ea09', minsnr=3.0,<br />
parang = True, gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq','3C129_pband.K1'],<br />
interp=['','','','nearest,nearestflag'])<br />
<br />
</source><br />
<br />
After solving, we inspect the bandpass calibration amplitudes and phase, and flag any obvious residual outliers. Outliers may be found above or below the average bandpass curves, and tend to arise in the same channel (frequency) ranges for all antennas. Don't spend more than 1 minute per antenna for this tutorial and don't worry about missing some bad points; these bad channels will likely get flagged in a later stage anyway. Note that the bandpass amplitudes (Fig. 6) and phases may differ between polarizations of the same antenna. Also note that the bandpass phases across frequency should not have an overall gradient (this should be removed by the delay calibration). Also note that bandpass phases may wrap around from +/-180 to -/+180 degrees. If that is problematic, re-run the phase <i>plotcal()</i> with <i>plotrange = [ 200.,500.,0.,360. ]</i>. Also note that there are no bandpass solutions for antennas ea19 and ea28, and only a few solutions for ea14.<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='amp', iteration='antenna', markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='phase', iteration='antenna', plotrange=[ 200.,500.,-180.,180. ], markersize = 2.0)<br />
</source><br />
{|<br />
| [[File:3C147_bp0_ea01.png|200px|thumb|right|Figure 6a: Initial bandpass amplitudes for antenna ea01.]]<br />
| [[File:3C147_bp0_phase_ea01.png|200px|thumb|right|Figure 6b: Initial bandpass phase for antenna ea01.]]<br />
|}<br />
<br />
Flagging is made easier by maximizing the plotcal() window. We don't set the scale for plotting the bandpass amplitudes, which is convenient when flagging outliers as it will auto-rescale. While plotting the bandpass phases we do fix the scale, as it provides a better feeling for the relevant magnitude of phase outliers. Flagging of outliers is done using the <i>Mark Region</i> and <i>Flag</i> buttons. For more information on interactive flagging, see the topical guide [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Flagging VLA Data in CASA]. We are using this mechanism to efficiently flag data manually, since it is done per antenna rather than per baseline. These 'flags' will not be permanent until we call <i>applycal()</i> later on, but are effective while applying the calibration on the fly, as we will do in subsequent calibration steps.<br />
<br />
== Gain calibration ==<br />
<br />
With the delay and bandpass calibration in place, we will now look more closely at the time-variable behaviour of the VLA. We use the same scan on our primary calibrator, 3C147, over the full effective bandwidth to determine gain calibrations: one complex value per antenna, per polarization, per integration time. The delay and bandpass tables are applied on the fly.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal( vis='3C129_pband.ms', caltable='3C129_pband.G1', field='3C147', solint = 'int',<br />
refant = 'ea09', minsnr = 3.0, gaintype = 'G', calmode = 'ap',<br />
gaintable = ['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim', '3C129_pband.K1', '3C129_pband.B1' ],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag' ], parang = True )<br />
<br />
</source><br />
{|<br />
| [[File:3C147_Gamp_ea01.png|200px|thumb|right|Figure 7a: Initial gain amplitude solutions for antenna ea01.]]<br />
| [[File:3C147_Gphase_ea01.png|200px|thumb|right|Figure 7b: Initial gain phase solutions for antenna ea01.]]<br />
|}<br />
<br />
Similar to the bandpass flagging, we will flag outlier gain solutions in both amplitude and phase. For the amplitudes, the final scatter around an average close to one should be a few percent (see Fig. 7). Note antennas ea08, ea09 have particularly noisy solutions. Remember that the gain solutions for both polarizations can be flagged together, as we will lose the equivalent data of the surviving polarization anyway.<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='amp', iteration='antenna', markersize=2.0 )<br />
#<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='phase', iteration='antenna', <br />
plotrange=[ None,None,-180.,180.], markersize=2.0)<br />
</source><br />
<br />
Note that when plotting gains against time, it is convenient to use auto-scaling of the time axis by putting <i>None</i> in the first two fields of <i>plotrange</i>.<br />
<br />
Next, we will prepare a smoothed and interpolated version of the gain calibration table, which will be applied later to the target field data. This prevents flagging of target field data when one of the edges of a calibrator scan is flagged for one or more antennas. The result can be inspected with <i>plotcal</i>.<br />
<source lang='python'><br />
# In CASA<br />
smoothcal(vis='3C129_pband.ms', tablein='3C129_pband.G1', caltable='3C129_pband.Gs1', smoothtype='median', smoothtime = 60.*60.)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='amp', iteration='antenna', plotrange=[ None,None,0.8,1.2 ], markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='phase', iteration='antenna', plotrange=[ None,None,-10.,10. ], markersize=2.0)<br />
<br />
</source><br />
<br />
== Transfer of calibrations to the target field ==<br />
<br />
So far we have used one scan on the primary calibrator 3C147 to derive various calibration tables. These calibrations will now be applied to all scans of all sources in our measurement set, which includes 3C147 itself and our target field 3C129. As described above, the task <i>applycal()</i> creates a <i>CORRECTED_DATA</i> column in which the calibrated visibilities get stored. This may take a while.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='3C129_pband.ms', parang=True, applymode='calflagstrict', flagbackup=True, gaintable=['3C129_pband.antpos','3C129_pband.rq',<br />
'3C129_pband.tecim','3C129_pband.K1','3C129_pband.B1','3C129_pband.Gs1'],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag', 'nearest,nearestflag'])<br />
</source><br />
<br />
Now we split off the calibrated target field data, meaning that the visibilities of source 3C129 get copied from the <i>CORRECTED_DATA</i> column to the <i>DATA</i> column of a new measurement set. This is convenient for further processing. <br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='3C129_pband.ms', outputvis='3C129_pband_target.ms', datacolumn='corrected', field='3C129')<br />
</source><br />
<br />
It is also helpful to perform additional flagging for any residual RFI left behind in our target after all our calibration steps have been completed and we begin the imaging of our data.<br />
<br />
<source land='python'><br />
flagdata(vis='3C129_pband_target.ms', mode='rflag')<br />
flagdata(vis='3C129_pband_target.ms', mode='extend')<br />
</source><br />
<br />
<br />
== Statwt ==<br />
Before we go ahead and image our target field. We would like to weight any residual low level RFI down. This is done using the task <i>statwt</i>. The task runs through the data and computes the rms of the data and weighting down outliers. This makes a big impact for imaging of a target in the presence of low level residual RFI. For our purposes we will bin the data in a time interval larger than the integration time and run statwt as follows.<br />
<br />
<source lang='python'> <br />
statwt(vis='3C129_pband_target.ms', datacolumn='data', timebin=30.)<br />
</source><br />
<br />
With data weights for outliers being set we will now proceed with the imaging and self-calibration of our data.<br />
==Imaging==<br />
<br />
At this point, we're ready to make a first image of our target field. Imaging in CASA is done by means of the <i>tCLEAN</i> task. The task implements algorithms for wide field imaging, such as W-Projection (Cornwell et al. 2008 http://arxiv.org/abs/0807.4161) and Multi Term Multi Frequency Synthesis ( Rau et al. 2011 http://arxiv.org/abs/1106.2745) both of which we will utilize to make our initial image. Even though our source of concern, the radio galaxy, lies in the centre of our field. The observations were carried out in B configuration resulting in an effective maximum resolution of approximately 5x5 arcsecs. To effectively model the source spectral index in the sky, we will utilize the MTMFS algorithm and use the W-Projection algorithm to make a wide field image (The full beam at P Band is about 3 degrees in diameter). So, going by these requirements, we can now compute the required cell and image size. Consulting the resolution guide of NRAO science page, we can see that the expected HPBW for B - Configuration, P -Band Observation is 18.5 arcsec. So, we can sample it well by using a 5 arcsec cellsize. Having done that, if we decide to make a wide field image to account for all the point sources we will set the image size parameter to 4096 pixels. To enable the wide-field algorithm, we set the <i>gridder='wproject'</i>: this invokes the W-Projection algorithm, upon which we set the number of W-Projection planes to be 128. We set the deconvolution mode as Multi Term Multi Scale Multi Frequency Synthesis using the <i>deconvolver='mtmfs'</i> parameter along with the number of Taylor terms to be considered during imaging to be 2 and choosing three different scales to model the point source and extended emission in our field well, we do this by setting <i> scales=[0,20,30]</i>. This allows for the source spectral variation to be modeled by a second order polynomial. We launch the interactive tclean process, but only for <i>spw='3~8'</i> to begin with. Note, running the first clean call will take at least ~10 min to process.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_initial_clean', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30],<br />
pblimit=0.01, savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
</source><br />
<br />
{|<br />
| [[File:3C129_radio_gal_residual.png|200px|thumb|right|Figure 10a: Resulting residuals of 3C129 after first call of interactive <i>clean</i>.]]<br />
| [[File:3C129_radio_gal_clean.png|200px|thumb|right|Figure 10b: Resulting image of 3C129 after completing the first <i>clean</i>.]]<br />
|}<br />
The <i>tCLEAN</i> command launches an interactive session after a 100 iterations of clean and produces a wide field map with two sources at the center and a lot of bright sources far out in the field. As it is a snapshot image, the bright sources have significant side lobes and so tight clean boxing can help. This can done in the interactive viewer interface that pops up. If we proceed with interactive clean with subsequent steps to keep boxing out the strong sources that pop up in the image, we finally see the extended emission from the target radio galaxy begin to emerge. Continue boxing and cleaning to ensure that the residuals of the boxed cleaning look noise like (~10000 clean iterations). We now stop the interactive task and look at the final image it produced.<br />
<br />
<source lang='python'><br />
# In CASA<br />
viewer('3C129_initial_clean.image.tt0')<br />
</source><br />
<br />
The recovered structure at the center of the map should resemble what is shown in Fig. 10b. If the image looks worse then boxing of bright features in the map needs to be done more carefully. Note the image has some sources still showing strong side lobes and imaging artifacts around them. We expected this as we have carried out phase calibration only on our flux calibrator and have just transferred the solutions over to our target field. Since we used the <i>usescratch=True</i>, the MODEL_DATA column in the measurement set now contains the initial image model, which we will self calibrate against to produce a better image. Also in clean, do notice that the spw's utilized are the cleanest spectral windows that are totally RFI free.<br />
<br />
==Self Calibration ==<br />
<br />
We now proceed to compute gain phase solutions for our target field using the <i>gaincal</i> task as the first step in self-calibration.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScG0', field='3C129', solint='inf', refant='ea09', <br />
spw='3~8',minsnr=3.0, gaintype='G', parang=True, calmode='p')<br />
<br />
bandpass( vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScB0', field='3C129', solint='inf', refant='ea09', minsnr=3.0, spw='3~8',<br />
parang = True, gaintable=['3C129_pband_target.ScG0'],<br />
interp=['','','','nearest,nearestflag'])<br />
applycal(vis='3C129_pband_target.ms', gaintable=['3C129_pband_target.ScG0','3C129_pband_target.ScB0'], spw='3~8', applymode='calflagstrict')<br />
</source><br />
<br />
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the <i>tCLEAN</i> command once again.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_clean_sc0', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30],<br />
pblimit=0.01,savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
<br />
viewer('3C129_clean_sc0.image.tt0')<br />
</source><br />
[[File:3C129_radio_gal_cselfcal0.png|200px|thumb|right|Figure 11: Resulting image of 3C129 after completing a first selfcal.]]<br />
<br />
On boxing and cleaning, we already notice that the imaging artifacts have reduced significantly. We also see that the target source appears to contain more structure and a greater amount of flux. Further self calibration iterations involving an amplitude & phase gain calibration, and target bandpass calibration, are all possible steps that can be explored. <br />
<br />
== Appendix: Some P-band data issues you may want to know about ==<br />
<br />
Unfortunately, there are a few known issues with JVLA P-band observing. Most problems were discovered during the P-band commissioning period and have been fixed for newer data sets, but the archive keeps part of this history alive. Here we will go over some of those issues, and (if possible) provide ways of fixing them.<br />
<br />
=== Polarization labeling ===<br />
<br />
For a long period, P-band feeds have been labelled as being circular (R and L), while they are linear (X and Y). It is still possible to generate data with incorrect polarization labeling, however for standard observations this should not be the case anymore. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem. This may seem mostly harmless, but does make a difference for polarization calibration. The following (non-standard) task will check and fix this (and another related problem of swapped polarizations):<br />
<br />
Download [http://casa.nrao.edu/Data/EVLA/Pband/casa_vla_lowband.tar.gz this tarball] in your CASA working directory, open a shell there, and type the following commands:<br />
<br />
<source lang='bash'><br />
tar -xzvf casa_vla_lowband.tar.gz<br />
<br />
buildmytasks<br />
</source><br />
<br />
Now go back to your CASA session and type:<br />
<br />
<source lang='python'><br />
# In CASA<br />
execfile('mytasks.py')<br />
#<br />
fixlowband(vis='TSUB0001_pband_prehs.ms')<br />
</source><br />
<br />
Re-run the <i>listobs()</i> task to check if the fix was correctly applied.<br />
<br />
=== Swapped polarizations ===<br />
<br />
Throughout the whole history of the new VLA low-band system, even up to this day, there have been mistakes in the cabling of the full signal chain. This results in that some antennas have the X-polarization signal come in as Y and vice versa. The way to notice this in visibility data is that for baselines with one swapped antenna most power will be in the cross-hand correlations (XY and YX) rather than the parallel-hand correlations (XX and YY). Once antennas with this feature are manually identified, a contributed task <i>swappol()</i> is available to fix this problem (see under polarization labelling).<br />
<br />
=== Double data descriptor entries ===<br />
<br />
The data description table is part of the measurement set, and provides a link between the recorded visibilities, the spectral window information and the polarization information. For some data sets, the data description table contains two entries for each P-band spectral window, one pointing towards a circular (RL) polarization definition, and one pointing towards a linear (XY) polarization definition. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem.<br />
<br />
=== Continuous Radio Frequency Interference (RFI) ===<br />
<br />
The wide bandwidth of the low-band receiver is (unfortunately) guaranteed to contain significant amounts of RFI. There are a few RFI sources that are active all the time, and are visible in all array configurations. The default P-band observing setup of 16 x 16 MHz tries to capture as much of the continuous RFI in as few spectral windows as possible, allowing for a simple RFI mitigation strategy in which these spectral windows (spws 1,2 and 9,10, but possibly more) can be immediately flagged.<br />
<br />
=== Two spectral window setup ===<br />
<br />
In the early commissioning period, the default setup for P-band observing was to use 2 spectral windows of 1024 channels each to cover 256 MHz of bandwidth from 230 - 486 MHz. However, it was noticed that strong, narrow-band RFI events were causing data to be lost for the whole spectral window in which they occured. To make the system more robust against data loss from these events, the frequency range was divided into 16 x 16 MHz, and shifted downwards by 6 MHz to capture ever-present RFI into as few spectral windows as possible. If your data has only two P-band spectral windows, please be aware that a higher data loss due to RFI is possible. There is no way to repair this.<br />
<br />
=== Bandpass ripples ===<br />
<br />
Due to signal reflections in cables and within the VLA dish, sinusoidal amplitude and phase modulations are always present in P-band data. This is most easily seen in bandpass calibration plots of amplitude versus frequency. In some cases, mostly due to cable & connector problems, these modulations can be very strong (~up to 50 percent of the average amplitude level). These modulations tend not to vary over the duration of an observation and can therefore be removed through bandpass and polarization calibration. If they are found to be variable (e.g., by inspecting bandpass solutions for separate calibrator scans), the offending antenna / polarization should be flagged.<br />
<br />
=== High cross-polarization ===<br />
<br />
On each antenna, the P-band feed (dipole) is visually aligned with respect to the primary focus support legs. This is normally done within 5-10 degrees accuracy. On rare occasions the dipole on one antenna has accidentally rotated to much larger angles because the locking bolt on the back of the feed was not completely tightened. The result is a high cross-polarization in the baseline visibilities that include this antenna. If this is the case, it is safest to flag this data, since there is a possibility that the dipole has rotated during the observations.<br />
<br />
=== Instrumental Polarization Calibration ===<br />
<br />
At present polarization P-band observations are in RSRO mode only. While polarization leakage calibration has been highlighted in the past reports it does not directly impact imaging in Stokes I and hence has been dropped from this casaguide.<br />
<br />
== Useful links ==<br />
<br />
* [http://www.aoc.nrao.edu/lbr The EVLA Low-Band Receiver System Status & User Pages]<br />
* [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/topical-guides/vlofreq VLA Very Low Frequency Observing]<br />
<br />
<br />
{{Checked 5.4.0}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0&diff=26246VLA Radio galaxy 3C 129: P-band continuum tutorial-CASA5.4.02018-12-01T23:31:34Z<p>Emomjian: </p>
<hr />
<div><!--By [http://www.aoc.nrao.edu/~pjaganna/Site/Home.html Preshanth Jagannathan] & [https://safe.nrao.edu/wiki/bin/view/Main/HuibIntemaWebHome Huib Intema] & Frank Schinzel--><br />
<br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b><p></p> <br />
<b> Note: This guide may take up to 2-days to complete </b><p></p> <br />
If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
<br />
This webpage provides a basic description on how to use CASA to reduce data from the upper part of the new VLA low-band system, also known as P-band, covering roughly 220-480 MHz. The goal is to make a wide-field continuum stokes I image of a typical blank field using the full effective bandwidth. The dataset that will be downloaded in this guide has a size of 26.46 GB and will require a total of ~140 GB of disk space if all generated files are kept. <br />
<br />
''Note: Starting with CASA 5.0 IPython 5.1.0 is used which for interactive CASA sessions. This does not allow to readily paste and execute multiple lines of code like in previous versions of the interpreter. If this is needed at any point in this tutorial, we use "%cpaste" to allow the copy/paste of loops, conditions, etc.. Press "Enter" (or "Return") after "--" at the end, this stops (or exits) the copy/paste prompt and will return you to the CASA prompt.''<br />
<br />
== Obtaining the raw data ==<br />
<br />
For this guide, we'll use some test data that was taken in B-configuration. To get a copy of the raw data, download [https://casa.nrao.edu/Data/EVLA/Pband/P_band_3C129.tgz P_band_3C129.tgz] onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
<br />
Alternatively, the raw data can also be obtained from the NRAO archive, go to the [https://archive.nrao.edu/archive/advquery.jsp NRAO advance archive query page], and enter the following search parameter in the field <i>Archive File ID</i> (leave the rest on default):<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
imag-test-copy.57080.956837025464<br />
</pre><br />
</div><br />
<br />
The search should return a single row for Archive File: imag-test-copy.57080.956837025464, taken on Feb. 27, 2015 (or "15-Feb-27 22:57:52"). Check the box next to it. Next, fill in your email and select the "SDM-BDF dataset (all files)" option. Note, you may find it easier to download a tar file, in which case, you'll want to check the box for Create MS or SDM tar file. Click "Get My Data" and follow the on-screen prompts to reach the next page, where you should click "Retrieve over internet". The next page should report that the data staging is in progress. Wait until you receive an e-mail reporting that your archive data is copied, which should take a few minutes. Download the data onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing <br />
<source lang='bash'><br />
casa<br />
</source><br />
on the (linux) command line. This should start a CASA interactive python (iPython) session, and open a separate log window. The CASA version is reported at startup, both in the python session and the log window.<br />
<br />
== Importing the raw data into CASA ==<br />
<br />
We will begin by importing our data into the measurement set format (CASA standard) from the binary format (SDM-BDF) as downloaded from the archive. We do this by means of the importasdm task. It is important to note that the importevla task has been deprecated.<br />
<br />
<source lang='python'><br />
# In CASA<br />
importasdm(asdm='imag-test-copy.57080.956837025464', vis ='3C129.ms', savecmds=True, outfile='importflags.txt')<br />
<br />
</source><br />
<br />
This task calls the external tool <i>asdm2MS</i> to perform the conversion. Together with the visibility data, also the flag commands from the VLA online system are imported. These flags are not directly applied, but stored in the <i>importflags.txt</i> file and applied later.<br />
<br />
== Preliminary data inspection ==<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms', verbose = True, listfile = '3C129.listobs')<br />
</source><br />
<br />
Alternatively, you can also output to the logger by running the command without the listfile argument. The sample listobs output shown has flag <i>verbose=False</i> for the sake of brevity.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/fschinze/P_band_tutorial/3C129.ms MS Version 2<br />
================================================================================<br />
Observer: Project: uid://evla/sdm/X2<br />
Observation: EVLA(27 antennas)<br />
Data records: 4714983 Total elapsed time = 1422 seconds<br />
Observed from 27-Feb-2015/22:57:54.0 to 27-Feb-2015/23:21:36.0 (UTC)<br />
<br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C147 05:42:36.127300 +49.51.07.17800 J2000 0 1307124<br />
1 NONE 3C129 04:48:58.200000 +45.02.01.00000 J2000 1 3407859<br />
Spectral Windows: (19 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 EVLA_P#A0C0#0 128 TOPO 224.000 125.000 16000.0 231.9375 12 XX XY YX YY<br />
1 EVLA_P#A0C0#1 128 TOPO 240.000 125.000 16000.0 247.9375 12 XX XY YX YY<br />
2 EVLA_P#A0C0#2 128 TOPO 256.000 125.000 16000.0 263.9375 12 XX XY YX YY<br />
3 EVLA_P#A0C0#3 128 TOPO 272.000 125.000 16000.0 279.9375 12 XX XY YX YY<br />
4 EVLA_P#A0C0#4 128 TOPO 288.000 125.000 16000.0 295.9375 12 XX XY YX YY<br />
5 EVLA_P#A0C0#5 128 TOPO 304.000 125.000 16000.0 311.9375 12 XX XY YX YY<br />
6 EVLA_P#A0C0#6 128 TOPO 320.000 125.000 16000.0 327.9375 12 XX XY YX YY<br />
7 EVLA_P#A0C0#7 128 TOPO 336.000 125.000 16000.0 343.9375 12 XX XY YX YY<br />
8 EVLA_P#A0C0#8 128 TOPO 352.000 125.000 16000.0 359.9375 12 XX XY YX YY<br />
9 EVLA_P#A0C0#9 128 TOPO 368.000 125.000 16000.0 375.9375 12 XX XY YX YY<br />
10 EVLA_P#A0C0#10 128 TOPO 384.000 125.000 16000.0 391.9375 12 XX XY YX YY<br />
11 EVLA_P#A0C0#11 128 TOPO 400.000 125.000 16000.0 407.9375 12 XX XY YX YY<br />
12 EVLA_P#A0C0#12 128 TOPO 416.000 125.000 16000.0 423.9375 12 XX XY YX YY<br />
13 EVLA_P#A0C0#13 128 TOPO 432.000 125.000 16000.0 439.9375 12 XX XY YX YY<br />
14 EVLA_P#A0C0#14 128 TOPO 448.000 125.000 16000.0 455.9375 12 XX XY YX YY<br />
15 EVLA_P#A0C0#15 128 TOPO 464.000 125.000 16000.0 471.9375 12 XX XY YX YY<br />
16 EVLA_P#B0D0#16 64 TOPO 310.000 500.000 32000.0 325.7500 15 XX XY YX YY<br />
17 EVLA_P#B0D0#17 1024 TOPO 310.000 62.500 64000.0 341.9688 15 XX XY YX YY<br />
18 EVLA_P#B0D0#18 64 TOPO 342.000 500.000 32000.0 357.7500 15 XX XY YX YY<br />
Antennas: 27 'name'='station'<br />
ID= 0-3: 'ea01'='N12', 'ea02'='N16', 'ea03'='W28', 'ea05'='E20',<br />
ID= 4-7: 'ea06'='N20', 'ea07'='N32', 'ea08'='E28', 'ea09'='W08',<br />
ID= 8-11: 'ea10'='E36', 'ea11'='W12', 'ea12'='N36', 'ea13'='W04',<br />
ID= 12-15: 'ea14'='E08', 'ea15'='E24', 'ea16'='W24', 'ea17'='E04',<br />
ID= 16-19: 'ea18'='W36', 'ea19'='MAS', 'ea20'='N04', 'ea21'='E32',<br />
ID= 20-23: 'ea22'='N24', 'ea23'='E16', 'ea24'='W32', 'ea25'='W20',<br />
ID= 24-26: 'ea26'='E12', 'ea27'='N08', 'ea28'='N28'<br />
<br />
</pre><br />
<br />
You can inspect the output text file <i>3C129.listobs</i> in your favorite text editor (e.g., <i>gedit</i>). When taking some time to familiarize yourself with the output format, you will see that:<br />
* The observation consists of 5 scans, namely:<br />
# 3C147 (hardware setup)<br />
# 3C147 (setting requantizers)<br />
# 3C147 (primary calibrator)<br />
# 3C129 (target)<br />
# 3C129 (target)<br />
* The frequency coverage is 222 - 478 MHz, divided into 16 x 16 MHz spectral windows, each having 128 x 0.125 MHz channels. There are also two spectral windows with 64x0.5 MHz and one spectral window with 1024x0.0625 MHz. <br />
* Visibilities are recorded every 2 seconds in full polarization (4 polarization products labelled XX,XY,YX,YY).<br />
* Of the 28 VLA antennas labelled ea01 - ea028, antenna ea04 is not participating in this observation.<br />
* Source, antenna and spectral window IDs start at zero, but scan IDs start at one.<br />
<br />
<br />
[[File:3C129_pband_plotants.png|200px|thumb|right|Figure 1: plotants figure]] <br />
<br />
The array configuration can be inspected using:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotants(vis='3C129.ms', figfile='3C129_pband_plotants.png')<br />
</source><br />
<br />
== Initial processing steps - Antenna Position, Requantizer Gain, Ionospheric Correction ==<br />
<br />
When importing the data, we saved the flags from the online system to an ASCII text file. This gives us the opportunity to review the flag commands before applying them. For our data set, the flag file is called <i>importflags.txt</i>. Please load this file into your favorite text editor. The bulk of the flag commands refer to times when the VLA is slewing (<i>ANTENNA_NOT_ON_SOURCE</i>) or when the movable secondary reflector of the VLA's Cassegrain system is not in place (<i>SUBREFLECTOR_ERROR</i>). The latter can cause antenna gain variations (amplitude and phase), so it is safest to apply all the flags. The final two rows in the flag file will remove visibilities that are pure zero, and flag antennas that are partly blocked by other antennas (shadowing) which occurs mostly in compact configurations when observing along a VLA arm).<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms', mode='list', inpfile='importflags.txt', action='apply', reason='any', flagbackup=True)<br />
</source><br />
This call, before applying the new flags to the data, created a new flagbackup file called flagdata_1 which can be used by the task <i>flagmanager</i> to restore the state of the measurement set prior applying the flags from <i>importflags.txt</i>.<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
Now that we have applied the import flags, we will correct for the antenna position offsets. Antenna positional errors translates to an error in the measured visibilities and need to be accounted for before we proceed with any of the other calibration steps. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltable='3C129_pband.antpos',caltype='antpos')<br />
</source><br />
<br />
The logger output shows that there are 13 antennas with positional offsets: <br />
<pre><br />
<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea03 : 0.00290 0.00000 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea08 : 0.00000 0.00390 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea10 : -0.00170 0.00000 0.00400<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea11 : 0.00000 0.00160 -0.00170<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea13 : 0.00000 0.00050 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea16 : -0.00180 0.00680 0.00230<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea18 : 0.00470 0.00350 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea21 : -0.00160 0.00000 0.00210<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea23 : 0.00140 0.00200 -0.00150<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea24 : 0.00380 0.00000 -0.00340<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea25 : 0.00330 0.00000 -0.00110<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea26 : 0.00000 0.00190 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea27 : -0.00120 0.00070 0.00000<br />
<br />
<br />
</pre><br />
<br />
=== Ionospheric TEC Corrections ===<br />
[[File:Pband_vla_tecu_new.png|200px|thumb|right|Figure 2: TEC value vs time generated by <i>tec_maps.create()</i>]] <br />
<br />
Low frequency observations are affected by the ionosphere. A delay in the signal path is introduced between the two polarization of light that varies both as a function of time and line of sight (direction dependent). The delay is proportional to the Total Electron Content (TEC) along the line of sight and is inversely proportional to the square of the frequency. GPS measurements at two different frequencies provides us with an estimate of the TEC per square meter. This correction has been implemented in CASA which we shall apply as a calibration table by means of the <i>gencal</i> task. The task requires a TEC map that we will generate utilizing CASA recipes. For more information, see the section on [https://casa.nrao.edu/casadocs/casa-5.0.0/synthesis-calibration/preparing-for-calibration/#title5 ionospheric corrections] in the CASA documentation archives. For some versions of CASA, the first plot to the right (Figure 2) may not be generated if you have already run plotants: to generate this plot, you will need to quit CASA and start it again. If the IONEX data file was not downloaded you might be missing the curl package in the environment you are running CASA in. In this case you can provide the required file manually by downloading: ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/2015/058/igsg0580.15i.Z, (for the tutorial it is also also available in an unzipped version [http://casa.nrao.edu/Data/EVLA/Pband/igsg0580.15i here]) unziping it, and placing it in the directory from which you started CASA. The current implementation in CASA is in the development stage, however since the CASA 5.0 release both the Faraday rotation correction and dispersive delay corrections for the ionosphere are enabled. In previous releases only Faraday rotation corrections are enabled. In this example we will generate the calibration table and apply it throughout calibration. <br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
#<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='3C129.ms',doplot=True)<br />
#[NOTE: If you are using CASA 5.0.0, or earlier this should be: tec_image, tec_rms_image = tec_maps.create(vis='3C129.ms', doplot=True)]<br />
#<br />
gencal(vis='3C129.ms',caltable='3C129_pband.tecim',caltype='tecim',infile=tec_image)<br />
# Spatial plots like shown in Figure 3 can be looked at using the CASA viewer for the TEC values<br />
viewer('3C129.ms.IGS_TEC.im')<br />
# and for TEC rms<br />
viewer('3C129.ms.IGS_RMS_TEC.im')<br />
</source><br />
<br />
[[File:Pband_tec_new.png|200px|thumb|right|Figure 3: TEC map movie generated by the TEC tasks.]] <br />
<br />
A word of caution regarding the TEC map generation. The IGS website updates measurements only two weeks after the date of observation.<br />
<br />
=== Requantizer Gains ===<br />
<br />
The next step will correct the visibility amplitudes for the signal leveling (requantizer gains) that occurs at the inputs of the WIDAR correlator. These levels (per antenna, per polarization, per spectral window) are stored with the measurement set (in the <i>SYSPOWER</i> sub-table). This step is currently not essential, since the levels get set only once at the start of an observation and bandpass calibration will correct for this. But it will make your bandpass plots look better if you have multiple spectral windows. And, more importantly, in general it is possible to trigger the setting of requantizers throughout an observation. This is the case for new observations, when a P-band scan is preceded by a P-band setup scan, which itself is preceded by a scan with a different telescope configuration. <br />
<br />
The correction is done by means of the <i>gencal</i> task, where information in the <i>SYSPOWER</i> sub-table gets translated into a gain table. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltype='rq',caltable='3C129_pband.rq')<br />
</source><br />
<br />
== Dead antennas ==<br />
<br />
Now it is time to have a first visual look at the uncalibrated visibility data. It is important to identify dead antennas / polarizations early on, so we can exclude them from further data processing (which is the most efficient data reduction). A convenient way of doing this is through the <i>plotms</i> task on bright calibrator(s) (see the <i>listobs()</i> output), in our case 3C147 (see Fig. 4a).[[File:3C129_pband_3C147_prebp.png|200px|thumb|right|Figure 4a: <i>plotms</i> output before RFI flagging]]<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY', field='3C147', <br />
plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',iteraxis='baseline', <br />
plotfile='3C129_pband_3C147_prebp.png')<br />
</source><br />
<br />
This will load the <i>plotms</i> window that has queued up all the baselines of antenna ea01 and is currently displaying the first amplitude vs frequency plot with the colors representing the different spectral windows. Note that we are only plotting the XX and YY correlation of the antennas. This is because we expect most of the power in the linear cross correlation products to be in these two correlations. Notice that some spectral windows in particular are badly affected due to radio frequency interference (RFI). This is the issue that we will deal with after identifying dead antennas. If you scroll through the baselines using the green forward button in the <i>plotms</i> viewer, you will notice that the amplitude is particularly low when you encounter ea19. If we make a quick plot of the baselines made by ea19 (put <i>ea19</i> in the antenna field) we can see that there is no power in any of the polarizations indicating this antenna was dead and can be flagged out. It is also good to check for high power levels in cross-hand correlations in comparison to the parallel-hand correlations, which would uncover swapped polarization labels and severely mis-aligned dipoles. At the end of this tutorial some hints on solving for swapped or mislabeled dipole polarizations are given. In addition to the dead antenna, we also make sure that the first two scans are flagged, which were not labelled as setup scans but were used to set the attenuators and requantizers.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms',mode='manual',antenna='ea19')<br />
#<br />
flagdata(vis='3C129.ms',mode='manual',scan='1~2')<br />
</source><br />
<br />
We also noticed sharp RFI peaks during our data examination. To prevent Gibbs ringing, it is best to hanning smooth the data at this juncture before we proceed further with automatic flagging. Now is a good time to get a cup of coffee while hanning smoothing the data. [[File:3C129_pband_3C147_prebp_hanning.png|200px|thumb|right|Figure 4b: <i>plotms</i> output after Hanning smooth and first flagging.]]<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='3C129.ms',outputvis='3C129_pband.ms',datacolumn='data')<br />
</source><br />
<br />
We should replot the primary calibrator to see the effect of hanning smoothing on the data and the RFI (Fig. 4b). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning.png')<br />
<br />
</source><br />
<br />
== Automatic flagging ==<br />
<br />
<br />
It is worthwhile noting at this juncture that the standard P-band setup will ensure that the worst of the RFI is restricted to four of the spectral windows. This particular dataset is an example of higher than average levels of RFI across the band. Before throwing away bad spectral windows by hand, we'll let the automated flaggers in CASA have a go at it. The main task for this is <i>flagdata()</i>. While working on your data, <i>flagdata()</i> produces an abundance of output, not all of which is easy to understand. To help with this, we can get the flag status of our data before and after auto-flagging by running <i>flagdata()</i> in the <i>summary</i> mode. This first call provides the flag statistics of the flagging up to now, before any auto-flagging:<br />
<source lang='python'><br />
summary_1 = flagdata(vis='3C129_pband.ms', mode='summary')<br />
</source><br />
<br />
This returns a python dictionary with flagged versus total visibilities along various axes (antenna, scan, spw, field, correlation, etc.). For example, if we want to know the percentage flagged per scan, run the following (note that the scans may not appear in sorted order):<br />
<source lang='python'><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for id, stats in summary_1[ axis ].iteritems():<br />
print '%s %s: %5.1f percent flagged' % ( axis, id, 100. * stats[ 'flagged' ] / stats[ 'total' ] )<br />
--<br />
</source><br />
<br />
For this example, you will notice that scans 1 and 2 (the dummy scans on 3C147) are 100 percent flagged, which is what we did during the initial flagging.<br />
<br />
We will run <i>flagdata()</i> in the <i>tfcrop</i> mode, which will (per scan, per baseline, per spectral window, per polarization) look for visibility amplitude outliers. It uses a 5-piece polynomial in an attempt to remove any intrinsic bandpass (amplitude) structure (we did not calibrate for bandpass yet). We run the task twice to allow for slightly deeper flagging and separate treatment of the parallel and cross-hand correlations. For the first run, we tell CASA to make a backup of our visibility flag status up to here, giving us an option to restore them if we choose over-aggresive flagging parameters and consequently over-flag our data. The flag backup file name is reported in the log window, and can be found in the <i>3C129_pband.ms.flagversions</i> directory.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=4., freqcutoff=3., maxnpieces=5,<br />
action='apply', display='report', flagbackup=True, combinescans=True, ntime='3600s', correlation='ABS_XY,ABS_YX')<br />
#<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=3., freqcutoff=3., maxnpieces=2,<br />
action='apply', display='report', flagbackup=False, combinescans=True, ntime='3600s', correlation='ABS_XX,ABS_YY')<br />
#<br />
flagdata(vis='3C129_pband.ms', mode='extend')<br />
</source><br />
<br />
As a note of caution: When executing flagdata with <i>mode='extend'</i>, all flags that were ever applied to the dataset will get extended. Repeat execution of this command will incrementally flag more and more around already flagged data.<br />
<br />
Let's get another flag summary to see how much extra data was flagged: <br />
<source lang='python'><br />
# In CASA<br />
summary_2 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
# <br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_2[ axis ].iteritems():<br />
old_stats = summary_1[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning_automatic.png')<br />
<br />
</source><br />
[[File:3C129_pband_3C147_prbp_tfcropped.png|200px|thumb|right|Figure 4c: <i>plotms</i> output after RFI flagging with tfcrop]]<br />
There is quite a bit of RFI that gets missed in RFI-rich spectral windows as can be seen in Fig. 4c. The solution to this is to provide the flagging routines with more contrast between healthy and affected data. For this, we will perform a preliminary bandpass calibration to take out the bandpass shape. We will do some coarse preliminary calibration and apply it to the calibrator before flagging for RFI once more.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.G0', gaintype='G', calmode='p', solint='int', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim'])<br />
#<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K0', gaintype='K', solint='inf', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0'])<br />
#<br />
bandpass(vis='3C129_pband.ms', caltable='3C129_pband.B0', solint='inf', field='3C147',refant='ea09', minsnr=2.0,<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0'])<br />
#<br />
applycal(vis='3C129_pband.ms', field='3C147', applymode='calflagstrict',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0','3C129_pband.B0'] )<br />
<br />
</source><br />
<br />
On the command line you should see a notification stating: "FJones (iononsphere) apply now provisionally includes disp. delay." which indicates that both Faraday rotation as well as dispersive delay corrections for the ionosphere are applied.<br />
<br />
[[File:3c147_postbandpass_rflag.png|200px|thumb|right|Figure 4d: <i>plotms</i> output after RFI flagging with <i>mode='rflag'</i>]]<br />
<br />
We will now flag the corrected data column that contains the coarsely calibrated visibilities which provide better contrast to the flagging algorithms to remove the RFI present. We will begin by running the <i>RFLAG</i> algorithm in the task <i>flagdata</i>. We will again use summary mode to see how much more of the observed data was flagged.<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
summary_3 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
#<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_3[ axis ].iteritems():<br />
old_stats = summary_2[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C147_postbandpass_rflag.png', ydatacolumn='corrected')<br />
</source><br />
<br />
Taking another visual look at the data using <i>plotms</i>, we examine the effect of the automated flagging on the calibrator (Fig. 4d). We find that most spectral windows are RFI free and that about 60% of the data was flagged. We typically expect between 40-60% of flagging due to interference in a significant fraction of the band. In this example, a more careful RFI flagging approach is likely to recover more usable spectrum.<br />
<br />
==Flux Density Calibration==<br />
The initial flagging has cleaned up most of the stray RFI across the band. This allows us to proceed with the actual calibration of the data. Before we get to the calibration tables, it's essential to do the flux density calibration of our calibrator 3C147. This is done by the <i>setjy</i> task. Before we run the task we first clear the preliminary calibration that was carried out to enable better flagging. We do that by running the <i>clearcal</i> task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
clearcal(vis='3C129_pband.ms')<br />
#<br />
setjy(vis='3C129_pband.ms', standard='Scaife-Heald 2012', field='3C147')<br />
</source><br />
<br />
== Delay and bandpass calibration ==<br />
<br />
First, we use the full bandwidth on a single scan on the primary calibrator to determine a single delay per antenna, per polarization. This will determine a single (approximate) phase slope across frequency, mainly caused by propagation effects in the (time-variable) ionosphere, cable length differences, and electronics in the signal paths from antenna feeds to correlator. Note that for a single short (5-10 minutes) scan on the calibrator, we can get away with solving for a time-invariant delay per antenna, per polarization. We will use scan 3 on 3C147.<br />
[[File:3C147_init_delay.png|200px|thumb|right|Figure 5: Initial per antenna delays derived by <i>gaincal</i>.]]<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K1', field='3C147', solint='inf', refant='ea09', gaintype='K',<br />
gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq'],parang=True)<br />
#<br />
plotcal(caltable='3C129_pband.K1', xaxis='antenna', yaxis='delay', markersize=2.0, <br />
plotrange=[0,30,-50.,50.],figfile='3C147_delay.png')<br />
</source><br />
<br />
Since we picked ea09 as our reference antenna, the delays for this antenna ID (=0) are arbitrarily set to zero. All other delays should be within 30 nanosec or so (which they should be for this data set). Larger values should be treated with suspicion, possibly indicating a problem with the antenna or polarization. We will use the bandpass calibration to verify this in the next step.<br />
<br />
To determine the bandpass calibration, we use the same source and scan and apply the delay calibration before solving for the bandpass. Note that we request a minimum SNR of 3, which will make the solve fail for the worst, but not all, channels.<br />
<source lang='python'><br />
# In CASA<br />
bandpass( vis='3C129_pband.ms', caltable='3C129_pband.B1', field='3C147', solint='inf', refant='ea09', minsnr=3.0,<br />
parang = True, gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq','3C129_pband.K1'],<br />
interp=['','','','nearest,nearestflag'])<br />
<br />
</source><br />
<br />
After solving, we inspect the bandpass calibration amplitudes and phase, and flag any obvious residual outliers. Outliers may be found above or below the average bandpass curves, and tend to arise in the same channel (frequency) ranges for all antennas. Don't spend more than 1 minute per antenna for this tutorial and don't worry about missing some bad points; these bad channels will likely get flagged in a later stage anyway. Note that the bandpass amplitudes (Fig. 6) and phases may differ between polarizations of the same antenna. Also note that the bandpass phases across frequency should not have an overall gradient (this should be removed by the delay calibration). Also note that bandpass phases may wrap around from +/-180 to -/+180 degrees. If that is problematic, re-run the phase <i>plotcal()</i> with <i>plotrange = [ 200.,500.,0.,360. ]</i>. Also note that there are no bandpass solutions for antennas ea19 and ea28, and only a few solutions for ea14.<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='amp', iteration='antenna', markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='phase', iteration='antenna', plotrange=[ 200.,500.,-180.,180. ], markersize = 2.0)<br />
</source><br />
{|<br />
| [[File:3C147_bp0_ea01.png|200px|thumb|right|Figure 6a: Initial bandpass amplitudes for antenna ea01.]]<br />
| [[File:3C147_bp0_phase_ea01.png|200px|thumb|right|Figure 6b: Initial bandpass phase for antenna ea01.]]<br />
|}<br />
<br />
Flagging is made easier by maximizing the plotcal() window. We don't set the scale for plotting the bandpass amplitudes, which is convenient when flagging outliers as it will auto-rescale. While plotting the bandpass phases we do fix the scale, as it provides a better feeling for the relevant magnitude of phase outliers. Flagging of outliers is done using the <i>Mark Region</i> and <i>Flag</i> buttons. For more information on interactive flagging, see the topical guide [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Flagging VLA Data in CASA]. We are using this mechanism to efficiently flag data manually, since it is done per antenna rather than per baseline. These 'flags' will not be permanent until we call <i>applycal()</i> later on, but are effective while applying the calibration on the fly, as we will do in subsequent calibration steps.<br />
<br />
== Gain calibration ==<br />
<br />
With the delay and bandpass calibration in place, we will now look more closely at the time-variable behaviour of the VLA. We use the same scan on our primary calibrator, 3C147, over the full effective bandwidth to determine gain calibrations: one complex value per antenna, per polarization, per integration time. The delay and bandpass tables are applied on the fly.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal( vis='3C129_pband.ms', caltable='3C129_pband.G1', field='3C147', solint = 'int',<br />
refant = 'ea09', minsnr = 3.0, gaintype = 'G', calmode = 'ap',<br />
gaintable = ['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim', '3C129_pband.K1', '3C129_pband.B1' ],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag' ], parang = True )<br />
<br />
</source><br />
{|<br />
| [[File:3C147_Gamp_ea01.png|200px|thumb|right|Figure 7a: Initial gain amplitude solutions for antenna ea01.]]<br />
| [[File:3C147_Gphase_ea01.png|200px|thumb|right|Figure 7b: Initial gain phase solutions for antenna ea01.]]<br />
|}<br />
<br />
Similar to the bandpass flagging, we will flag outlier gain solutions in both amplitude and phase. For the amplitudes, the final scatter around an average close to one should be a few percent (see Fig. 7). Note antennas ea08, ea09 have particularly noisy solutions. Remember that the gain solutions for both polarizations can be flagged together, as we will lose the equivalent data of the surviving polarization anyway.<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='amp', iteration='antenna', markersize=2.0 )<br />
#<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='phase', iteration='antenna', <br />
plotrange=[ None,None,-180.,180.], markersize=2.0)<br />
</source><br />
<br />
Note that when plotting gains against time, it is convenient to use auto-scaling of the time axis by putting <i>None</i> in the first two fields of <i>plotrange</i>.<br />
<br />
Next, we will prepare a smoothed and interpolated version of the gain calibration table, which will be applied later to the target field data. This prevents flagging of target field data when one of the edges of a calibrator scan is flagged for one or more antennas. The result can be inspected with <i>plotcal</i>.<br />
<source lang='python'><br />
# In CASA<br />
smoothcal(vis='3C129_pband.ms', tablein='3C129_pband.G1', caltable='3C129_pband.Gs1', smoothtype='median', smoothtime = 60.*60.)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='amp', iteration='antenna', plotrange=[ None,None,0.8,1.2 ], markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='phase', iteration='antenna', plotrange=[ None,None,-10.,10. ], markersize=2.0)<br />
<br />
</source><br />
<br />
== Transfer of calibrations to the target field ==<br />
<br />
So far we have used one scan on the primary calibrator 3C147 to derive various calibration tables. These calibrations will now be applied to all scans of all sources in our measurement set, which includes 3C147 itself and our target field 3C129. As described above, the task <i>applycal()</i> creates a <i>CORRECTED_DATA</i> column in which the calibrated visibilities get stored. This may take a while.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='3C129_pband.ms', parang=True, applymode='calflagstrict', flagbackup=True, gaintable=['3C129_pband.antpos','3C129_pband.rq',<br />
'3C129_pband.tecim','3C129_pband.K1','3C129_pband.B1','3C129_pband.Gs1'],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag', 'nearest,nearestflag'])<br />
</source><br />
<br />
Now we split off the calibrated target field data, meaning that the visibilities of source 3C129 get copied from the <i>CORRECTED_DATA</i> column to the <i>DATA</i> column of a new measurement set. This is convenient for further processing. <br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='3C129_pband.ms', outputvis='3C129_pband_target.ms', datacolumn='corrected', field='3C129')<br />
</source><br />
<br />
It is also helpful to perform additional flagging for any residual RFI left behind in our target after all our calibration steps have been completed and we begin the imaging of our data.<br />
<br />
<source land='python'><br />
flagdata(vis='3C129_pband_target.ms', mode='rflag')<br />
flagdata(vis='3C129_pband_target.ms', mode='extend')<br />
</source><br />
<br />
<br />
== Statwt ==<br />
Before we go ahead and image our target field. We would like to weight any residual low level RFI down. This is done using the task <i>statwt</i>. The task runs through the data and computes the rms of the data and weighting down outliers. This makes a big impact for imaging of a target in the presence of low level residual RFI. For our purposes we will bin the data in a time interval larger than the integration time and run statwt as follows.<br />
<br />
<source lang='python'> <br />
statwt(vis='3C129_pband_target.ms', datacolumn='data', timebin=30.)<br />
</source><br />
<br />
With data weights for outliers being set we will now proceed with the imaging and self-calibration of our data.<br />
==Imaging==<br />
<br />
At this point, we're ready to make a first image of our target field. Imaging in CASA is done by means of the <i>tCLEAN</i> task. The task implements algorithms for wide field imaging, such as W-Projection (Cornwell et al. 2008 http://arxiv.org/abs/0807.4161) and Multi Term Multi Frequency Synthesis ( Rau et al. 2011 http://arxiv.org/abs/1106.2745) both of which we will utilize to make our initial image. Even though our source of concern, the radio galaxy, lies in the centre of our field. The observations were carried out in B configuration resulting in an effective maximum resolution of approximately 5x5 arcsecs. To effectively model the source spectral index in the sky, we will utilize the MTMFS algorithm and use the W-Projection algorithm to make a wide field image (The full beam at P Band is about 3 degrees in diameter). So, going by these requirements, we can now compute the required cell and image size. Consulting the resolution guide of NRAO science page, we can see that the expected HPBW for B - Configuration, P -Band Observation is 18.5 arcsec. So, we can sample it well by using a 5 arcsec cellsize. Having done that, if we decide to make a wide field image to account for all the point sources we will set the image size parameter to 4096 pixels. To enable the wide-field algorithm, we set the <i>gridder='wproject'</i>: this invokes the W-Projection algorithm, upon which we set the number of W-Projection planes to be 128. We set the deconvolution mode as Multi Term Multi Scale Multi Frequency Synthesis using the <i>deconvolver='mtmfs'</i> parameter along with the number of Taylor terms to be considered during imaging to be 2 and choosing three different scales to model the point source and extended emission in our field well, we do this by setting <i> scales=[0,20,30]</i>. This allows for the source spectral variation to be modeled by a second order polynomial. We launch the interactive tclean process, but only for <i>spw='3~8'</i> to begin with. Note, running the first clean call will take at least ~10 min to process.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_initial_clean', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30],<br />
pblimit=0.01, savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
</source><br />
<br />
{|<br />
| [[File:3C129_radio_gal_residual.png|200px|thumb|right|Figure 10a: Resulting residuals of 3C129 after first call of interactive <i>clean</i>.]]<br />
| [[File:3C129_radio_gal_clean.png|200px|thumb|right|Figure 10b: Resulting image of 3C129 after completing the first <i>clean</i>.]]<br />
|}<br />
The <i>tCLEAN</i> command launches an interactive session after a 100 iterations of clean and produces a wide field map with two sources at the center and a lot of bright sources far out in the field. As it is a snapshot image, the bright sources have significant side lobes and so tight clean boxing can help. This can done in the interactive viewer interface that pops up. If we proceed with interactive clean with subsequent steps to keep boxing out the strong sources that pop up in the image, we finally see the extended emission from the target radio galaxy begin to emerge. Continue boxing and cleaning to ensure that the residuals of the boxed cleaning look noise like (~10000 clean iterations). We now stop the interactive task and look at the final image it produced.<br />
<br />
<source lang='python'><br />
# In CASA<br />
viewer('3C129_initial_clean.image.tt0')<br />
</source><br />
<br />
The recovered structure at the center of the map should resemble what is shown in Fig. 10b. If the image looks worse then boxing of bright features in the map needs to be done more carefully. Note the image has some sources still showing strong side lobes and imaging artifacts around them. We expected this as we have carried out phase calibration only on our flux calibrator and have just transferred the solutions over to our target field. Since we used the <i>usescratch=True</i>, the MODEL_DATA column in the measurement set now contains the initial image model, which we will self calibrate against to produce a better image. Also in clean, do notice that the spw's utilized are the cleanest spectral windows that are totally RFI free.<br />
<br />
==Self Calibration ==<br />
<br />
We now proceed to compute gain phase solutions for our target field using the <i>gaincal</i> task as the first step in self-calibration.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScG0', field='3C129', solint='inf', refant='ea09', <br />
spw='3~8',minsnr=3.0, gaintype='G', parang=True, calmode='p')<br />
<br />
bandpass( vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScB0', field='3C129', solint='inf', refant='ea09', minsnr=3.0, spw='3~8',<br />
parang = True, gaintable=['3C129_pband_target.ScG0'],<br />
interp=['','','','nearest,nearestflag'])<br />
applycal(vis='3C129_pband_target.ms', gaintable=['3C129_pband_target.ScG0','3C129_pband_target.ScB0'], spw='3~8', applymode='calflagstrict')<br />
</source><br />
<br />
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the <i>tCLEAN</i> command once again.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_clean_sc0', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30],<br />
pblimit=0.01,savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
<br />
viewer('3C129_clean_sc0.image.tt0')<br />
</source><br />
[[File:3C129_radio_gal_cselfcal0.png|200px|thumb|right|Figure 11: Resulting image of 3C129 after completing a first selfcal.]]<br />
<br />
On boxing and cleaning, we already notice that the imaging artifacts have reduced significantly. We also see that the target source appears to contain more structure and a greater amount of flux. Further self calibration iterations involving an amplitude & phase gain calibration, and target bandpass calibration, are all possible steps that can be explored. <br />
<br />
== Appendix: Some P-band data issues you may want to know about ==<br />
<br />
Unfortunately, there are a few known issues with JVLA P-band observing. Most problems were discovered during the P-band commissioning period and have been fixed for newer data sets, but the archive keeps part of this history alive. Here we will go over some of those issues, and (if possible) provide ways of fixing them.<br />
<br />
=== Polarization labeling ===<br />
<br />
For a long period, P-band feeds have been labelled as being circular (R and L), while they are linear (X and Y). It is still possible to generate data with incorrect polarization labeling, however for standard observations this should not be the case anymore. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem. This may seem mostly harmless, but does make a difference for polarization calibration. The following (non-standard) task will check and fix this (and another related problem of swapped polarizations):<br />
<br />
Download [http://casa.nrao.edu/Data/EVLA/Pband/casa_vla_lowband.tar.gz this tarball] in your CASA working directory, open a shell there, and type the following commands:<br />
<br />
<source lang='bash'><br />
tar -xzvf casa_vla_lowband.tar.gz<br />
<br />
buildmytasks<br />
</source><br />
<br />
Now go back to your CASA session and type:<br />
<br />
<source lang='python'><br />
# In CASA<br />
execfile('mytasks.py')<br />
#<br />
fixlowband(vis='TSUB0001_pband_prehs.ms')<br />
</source><br />
<br />
Re-run the <i>listobs()</i> task to check if the fix was correctly applied.<br />
<br />
=== Swapped polarizations ===<br />
<br />
Throughout the whole history of the new VLA low-band system, even up to this day, there have been mistakes in the cabling of the full signal chain. This results in that some antennas have the X-polarization signal come in as Y and vice versa. The way to notice this in visibility data is that for baselines with one swapped antenna most power will be in the cross-hand correlations (XY and YX) rather than the parallel-hand correlations (XX and YY). Once antennas with this feature are manually identified, a contributed task <i>swappol()</i> is available to fix this problem (see under polarization labelling).<br />
<br />
=== Double data descriptor entries ===<br />
<br />
The data description table is part of the measurement set, and provides a link between the recorded visibilities, the spectral window information and the polarization information. For some data sets, the data description table contains two entries for each P-band spectral window, one pointing towards a circular (RL) polarization definition, and one pointing towards a linear (XY) polarization definition. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem.<br />
<br />
=== Continuous Radio Frequency Interference (RFI) ===<br />
<br />
The wide bandwidth of the low-band receiver is (unfortunately) guaranteed to contain significant amounts of RFI. There are a few RFI sources that are active all the time, and are visible in all array configurations. The default P-band observing setup of 16 x 16 MHz tries to capture as much of the continuous RFI in as few spectral windows as possible, allowing for a simple RFI mitigation strategy in which these spectral windows (spws 1,2 and 9,10, but possibly more) can be immediately flagged.<br />
<br />
=== Two spectral window setup ===<br />
<br />
In the early commissioning period, the default setup for P-band observing was to use 2 spectral windows of 1024 channels each to cover 256 MHz of bandwidth from 230 - 486 MHz. However, it was noticed that strong, narrow-band RFI events were causing data to be lost for the whole spectral window in which they occured. To make the system more robust against data loss from these events, the frequency range was divided into 16 x 16 MHz, and shifted downwards by 6 MHz to capture ever-present RFI into as few spectral windows as possible. If your data has only two P-band spectral windows, please be aware that a higher data loss due to RFI is possible. There is no way to repair this.<br />
<br />
=== Bandpass ripples ===<br />
<br />
Due to signal reflections in cables and within the VLA dish, sinusoidal amplitude and phase modulations are always present in P-band data. This is most easily seen in bandpass calibration plots of amplitude versus frequency. In some cases, mostly due to cable & connector problems, these modulations can be very strong (~up to 50 percent of the average amplitude level). These modulations tend not to vary over the duration of an observation and can therefore be removed through bandpass and polarization calibration. If they are found to be variable (e.g., by inspecting bandpass solutions for separate calibrator scans), the offending antenna / polarization should be flagged.<br />
<br />
=== High cross-polarization ===<br />
<br />
On each antenna, the P-band feed (dipole) is visually aligned with respect to the primary focus support legs. This is normally done within 5-10 degrees accuracy. On rare occasions the dipole on one antenna has accidentally rotated to much larger angles because the locking bolt on the back of the feed was not completely tightened. The result is a high cross-polarization in the baseline visibilities that include this antenna. If this is the case, it is safest to flag this data, since there is a possibility that the dipole has rotated during the observations.<br />
<br />
=== Instrumental Polarization Calibration ===<br />
<br />
At present polarization P-band observations are in RSRO mode only. While polarization leakage calibration has been highlighted in the past reports it does not directly impact imaging in Stokes I and hence has been dropped from this casaguide.<br />
<br />
== Useful links ==<br />
<br />
* [http://www.aoc.nrao.edu/lbr The EVLA Low-Band Receiver System Status & User Pages]<br />
* [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/topical-guides/vlofreq VLA Very Low Frequency Observing]<br />
<br />
<br />
{{Checked 5.4.1}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0&diff=26245VLA Radio galaxy 3C 129: P-band continuum tutorial-CASA5.4.02018-12-01T23:29:17Z<p>Emomjian: </p>
<hr />
<div><!--By [http://www.aoc.nrao.edu/~pjaganna/Site/Home.html Preshanth Jagannathan] & [https://safe.nrao.edu/wiki/bin/view/Main/HuibIntemaWebHome Huib Intema] & Frank Schinzel--><br />
<br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b><p></p> <br />
<b> Note: This guide may take up to 2-days to complete </b><p></p> <br />
If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
<br />
This webpage provides a basic description on how to use CASA to reduce data from the upper part of the new VLA low-band system, also known as P-band, covering roughly 220-480 MHz. The goal is to make a wide-field continuum stokes I image of a typical blank field using the full effective bandwidth. The dataset that will be downloaded in this guide has a size of 26.46 GB and will require a total of ~140 GB of disk space if all generated files are kept. <br />
<br />
''Note: Starting with CASA 5.0 IPython 5.1.0 is used which for interactive CASA sessions. This does not allow to readily paste and execute multiple lines of code like in previous versions of the interpreter. If this is needed at any point in this tutorial, we use "%cpaste" to allow the copy/paste of loops, conditions, etc.. Press "Enter" (or "Return") after "--" at the end, this stops (or exits) the copy/paste prompt and will return you to the CASA prompt.''<br />
<br />
== Obtaining the raw data ==<br />
<br />
For this guide, we'll use some test data that was taken in B-configuration. To get a copy of the raw data, download [https://casa.nrao.edu/Data/EVLA/Pband/P_band_3C129.tgz P_band_3C129.tgz] onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
<br />
Alternatively, the raw data can also be obtained from the NRAO archive, go to the [https://archive.nrao.edu/archive/advquery.jsp NRAO advance archive query page], and enter the following search parameter in the field <i>Archive File ID</i> (leave the rest on default):<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
imag-test-copy.57080.956837025464<br />
</pre><br />
</div><br />
<br />
The search should return a single row for Archive File: imag-test-copy.57080.956837025464, taken on Feb. 27, 2015 (or "15-Feb-27 22:57:52"). Check the box next to it. Next, fill in your email and select the "SDM-BDF dataset (all files)" option. Note, you may find it easier to download a tar file, in which case, you'll want to check the box for Create MS or SDM tar file. Click "Get My Data" and follow the on-screen prompts to reach the next page, where you should click "Retrieve over internet". The next page should report that the data staging is in progress. Wait until you receive an e-mail reporting that your archive data is copied, which should take a few minutes. Download the data onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing <br />
<source lang='bash'><br />
casa<br />
</source><br />
on the (linux) command line. This should start a CASA interactive python (iPython) session, and open a separate log window. The CASA version is reported at startup, both in the python session and the log window.<br />
<br />
== Importing the raw data into CASA ==<br />
<br />
We will begin by importing our data into the measurement set format (CASA standard) from the binary format (SDM-BDF) as downloaded from the archive. We do this by means of the importasdm task. It is important to note that the importevla task has been deprecated.<br />
<br />
<source lang='python'><br />
# In CASA<br />
importasdm(asdm='imag-test-copy.57080.956837025464', vis ='3C129.ms', savecmds=True, outfile='importflags.txt')<br />
<br />
</source><br />
<br />
This task calls the external tool <i>asdm2MS</i> to perform the conversion. Together with the visibility data, also the flag commands from the VLA online system are imported. These flags are not directly applied, but stored in the <i>importflags.txt</i> file and applied later.<br />
<br />
== Preliminary data inspection ==<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms', verbose = True, listfile = '3C129.listobs')<br />
</source><br />
<br />
Alternatively, you can also output to the logger by running the command without the listfile argument. The sample listobs output shown has flag <i>verbose=False</i> for the sake of brevity.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/fschinze/P_band_tutorial/3C129.ms MS Version 2<br />
================================================================================<br />
Observer: Project: uid://evla/sdm/X2<br />
Observation: EVLA(27 antennas)<br />
Data records: 4714983 Total elapsed time = 1422 seconds<br />
Observed from 27-Feb-2015/22:57:54.0 to 27-Feb-2015/23:21:36.0 (UTC)<br />
<br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C147 05:42:36.127300 +49.51.07.17800 J2000 0 1307124<br />
1 NONE 3C129 04:48:58.200000 +45.02.01.00000 J2000 1 3407859<br />
Spectral Windows: (19 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 EVLA_P#A0C0#0 128 TOPO 224.000 125.000 16000.0 231.9375 12 XX XY YX YY<br />
1 EVLA_P#A0C0#1 128 TOPO 240.000 125.000 16000.0 247.9375 12 XX XY YX YY<br />
2 EVLA_P#A0C0#2 128 TOPO 256.000 125.000 16000.0 263.9375 12 XX XY YX YY<br />
3 EVLA_P#A0C0#3 128 TOPO 272.000 125.000 16000.0 279.9375 12 XX XY YX YY<br />
4 EVLA_P#A0C0#4 128 TOPO 288.000 125.000 16000.0 295.9375 12 XX XY YX YY<br />
5 EVLA_P#A0C0#5 128 TOPO 304.000 125.000 16000.0 311.9375 12 XX XY YX YY<br />
6 EVLA_P#A0C0#6 128 TOPO 320.000 125.000 16000.0 327.9375 12 XX XY YX YY<br />
7 EVLA_P#A0C0#7 128 TOPO 336.000 125.000 16000.0 343.9375 12 XX XY YX YY<br />
8 EVLA_P#A0C0#8 128 TOPO 352.000 125.000 16000.0 359.9375 12 XX XY YX YY<br />
9 EVLA_P#A0C0#9 128 TOPO 368.000 125.000 16000.0 375.9375 12 XX XY YX YY<br />
10 EVLA_P#A0C0#10 128 TOPO 384.000 125.000 16000.0 391.9375 12 XX XY YX YY<br />
11 EVLA_P#A0C0#11 128 TOPO 400.000 125.000 16000.0 407.9375 12 XX XY YX YY<br />
12 EVLA_P#A0C0#12 128 TOPO 416.000 125.000 16000.0 423.9375 12 XX XY YX YY<br />
13 EVLA_P#A0C0#13 128 TOPO 432.000 125.000 16000.0 439.9375 12 XX XY YX YY<br />
14 EVLA_P#A0C0#14 128 TOPO 448.000 125.000 16000.0 455.9375 12 XX XY YX YY<br />
15 EVLA_P#A0C0#15 128 TOPO 464.000 125.000 16000.0 471.9375 12 XX XY YX YY<br />
16 EVLA_P#B0D0#16 64 TOPO 310.000 500.000 32000.0 325.7500 15 XX XY YX YY<br />
17 EVLA_P#B0D0#17 1024 TOPO 310.000 62.500 64000.0 341.9688 15 XX XY YX YY<br />
18 EVLA_P#B0D0#18 64 TOPO 342.000 500.000 32000.0 357.7500 15 XX XY YX YY<br />
Antennas: 27 'name'='station'<br />
ID= 0-3: 'ea01'='N12', 'ea02'='N16', 'ea03'='W28', 'ea05'='E20',<br />
ID= 4-7: 'ea06'='N20', 'ea07'='N32', 'ea08'='E28', 'ea09'='W08',<br />
ID= 8-11: 'ea10'='E36', 'ea11'='W12', 'ea12'='N36', 'ea13'='W04',<br />
ID= 12-15: 'ea14'='E08', 'ea15'='E24', 'ea16'='W24', 'ea17'='E04',<br />
ID= 16-19: 'ea18'='W36', 'ea19'='MAS', 'ea20'='N04', 'ea21'='E32',<br />
ID= 20-23: 'ea22'='N24', 'ea23'='E16', 'ea24'='W32', 'ea25'='W20',<br />
ID= 24-26: 'ea26'='E12', 'ea27'='N08', 'ea28'='N28'<br />
<br />
</pre><br />
<br />
You can inspect the output text file <i>3C129.listobs</i> in your favorite text editor (e.g., <i>gedit</i>). When taking some time to familiarize yourself with the output format, you will see that:<br />
* The observation consists of 5 scans, namely:<br />
# 3C147 (hardware setup)<br />
# 3C147 (setting requantizers)<br />
# 3C147 (primary calibrator)<br />
# 3C129 (target)<br />
# 3C129 (target)<br />
* The frequency coverage is 222 - 478 MHz, divided into 16 x 16 MHz spectral windows, each having 128 x 0.125 MHz channels. There are also two spectral windows with 64x0.5 MHz and one spectral window with 1024x0.0625 MHz. <br />
* Visibilities are recorded every 2 seconds in full polarization (4 polarization products labelled XX,XY,YX,YY).<br />
* Of the 28 VLA antennas labelled ea01 - ea028, antenna ea04 is not participating in this observation.<br />
* Source, antenna and spectral window IDs start at zero, but scan IDs start at one.<br />
<br />
<br />
[[File:3C129_pband_plotants.png|200px|thumb|right|Figure 1: plotants figure]] <br />
<br />
The array configuration can be inspected using:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotants(vis='3C129.ms', figfile='3C129_pband_plotants.png')<br />
</source><br />
<br />
== Initial processing steps - Antenna Position, Requantizer Gain, Ionospheric Correction ==<br />
<br />
When importing the data, we saved the flags from the online system to an ASCII text file. This gives us the opportunity to review the flag commands before applying them. For our data set, the flag file is called <i>importflags.txt</i>. Please load this file into your favorite text editor. The bulk of the flag commands refer to times when the VLA is slewing (<i>ANTENNA_NOT_ON_SOURCE</i>) or when the movable secondary reflector of the VLA's Cassegrain system is not in place (<i>SUBREFLECTOR_ERROR</i>). The latter can cause antenna gain variations (amplitude and phase), so it is safest to apply all the flags. The final two rows in the flag file will remove visibilities that are pure zero, and flag antennas that are partly blocked by other antennas (shadowing) which occurs mostly in compact configurations when observing along a VLA arm).<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms', mode='list', inpfile='importflags.txt', action='apply', reason='any', flagbackup=True)<br />
</source><br />
This call, before applying the new flags to the data, created a new flagbackup file called flagdata_1 which can be used by the task <i>flagmanager</i> to restore the state of the measurement set prior applying the flags from <i>importflags.txt</i>.<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
Now that we have applied the import flags, we will correct for the antenna position offsets. Antenna positional errors translates to an error in the measured visibilities and need to be accounted for before we proceed with any of the other calibration steps. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltable='3C129_pband.antpos',caltype='antpos')<br />
</source><br />
<br />
The logger output shows that there are 13 antennas with positional offsets: <br />
<pre><br />
<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea03 : 0.00290 0.00000 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea08 : 0.00000 0.00390 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea10 : -0.00170 0.00000 0.00400<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea11 : 0.00000 0.00160 -0.00170<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea13 : 0.00000 0.00050 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea16 : -0.00180 0.00680 0.00230<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea18 : 0.00470 0.00350 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea21 : -0.00160 0.00000 0.00210<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea23 : 0.00140 0.00200 -0.00150<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea24 : 0.00380 0.00000 -0.00340<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea25 : 0.00330 0.00000 -0.00110<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea26 : 0.00000 0.00190 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea27 : -0.00120 0.00070 0.00000<br />
<br />
<br />
</pre><br />
<br />
=== Ionospheric TEC Corrections ===<br />
[[File:Pband_vla_tecu_new.png|200px|thumb|right|Figure 2: TEC value vs time generated by <i>tec_maps.create()</i>]] <br />
<br />
Low frequency observations are affected by the ionosphere. A delay in the signal path is introduced between the two polarization of light that varies both as a function of time and line of sight (direction dependent). The delay is proportional to the Total Electron Content (TEC) along the line of sight and is inversely proportional to the square of the frequency. GPS measurements at two different frequencies provides us with an estimate of the TEC per square meter. This correction has been implemented in CASA which we shall apply as a calibration table by means of the <i>gencal</i> task. The task requires a TEC map that we will generate utilizing CASA recipes. For more information, see the section on [https://casa.nrao.edu/casadocs/casa-5.0.0/synthesis-calibration/preparing-for-calibration/#title5 ionospheric corrections] in the CASA documentation archives. For some versions of CASA, the first plot to the right (Figure 2) may not be generated if you have already run plotants: to generate this plot, you will need to quit CASA and start it again. If the IONEX data file was not downloaded you might be missing the curl package in the environment you are running CASA in. In this case you can provide the required file manually by downloading: ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/2015/058/igsg0580.15i.Z, (for the tutorial it is also also available in an unzipped version [http://casa.nrao.edu/Data/EVLA/Pband/igsg0580.15i here]) unziping it, and placing it in the directory from which you started CASA. The current implementation in CASA is in the development stage, however since the CASA 5.0 release both the Faraday rotation correction and dispersive delay corrections for the ionosphere are enabled. In previous releases only Faraday rotation corrections are enabled. In this example we will generate the calibration table and apply it throughout calibration. <br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
#<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='3C129.ms',doplot=True)<br />
#[NOTE: If you are using CASA 5.0.0, or earlier this should be: tec_image, tec_rms_image = tec_maps.create(vis='3C129.ms', doplot=True)]<br />
#<br />
gencal(vis='3C129.ms',caltable='3C129_pband.tecim',caltype='tecim',infile=tec_image)<br />
# Spatial plots like shown in Figure 3 can be looked at using the CASA viewer for the TEC values<br />
viewer('3C129.ms.IGS_TEC.im')<br />
# and for TEC rms<br />
viewer('3C129.ms.IGS_RMS_TEC.im')<br />
</source><br />
<br />
[[File:Pband_tec_new.png|200px|thumb|right|Figure 3: TEC map movie generated by the TEC tasks.]] <br />
<br />
A word of caution regarding the TEC map generation. The IGS website updates measurements only two weeks after the date of observation.<br />
<br />
=== Requantizer Gains ===<br />
<br />
The next step will correct the visibility amplitudes for the signal leveling (requantizer gains) that occurs at the inputs of the WIDAR correlator. These levels (per antenna, per polarization, per spectral window) are stored with the measurement set (in the <i>SYSPOWER</i> sub-table). This step is currently not essential, since the levels get set only once at the start of an observation and bandpass calibration will correct for this. But it will make your bandpass plots look better if you have multiple spectral windows. And, more importantly, in general it is possible to trigger the setting of requantizers throughout an observation. This is the case for new observations, when a P-band scan is preceded by a P-band setup scan, which itself is preceded by a scan with a different telescope configuration. <br />
<br />
The correction is done by means of the <i>gencal</i> task, where information in the <i>SYSPOWER</i> sub-table gets translated into a gain table. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltype='rq',caltable='3C129_pband.rq')<br />
</source><br />
<br />
== Dead antennas ==<br />
<br />
Now it is time to have a first visual look at the uncalibrated visibility data. It is important to identify dead antennas / polarizations early on, so we can exclude them from further data processing (which is the most efficient data reduction). A convenient way of doing this is through the <i>plotms</i> task on bright calibrator(s) (see the <i>listobs()</i> output), in our case 3C147 (see Fig. 4a).[[File:3C129_pband_3C147_prebp.png|200px|thumb|right|Figure 4a: <i>plotms</i> output before RFI flagging]]<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY', field='3C147', <br />
plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',iteraxis='baseline', <br />
plotfile='3C129_pband_3C147_prebp.png')<br />
</source><br />
<br />
This will load the <i>plotms</i> window that has queued up all the baselines of antenna ea01 and is currently displaying the first amplitude vs frequency plot with the colors representing the different spectral windows. Note that we are only plotting the XX and YY correlation of the antennas. This is because we expect most of the power in the linear cross correlation products to be in these two correlations. Notice that some spectral windows in particular are badly affected due to radio frequency interference (RFI). This is the issue that we will deal with after identifying dead antennas. If you scroll through the baselines using the green forward button in the <i>plotms</i> viewer, you will notice that the amplitude is particularly low when you encounter ea19. If we make a quick plot of the baselines made by ea19 (put <i>ea19</i> in the antenna field) we can see that there is no power in any of the polarizations indicating this antenna was dead and can be flagged out. It is also good to check for high power levels in cross-hand correlations in comparison to the parallel-hand correlations, which would uncover swapped polarization labels and severely mis-aligned dipoles. At the end of this tutorial some hints on solving for swapped or mislabeled dipole polarizations are given. In addition to the dead antenna, we also make sure that the first two scans are flagged, which were not labelled as setup scans but were used to set the attenuators and requantizers.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms',mode='manual',antenna='ea19')<br />
#<br />
flagdata(vis='3C129.ms',mode='manual',scan='1~2')<br />
</source><br />
<br />
We also noticed sharp RFI peaks during our data examination. To prevent Gibbs ringing, it is best to hanning smooth the data at this juncture before we proceed further with automatic flagging. Now is a good time to get a cup of coffee while hanning smoothing the data. [[File:3C129_pband_3C147_prebp_hanning.png|200px|thumb|right|Figure 4b: <i>plotms</i> output after Hanning smooth and first flagging.]]<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='3C129.ms',outputvis='3C129_pband.ms',datacolumn='data')<br />
</source><br />
<br />
We should replot the primary calibrator to see the effect of hanning smoothing on the data and the RFI (Fig. 4b). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning.png')<br />
<br />
</source><br />
<br />
== Automatic flagging ==<br />
<br />
<br />
It is worthwhile noting at this juncture that the standard P-band setup will ensure that the worst of the RFI is restricted to four of the spectral windows. This particular dataset is an example of higher than average levels of RFI across the band. Before throwing away bad spectral windows by hand, we'll let the automated flaggers in CASA have a go at it. The main task for this is <i>flagdata()</i>. While working on your data, <i>flagdata()</i> produces an abundance of output, not all of which is easy to understand. To help with this, we can get the flag status of our data before and after auto-flagging by running <i>flagdata()</i> in the <i>summary</i> mode. This first call provides the flag statistics of the flagging up to now, before any auto-flagging:<br />
<source lang='python'><br />
summary_1 = flagdata(vis='3C129_pband.ms', mode='summary')<br />
</source><br />
<br />
This returns a python dictionary with flagged versus total visibilities along various axes (antenna, scan, spw, field, correlation, etc.). For example, if we want to know the percentage flagged per scan, run the following (note that the scans may not appear in sorted order):<br />
<source lang='python'><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for id, stats in summary_1[ axis ].iteritems():<br />
print '%s %s: %5.1f percent flagged' % ( axis, id, 100. * stats[ 'flagged' ] / stats[ 'total' ] )<br />
--<br />
</source><br />
<br />
For this example, you will notice that scans 1 and 2 (the dummy scans on 3C147) are 100 percent flagged, which is what we did during the initial flagging.<br />
<br />
We will run <i>flagdata()</i> in the <i>tfcrop</i> mode, which will (per scan, per baseline, per spectral window, per polarization) look for visibility amplitude outliers. It uses a 5-piece polynomial in an attempt to remove any intrinsic bandpass (amplitude) structure (we did not calibrate for bandpass yet). We run the task twice to allow for slightly deeper flagging and separate treatment of the parallel and cross-hand correlations. For the first run, we tell CASA to make a backup of our visibility flag status up to here, giving us an option to restore them if we choose over-aggresive flagging parameters and consequently over-flag our data. The flag backup file name is reported in the log window, and can be found in the <i>3C129_pband.ms.flagversions</i> directory.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=4., freqcutoff=3., maxnpieces=5,<br />
action='apply', display='report', flagbackup=True, combinescans=True, ntime='3600s', correlation='ABS_XY,ABS_YX')<br />
#<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=3., freqcutoff=3., maxnpieces=2,<br />
action='apply', display='report', flagbackup=False, combinescans=True, ntime='3600s', correlation='ABS_XX,ABS_YY')<br />
#<br />
flagdata(vis='3C129_pband.ms', mode='extend')<br />
</source><br />
<br />
As a note of caution: When executing flagdata with <i>mode='extend'</i>, all flags that were ever applied to the dataset will get extended. Repeat execution of this command will incrementally flag more and more around already flagged data.<br />
<br />
Let's get another flag summary to see how much extra data was flagged: <br />
<source lang='python'><br />
# In CASA<br />
summary_2 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
# <br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_2[ axis ].iteritems():<br />
old_stats = summary_1[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning_automatic.png')<br />
<br />
</source><br />
[[File:3C129_pband_3C147_prbp_tfcropped.png|200px|thumb|right|Figure 4c: <i>plotms</i> output after RFI flagging with tfcrop]]<br />
There is quite a bit of RFI that gets missed in RFI-rich spectral windows as can be seen in Fig. 4c. The solution to this is to provide the flagging routines with more contrast between healthy and affected data. For this, we will perform a preliminary bandpass calibration to take out the bandpass shape. We will do some coarse preliminary calibration and apply it to the calibrator before flagging for RFI once more.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.G0', gaintype='G', calmode='p', solint='int', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim'])<br />
#<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K0', gaintype='K', solint='inf', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0'])<br />
#<br />
bandpass(vis='3C129_pband.ms', caltable='3C129_pband.B0', solint='inf', field='3C147',refant='ea09', minsnr=2.0,<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0'])<br />
#<br />
applycal(vis='3C129_pband.ms', field='3C147', applymode='calflagstrict',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0','3C129_pband.B0'] )<br />
<br />
</source><br />
<br />
On the command line you should see a notification stating: "FJones (iononsphere) apply now provisionally includes disp. delay." which indicates that both Faraday rotation as well as dispersive delay corrections for the ionosphere are applied.<br />
<br />
[[File:3c147_postbandpass_rflag.png|200px|thumb|right|Figure 4d: <i>plotms</i> output after RFI flagging with <i>mode='rflag'</i>]]<br />
<br />
We will now flag the corrected data column that contains the coarsely calibrated visibilities which provide better contrast to the flagging algorithms to remove the RFI present. We will begin by running the <i>RFLAG</i> algorithm in the task <i>flagdata</i>. We will again use summary mode to see how much more of the observed data was flagged.<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
summary_3 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
#<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_3[ axis ].iteritems():<br />
old_stats = summary_2[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C147_postbandpass_rflag.png', ydatacolumn='corrected')<br />
</source><br />
<br />
Taking another visual look at the data using <i>plotms</i>, we examine the effect of the automated flagging on the calibrator (Fig. 4d). We find that most spectral windows are RFI free and that about 60% of the data was flagged. We typically expect between 40-60% of flagging due to interference in a significant fraction of the band. In this example, a more careful RFI flagging approach is likely to recover more usable spectrum.<br />
<br />
==Flux Density Calibration==<br />
The initial flagging has cleaned up most of the stray RFI across the band. This allows us to proceed with the actual calibration of the data. Before we get to the calibration tables, it's essential to do the flux density calibration of our calibrator 3C147. This is done by the <i>setjy</i> task. Before we run the task we first clear the preliminary calibration that was carried out to enable better flagging. We do that by running the <i>clearcal</i> task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
clearcal(vis='3C129_pband.ms')<br />
#<br />
setjy(vis='3C129_pband.ms', standard='Scaife-Heald 2012', field='3C147')<br />
</source><br />
<br />
== Delay and bandpass calibration ==<br />
<br />
First, we use the full bandwidth on a single scan on the primary calibrator to determine a single delay per antenna, per polarization. This will determine a single (approximate) phase slope across frequency, mainly caused by propagation effects in the (time-variable) ionosphere, cable length differences, and electronics in the signal paths from antenna feeds to correlator. Note that for a single short (5-10 minutes) scan on the calibrator, we can get away with solving for a time-invariant delay per antenna, per polarization. We will use scan 3 on 3C147.<br />
[[File:3C147_init_delay.png|200px|thumb|right|Figure 5: Initial per antenna delays derived by <i>gaincal</i>.]]<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K1', field='3C147', solint='inf', refant='ea09', gaintype='K',<br />
gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq'],parang=True)<br />
#<br />
plotcal(caltable='3C129_pband.K1', xaxis='antenna', yaxis='delay', markersize=2.0, <br />
plotrange=[0,30,-50.,50.],figfile='3C147_delay.png')<br />
</source><br />
<br />
Since we picked ea09 as our reference antenna, the delays for this antenna ID (=0) are arbitrarily set to zero. All other delays should be within 30 nanosec or so (which they should be for this data set). Larger values should be treated with suspicion, possibly indicating a problem with the antenna or polarization. We will use the bandpass calibration to verify this in the next step.<br />
<br />
To determine the bandpass calibration, we use the same source and scan and apply the delay calibration before solving for the bandpass. Note that we request a minimum SNR of 3, which will make the solve fail for the worst, but not all, channels.<br />
<source lang='python'><br />
# In CASA<br />
bandpass( vis='3C129_pband.ms', caltable='3C129_pband.B1', field='3C147', solint='inf', refant='ea09', minsnr=3.0,<br />
parang = True, gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq','3C129_pband.K1'],<br />
interp=['','','','nearest,nearestflag'])<br />
<br />
</source><br />
<br />
After solving, we inspect the bandpass calibration amplitudes and phase, and flag any obvious residual outliers. Outliers may be found above or below the average bandpass curves, and tend to arise in the same channel (frequency) ranges for all antennas. Don't spend more than 1 minute per antenna for this tutorial and don't worry about missing some bad points; these bad channels will likely get flagged in a later stage anyway. Note that the bandpass amplitudes (Fig. 6) and phases may differ between polarizations of the same antenna. Also note that the bandpass phases across frequency should not have an overall gradient (this should be removed by the delay calibration). Also note that bandpass phases may wrap around from +/-180 to -/+180 degrees. If that is problematic, re-run the phase <i>plotcal()</i> with <i>plotrange = [ 200.,500.,0.,360. ]</i>. Also note that there are no bandpass solutions for antennas ea19 and ea28, and only a few solutions for ea14.<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='amp', iteration='antenna', markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='phase', iteration='antenna', plotrange=[ 200.,500.,-180.,180. ], markersize = 2.0)<br />
</source><br />
{|<br />
| [[File:3C147_bp0_ea01.png|200px|thumb|right|Figure 6a: Initial bandpass amplitudes for antenna ea01.]]<br />
| [[File:3C147_bp0_phase_ea01.png|200px|thumb|right|Figure 6b: Initial bandpass phase for antenna ea01.]]<br />
|}<br />
<br />
Flagging is made easier by maximizing the plotcal() window. We don't set the scale for plotting the bandpass amplitudes, which is convenient when flagging outliers as it will auto-rescale. While plotting the bandpass phases we do fix the scale, as it provides a better feeling for the relevant magnitude of phase outliers. Flagging of outliers is done using the <i>Mark Region</i> and <i>Flag</i> buttons. For more information on interactive flagging, see the topical guide [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Flagging VLA Data in CASA]. We are using this mechanism to efficiently flag data manually, since it is done per antenna rather than per baseline. These 'flags' will not be permanent until we call <i>applycal()</i> later on, but are effective while applying the calibration on the fly, as we will do in subsequent calibration steps.<br />
<br />
== Gain calibration ==<br />
<br />
With the delay and bandpass calibration in place, we will now look more closely at the time-variable behaviour of the VLA. We use the same scan on our primary calibrator, 3C147, over the full effective bandwidth to determine gain calibrations: one complex value per antenna, per polarization, per integration time. The delay and bandpass tables are applied on the fly.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal( vis='3C129_pband.ms', caltable='3C129_pband.G1', field='3C147', solint = 'int',<br />
refant = 'ea09', minsnr = 3.0, gaintype = 'G', calmode = 'ap',<br />
gaintable = ['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim', '3C129_pband.K1', '3C129_pband.B1' ],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag' ], parang = True )<br />
<br />
</source><br />
{|<br />
| [[File:3C147_Gamp_ea01.png|200px|thumb|right|Figure 7a: Initial gain amplitude solutions for antenna ea01.]]<br />
| [[File:3C147_Gphase_ea01.png|200px|thumb|right|Figure 7b: Initial gain phase solutions for antenna ea01.]]<br />
|}<br />
<br />
Similar to the bandpass flagging, we will flag outlier gain solutions in both amplitude and phase. For the amplitudes, the final scatter around an average close to one should be a few percent (see Fig. 7). Note antennas ea08, ea09 have particularly noisy solutions. Remember that the gain solutions for both polarizations can be flagged together, as we will lose the equivalent data of the surviving polarization anyway.<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='amp', iteration='antenna', markersize=2.0 )<br />
#<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='phase', iteration='antenna', <br />
plotrange=[ None,None,-180.,180.], markersize=2.0)<br />
</source><br />
<br />
Note that when plotting gains against time, it is convenient to use auto-scaling of the time axis by putting <i>None</i> in the first two fields of <i>plotrange</i>.<br />
<br />
Next, we will prepare a smoothed and interpolated version of the gain calibration table, which will be applied later to the target field data. This prevents flagging of target field data when one of the edges of a calibrator scan is flagged for one or more antennas. The result can be inspected with <i>plotcal</i>.<br />
<source lang='python'><br />
# In CASA<br />
smoothcal(vis='3C129_pband.ms', tablein='3C129_pband.G1', caltable='3C129_pband.Gs1', smoothtype='median', smoothtime = 60.*60.)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='amp', iteration='antenna', plotrange=[ None,None,0.8,1.2 ], markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='phase', iteration='antenna', plotrange=[ None,None,-10.,10. ], markersize=2.0)<br />
<br />
</source><br />
<br />
== Transfer of calibrations to the target field ==<br />
<br />
So far we have used one scan on the primary calibrator 3C147 to derive various calibration tables. These calibrations will now be applied to all scans of all sources in our measurement set, which includes 3C147 itself and our target field 3C129. As described above, the task <i>applycal()</i> creates a <i>CORRECTED_DATA</i> column in which the calibrated visibilities get stored. This may take a while.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='3C129_pband.ms', parang=True, applymode='calflagstrict', flagbackup=True, gaintable=['3C129_pband.antpos','3C129_pband.rq',<br />
'3C129_pband.tecim','3C129_pband.K1','3C129_pband.B1','3C129_pband.Gs1'],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag', 'nearest,nearestflag'])<br />
</source><br />
<br />
Now we split off the calibrated target field data, meaning that the visibilities of source 3C129 get copied from the <i>CORRECTED_DATA</i> column to the <i>DATA</i> column of a new measurement set. This is convenient for further processing. <br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='3C129_pband.ms', outputvis='3C129_pband_target.ms', datacolumn='corrected', field='3C129')<br />
</source><br />
<br />
It is also helpful to perform additional flagging for any residual RFI left behind in our target after all our calibration steps have been completed and we begin the imaging of our data.<br />
<br />
<source land='python'><br />
flagdata(vis='3C129_pband_target.ms', mode='rflag')<br />
flagdata(vis='3C129_pband_target.ms', mode='extend')<br />
</source><br />
<br />
<br />
== Statwt ==<br />
Before we go ahead and image our target field. We would like to weight any residual low level RFI down. This is done using the task <i>statwt</i>. The task runs through the data and computes the rms of the data and weighting down outliers. This makes a big impact for imaging of a target in the presence of low level residual RFI. For our purposes we will bin the data in a time interval larger than the integration time and run statwt as follows.<br />
<br />
<source lang='python'> <br />
statwt(vis='3C129_pband_target.ms', datacolumn='data', timebin=30.)<br />
</source><br />
<br />
With data weights for outliers being set we will now proceed with the imaging and self-calibration of our data.<br />
==Imaging==<br />
<br />
At this point, we're ready to make a first image of our target field. Imaging in CASA is done by means of the <i>tCLEAN</i> task. The task implements algorithms for wide field imaging, such as W-Projection (Cornwell et al. 2008 http://arxiv.org/abs/0807.4161) and Multi Term Multi Frequency Synthesis ( Rau et al. 2011 http://arxiv.org/abs/1106.2745) both of which we will utilize to make our initial image. Even though our source of concern, the radio galaxy, lies in the centre of our field. The observations were carried out in B configuration resulting in an effective maximum resolution of approximately 5x5 arcsecs. To effectively model the source spectral index in the sky, we will utilize the MTMFS algorithm and use the W-Projection algorithm to make a wide field image (The full beam at P Band is about 3 degrees in diameter). So, going by these requirements, we can now compute the required cell and image size. Consulting the resolution guide of NRAO science page, we can see that the expected HPBW for B - Configuration, P -Band Observation is 18.5 arcsec. So, we can sample it well by using a 5 arcsec cellsize. Having done that, if we decide to make a wide field image to account for all the point sources we will set the image size parameter to 4096 pixels. To enable the wide-field algorithm, we set the <i>gridder='wproject'</i>: this invokes the W-Projection algorithm, upon which we set the number of W-Projection planes to be 128. We set the deconvolution mode as Multi Term Multi Scale Multi Frequency Synthesis using the <i>deconvolver='mtmfs'</i> parameter along with the number of Taylor terms to be considered during imaging to be 2 and choosing three different scales to model the point source and extended emission in our field well, we do this by setting <i> scales=[0,20,30]</i>. This allows for the source spectral variation to be modeled by a second order polynomial. We launch the interactive tclean process, but only for <i>spw='3~8'</i> to begin with. Note, running the first clean call will take at least ~10 min to process.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_initial_clean', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30]<br />
pblimit=0.01, savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
</source><br />
<br />
{|<br />
| [[File:3C129_radio_gal_residual.png|200px|thumb|right|Figure 10a: Resulting residuals of 3C129 after first call of interactive <i>clean</i>.]]<br />
| [[File:3C129_radio_gal_clean.png|200px|thumb|right|Figure 10b: Resulting image of 3C129 after completing the first <i>clean</i>.]]<br />
|}<br />
The <i>tCLEAN</i> command launches an interactive session after a 100 iterations of clean and produces a wide field map with two sources at the center and a lot of bright sources far out in the field. As it is a snapshot image, the bright sources have significant side lobes and so tight clean boxing can help. This can done in the interactive viewer interface that pops up. If we proceed with interactive clean with subsequent steps to keep boxing out the strong sources that pop up in the image, we finally see the extended emission from the target radio galaxy begin to emerge. Continue boxing and cleaning to ensure that the residuals of the boxed cleaning look noise like (~10000 clean iterations). We now stop the interactive task and look at the final image it produced.<br />
<br />
<source lang='python'><br />
# In CASA<br />
viewer('3C129_initial_clean.image.tt0')<br />
</source><br />
<br />
The recovered structure at the center of the map should resemble what is shown in Fig. 10b. If the image looks worse then boxing of bright features in the map needs to be done more carefully. Note the image has some sources still showing strong side lobes and imaging artifacts around them. We expected this as we have carried out phase calibration only on our flux calibrator and have just transferred the solutions over to our target field. Since we used the <i>usescratch=True</i>, the MODEL_DATA column in the measurement set now contains the initial image model, which we will self calibrate against to produce a better image. Also in clean, do notice that the spw's utilized are the cleanest spectral windows that are totally RFI free.<br />
<br />
==Self Calibration ==<br />
<br />
We now proceed to compute gain phase solutions for our target field using the <i>gaincal</i> task as the first step in self-calibration.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScG0', field='3C129', solint='inf', refant='ea09', <br />
spw='3~8',minsnr=3.0, gaintype='G', parang=True, calmode='p')<br />
<br />
bandpass( vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScB0', field='3C129', solint='inf', refant='ea09', minsnr=3.0, spw='3~8',<br />
parang = True, gaintable=['3C129_pband_target.ScG0'],<br />
interp=['','','','nearest,nearestflag'])<br />
applycal(vis='3C129_pband_target.ms', gaintable=['3C129_pband_target.ScG0','3C129_pband_target.ScB0'], spw='3~8', applymode='calflagstrict')<br />
</source><br />
<br />
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the <i>tCLEAN</i> command once again.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_clean_sc0', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30]<br />
pblimit=0.01,savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
<br />
viewer('3C129_clean_sc0.image.tt0')<br />
</source><br />
[[File:3C129_radio_gal_cselfcal0.png|200px|thumb|right|Figure 11: Resulting image of 3C129 after completing a first selfcal.]]<br />
<br />
On boxing and cleaning, we already notice that the imaging artifacts have reduced significantly. We also see that the target source appears to contain more structure and a greater amount of flux. Further self calibration iterations involving an amplitude & phase gain calibration, and target bandpass calibration, are all possible steps that can be explored. <br />
<br />
== Appendix: Some P-band data issues you may want to know about ==<br />
<br />
Unfortunately, there are a few known issues with JVLA P-band observing. Most problems were discovered during the P-band commissioning period and have been fixed for newer data sets, but the archive keeps part of this history alive. Here we will go over some of those issues, and (if possible) provide ways of fixing them.<br />
<br />
=== Polarization labeling ===<br />
<br />
For a long period, P-band feeds have been labelled as being circular (R and L), while they are linear (X and Y). It is still possible to generate data with incorrect polarization labeling, however for standard observations this should not be the case anymore. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem. This may seem mostly harmless, but does make a difference for polarization calibration. The following (non-standard) task will check and fix this (and another related problem of swapped polarizations):<br />
<br />
Download [http://casa.nrao.edu/Data/EVLA/Pband/casa_vla_lowband.tar.gz this tarball] in your CASA working directory, open a shell there, and type the following commands:<br />
<br />
<source lang='bash'><br />
tar -xzvf casa_vla_lowband.tar.gz<br />
<br />
buildmytasks<br />
</source><br />
<br />
Now go back to your CASA session and type:<br />
<br />
<source lang='python'><br />
# In CASA<br />
execfile('mytasks.py')<br />
#<br />
fixlowband(vis='TSUB0001_pband_prehs.ms')<br />
</source><br />
<br />
Re-run the <i>listobs()</i> task to check if the fix was correctly applied.<br />
<br />
=== Swapped polarizations ===<br />
<br />
Throughout the whole history of the new VLA low-band system, even up to this day, there have been mistakes in the cabling of the full signal chain. This results in that some antennas have the X-polarization signal come in as Y and vice versa. The way to notice this in visibility data is that for baselines with one swapped antenna most power will be in the cross-hand correlations (XY and YX) rather than the parallel-hand correlations (XX and YY). Once antennas with this feature are manually identified, a contributed task <i>swappol()</i> is available to fix this problem (see under polarization labelling).<br />
<br />
=== Double data descriptor entries ===<br />
<br />
The data description table is part of the measurement set, and provides a link between the recorded visibilities, the spectral window information and the polarization information. For some data sets, the data description table contains two entries for each P-band spectral window, one pointing towards a circular (RL) polarization definition, and one pointing towards a linear (XY) polarization definition. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem.<br />
<br />
=== Continuous Radio Frequency Interference (RFI) ===<br />
<br />
The wide bandwidth of the low-band receiver is (unfortunately) guaranteed to contain significant amounts of RFI. There are a few RFI sources that are active all the time, and are visible in all array configurations. The default P-band observing setup of 16 x 16 MHz tries to capture as much of the continuous RFI in as few spectral windows as possible, allowing for a simple RFI mitigation strategy in which these spectral windows (spws 1,2 and 9,10, but possibly more) can be immediately flagged.<br />
<br />
=== Two spectral window setup ===<br />
<br />
In the early commissioning period, the default setup for P-band observing was to use 2 spectral windows of 1024 channels each to cover 256 MHz of bandwidth from 230 - 486 MHz. However, it was noticed that strong, narrow-band RFI events were causing data to be lost for the whole spectral window in which they occured. To make the system more robust against data loss from these events, the frequency range was divided into 16 x 16 MHz, and shifted downwards by 6 MHz to capture ever-present RFI into as few spectral windows as possible. If your data has only two P-band spectral windows, please be aware that a higher data loss due to RFI is possible. There is no way to repair this.<br />
<br />
=== Bandpass ripples ===<br />
<br />
Due to signal reflections in cables and within the VLA dish, sinusoidal amplitude and phase modulations are always present in P-band data. This is most easily seen in bandpass calibration plots of amplitude versus frequency. In some cases, mostly due to cable & connector problems, these modulations can be very strong (~up to 50 percent of the average amplitude level). These modulations tend not to vary over the duration of an observation and can therefore be removed through bandpass and polarization calibration. If they are found to be variable (e.g., by inspecting bandpass solutions for separate calibrator scans), the offending antenna / polarization should be flagged.<br />
<br />
=== High cross-polarization ===<br />
<br />
On each antenna, the P-band feed (dipole) is visually aligned with respect to the primary focus support legs. This is normally done within 5-10 degrees accuracy. On rare occasions the dipole on one antenna has accidentally rotated to much larger angles because the locking bolt on the back of the feed was not completely tightened. The result is a high cross-polarization in the baseline visibilities that include this antenna. If this is the case, it is safest to flag this data, since there is a possibility that the dipole has rotated during the observations.<br />
<br />
=== Instrumental Polarization Calibration ===<br />
<br />
At present polarization P-band observations are in RSRO mode only. While polarization leakage calibration has been highlighted in the past reports it does not directly impact imaging in Stokes I and hence has been dropped from this casaguide.<br />
<br />
== Useful links ==<br />
<br />
* [http://www.aoc.nrao.edu/lbr The EVLA Low-Band Receiver System Status & User Pages]<br />
* [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/topical-guides/vlofreq VLA Very Low Frequency Observing]<br />
<br />
<br />
{{Checked 5.4.1}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0&diff=26244VLA Radio galaxy 3C 129: P-band continuum tutorial-CASA5.4.02018-12-01T23:01:33Z<p>Emomjian: </p>
<hr />
<div><!--By [http://www.aoc.nrao.edu/~pjaganna/Site/Home.html Preshanth Jagannathan] & [https://safe.nrao.edu/wiki/bin/view/Main/HuibIntemaWebHome Huib Intema] & Frank Schinzel--><br />
<br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b><p></p> <br />
<b> Note: This guide may take up to 2-days to complete </b><p></p> <br />
If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
<br />
This webpage provides a basic description on how to use CASA to reduce data from the upper part of the new VLA low-band system, also known as P-band, covering roughly 220-480 MHz. The goal is to make a wide-field continuum stokes I image of a typical blank field using the full effective bandwidth. The dataset that will be downloaded in this guide has a size of 26.46 GB and will require a total of ~140 GB of disk space if all generated files are kept. <br />
<br />
''Note: Starting with CASA 5.0 IPython 5.1.0 is used which for interactive CASA sessions. This does not allow to readily paste and execute multiple lines of code like in previous versions of the interpreter. If this is needed at any point in this tutorial, we use "%cpaste" to allow the copy/paste of loops, conditions, etc.. Press "Enter" (or "Return") after "--" at the end, this stops (or exits) the copy/paste prompt and will return you to the CASA prompt.''<br />
<br />
== Obtaining the raw data ==<br />
<br />
For this guide, we'll use some test data that was taken in B-configuration. To get a copy of the raw data, download [https://casa.nrao.edu/Data/EVLA/Pband/P_band_3C129.tgz P_band_3C129.tgz] onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
<br />
Alternatively, the raw data can also be obtained from the NRAO archive, go to the [https://archive.nrao.edu/archive/advquery.jsp NRAO advance archive query page], and enter the following search parameter in the field <i>Archive File ID</i> (leave the rest on default):<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
imag-test-copy.57080.956837025464<br />
</pre><br />
</div><br />
<br />
The search should return a single row for Archive File: imag-test-copy.57080.956837025464, taken on Feb. 27, 2015 (or "15-Feb-27 22:57:52"). Check the box next to it. Next, fill in your email and select the "SDM-BDF dataset (all files)" option. Note, you may find it easier to download a tar file, in which case, you'll want to check the box for Create MS or SDM tar file. Click "Get My Data" and follow the on-screen prompts to reach the next page, where you should click "Retrieve over internet". The next page should report that the data staging is in progress. Wait until you receive an e-mail reporting that your archive data is copied, which should take a few minutes. Download the data onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing <br />
<source lang='bash'><br />
casa<br />
</source><br />
on the (linux) command line. This should start a CASA interactive python (iPython) session, and open a separate log window. The CASA version is reported at startup, both in the python session and the log window.<br />
<br />
== Importing the raw data into CASA ==<br />
<br />
We will begin by importing our data into the measurement set format (CASA standard) from the binary format (SDM-BDF) as downloaded from the archive. We do this by means of the importasdm task. It is important to note that the importevla task has been deprecated.<br />
<br />
<source lang='python'><br />
# In CASA<br />
importasdm(asdm='imag-test-copy.57080.956837025464', vis ='3C129.ms', savecmds=True, outfile='importflags.txt')<br />
<br />
</source><br />
<br />
This task calls the external tool <i>asdm2MS</i> to perform the conversion. Together with the visibility data, also the flag commands from the VLA online system are imported. These flags are not directly applied, but stored in the <i>importflags.txt</i> file and applied later.<br />
<br />
== Preliminary data inspection ==<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms', verbose = True, listfile = '3C129.listobs')<br />
</source><br />
<br />
Alternatively, you can also output to the logger by running the command without the listfile argument. The sample listobs output shown has flag <i>verbose=False</i> for the sake of brevity.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/fschinze/P_band_tutorial/3C129.ms MS Version 2<br />
================================================================================<br />
Observer: Project: uid://evla/sdm/X2<br />
Observation: EVLA(27 antennas)<br />
Data records: 4714983 Total elapsed time = 1422 seconds<br />
Observed from 27-Feb-2015/22:57:54.0 to 27-Feb-2015/23:21:36.0 (UTC)<br />
<br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C147 05:42:36.127300 +49.51.07.17800 J2000 0 1307124<br />
1 NONE 3C129 04:48:58.200000 +45.02.01.00000 J2000 1 3407859<br />
Spectral Windows: (19 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 EVLA_P#A0C0#0 128 TOPO 224.000 125.000 16000.0 231.9375 12 XX XY YX YY<br />
1 EVLA_P#A0C0#1 128 TOPO 240.000 125.000 16000.0 247.9375 12 XX XY YX YY<br />
2 EVLA_P#A0C0#2 128 TOPO 256.000 125.000 16000.0 263.9375 12 XX XY YX YY<br />
3 EVLA_P#A0C0#3 128 TOPO 272.000 125.000 16000.0 279.9375 12 XX XY YX YY<br />
4 EVLA_P#A0C0#4 128 TOPO 288.000 125.000 16000.0 295.9375 12 XX XY YX YY<br />
5 EVLA_P#A0C0#5 128 TOPO 304.000 125.000 16000.0 311.9375 12 XX XY YX YY<br />
6 EVLA_P#A0C0#6 128 TOPO 320.000 125.000 16000.0 327.9375 12 XX XY YX YY<br />
7 EVLA_P#A0C0#7 128 TOPO 336.000 125.000 16000.0 343.9375 12 XX XY YX YY<br />
8 EVLA_P#A0C0#8 128 TOPO 352.000 125.000 16000.0 359.9375 12 XX XY YX YY<br />
9 EVLA_P#A0C0#9 128 TOPO 368.000 125.000 16000.0 375.9375 12 XX XY YX YY<br />
10 EVLA_P#A0C0#10 128 TOPO 384.000 125.000 16000.0 391.9375 12 XX XY YX YY<br />
11 EVLA_P#A0C0#11 128 TOPO 400.000 125.000 16000.0 407.9375 12 XX XY YX YY<br />
12 EVLA_P#A0C0#12 128 TOPO 416.000 125.000 16000.0 423.9375 12 XX XY YX YY<br />
13 EVLA_P#A0C0#13 128 TOPO 432.000 125.000 16000.0 439.9375 12 XX XY YX YY<br />
14 EVLA_P#A0C0#14 128 TOPO 448.000 125.000 16000.0 455.9375 12 XX XY YX YY<br />
15 EVLA_P#A0C0#15 128 TOPO 464.000 125.000 16000.0 471.9375 12 XX XY YX YY<br />
16 EVLA_P#B0D0#16 64 TOPO 310.000 500.000 32000.0 325.7500 15 XX XY YX YY<br />
17 EVLA_P#B0D0#17 1024 TOPO 310.000 62.500 64000.0 341.9688 15 XX XY YX YY<br />
18 EVLA_P#B0D0#18 64 TOPO 342.000 500.000 32000.0 357.7500 15 XX XY YX YY<br />
Antennas: 27 'name'='station'<br />
ID= 0-3: 'ea01'='N12', 'ea02'='N16', 'ea03'='W28', 'ea05'='E20',<br />
ID= 4-7: 'ea06'='N20', 'ea07'='N32', 'ea08'='E28', 'ea09'='W08',<br />
ID= 8-11: 'ea10'='E36', 'ea11'='W12', 'ea12'='N36', 'ea13'='W04',<br />
ID= 12-15: 'ea14'='E08', 'ea15'='E24', 'ea16'='W24', 'ea17'='E04',<br />
ID= 16-19: 'ea18'='W36', 'ea19'='MAS', 'ea20'='N04', 'ea21'='E32',<br />
ID= 20-23: 'ea22'='N24', 'ea23'='E16', 'ea24'='W32', 'ea25'='W20',<br />
ID= 24-26: 'ea26'='E12', 'ea27'='N08', 'ea28'='N28'<br />
<br />
</pre><br />
<br />
You can inspect the output text file <i>3C129.listobs</i> in your favorite text editor (e.g., <i>gedit</i>). When taking some time to familiarize yourself with the output format, you will see that:<br />
* The observation consists of 5 scans, namely:<br />
# 3C147 (hardware setup)<br />
# 3C147 (setting requantizers)<br />
# 3C147 (primary calibrator)<br />
# 3C129 (target)<br />
# 3C129 (target)<br />
* The frequency coverage is 222 - 478 MHz, divided into 16 x 16 MHz spectral windows, each having 128 x 0.125 MHz channels. There are also two spectral windows with 64x0.5 MHz and one spectral window with 1024x0.0625 MHz. <br />
* Visibilities are recorded every 2 seconds in full polarization (4 polarization products labelled XX,XY,YX,YY).<br />
* Of the 28 VLA antennas labelled ea01 - ea028, antenna ea04 is not participating in this observation.<br />
* Source, antenna and spectral window IDs start at zero, but scan IDs start at one.<br />
<br />
<br />
[[File:3C129_pband_plotants.png|200px|thumb|right|Figure 1: plotants figure]] <br />
<br />
The array configuration can be inspected using:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotants(vis='3C129.ms', figfile='3C129_pband_plotants.png')<br />
</source><br />
<br />
== Initial processing steps - Antenna Position, Requantizer Gain, Ionospheric Correction ==<br />
<br />
When importing the data, we saved the flags from the online system to an ASCII text file. This gives us the opportunity to review the flag commands before applying them. For our data set, the flag file is called <i>importflags.txt</i>. Please load this file into your favorite text editor. The bulk of the flag commands refer to times when the VLA is slewing (<i>ANTENNA_NOT_ON_SOURCE</i>) or when the movable secondary reflector of the VLA's Cassegrain system is not in place (<i>SUBREFLECTOR_ERROR</i>). The latter can cause antenna gain variations (amplitude and phase), so it is safest to apply all the flags. The final two rows in the flag file will remove visibilities that are pure zero, and flag antennas that are partly blocked by other antennas (shadowing) which occurs mostly in compact configurations when observing along a VLA arm).<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms', mode='list', inpfile='importflags.txt', action='apply', reason='any', flagbackup=True)<br />
</source><br />
This call, before applying the new flags to the data, created a new flagbackup file called flagdata_1 which can be used by the task <i>flagmanager</i> to restore the state of the measurement set prior applying the flags from <i>importflags.txt</i>.<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
Now that we have applied the import flags, we will correct for the antenna position offsets. Antenna positional errors translates to an error in the measured visibilities and need to be accounted for before we proceed with any of the other calibration steps. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltable='3C129_pband.antpos',caltype='antpos')<br />
</source><br />
<br />
The logger output shows that there are 13 antennas with positional offsets: <br />
<pre><br />
<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea03 : 0.00290 0.00000 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea08 : 0.00000 0.00390 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea10 : -0.00170 0.00000 0.00400<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea11 : 0.00000 0.00160 -0.00170<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea13 : 0.00000 0.00050 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea16 : -0.00180 0.00680 0.00230<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea18 : 0.00470 0.00350 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea21 : -0.00160 0.00000 0.00210<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea23 : 0.00140 0.00200 -0.00150<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea24 : 0.00380 0.00000 -0.00340<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea25 : 0.00330 0.00000 -0.00110<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea26 : 0.00000 0.00190 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea27 : -0.00120 0.00070 0.00000<br />
<br />
<br />
</pre><br />
<br />
=== Ionospheric TEC Corrections ===<br />
[[File:Pband_vla_tecu_new.png|200px|thumb|right|Figure 2: TEC value vs time generated by <i>tec_maps.create()</i>]] <br />
<br />
Low frequency observations are affected by the ionosphere. A delay in the signal path is introduced between the two polarization of light that varies both as a function of time and line of sight (direction dependent). The delay is proportional to the Total Electron Content (TEC) along the line of sight and is inversely proportional to the square of the frequency. GPS measurements at two different frequencies provides us with an estimate of the TEC per square meter. This correction has been implemented in CASA which we shall apply as a calibration table by means of the <i>gencal</i> task. The task requires a TEC map that we will generate utilizing CASA recipes. For more information, see the section on [https://casa.nrao.edu/casadocs/casa-5.0.0/synthesis-calibration/preparing-for-calibration/#title5 ionospheric corrections] in the CASA documentation archives. For some versions of CASA, the first plot to the right (Figure 2) may not be generated if you have already run plotants: to generate this plot, you will need to quit CASA and start it again. If the IONEX data file was not downloaded you might be missing the curl package in the environment you are running CASA in. In this case you can provide the required file manually by downloading: ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/2015/058/igsg0580.15i.Z, (for the tutorial it is also also available in an unzipped version [http://casa.nrao.edu/Data/EVLA/Pband/igsg0580.15i here]) unziping it, and placing it in the directory from which you started CASA. The current implementation in CASA is in the development stage, however since the CASA 5.0 release both the Faraday rotation correction and dispersive delay corrections for the ionosphere are enabled. In previous releases only Faraday rotation corrections are enabled. In this example we will generate the calibration table and apply it throughout calibration. <br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
#<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='3C129.ms',doplot=True)<br />
#[NOTE: If you are using CASA 5.0.0, or earlier this should be: tec_image, tec_rms_image = tec_maps.create(vis='3C129.ms', doplot=True)]<br />
#<br />
gencal(vis='3C129.ms',caltable='3C129_pband.tecim',caltype='tecim',infile=tec_image)<br />
# Spatial plots like shown in Figure 3 can be looked at using the CASA viewer for the TEC values<br />
viewer('3C129.ms.IGS_TEC.im')<br />
# and for TEC rms<br />
viewer('3C129.ms.IGS_RMS_TEC.im')<br />
</source><br />
<br />
[[File:Pband_tec_new.png|200px|thumb|right|Figure 3: TEC map movie generated by the TEC tasks.]] <br />
<br />
A word of caution regarding the TEC map generation. The IGS website updates measurements only two weeks after the date of observation.<br />
<br />
=== Requantizer Gains ===<br />
<br />
The next step will correct the visibility amplitudes for the signal leveling (requantizer gains) that occurs at the inputs of the WIDAR correlator. These levels (per antenna, per polarization, per spectral window) are stored with the measurement set (in the <i>SYSPOWER</i> sub-table). This step is currently not essential, since the levels get set only once at the start of an observation and bandpass calibration will correct for this. But it will make your bandpass plots look better if you have multiple spectral windows. And, more importantly, in general it is possible to trigger the setting of requantizers throughout an observation. This is the case for new observations, when a P-band scan is preceded by a P-band setup scan, which itself is preceded by a scan with a different telescope configuration. <br />
<br />
The correction is done by means of the <i>gencal</i> task, where information in the <i>SYSPOWER</i> sub-table gets translated into a gain table. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltype='rq',caltable='3C129_pband.rq')<br />
</source><br />
<br />
== Dead antennas ==<br />
<br />
Now it is time to have a first visual look at the uncalibrated visibility data. It is important to identify dead antennas / polarizations early on, so we can exclude them from further data processing (which is the most efficient data reduction). A convenient way of doing this is through the <i>plotms</i> task on bright calibrator(s) (see the <i>listobs()</i> output), in our case 3C147 (see Fig. 4a).[[File:3C129_pband_3C147_prebp.png|200px|thumb|right|Figure 4a: <i>plotms</i> output before RFI flagging]]<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY', field='3C147', <br />
plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',iteraxis='baseline', <br />
plotfile='3C129_pband_3C147_prebp.png')<br />
</source><br />
<br />
This will load the <i>plotms</i> window that has queued up all the baselines of antenna ea01 and is currently displaying the first amplitude vs frequency plot with the colors representing the different spectral windows. Note that we are only plotting the XX and YY correlation of the antennas. This is because we expect most of the power in the linear cross correlation products to be in these two correlations. Notice that some spectral windows in particular are badly affected due to radio frequency interference (RFI). This is the issue that we will deal with after identifying dead antennas. If you scroll through the baselines using the green forward button in the <i>plotms</i> viewer, you will notice that the amplitude is particularly low when you encounter ea19. If we make a quick plot of the baselines made by ea19 (put <i>ea19</i> in the antenna field) we can see that there is no power in any of the polarizations indicating this antenna was dead and can be flagged out. It is also good to check for high power levels in cross-hand correlations in comparison to the parallel-hand correlations, which would uncover swapped polarization labels and severely mis-aligned dipoles. At the end of this tutorial some hints on solving for swapped or mislabeled dipole polarizations are given. In addition to the dead antenna, we also make sure that the first two scans are flagged, which were not labelled as setup scans but were used to set the attenuators and requantizers.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms',mode='manual',antenna='ea19')<br />
#<br />
flagdata(vis='3C129.ms',mode='manual',scan='1~2')<br />
</source><br />
<br />
We also noticed sharp RFI peaks during our data examination. To prevent Gibbs ringing, it is best to hanning smooth the data at this juncture before we proceed further with automatic flagging. Now is a good time to get a cup of coffee while hanning smoothing the data. [[File:3C129_pband_3C147_prebp_hanning.png|200px|thumb|right|Figure 4b: <i>plotms</i> output after Hanning smooth and first flagging.]]<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='3C129.ms',outputvis='3C129_pband.ms',datacolumn='data')<br />
</source><br />
<br />
We should replot the primary calibrator to see the effect of hanning smoothing on the data and the RFI (Fig. 4b). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning.png')<br />
<br />
</source><br />
<br />
== Automatic flagging ==<br />
<br />
<br />
It is worthwhile noting at this juncture that the standard P-band setup will ensure that the worst of the RFI is restricted to four of the spectral windows. This particular dataset is an example of higher than average levels of RFI across the band. Before throwing away bad spectral windows by hand, we'll let the automated flaggers in CASA have a go at it. The main task for this is <i>flagdata()</i>. While working on your data, <i>flagdata()</i> produces an abundance of output, not all of which is easy to understand. To help with this, we can get the flag status of our data before and after auto-flagging by running <i>flagdata()</i> in the <i>summary</i> mode. This first call provides the flag statistics of the flagging up to now, before any auto-flagging:<br />
<source lang='python'><br />
summary_1 = flagdata(vis='3C129_pband.ms', mode='summary')<br />
</source><br />
<br />
This returns a python dictionary with flagged versus total visibilities along various axes (antenna, scan, spw, field, correlation, etc.). For example, if we want to know the percentage flagged per scan, run the following (note that the scans may not appear in sorted order):<br />
<source lang='python'><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for id, stats in summary_1[ axis ].iteritems():<br />
print '%s %s: %5.1f percent flagged' % ( axis, id, 100. * stats[ 'flagged' ] / stats[ 'total' ] )<br />
--<br />
</source><br />
<br />
For this example, you will notice that scans 1 and 2 (the dummy scans on 3C147) are 100 percent flagged, which is what we did during the initial flagging.<br />
<br />
We will run <i>flagdata()</i> in the <i>tfcrop</i> mode, which will (per scan, per baseline, per spectral window, per polarization) look for visibility amplitude outliers. It uses a 5-piece polynomial in an attempt to remove any intrinsic bandpass (amplitude) structure (we did not calibrate for bandpass yet). We run the task twice to allow for slightly deeper flagging and separate treatment of the parallel and cross-hand correlations. For the first run, we tell CASA to make a backup of our visibility flag status up to here, giving us an option to restore them if we choose over-aggresive flagging parameters and consequently over-flag our data. The flag backup file name is reported in the log window, and can be found in the <i>3C129_pband.ms.flagversions</i> directory.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=4., freqcutoff=3., maxnpieces=5,<br />
action='apply', display='report', flagbackup=True, combinescans=True, ntime='3600s', correlation='ABS_XY,ABS_YX')<br />
#<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=3., freqcutoff=3., maxnpieces=2,<br />
action='apply', display='report', flagbackup=False, combinescans=True, ntime='3600s', correlation='ABS_XX,ABS_YY')<br />
#<br />
flagdata(vis='3C129_pband.ms', mode='extend')<br />
</source><br />
<br />
As a note of caution: When executing flagdata with <i>mode='extend'</i>, all flags that were ever applied to the dataset will get extended. Repeat execution of this command will incrementally flag more and more around already flagged data.<br />
<br />
Let's get another flag summary to see how much extra data was flagged: <br />
<source lang='python'><br />
# In CASA<br />
summary_2 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
# <br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_2[ axis ].iteritems():<br />
old_stats = summary_1[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning_automatic.png')<br />
<br />
</source><br />
[[File:3C129_pband_3C147_prbp_tfcropped.png|200px|thumb|right|Figure 4c: <i>plotms</i> output after RFI flagging with tfcrop]]<br />
There is quite a bit of RFI that gets missed in RFI-rich spectral windows as can be seen in Fig. 4c. The solution to this is to provide the flagging routines with more contrast between healthy and affected data. For this, we will perform a preliminary bandpass calibration to take out the bandpass shape. We will do some coarse preliminary calibration and apply it to the calibrator before flagging for RFI once more.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.G0', gaintype='G', calmode='p', solint='int', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim'])<br />
#<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K0', gaintype='K', solint='inf', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0'])<br />
#<br />
bandpass(vis='3C129_pband.ms', caltable='3C129_pband.B0', solint='inf', field='3C147',refant='ea09', minsnr=2.0,<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0'])<br />
#<br />
applycal(vis='3C129_pband.ms', field='3C147', applymode='calflagstrict',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0','3C129_pband.B0'] )<br />
<br />
</source><br />
<br />
On the command line you should see a notification stating: "FJones (iononsphere) apply now provisionally includes disp. delay." which indicates that both Faraday rotation as well as dispersive delay corrections for the ionosphere are applied.<br />
<br />
[[File:3c147_postbandpass_rflag.png|200px|thumb|right|Figure 4d: <i>plotms</i> output after RFI flagging with <i>mode='rflag'</i>]]<br />
<br />
We will now flag the corrected data column that contains the coarsely calibrated visibilities which provide better contrast to the flagging algorithms to remove the RFI present. We will begin by running the <i>RFLAG</i> algorithm in the task <i>flagdata</i>. We will again use summary mode to see how much more of the observed data was flagged.<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
summary_3 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
#<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_3[ axis ].iteritems():<br />
old_stats = summary_2[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C147_postbandpass_rflag.png', ydatacolumn='corrected')<br />
</source><br />
<br />
Taking another visual look at the data using <i>plotms</i>, we examine the effect of the automated flagging on the calibrator (Fig. 4d). We find that most spectral windows are RFI free and that about 60% of the data was flagged. We typically expect between 40-60% of flagging due to interference in a significant fraction of the band. In this example, a more careful RFI flagging approach is likely to recover more usable spectrum.<br />
<br />
==Flux Density Calibration==<br />
The initial flagging has cleaned up most of the stray RFI across the band. This allows us to proceed with the actual calibration of the data. Before we get to the calibration tables, it's essential to do the flux density calibration of our calibrator 3C147. This is done by the <i>setjy</i> task. Before we run the task we first clear the preliminary calibration that was carried out to enable better flagging. We do that by running the <i>clearcal</i> task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
clearcal(vis='3C129_pband.ms')<br />
#<br />
setjy(vis='3C129_pband.ms', standard='Scaife-Heald 2012', field='3C147')<br />
</source><br />
<br />
== Delay and bandpass calibration ==<br />
<br />
First, we use the full bandwidth on a single scan on the primary calibrator to determine a single delay per antenna, per polarization. This will determine a single (approximate) phase slope across frequency, mainly caused by propagation effects in the (time-variable) ionosphere, cable length differences, and electronics in the signal paths from antenna feeds to correlator. Note that for a single short (5-10 minutes) scan on the calibrator, we can get away with solving for a time-invariant delay per antenna, per polarization. We will use scan 3 on 3C147.<br />
[[File:3C147_init_delay.png|200px|thumb|right|Figure 5: Initial per antenna delays derived by <i>gaincal</i>.]]<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K1', field='3C147', solint='inf', refant='ea09', gaintype='K',<br />
gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq'],parang=True)<br />
#<br />
plotcal(caltable='3C129_pband.K1', xaxis='antenna', yaxis='delay', markersize=2.0, <br />
plotrange=[0,30,-50.,50.],figfile='3C147_delay.png')<br />
</source><br />
<br />
Since we picked ea09 as our reference antenna, the delays for this antenna ID (=0) are arbitrarily set to zero. All other delays should be within 30 nanosec or so (which they should be for this data set). Larger values should be treated with suspicion, possibly indicating a problem with the antenna or polarization. We will use the bandpass calibration to verify this in the next step.<br />
<br />
To determine the bandpass calibration, we use the same source and scan and apply the delay calibration before solving for the bandpass. Note that we request a minimum SNR of 3, which will make the solve fail for the worst, but not all, channels.<br />
<source lang='python'><br />
# In CASA<br />
bandpass( vis='3C129_pband.ms', caltable='3C129_pband.B1', field='3C147', solint='inf', refant='ea09', minsnr=3.0,<br />
parang = True, gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq','3C129_pband.K1'],<br />
interp=['','','','nearest,nearestflag'])<br />
<br />
</source><br />
<br />
After solving, we inspect the bandpass calibration amplitudes and phase, and flag any obvious residual outliers. Outliers may be found above or below the average bandpass curves, and tend to arise in the same channel (frequency) ranges for all antennas. Don't spend more than 1 minute per antenna for this tutorial and don't worry about missing some bad points; these bad channels will likely get flagged in a later stage anyway. Note that the bandpass amplitudes (Fig. 6) and phases may differ between polarizations of the same antenna. Also note that the bandpass phases across frequency should not have an overall gradient (this should be removed by the delay calibration). Also note that bandpass phases may wrap around from +/-180 to -/+180 degrees. If that is problematic, re-run the phase <i>plotcal()</i> with <i>plotrange = [ 200.,500.,0.,360. ]</i>. Also note that there are no bandpass solutions for antennas ea19 and ea28, and only a few solutions for ea14.<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='amp', iteration='antenna', markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='phase', iteration='antenna', plotrange=[ 200.,500.,-180.,180. ], markersize = 2.0)<br />
</source><br />
{|<br />
| [[File:3C147_bp0_ea01.png|200px|thumb|right|Figure 6a: Initial bandpass amplitudes for antenna ea01.]]<br />
| [[File:3C147_bp0_phase_ea01.png|200px|thumb|right|Figure 6b: Initial bandpass phase for antenna ea01.]]<br />
|}<br />
<br />
Flagging is made easier by maximizing the plotcal() window. We don't set the scale for plotting the bandpass amplitudes, which is convenient when flagging outliers as it will auto-rescale. While plotting the bandpass phases we do fix the scale, as it provides a better feeling for the relevant magnitude of phase outliers. Flagging of outliers is done using the <i>Mark Region</i> and <i>Flag</i> buttons. For more information on interactive flagging, see the topical guide [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Flagging VLA Data in CASA]. We are using this mechanism to efficiently flag data manually, since it is done per antenna rather than per baseline. These 'flags' will not be permanent until we call <i>applycal()</i> later on, but are effective while applying the calibration on the fly, as we will do in subsequent calibration steps.<br />
<br />
== Gain calibration ==<br />
<br />
With the delay and bandpass calibration in place, we will now look more closely at the time-variable behaviour of the VLA. We use the same scan on our primary calibrator, 3C147, over the full effective bandwidth to determine gain calibrations: one complex value per antenna, per polarization, per integration time. The delay and bandpass tables are applied on the fly.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal( vis='3C129_pband.ms', caltable='3C129_pband.G1', field='3C147', solint = 'int',<br />
refant = 'ea09', minsnr = 3.0, gaintype = 'G', calmode = 'ap',<br />
gaintable = ['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim', '3C129_pband.K1', '3C129_pband.B1' ],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag' ], parang = True )<br />
<br />
</source><br />
{|<br />
| [[File:3C147_Gamp_ea01.png|200px|thumb|right|Figure 7a: Initial gain amplitude solutions for antenna ea01.]]<br />
| [[File:3C147_Gphase_ea01.png|200px|thumb|right|Figure 7b: Initial gain phase solutions for antenna ea01.]]<br />
|}<br />
<br />
Similar to the bandpass flagging, we will flag outlier gain solutions in both amplitude and phase. For the amplitudes, the final scatter around an average close to one should be a few percent (see Fig. 7). Note antennas ea08, ea09 have particularly noisy solutions. Remember that the gain solutions for both polarizations can be flagged together, as we will lose the equivalent data of the surviving polarization anyway.<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='amp', iteration='antenna', markersize=2.0 )<br />
#<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='phase', iteration='antenna', <br />
plotrange=[ None,None,-180.,180.], markersize=2.0)<br />
</source><br />
<br />
Note that when plotting gains against time, it is convenient to use auto-scaling of the time axis by putting <i>None</i> in the first two fields of <i>plotrange</i>.<br />
<br />
Next, we will prepare a smoothed and interpolated version of the gain calibration table, which will be applied later to the target field data. This prevents flagging of target field data when one of the edges of a calibrator scan is flagged for one or more antennas. The result can be inspected with <i>plotcal</i>.<br />
<source lang='python'><br />
# In CASA<br />
smoothcal(vis='3C129_pband.ms', tablein='3C129_pband.G1', caltable='3C129_pband.Gs1', smoothtype='median', smoothtime = 60.*60.)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='amp', iteration='antenna', plotrange=[ None,None,0.8,1.2 ], markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='phase', iteration='antenna', plotrange=[ None,None,-10.,10. ], markersize=2.0)<br />
<br />
</source><br />
<br />
== Transfer of calibrations to the target field ==<br />
<br />
So far we have used one scan on the primary calibrator 3C147 to derive various calibration tables. These calibrations will now be applied to all scans of all sources in our measurement set, which includes 3C147 itself and our target field 3C129. As described above, the task <i>applycal()</i> creates a <i>CORRECTED_DATA</i> column in which the calibrated visibilities get stored. This may take a while.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='3C129_pband.ms', parang=True, applymode='calflagstrict', flagbackup=True, gaintable=['3C129_pband.antpos','3C129_pband.rq',<br />
'3C129_pband.tecim','3C129_pband.K1','3C129_pband.B1','3C129_pband.Gs1'],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag', 'nearest,nearestflag'])<br />
</source><br />
<br />
Now we split off the calibrated target field data, meaning that the visibilities of source 3C129 get copied from the <i>CORRECTED_DATA</i> column to the <i>DATA</i> column of a new measurement set. This is convenient for further processing. <br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='3C129_pband.ms', outputvis='3C129_pband_target.ms', datacolumn='corrected', field='3C129')<br />
</source><br />
<br />
It is also helpful to perform additional flagging for any residual RFI left behind in our target after all our calibration steps have been completed and we begin the imaging of our data.<br />
<br />
<source land='python'><br />
flagdata(vis='3C129_pband_target.ms', mode='rflag')<br />
flagdata(vis='3C129_pband_target.ms', mode='extend')<br />
</source><br />
<br />
<br />
== Statwt ==<br />
Before we go ahead and image our target field. We would like to weight any residual low level RFI down. This is done using the task <i>statwt</i>. The task runs through the data and computes the rms of the data and weighting down outliers. This makes a big impact for imaging of a target in the presence of low level residual RFI. For our purposes we will bin the data in a time interval larger than the integration time and run statwt as follows.<br />
<br />
<source lang='python'> <br />
statwt(vis='3C129_pband_target.ms', datacolumn='data', timebin=30.)<br />
</source><br />
<br />
With data weights for outliers being set we will now proceed with the imaging and self-calibration of our data.<br />
==Imaging==<br />
<br />
At this point, we're ready to make a first image of our target field. Imaging in CASA is done by means of the <i>tCLEAN</i> task. The task implements algorithms for wide field imaging, such as W-Projection (Cornwell et al. 2008 http://arxiv.org/abs/0807.4161) and Multi Term Multi Frequency Synthesis ( Rau et al. 2011 http://arxiv.org/abs/1106.2745) both of which we will utilize to make our initial image. Even though our source of concern, the radio galaxy, lies in the centre of our field. The observations were carried out in B configuration resulting in an effective maximum resolution of approximately 5x5 arcsecs. To effectively model the source spectral index in the sky, we will utilize the MTMFS algorithm and use the W-Projection algorithm to make a wide field image (The full beam at P Band is about 3 degrees in diameter). So, going by these requirements, we can now compute the required cell and image size. Consulting the resolution guide of NRAO science page, we can see that the expected HPBW for B - Configuration, P -Band Observation is 18.5 arcsec. So, we can sample it well by using a 5 arcsec cellsize. Having done that, if we decide to make a wide field image to account for all the point sources we will set the image size parameter to 4096 pixels. To enable the wide-field algorithm, we set the <i>gridder='wproject'</i>: this invokes the W-Projection algorithm, upon which we set the number of W-Projection planes to be 128. We set the deconvolution mode as Multi Term Multi Scale Multi Frequency Synthesis using the <i>deconvolver='mtmfs'</i> parameter along with the number of Taylor terms to be considered during imaging to be 2 and choosing three different scales to model the point source and extended emission in our field well, we do this by setting <i> scales=[0,20,30]</i>. This allows for the source spectral variation to be modeled by a second order polynomial. We launch the interactive tclean process, but only for <i>spw='3~8'</i> to begin with. Note, running the first clean call will take at least ~10 min to process.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_initial_clean', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30]<br />
minpb=0.01, savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
</source><br />
<br />
{|<br />
| [[File:3C129_radio_gal_residual.png|200px|thumb|right|Figure 10a: Resulting residuals of 3C129 after first call of interactive <i>clean</i>.]]<br />
| [[File:3C129_radio_gal_clean.png|200px|thumb|right|Figure 10b: Resulting image of 3C129 after completing the first <i>clean</i>.]]<br />
|}<br />
The <i>tCLEAN</i> command launches an interactive session after a 100 iterations of clean and produces a wide field map with two sources at the center and a lot of bright sources far out in the field. As it is a snapshot image, the bright sources have significant side lobes and so tight clean boxing can help. This can done in the interactive viewer interface that pops up. If we proceed with interactive clean with subsequent steps to keep boxing out the strong sources that pop up in the image, we finally see the extended emission from the target radio galaxy begin to emerge. Continue boxing and cleaning to ensure that the residuals of the boxed cleaning look noise like (~10000 clean iterations). We now stop the interactive task and look at the final image it produced.<br />
<br />
<source lang='python'><br />
# In CASA<br />
viewer('3C129_initial_clean.image.tt0')<br />
</source><br />
<br />
The recovered structure at the center of the map should resemble what is shown in Fig. 10b. If the image looks worse then boxing of bright features in the map needs to be done more carefully. Note the image has some sources still showing strong side lobes and imaging artifacts around them. We expected this as we have carried out phase calibration only on our flux calibrator and have just transferred the solutions over to our target field. Since we used the <i>usescratch=True</i>, the MODEL_DATA column in the measurement set now contains the initial image model, which we will self calibrate against to produce a better image. Also in clean, do notice that the spw's utilized are the cleanest spectral windows that are totally RFI free.<br />
<br />
==Self Calibration ==<br />
<br />
We now proceed to compute gain phase solutions for our target field using the <i>gaincal</i> task as the first step in self-calibration.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScG0', field='3C129', solint='inf', refant='ea09', <br />
spw='3~8',minsnr=3.0, gaintype='G', parang=True, calmode='p')<br />
<br />
bandpass( vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScB0', field='3C129', solint='inf', refant='ea09', minsnr=3.0, spw='3~8',<br />
parang = True, gaintable=['3C129_pband_target.ScG0'],<br />
interp=['','','','nearest,nearestflag'])<br />
applycal(vis='3C129_pband_target.ms', gaintable=['3C129_pband_target.ScG0','3C129_pband_target.ScB0'], spw='3~8', applymode='calflagstrict')<br />
</source><br />
<br />
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the <i>tCLEAN</i> command once again.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_clean_sc0', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30]<br />
minpb=0.01,savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
<br />
viewer('3C129_clean_sc0.image.tt0')<br />
</source><br />
[[File:3C129_radio_gal_cselfcal0.png|200px|thumb|right|Figure 11: Resulting image of 3C129 after completing a first selfcal.]]<br />
<br />
On boxing and cleaning, we already notice that the imaging artifacts have reduced significantly. We also see that the target source appears to contain more structure and a greater amount of flux. Further self calibration iterations involving an amplitude & phase gain calibration, and target bandpass calibration, are all possible steps that can be explored. <br />
<br />
== Appendix: Some P-band data issues you may want to know about ==<br />
<br />
Unfortunately, there are a few known issues with JVLA P-band observing. Most problems were discovered during the P-band commissioning period and have been fixed for newer data sets, but the archive keeps part of this history alive. Here we will go over some of those issues, and (if possible) provide ways of fixing them.<br />
<br />
=== Polarization labeling ===<br />
<br />
For a long period, P-band feeds have been labelled as being circular (R and L), while they are linear (X and Y). It is still possible to generate data with incorrect polarization labeling, however for standard observations this should not be the case anymore. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem. This may seem mostly harmless, but does make a difference for polarization calibration. The following (non-standard) task will check and fix this (and another related problem of swapped polarizations):<br />
<br />
Download [http://casa.nrao.edu/Data/EVLA/Pband/casa_vla_lowband.tar.gz this tarball] in your CASA working directory, open a shell there, and type the following commands:<br />
<br />
<source lang='bash'><br />
tar -xzvf casa_vla_lowband.tar.gz<br />
<br />
buildmytasks<br />
</source><br />
<br />
Now go back to your CASA session and type:<br />
<br />
<source lang='python'><br />
# In CASA<br />
execfile('mytasks.py')<br />
#<br />
fixlowband(vis='TSUB0001_pband_prehs.ms')<br />
</source><br />
<br />
Re-run the <i>listobs()</i> task to check if the fix was correctly applied.<br />
<br />
=== Swapped polarizations ===<br />
<br />
Throughout the whole history of the new VLA low-band system, even up to this day, there have been mistakes in the cabling of the full signal chain. This results in that some antennas have the X-polarization signal come in as Y and vice versa. The way to notice this in visibility data is that for baselines with one swapped antenna most power will be in the cross-hand correlations (XY and YX) rather than the parallel-hand correlations (XX and YY). Once antennas with this feature are manually identified, a contributed task <i>swappol()</i> is available to fix this problem (see under polarization labelling).<br />
<br />
=== Double data descriptor entries ===<br />
<br />
The data description table is part of the measurement set, and provides a link between the recorded visibilities, the spectral window information and the polarization information. For some data sets, the data description table contains two entries for each P-band spectral window, one pointing towards a circular (RL) polarization definition, and one pointing towards a linear (XY) polarization definition. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem.<br />
<br />
=== Continuous Radio Frequency Interference (RFI) ===<br />
<br />
The wide bandwidth of the low-band receiver is (unfortunately) guaranteed to contain significant amounts of RFI. There are a few RFI sources that are active all the time, and are visible in all array configurations. The default P-band observing setup of 16 x 16 MHz tries to capture as much of the continuous RFI in as few spectral windows as possible, allowing for a simple RFI mitigation strategy in which these spectral windows (spws 1,2 and 9,10, but possibly more) can be immediately flagged.<br />
<br />
=== Two spectral window setup ===<br />
<br />
In the early commissioning period, the default setup for P-band observing was to use 2 spectral windows of 1024 channels each to cover 256 MHz of bandwidth from 230 - 486 MHz. However, it was noticed that strong, narrow-band RFI events were causing data to be lost for the whole spectral window in which they occured. To make the system more robust against data loss from these events, the frequency range was divided into 16 x 16 MHz, and shifted downwards by 6 MHz to capture ever-present RFI into as few spectral windows as possible. If your data has only two P-band spectral windows, please be aware that a higher data loss due to RFI is possible. There is no way to repair this.<br />
<br />
=== Bandpass ripples ===<br />
<br />
Due to signal reflections in cables and within the VLA dish, sinusoidal amplitude and phase modulations are always present in P-band data. This is most easily seen in bandpass calibration plots of amplitude versus frequency. In some cases, mostly due to cable & connector problems, these modulations can be very strong (~up to 50 percent of the average amplitude level). These modulations tend not to vary over the duration of an observation and can therefore be removed through bandpass and polarization calibration. If they are found to be variable (e.g., by inspecting bandpass solutions for separate calibrator scans), the offending antenna / polarization should be flagged.<br />
<br />
=== High cross-polarization ===<br />
<br />
On each antenna, the P-band feed (dipole) is visually aligned with respect to the primary focus support legs. This is normally done within 5-10 degrees accuracy. On rare occasions the dipole on one antenna has accidentally rotated to much larger angles because the locking bolt on the back of the feed was not completely tightened. The result is a high cross-polarization in the baseline visibilities that include this antenna. If this is the case, it is safest to flag this data, since there is a possibility that the dipole has rotated during the observations.<br />
<br />
=== Instrumental Polarization Calibration ===<br />
<br />
At present polarization P-band observations are in RSRO mode only. While polarization leakage calibration has been highlighted in the past reports it does not directly impact imaging in Stokes I and hence has been dropped from this casaguide.<br />
<br />
== Useful links ==<br />
<br />
* [http://www.aoc.nrao.edu/lbr The EVLA Low-Band Receiver System Status & User Pages]<br />
* [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/topical-guides/vlofreq VLA Very Low Frequency Observing]<br />
<br />
<br />
{{Checked 5.4.1}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0&diff=26243VLA Radio galaxy 3C 129: P-band continuum tutorial-CASA5.4.02018-12-01T22:53:29Z<p>Emomjian: </p>
<hr />
<div><!--By [http://www.aoc.nrao.edu/~pjaganna/Site/Home.html Preshanth Jagannathan] & [https://safe.nrao.edu/wiki/bin/view/Main/HuibIntemaWebHome Huib Intema] & Frank Schinzel--><br />
<br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b><p></p> <br />
<b> Note: This guide may take up to 2-days to complete </b><p></p> <br />
If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
<br />
This webpage provides a basic description on how to use CASA to reduce data from the upper part of the new VLA low-band system, also known as P-band, covering roughly 220-480 MHz. The goal is to make a wide-field continuum stokes I image of a typical blank field using the full effective bandwidth. The dataset that will be downloaded in this guide has a size of 26.46 GB and will require a total of ~140 GB of disk space if all generated files are kept. <br />
<br />
''Note: Starting with CASA 5.0 IPython 5.1.0 is used which for interactive CASA sessions. This does not allow to readily paste and execute multiple lines of code like in previous versions of the interpreter. If this is needed at any point in this tutorial, we use "%cpaste" to allow the copy/paste of loops, conditions, etc.. Press "Enter" (or "Return") after "--" at the end, this stops (or exits) the copy/paste prompt and will return you to the CASA prompt.''<br />
<br />
== Obtaining the raw data ==<br />
<br />
For this guide, we'll use some test data that was taken in B-configuration. To get a copy of the raw data, download [https://casa.nrao.edu/Data/EVLA/Pband/P_band_3C129.tgz P_band_3C129.tgz] onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
<br />
Alternatively, the raw data can also be obtained from the NRAO archive, go to the [https://archive.nrao.edu/archive/advquery.jsp NRAO advance archive query page], and enter the following search parameter in the field <i>Archive File ID</i> (leave the rest on default):<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
imag-test-copy.57080.956837025464<br />
</pre><br />
</div><br />
<br />
The search should return a single row for Archive File: imag-test-copy.57080.956837025464, taken on Feb. 27, 2015 (or "15-Feb-27 22:57:52"). Check the box next to it. Next, fill in your email and select the "SDM-BDF dataset (all files)" option. Note, you may find it easier to download a tar file, in which case, you'll want to check the box for Create MS or SDM tar file. Click "Get My Data" and follow the on-screen prompts to reach the next page, where you should click "Retrieve over internet". The next page should report that the data staging is in progress. Wait until you receive an e-mail reporting that your archive data is copied, which should take a few minutes. Download the data onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing <br />
<source lang='bash'><br />
casa<br />
</source><br />
on the (linux) command line. This should start a CASA interactive python (iPython) session, and open a separate log window. The CASA version is reported at startup, both in the python session and the log window.<br />
<br />
== Importing the raw data into CASA ==<br />
<br />
We will begin by importing our data into the measurement set format (CASA standard) from the binary format (SDM-BDF) as downloaded from the archive. We do this by means of the importasdm task. It is important to note that the importevla task has been deprecated.<br />
<br />
<source lang='python'><br />
# In CASA<br />
importasdm(asdm='imag-test-copy.57080.956837025464', vis ='3C129.ms', savecmds=True, outfile='importflags.txt')<br />
<br />
</source><br />
<br />
This task calls the external tool <i>asdm2MS</i> to perform the conversion. Together with the visibility data, also the flag commands from the VLA online system are imported. These flags are not directly applied, but stored in the <i>importflags.txt</i> file and applied later.<br />
<br />
== Preliminary data inspection ==<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms', verbose = True, listfile = '3C129.listobs')<br />
</source><br />
<br />
Alternatively, you can also output to the logger by running the command without the listfile argument. The sample listobs output shown has flag <i>verbose=False</i> for the sake of brevity.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/fschinze/P_band_tutorial/3C129.ms MS Version 2<br />
================================================================================<br />
Observer: Project: uid://evla/sdm/X2<br />
Observation: EVLA(27 antennas)<br />
Data records: 4714983 Total elapsed time = 1422 seconds<br />
Observed from 27-Feb-2015/22:57:54.0 to 27-Feb-2015/23:21:36.0 (UTC)<br />
<br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C147 05:42:36.127300 +49.51.07.17800 J2000 0 1307124<br />
1 NONE 3C129 04:48:58.200000 +45.02.01.00000 J2000 1 3407859<br />
Spectral Windows: (19 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 EVLA_P#A0C0#0 128 TOPO 224.000 125.000 16000.0 231.9375 12 XX XY YX YY<br />
1 EVLA_P#A0C0#1 128 TOPO 240.000 125.000 16000.0 247.9375 12 XX XY YX YY<br />
2 EVLA_P#A0C0#2 128 TOPO 256.000 125.000 16000.0 263.9375 12 XX XY YX YY<br />
3 EVLA_P#A0C0#3 128 TOPO 272.000 125.000 16000.0 279.9375 12 XX XY YX YY<br />
4 EVLA_P#A0C0#4 128 TOPO 288.000 125.000 16000.0 295.9375 12 XX XY YX YY<br />
5 EVLA_P#A0C0#5 128 TOPO 304.000 125.000 16000.0 311.9375 12 XX XY YX YY<br />
6 EVLA_P#A0C0#6 128 TOPO 320.000 125.000 16000.0 327.9375 12 XX XY YX YY<br />
7 EVLA_P#A0C0#7 128 TOPO 336.000 125.000 16000.0 343.9375 12 XX XY YX YY<br />
8 EVLA_P#A0C0#8 128 TOPO 352.000 125.000 16000.0 359.9375 12 XX XY YX YY<br />
9 EVLA_P#A0C0#9 128 TOPO 368.000 125.000 16000.0 375.9375 12 XX XY YX YY<br />
10 EVLA_P#A0C0#10 128 TOPO 384.000 125.000 16000.0 391.9375 12 XX XY YX YY<br />
11 EVLA_P#A0C0#11 128 TOPO 400.000 125.000 16000.0 407.9375 12 XX XY YX YY<br />
12 EVLA_P#A0C0#12 128 TOPO 416.000 125.000 16000.0 423.9375 12 XX XY YX YY<br />
13 EVLA_P#A0C0#13 128 TOPO 432.000 125.000 16000.0 439.9375 12 XX XY YX YY<br />
14 EVLA_P#A0C0#14 128 TOPO 448.000 125.000 16000.0 455.9375 12 XX XY YX YY<br />
15 EVLA_P#A0C0#15 128 TOPO 464.000 125.000 16000.0 471.9375 12 XX XY YX YY<br />
16 EVLA_P#B0D0#16 64 TOPO 310.000 500.000 32000.0 325.7500 15 XX XY YX YY<br />
17 EVLA_P#B0D0#17 1024 TOPO 310.000 62.500 64000.0 341.9688 15 XX XY YX YY<br />
18 EVLA_P#B0D0#18 64 TOPO 342.000 500.000 32000.0 357.7500 15 XX XY YX YY<br />
Antennas: 27 'name'='station'<br />
ID= 0-3: 'ea01'='N12', 'ea02'='N16', 'ea03'='W28', 'ea05'='E20',<br />
ID= 4-7: 'ea06'='N20', 'ea07'='N32', 'ea08'='E28', 'ea09'='W08',<br />
ID= 8-11: 'ea10'='E36', 'ea11'='W12', 'ea12'='N36', 'ea13'='W04',<br />
ID= 12-15: 'ea14'='E08', 'ea15'='E24', 'ea16'='W24', 'ea17'='E04',<br />
ID= 16-19: 'ea18'='W36', 'ea19'='MAS', 'ea20'='N04', 'ea21'='E32',<br />
ID= 20-23: 'ea22'='N24', 'ea23'='E16', 'ea24'='W32', 'ea25'='W20',<br />
ID= 24-26: 'ea26'='E12', 'ea27'='N08', 'ea28'='N28'<br />
<br />
</pre><br />
<br />
You can inspect the output text file <i>3C129.listobs</i> in your favorite text editor (e.g., <i>gedit</i>). When taking some time to familiarize yourself with the output format, you will see that:<br />
* The observation consists of 5 scans, namely:<br />
# 3C147 (hardware setup)<br />
# 3C147 (setting requantizers)<br />
# 3C147 (primary calibrator)<br />
# 3C129 (target)<br />
# 3C129 (target)<br />
* The frequency coverage is 222 - 478 MHz, divided into 16 x 16 MHz spectral windows, each having 128 x 0.125 MHz channels. There are also two spectral windows with 64x0.5 MHz and one spectral window with 1024x0.0625 MHz. <br />
* Visibilities are recorded every 2 seconds in full polarization (4 polarization products labelled XX,XY,YX,YY).<br />
* Of the 28 VLA antennas labelled ea01 - ea028, antenna ea04 is not participating in this observation.<br />
* Source, antenna and spectral window IDs start at zero, but scan IDs start at one.<br />
<br />
<br />
[[File:3C129_pband_plotants.png|200px|thumb|right|Figure 1: plotants figure]] <br />
<br />
The array configuration can be inspected using:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotants(vis='3C129.ms', figfile='3C129_pband_plotants.png')<br />
</source><br />
<br />
== Initial processing steps - Antenna Position, Requantizer Gain, Ionospheric Correction ==<br />
<br />
When importing the data, we saved the flags from the online system to an ASCII text file. This gives us the opportunity to review the flag commands before applying them. For our data set, the flag file is called <i>importflags.txt</i>. Please load this file into your favorite text editor. The bulk of the flag commands refer to times when the VLA is slewing (<i>ANTENNA_NOT_ON_SOURCE</i>) or when the movable secondary reflector of the VLA's Cassegrain system is not in place (<i>SUBREFLECTOR_ERROR</i>). The latter can cause antenna gain variations (amplitude and phase), so it is safest to apply all the flags. The final two rows in the flag file will remove visibilities that are pure zero, and flag antennas that are partly blocked by other antennas (shadowing) which occurs mostly in compact configurations when observing along a VLA arm).<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms', mode='list', inpfile='importflags.txt', action='apply', reason='any', flagbackup=True)<br />
</source><br />
This call, before applying the new flags to the data, created a new flagbackup file called flagdata_1 which can be used by the task <i>flagmanager</i> to restore the state of the measurement set prior applying the flags from <i>importflags.txt</i>.<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
Now that we have applied the import flags, we will correct for the antenna position offsets. Antenna positional errors translates to an error in the measured visibilities and need to be accounted for before we proceed with any of the other calibration steps. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltable='3C129_pband.antpos',caltype='antpos')<br />
</source><br />
<br />
The logger output shows that there are 13 antennas with positional offsets: <br />
<pre><br />
<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea03 : 0.00290 0.00000 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea08 : 0.00000 0.00390 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea10 : -0.00170 0.00000 0.00400<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea11 : 0.00000 0.00160 -0.00170<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea13 : 0.00000 0.00050 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea16 : -0.00180 0.00680 0.00230<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea18 : 0.00470 0.00350 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea21 : -0.00160 0.00000 0.00210<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea23 : 0.00140 0.00200 -0.00150<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea24 : 0.00380 0.00000 -0.00340<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea25 : 0.00330 0.00000 -0.00110<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea26 : 0.00000 0.00190 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea27 : -0.00120 0.00070 0.00000<br />
<br />
<br />
</pre><br />
<br />
=== Ionospheric TEC Corrections ===<br />
[[File:Pband_vla_tecu_new.png|200px|thumb|right|Figure 2: TEC value vs time generated by <i>tec_maps.create()</i>]] <br />
<br />
Low frequency observations are affected by the ionosphere. A delay in the signal path is introduced between the two polarization of light that varies both as a function of time and line of sight (direction dependent). The delay is proportional to the Total Electron Content (TEC) along the line of sight and is inversely proportional to the square of the frequency. GPS measurements at two different frequencies provides us with an estimate of the TEC per square meter. This correction has been implemented in CASA which we shall apply as a calibration table by means of the <i>gencal</i> task. The task requires a TEC map that we will generate utilizing CASA recipes. For more information, see the section on [https://casa.nrao.edu/casadocs/casa-5.0.0/synthesis-calibration/preparing-for-calibration/#title5 ionospheric corrections] in the CASA documentation archives. For some versions of CASA, the first plot to the right (Figure 2) may not be generated if you have already run plotants: to generate this plot, you will need to quit CASA and start it again. If the IONEX data file was not downloaded you might be missing the curl package in the environment you are running CASA in. In this case you can provide the required file manually by downloading: ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/2015/058/igsg0580.15i.Z, (for the tutorial it is also also available in an unzipped version [http://casa.nrao.edu/Data/EVLA/Pband/igsg0580.15i here]) unziping it, and placing it in the directory from which you started CASA. The current implementation in CASA is in the development stage, however since the CASA 5.0 release both the Faraday rotation correction and dispersive delay corrections for the ionosphere are enabled. In previous releases only Faraday rotation corrections are enabled. In this example we will generate the calibration table and apply it throughout calibration. <br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
#<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='3C129.ms',doplot=True)<br />
#[NOTE: If you are using CASA 5.0.0, or earlier this should be: tec_image, tec_rms_image = tec_maps.create(vis='3C129.ms', doplot=True)]<br />
#<br />
gencal(vis='3C129.ms',caltable='3C129_pband.tecim',caltype='tecim',infile=tec_image)<br />
# Spatial plots like shown in Figure 3 can be looked at using the CASA viewer for the TEC values<br />
viewer('3C129.ms.IGS_TEC.im')<br />
# and for TEC rms<br />
viewer('3C129.ms.IGS_RMS_TEC.im')<br />
</source><br />
<br />
[[File:Pband_tec_new.png|200px|thumb|right|Figure 3: TEC map movie generated by the TEC tasks.]] <br />
<br />
A word of caution regarding the TEC map generation. The IGS website updates measurements only two weeks after the date of observation.<br />
<br />
=== Requantizer Gains ===<br />
<br />
The next step will correct the visibility amplitudes for the signal leveling (requantizer gains) that occurs at the inputs of the WIDAR correlator. These levels (per antenna, per polarization, per spectral window) are stored with the measurement set (in the <i>SYSPOWER</i> sub-table). This step is currently not essential, since the levels get set only once at the start of an observation and bandpass calibration will correct for this. But it will make your bandpass plots look better if you have multiple spectral windows. And, more importantly, in general it is possible to trigger the setting of requantizers throughout an observation. This is the case for new observations, when a P-band scan is preceded by a P-band setup scan, which itself is preceded by a scan with a different telescope configuration. <br />
<br />
The correction is done by means of the <i>gencal</i> task, where information in the <i>SYSPOWER</i> sub-table gets translated into a gain table. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltype='rq',caltable='3C129_pband.rq')<br />
</source><br />
<br />
== Dead antennas ==<br />
<br />
Now it is time to have a first visual look at the uncalibrated visibility data. It is important to identify dead antennas / polarizations early on, so we can exclude them from further data processing (which is the most efficient data reduction). A convenient way of doing this is through the <i>plotms</i> task on bright calibrator(s) (see the <i>listobs()</i> output), in our case 3C147 (see Fig. 4a).[[File:3C129_pband_3C147_prebp.png|200px|thumb|right|Figure 4a: <i>plotms</i> output before RFI flagging]]<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY', field='3C147', <br />
plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',iteraxis='baseline', <br />
plotfile='3C129_pband_3C147_prebp.png')<br />
</source><br />
<br />
This will load the <i>plotms</i> window that has queued up all the baselines of antenna ea01 and is currently displaying the first amplitude vs frequency plot with the colors representing the different spectral windows. Note that we are only plotting the XX and YY correlation of the antennas. This is because we expect most of the power in the linear cross correlation products to be in these two correlations. Notice that some spectral windows in particular are badly affected due to radio frequency interference (RFI). This is the issue that we will deal with after identifying dead antennas. If you scroll through the baselines using the green forward button in the <i>plotms</i> viewer, you will notice that the amplitude is particularly low when you encounter ea19. If we make a quick plot of the baselines made by ea19 (put <i>ea19</i> in the antenna field) we can see that there is no power in any of the polarizations indicating this antenna was dead and can be flagged out. It is also good to check for high power levels in cross-hand correlations in comparison to the parallel-hand correlations, which would uncover swapped polarization labels and severely mis-aligned dipoles. At the end of this tutorial some hints on solving for swapped or mislabeled dipole polarizations are given. In addition to the dead antenna, we also make sure that the first two scans are flagged, which were not labelled as setup scans but were used to set the attenuators and requantizers.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms',mode='manual',antenna='ea19')<br />
#<br />
flagdata(vis='3C129.ms',mode='manual',scan='1~2')<br />
</source><br />
<br />
We also noticed sharp RFI peaks during our data examination. To prevent Gibbs ringing, it is best to hanning smooth the data at this juncture before we proceed further with automatic flagging. Now is a good time to get a cup of coffee while hanning smoothing the data. [[File:3C129_pband_3C147_prebp_hanning.png|200px|thumb|right|Figure 4b: <i>plotms</i> output after Hanning smooth and first flagging.]]<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='3C129.ms',outputvis='3C129_pband.ms',datacolumn='data')<br />
</source><br />
<br />
We should replot the primary calibrator to see the effect of hanning smoothing on the data and the RFI (Fig. 4b). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning.png')<br />
<br />
</source><br />
<br />
== Automatic flagging ==<br />
<br />
<br />
It is worthwhile noting at this juncture that the standard P-band setup will ensure that the worst of the RFI is restricted to four of the spectral windows. This particular dataset is an example of higher than average levels of RFI across the band. Before throwing away bad spectral windows by hand, we'll let the automated flaggers in CASA have a go at it. The main task for this is <i>flagdata()</i>. While working on your data, <i>flagdata()</i> produces an abundance of output, not all of which is easy to understand. To help with this, we can get the flag status of our data before and after auto-flagging by running <i>flagdata()</i> in the <i>summary</i> mode. This first call provides the flag statistics of the flagging up to now, before any auto-flagging:<br />
<source lang='python'><br />
summary_1 = flagdata(vis='3C129_pband.ms', mode='summary')<br />
</source><br />
<br />
This returns a python dictionary with flagged versus total visibilities along various axes (antenna, scan, spw, field, correlation, etc.). For example, if we want to know the percentage flagged per scan, run the following (note that the scans may not appear in sorted order):<br />
<source lang='python'><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for id, stats in summary_1[ axis ].iteritems():<br />
print '%s %s: %5.1f percent flagged' % ( axis, id, 100. * stats[ 'flagged' ] / stats[ 'total' ] )<br />
--<br />
</source><br />
<br />
For this example, you will notice that scans 1 and 2 (the dummy scans on 3C147) are 100 percent flagged, which is what we did during the initial flagging.<br />
<br />
We will run <i>flagdata()</i> in the <i>tfcrop</i> mode, which will (per scan, per baseline, per spectral window, per polarization) look for visibility amplitude outliers. It uses a 5-piece polynomial in an attempt to remove any intrinsic bandpass (amplitude) structure (we did not calibrate for bandpass yet). We run the task twice to allow for slightly deeper flagging and separate treatment of the parallel and cross-hand correlations. For the first run, we tell CASA to make a backup of our visibility flag status up to here, giving us an option to restore them if we choose over-aggresive flagging parameters and consequently over-flag our data. The flag backup file name is reported in the log window, and can be found in the <i>3C129_pband.ms.flagversions</i> directory.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=4., freqcutoff=3., maxnpieces=5,<br />
action='apply', display='report', flagbackup=True, combinescans=True, ntime='3600s', correlation='ABS_XY,ABS_YX')<br />
#<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=3., freqcutoff=3., maxnpieces=2,<br />
action='apply', display='report', flagbackup=False, combinescans=True, ntime='3600s', correlation='ABS_XX,ABS_YY')<br />
#<br />
flagdata(vis='3C129_pband.ms', mode='extend')<br />
</source><br />
<br />
As a note of caution: When executing flagdata with <i>mode='extend'</i>, all flags that were ever applied to the dataset will get extended. Repeat execution of this command will incrementally flag more and more around already flagged data.<br />
<br />
Let's get another flag summary to see how much extra data was flagged: <br />
<source lang='python'><br />
# In CASA<br />
summary_2 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
# <br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_2[ axis ].iteritems():<br />
old_stats = summary_1[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning_automatic.png')<br />
<br />
</source><br />
[[File:3C129_pband_3C147_prbp_tfcropped.png|200px|thumb|right|Figure 4c: <i>plotms</i> output after RFI flagging with tfcrop]]<br />
There is quite a bit of RFI that gets missed in RFI-rich spectral windows as can be seen in Fig. 4c. The solution to this is to provide the flagging routines with more contrast between healthy and affected data. For this, we will perform a preliminary bandpass calibration to take out the bandpass shape. We will do some coarse preliminary calibration and apply it to the calibrator before flagging for RFI once more.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.G0', gaintype='G', calmode='p', solint='int', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim'])<br />
#<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K0', gaintype='K', solint='inf', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0'])<br />
#<br />
bandpass(vis='3C129_pband.ms', caltable='3C129_pband.B0', solint='inf', field='3C147',refant='ea09', minsnr=2.0,<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0'])<br />
#<br />
applycal(vis='3C129_pband.ms', field='3C147', applymode='calflagstrict',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0','3C129_pband.B0'] )<br />
<br />
</source><br />
<br />
On the command line you should see a notification stating: "FJones (iononsphere) apply now provisionally includes disp. delay." which indicates that both Faraday rotation as well as dispersive delay corrections for the ionosphere are applied.<br />
<br />
[[File:3c147_postbandpass_rflag.png|200px|thumb|right|Figure 4d: <i>plotms</i> output after RFI flagging with <i>mode='rflag'</i>]]<br />
<br />
We will now flag the corrected data column that contains the coarsely calibrated visibilities which provide better contrast to the flagging algorithms to remove the RFI present. We will begin by running the <i>RFLAG</i> algorithm in the task <i>flagdata</i>. We will again use summary mode to see how much more of the observed data was flagged.<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
summary_3 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
#<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_3[ axis ].iteritems():<br />
old_stats = summary_2[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C147_postbandpass_rflag.png', ydatacolumn='corrected')<br />
</source><br />
<br />
Taking another visual look at the data using <i>plotms</i>, we examine the effect of the automated flagging on the calibrator (Fig. 4d). We find that most spectral windows are RFI free and that about 60% of the data was flagged. We typically expect between 40-60% of flagging due to interference in a significant fraction of the band. In this example, a more careful RFI flagging approach is likely to recover more usable spectrum.<br />
<br />
==Flux Density Calibration==<br />
The initial flagging has cleaned up most of the stray RFI across the band. This allows us to proceed with the actual calibration of the data. Before we get to the calibration tables, it's essential to do the flux density calibration of our calibrator 3C147. This is done by the <i>setjy</i> task. Before we run the task we first clear the preliminary calibration that was carried out to enable better flagging. We do that by running the <i>clearcal</i> task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
clearcal(vis='3C129_pband.ms')<br />
#<br />
setjy(vis='3C129_pband.ms', standard='Scaife-Heald 2012', field='3C147')<br />
</source><br />
<br />
== Delay and bandpass calibration ==<br />
<br />
First, we use the full bandwidth on a single scan on the primary calibrator to determine a single delay per antenna, per polarization. This will determine a single (approximate) phase slope across frequency, mainly caused by propagation effects in the (time-variable) ionosphere, cable length differences, and electronics in the signal paths from antenna feeds to correlator. Note that for a single short (5-10 minutes) scan on the calibrator, we can get away with solving for a time-invariant delay per antenna, per polarization. We will use scan 3 on 3C147.<br />
[[File:3C147_init_delay.png|200px|thumb|right|Figure 5: Initial per antenna delays derived by <i>gaincal</i>.]]<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K1', field='3C147', solint='inf', refant='ea09', gaintype='K',<br />
gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq'],parang=True)<br />
#<br />
plotcal(caltable='3C129_pband.K1', xaxis='antenna', yaxis='delay', markersize=2.0, <br />
plotrange=[0,30,-50.,50.],figfile='3C147_delay.png')<br />
</source><br />
<br />
Since we picked ea09 as our reference antenna, the delays for this antenna ID (=0) are arbitrarily set to zero. All other delays should be within 30 nanosec or so (which they should be for this data set). Larger values should be treated with suspicion, possibly indicating a problem with the antenna or polarization. We will use the bandpass calibration to verify this in the next step.<br />
<br />
To determine the bandpass calibration, we use the same source and scan and apply the delay calibration before solving for the bandpass. Note that we request a minimum SNR of 3, which will make the solve fail for the worst, but not all, channels.<br />
<source lang='python'><br />
# In CASA<br />
bandpass( vis='3C129_pband.ms', caltable='3C129_pband.B1', field='3C147', solint='inf', refant='ea09', minsnr=3.0,<br />
parang = True, gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq','3C129_pband.K1'],<br />
interp=['','','','nearest,nearestflag'])<br />
<br />
</source><br />
<br />
After solving, we inspect the bandpass calibration amplitudes and phase, and flag any obvious residual outliers. Outliers may be found above or below the average bandpass curves, and tend to arise in the same channel (frequency) ranges for all antennas. Don't spend more than 1 minute per antenna for this tutorial and don't worry about missing some bad points; these bad channels will likely get flagged in a later stage anyway. Note that the bandpass amplitudes (Fig. 6) and phases may differ between polarizations of the same antenna. Also note that the bandpass phases across frequency should not have an overall gradient (this should be removed by the delay calibration). Also note that bandpass phases may wrap around from +/-180 to -/+180 degrees. If that is problematic, re-run the phase <i>plotcal()</i> with <i>plotrange = [ 200.,500.,0.,360. ]</i>. Also note that there are no bandpass solutions for antennas ea19 and ea28, and only a few solutions for ea14.<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='amp', iteration='antenna', markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='phase', iteration='antenna', plotrange=[ 200.,500.,-180.,180. ], markersize = 2.0)<br />
</source><br />
{|<br />
| [[File:3C147_bp0_ea01.png|200px|thumb|right|Figure 6a: Initial bandpass amplitudes for antenna ea01.]]<br />
| [[File:3C147_bp0_phase_ea01.png|200px|thumb|right|Figure 6b: Initial bandpass phase for antenna ea01.]]<br />
|}<br />
<br />
Flagging is made easier by maximizing the plotcal() window. We don't set the scale for plotting the bandpass amplitudes, which is convenient when flagging outliers as it will auto-rescale. While plotting the bandpass phases we do fix the scale, as it provides a better feeling for the relevant magnitude of phase outliers. Flagging of outliers is done using the <i>Mark Region</i> and <i>Flag</i> buttons. For more information on interactive flagging, see the topical guide [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Flagging VLA Data in CASA]. We are using this mechanism to efficiently flag data manually, since it is done per antenna rather than per baseline. These 'flags' will not be permanent until we call <i>applycal()</i> later on, but are effective while applying the calibration on the fly, as we will do in subsequent calibration steps.<br />
<br />
== Gain calibration ==<br />
<br />
With the delay and bandpass calibration in place, we will now look more closely at the time-variable behaviour of the VLA. We use the same scan on our primary calibrator, 3C147, over the full effective bandwidth to determine gain calibrations: one complex value per antenna, per polarization, per integration time. The delay and bandpass tables are applied on the fly.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal( vis='3C129_pband.ms', caltable='3C129_pband.G1', field='3C147', solint = 'int',<br />
refant = 'ea09', minsnr = 3.0, gaintype = 'G', calmode = 'ap',<br />
gaintable = ['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim', '3C129_pband.K1', '3C129_pband.B1' ],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag' ], parang = True )<br />
<br />
</source><br />
{|<br />
| [[File:3C147_Gamp_ea01.png|200px|thumb|right|Figure 7a: Initial gain amplitude solutions for antenna ea01.]]<br />
| [[File:3C147_Gphase_ea01.png|200px|thumb|right|Figure 7b: Initial gain phase solutions for antenna ea01.]]<br />
|}<br />
<br />
Similar to the bandpass flagging, we will flag outlier gain solutions in both amplitude and phase. For the amplitudes, the final scatter around an average close to one should be a few percent (see Fig. 7). Note antennas ea08, ea09 have particularly noisy solutions. Remember that the gain solutions for both polarizations can be flagged together, as we will lose the equivalent data of the surviving polarization anyway.<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='amp', iteration='antenna', markersize=2.0 )<br />
#<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='phase', iteration='antenna', <br />
plotrange=[ None,None,-180.,180.], markersize=2.0)<br />
</source><br />
<br />
Note that when plotting gains against time, it is convenient to use auto-scaling of the time axis by putting <i>None</i> in the first two fields of <i>plotrange</i>.<br />
<br />
Next, we will prepare a smoothed and interpolated version of the gain calibration table, which will be applied later to the target field data. This prevents flagging of target field data when one of the edges of a calibrator scan is flagged for one or more antennas. The result can be inspected with <i>plotcal</i>.<br />
<source lang='python'><br />
# In CASA<br />
smoothcal(vis='3C129_pband.ms', tablein='3C129_pband.G1', caltable='3C129_pband.Gs1', smoothtype='median', smoothtime = 60.*60.)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='amp', iteration='antenna', plotrange=[ None,None,0.8,1.2 ], markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='phase', iteration='antenna', plotrange=[ None,None,-10.,10. ], markersize=2.0)<br />
<br />
</source><br />
<br />
== Transfer of calibrations to the target field ==<br />
<br />
So far we have used one scan on the primary calibrator 3C147 to derive various calibration tables. These calibrations will now be applied to all scans of all sources in our measurement set, which includes 3C147 itself and our target field 3C129. As described above, the task <i>applycal()</i> creates a <i>CORRECTED_DATA</i> column in which the calibrated visibilities get stored. This may take a while.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='3C129_pband.ms', parang=True, applymode='calflagstrict', flagbackup=True, gaintable=['3C129_pband.antpos','3C129_pband.rq',<br />
'3C129_pband.tecim','3C129_pband.K1','3C129_pband.B1','3C129_pband.Gs1'],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag', 'nearest,nearestflag'])<br />
</source><br />
<br />
Now we split off the calibrated target field data, meaning that the visibilities of source 3C129 get copied from the <i>CORRECTED_DATA</i> column to the <i>DATA</i> column of a new measurement set. This is convenient for further processing. <br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='3C129_pband.ms', outputvis='3C129_pband_target.ms', datacolumn='corrected', field='3C129')<br />
</source><br />
<br />
It is also helpful to perform additional flagging for any residual RFI left behind in our target after all our calibration steps have been completed and we begin the imaging of our data.<br />
<br />
<source land='python'><br />
flagdata(vis='3C129_pband_target.ms', mode='rflag')<br />
flagdata(vis='3C129_pband_target.ms', mode='extend')<br />
</source><br />
<br />
<br />
== Statwt ==<br />
Before we go ahead and image our target field. We would like to weight any residual low level RFI down. This is done using the task <i>statwt</i>. The task runs through the data and computes the rms of the data and weighting down outliers. This makes a big impact for imaging of a target in the presence of low level residual RFI. For our purposes we will bin the data in a time interval larger than the integration time and run statwt as follows.<br />
<br />
<source lang='python'> <br />
statwt(vis='3C129_pband_target.ms', datacolumn='data', timebin=30.)<br />
</source><br />
<br />
With data weights for outliers being set we will now proceed with the imaging and self-calibration of our data.<br />
==Imaging==<br />
<br />
At this point, we're ready to make a first image of our target field. Imaging in CASA is done by means of the <i>tCLEAN</i> task. The task implements algorithms for wide field imaging, such as W-Projection (Cornwell et al. 2008 http://arxiv.org/abs/0807.4161) and Multi Term Multi Frequency Synthesis ( Rau et al. 2011 http://arxiv.org/abs/1106.2745) both of which we will utilize to make our initial image. Even though our source of concern, the radio galaxy, lies in the centre of our field. The observations were carried out in B configuration resulting in an effective maximum resolution of approximately 5x5 arcsecs. To effectively model the source spectral index in the sky, we will utilize the MTMFS algorithm and use the W-Projection algorithm to make a wide field image (The full beam at P Band is about 3 degrees in diameter). So, going by these requirements, we can now compute the required cell and image size. Consulting the resolution guide of NRAO science page, we can see that the expected HPBW for B - Configuration, P -Band Observation is 18.5 arcsec. So, we can sample it well by using a 5 arcsec cellsize. Having done that, if we decide to make a wide field image to account for all the point sources we will set the image size parameter to 4096 pixels. To enable the wide-field algorithm, we set the <i>gridder='wproject'</i>: this invokes the W-Projection algorithm, upon which we set the number of W-Projection planes to be 128. We set the deconvolution mode as Multi Term Multi Scale Multi Frequency Synthesis using the <i>deconvolver='mtmfs'</i> parameter along with the number of Taylor terms to be considered during imaging to be 2 and choosing three different scales to model the point source and extended emission in our field well, we do this by setting <i> scales=[0,20,30]</i>. This allows for the source spectral variation to be modeled by a second order polynomial. We launch the interactive tclean process, but only for <i>spw='3~8'</i> to begin with. Note, running the first clean call will take at least ~10 min to process.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_initial_clean', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30]<br />
minpb=0.01, savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
</source><br />
<br />
{|<br />
| [[File:3C129_radio_gal_residual.png|200px|thumb|right|Figure 10a: Resulting residuals of 3C129 after first call of interactive <i>clean</i>.]]<br />
| [[File:3C129_radio_gal_clean.png|200px|thumb|right|Figure 10b: Resulting image of 3C129 after completing the first <i>clean</i>.]]<br />
|}<br />
The <i>tCLEAN</i> command launches an interactive session after a 100 iterations of clean and produces a wide field map with two sources at the center and a lot of bright sources far out in the field. As it is a snapshot image, the bright sources have significant side lobes and so tight clean boxing can help. This can done in the interactive viewer interface that pops up. If we proceed with interactive clean with subsequent steps to keep boxing out the strong sources that pop up in the image, we finally see the extended emission from the target radio galaxy begin to emerge. Continue boxing and cleaning to ensure that the residuals of the boxed cleaning look noise like (~10000 clean iterations). We now stop the interactive task and look at the final image it produced.<br />
<br />
<source lang='python'><br />
# In CASA<br />
viewer('3C129_initial_clean.image.tt0')<br />
</source><br />
<br />
The recovered structure at the center of the map should resemble what is shown in Fig. 10b. If the image looks worse then boxing of bright features in the map needs to be done more carefully. Note the image has some sources still showing strong side lobes and imaging artifacts around them. We expected this as we have carried out phase calibration only on our flux calibrator and have just transferred the solutions over to our target field. Since we used the <i>usescratch=True</i>, the MODEL_DATA column in the measurement set now contains the initial image model, which we will self calibrate against to produce a better image. Also in clean, do notice that the spw's utilized are the cleanest spectral windows that are totally RFI free.<br />
<br />
==Self Calibration ==<br />
<br />
We now proceed to compute gain phase solutions for our target field using the <i>gaincal</i> task as the first step in self-calibration.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScG0', field='3C129', solint='inf', refant='ea09', <br />
spw='3~8',minsnr=3.0, gaintype='G', parang=True, calmode='p')<br />
<br />
bandpass( vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScB0', field='3C129', solint='inf', refant='ea09', minsnr=3.0, spw='3~8',<br />
parang = True, gaintable=['3C129_pband_target.ScG0'],<br />
interp=['','','','nearest,nearestflag'])<br />
applycal(vis='3C129_pband_target.ms', gaintable=['3C129_pband_target.ScG0','3C129_pband_target.ScB0'], spw='3~8', applymode='calflagstrict')<br />
</source><br />
<br />
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the <i>tCLEAN</i> command once again.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_clean_sc0', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=False, scales=[0,20,30]<br />
minpb=0.01,savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
<br />
viewer('3C129_clean_sc0.image.tt0')<br />
</source><br />
[[File:3C129_radio_gal_cselfcal0.png|200px|thumb|right|Figure 11: Resulting image of 3C129 after completing a first selfcal.]]<br />
<br />
On boxing and cleaning, we already notice that the imaging artifacts have reduced significantly. We also see that the target source appears to contain more structure and a greater amount of flux. Further self calibration iterations involving an amplitude & phase gain calibration, and target bandpass calibration, are all possible steps that can be explored. <br />
<br />
== Appendix: Some P-band data issues you may want to know about ==<br />
<br />
Unfortunately, there are a few known issues with JVLA P-band observing. Most problems were discovered during the P-band commissioning period and have been fixed for newer data sets, but the archive keeps part of this history alive. Here we will go over some of those issues, and (if possible) provide ways of fixing them.<br />
<br />
=== Polarization labeling ===<br />
<br />
For a long period, P-band feeds have been labelled as being circular (R and L), while they are linear (X and Y). It is still possible to generate data with incorrect polarization labeling, however for standard observations this should not be the case anymore. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem. This may seem mostly harmless, but does make a difference for polarization calibration. The following (non-standard) task will check and fix this (and another related problem of swapped polarizations):<br />
<br />
Download [http://casa.nrao.edu/Data/EVLA/Pband/casa_vla_lowband.tar.gz this tarball] in your CASA working directory, open a shell there, and type the following commands:<br />
<br />
<source lang='bash'><br />
tar -xzvf casa_vla_lowband.tar.gz<br />
<br />
buildmytasks<br />
</source><br />
<br />
Now go back to your CASA session and type:<br />
<br />
<source lang='python'><br />
# In CASA<br />
execfile('mytasks.py')<br />
#<br />
fixlowband(vis='TSUB0001_pband_prehs.ms')<br />
</source><br />
<br />
Re-run the <i>listobs()</i> task to check if the fix was correctly applied.<br />
<br />
=== Swapped polarizations ===<br />
<br />
Throughout the whole history of the new VLA low-band system, even up to this day, there have been mistakes in the cabling of the full signal chain. This results in that some antennas have the X-polarization signal come in as Y and vice versa. The way to notice this in visibility data is that for baselines with one swapped antenna most power will be in the cross-hand correlations (XY and YX) rather than the parallel-hand correlations (XX and YY). Once antennas with this feature are manually identified, a contributed task <i>swappol()</i> is available to fix this problem (see under polarization labelling).<br />
<br />
=== Double data descriptor entries ===<br />
<br />
The data description table is part of the measurement set, and provides a link between the recorded visibilities, the spectral window information and the polarization information. For some data sets, the data description table contains two entries for each P-band spectral window, one pointing towards a circular (RL) polarization definition, and one pointing towards a linear (XY) polarization definition. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem.<br />
<br />
=== Continuous Radio Frequency Interference (RFI) ===<br />
<br />
The wide bandwidth of the low-band receiver is (unfortunately) guaranteed to contain significant amounts of RFI. There are a few RFI sources that are active all the time, and are visible in all array configurations. The default P-band observing setup of 16 x 16 MHz tries to capture as much of the continuous RFI in as few spectral windows as possible, allowing for a simple RFI mitigation strategy in which these spectral windows (spws 1,2 and 9,10, but possibly more) can be immediately flagged.<br />
<br />
=== Two spectral window setup ===<br />
<br />
In the early commissioning period, the default setup for P-band observing was to use 2 spectral windows of 1024 channels each to cover 256 MHz of bandwidth from 230 - 486 MHz. However, it was noticed that strong, narrow-band RFI events were causing data to be lost for the whole spectral window in which they occured. To make the system more robust against data loss from these events, the frequency range was divided into 16 x 16 MHz, and shifted downwards by 6 MHz to capture ever-present RFI into as few spectral windows as possible. If your data has only two P-band spectral windows, please be aware that a higher data loss due to RFI is possible. There is no way to repair this.<br />
<br />
=== Bandpass ripples ===<br />
<br />
Due to signal reflections in cables and within the VLA dish, sinusoidal amplitude and phase modulations are always present in P-band data. This is most easily seen in bandpass calibration plots of amplitude versus frequency. In some cases, mostly due to cable & connector problems, these modulations can be very strong (~up to 50 percent of the average amplitude level). These modulations tend not to vary over the duration of an observation and can therefore be removed through bandpass and polarization calibration. If they are found to be variable (e.g., by inspecting bandpass solutions for separate calibrator scans), the offending antenna / polarization should be flagged.<br />
<br />
=== High cross-polarization ===<br />
<br />
On each antenna, the P-band feed (dipole) is visually aligned with respect to the primary focus support legs. This is normally done within 5-10 degrees accuracy. On rare occasions the dipole on one antenna has accidentally rotated to much larger angles because the locking bolt on the back of the feed was not completely tightened. The result is a high cross-polarization in the baseline visibilities that include this antenna. If this is the case, it is safest to flag this data, since there is a possibility that the dipole has rotated during the observations.<br />
<br />
=== Instrumental Polarization Calibration ===<br />
<br />
At present polarization P-band observations are in RSRO mode only. While polarization leakage calibration has been highlighted in the past reports it does not directly impact imaging in Stokes I and hence has been dropped from this casaguide.<br />
<br />
== Useful links ==<br />
<br />
* [http://www.aoc.nrao.edu/lbr The EVLA Low-Band Receiver System Status & User Pages]<br />
* [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/topical-guides/vlofreq VLA Very Low Frequency Observing]<br />
<br />
<br />
{{Checked 5.4.1}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0&diff=26242VLA Radio galaxy 3C 129: P-band continuum tutorial-CASA5.4.02018-12-01T22:51:37Z<p>Emomjian: </p>
<hr />
<div><!--By [http://www.aoc.nrao.edu/~pjaganna/Site/Home.html Preshanth Jagannathan] & [https://safe.nrao.edu/wiki/bin/view/Main/HuibIntemaWebHome Huib Intema] & Frank Schinzel--><br />
<br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b><p></p> <br />
<b> Note: This guide may take up to 2-days to complete </b><p></p> <br />
If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
<br />
This webpage provides a basic description on how to use CASA to reduce data from the upper part of the new VLA low-band system, also known as P-band, covering roughly 220-480 MHz. The goal is to make a wide-field continuum stokes I image of a typical blank field using the full effective bandwidth. The dataset that will be downloaded in this guide has a size of 26.46 GB and will require a total of ~140 GB of disk space if all generated files are kept. <br />
<br />
''Note: Starting with CASA 5.0 IPython 5.1.0 is used which for interactive CASA sessions. This does not allow to readily paste and execute multiple lines of code like in previous versions of the interpreter. If this is needed at any point in this tutorial, we use "%cpaste" to allow the copy/paste of loops, conditions, etc.. Press "Enter" (or "Return") after "--" at the end, this stops (or exits) the copy/paste prompt and will return you to the CASA prompt.''<br />
<br />
== Obtaining the raw data ==<br />
<br />
For this guide, we'll use some test data that was taken in B-configuration. To get a copy of the raw data, download [https://casa.nrao.edu/Data/EVLA/Pband/P_band_3C129.tgz P_band_3C129.tgz] onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
<br />
Alternatively, the raw data can also be obtained from the NRAO archive, go to the [https://archive.nrao.edu/archive/advquery.jsp NRAO advance archive query page], and enter the following search parameter in the field <i>Archive File ID</i> (leave the rest on default):<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
imag-test-copy.57080.956837025464<br />
</pre><br />
</div><br />
<br />
The search should return a single row for Archive File: imag-test-copy.57080.956837025464, taken on Feb. 27, 2015 (or "15-Feb-27 22:57:52"). Check the box next to it. Next, fill in your email and select the "SDM-BDF dataset (all files)" option. Note, you may find it easier to download a tar file, in which case, you'll want to check the box for Create MS or SDM tar file. Click "Get My Data" and follow the on-screen prompts to reach the next page, where you should click "Retrieve over internet". The next page should report that the data staging is in progress. Wait until you receive an e-mail reporting that your archive data is copied, which should take a few minutes. Download the data onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing <br />
<source lang='bash'><br />
casa<br />
</source><br />
on the (linux) command line. This should start a CASA interactive python (iPython) session, and open a separate log window. The CASA version is reported at startup, both in the python session and the log window.<br />
<br />
== Importing the raw data into CASA ==<br />
<br />
We will begin by importing our data into the measurement set format (CASA standard) from the binary format (SDM-BDF) as downloaded from the archive. We do this by means of the importasdm task. It is important to note that the importevla task has been deprecated.<br />
<br />
<source lang='python'><br />
# In CASA<br />
importasdm(asdm='imag-test-copy.57080.956837025464', vis ='3C129.ms', savecmds=True, outfile='importflags.txt')<br />
<br />
</source><br />
<br />
This task calls the external tool <i>asdm2MS</i> to perform the conversion. Together with the visibility data, also the flag commands from the VLA online system are imported. These flags are not directly applied, but stored in the <i>importflags.txt</i> file and applied later.<br />
<br />
== Preliminary data inspection ==<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms', verbose = True, listfile = '3C129.listobs')<br />
</source><br />
<br />
Alternatively, you can also output to the logger by running the command without the listfile argument. The sample listobs output shown has flag <i>verbose=False</i> for the sake of brevity.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/fschinze/P_band_tutorial/3C129.ms MS Version 2<br />
================================================================================<br />
Observer: Project: uid://evla/sdm/X2<br />
Observation: EVLA(27 antennas)<br />
Data records: 4714983 Total elapsed time = 1422 seconds<br />
Observed from 27-Feb-2015/22:57:54.0 to 27-Feb-2015/23:21:36.0 (UTC)<br />
<br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C147 05:42:36.127300 +49.51.07.17800 J2000 0 1307124<br />
1 NONE 3C129 04:48:58.200000 +45.02.01.00000 J2000 1 3407859<br />
Spectral Windows: (19 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 EVLA_P#A0C0#0 128 TOPO 224.000 125.000 16000.0 231.9375 12 XX XY YX YY<br />
1 EVLA_P#A0C0#1 128 TOPO 240.000 125.000 16000.0 247.9375 12 XX XY YX YY<br />
2 EVLA_P#A0C0#2 128 TOPO 256.000 125.000 16000.0 263.9375 12 XX XY YX YY<br />
3 EVLA_P#A0C0#3 128 TOPO 272.000 125.000 16000.0 279.9375 12 XX XY YX YY<br />
4 EVLA_P#A0C0#4 128 TOPO 288.000 125.000 16000.0 295.9375 12 XX XY YX YY<br />
5 EVLA_P#A0C0#5 128 TOPO 304.000 125.000 16000.0 311.9375 12 XX XY YX YY<br />
6 EVLA_P#A0C0#6 128 TOPO 320.000 125.000 16000.0 327.9375 12 XX XY YX YY<br />
7 EVLA_P#A0C0#7 128 TOPO 336.000 125.000 16000.0 343.9375 12 XX XY YX YY<br />
8 EVLA_P#A0C0#8 128 TOPO 352.000 125.000 16000.0 359.9375 12 XX XY YX YY<br />
9 EVLA_P#A0C0#9 128 TOPO 368.000 125.000 16000.0 375.9375 12 XX XY YX YY<br />
10 EVLA_P#A0C0#10 128 TOPO 384.000 125.000 16000.0 391.9375 12 XX XY YX YY<br />
11 EVLA_P#A0C0#11 128 TOPO 400.000 125.000 16000.0 407.9375 12 XX XY YX YY<br />
12 EVLA_P#A0C0#12 128 TOPO 416.000 125.000 16000.0 423.9375 12 XX XY YX YY<br />
13 EVLA_P#A0C0#13 128 TOPO 432.000 125.000 16000.0 439.9375 12 XX XY YX YY<br />
14 EVLA_P#A0C0#14 128 TOPO 448.000 125.000 16000.0 455.9375 12 XX XY YX YY<br />
15 EVLA_P#A0C0#15 128 TOPO 464.000 125.000 16000.0 471.9375 12 XX XY YX YY<br />
16 EVLA_P#B0D0#16 64 TOPO 310.000 500.000 32000.0 325.7500 15 XX XY YX YY<br />
17 EVLA_P#B0D0#17 1024 TOPO 310.000 62.500 64000.0 341.9688 15 XX XY YX YY<br />
18 EVLA_P#B0D0#18 64 TOPO 342.000 500.000 32000.0 357.7500 15 XX XY YX YY<br />
Antennas: 27 'name'='station'<br />
ID= 0-3: 'ea01'='N12', 'ea02'='N16', 'ea03'='W28', 'ea05'='E20',<br />
ID= 4-7: 'ea06'='N20', 'ea07'='N32', 'ea08'='E28', 'ea09'='W08',<br />
ID= 8-11: 'ea10'='E36', 'ea11'='W12', 'ea12'='N36', 'ea13'='W04',<br />
ID= 12-15: 'ea14'='E08', 'ea15'='E24', 'ea16'='W24', 'ea17'='E04',<br />
ID= 16-19: 'ea18'='W36', 'ea19'='MAS', 'ea20'='N04', 'ea21'='E32',<br />
ID= 20-23: 'ea22'='N24', 'ea23'='E16', 'ea24'='W32', 'ea25'='W20',<br />
ID= 24-26: 'ea26'='E12', 'ea27'='N08', 'ea28'='N28'<br />
<br />
</pre><br />
<br />
You can inspect the output text file <i>3C129.listobs</i> in your favorite text editor (e.g., <i>gedit</i>). When taking some time to familiarize yourself with the output format, you will see that:<br />
* The observation consists of 5 scans, namely:<br />
# 3C147 (hardware setup)<br />
# 3C147 (setting requantizers)<br />
# 3C147 (primary calibrator)<br />
# 3C129 (target)<br />
# 3C129 (target)<br />
* The frequency coverage is 222 - 478 MHz, divided into 16 x 16 MHz spectral windows, each having 128 x 0.125 MHz channels. There are also two spectral windows with 64x0.5 MHz and one spectral window with 1024x0.0625 MHz. <br />
* Visibilities are recorded every 2 seconds in full polarization (4 polarization products labelled XX,XY,YX,YY).<br />
* Of the 28 VLA antennas labelled ea01 - ea028, antenna ea04 is not participating in this observation.<br />
* Source, antenna and spectral window IDs start at zero, but scan IDs start at one.<br />
<br />
<br />
[[File:3C129_pband_plotants.png|200px|thumb|right|Figure 1: plotants figure]] <br />
<br />
The array configuration can be inspected using:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotants(vis='3C129.ms', figfile='3C129_pband_plotants.png')<br />
</source><br />
<br />
== Initial processing steps - Antenna Position, Requantizer Gain, Ionospheric Correction ==<br />
<br />
When importing the data, we saved the flags from the online system to an ASCII text file. This gives us the opportunity to review the flag commands before applying them. For our data set, the flag file is called <i>importflags.txt</i>. Please load this file into your favorite text editor. The bulk of the flag commands refer to times when the VLA is slewing (<i>ANTENNA_NOT_ON_SOURCE</i>) or when the movable secondary reflector of the VLA's Cassegrain system is not in place (<i>SUBREFLECTOR_ERROR</i>). The latter can cause antenna gain variations (amplitude and phase), so it is safest to apply all the flags. The final two rows in the flag file will remove visibilities that are pure zero, and flag antennas that are partly blocked by other antennas (shadowing) which occurs mostly in compact configurations when observing along a VLA arm).<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms', mode='list', inpfile='importflags.txt', action='apply', reason='any', flagbackup=True)<br />
</source><br />
This call, before applying the new flags to the data, created a new flagbackup file called flagdata_1 which can be used by the task <i>flagmanager</i> to restore the state of the measurement set prior applying the flags from <i>importflags.txt</i>.<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
Now that we have applied the import flags, we will correct for the antenna position offsets. Antenna positional errors translates to an error in the measured visibilities and need to be accounted for before we proceed with any of the other calibration steps. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltable='3C129_pband.antpos',caltype='antpos')<br />
</source><br />
<br />
The logger output shows that there are 13 antennas with positional offsets: <br />
<pre><br />
<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea03 : 0.00290 0.00000 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea08 : 0.00000 0.00390 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea10 : -0.00170 0.00000 0.00400<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea11 : 0.00000 0.00160 -0.00170<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea13 : 0.00000 0.00050 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea16 : -0.00180 0.00680 0.00230<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea18 : 0.00470 0.00350 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea21 : -0.00160 0.00000 0.00210<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea23 : 0.00140 0.00200 -0.00150<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea24 : 0.00380 0.00000 -0.00340<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea25 : 0.00330 0.00000 -0.00110<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea26 : 0.00000 0.00190 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea27 : -0.00120 0.00070 0.00000<br />
<br />
<br />
</pre><br />
<br />
=== Ionospheric TEC Corrections ===<br />
[[File:Pband_vla_tecu_new.png|200px|thumb|right|Figure 2: TEC value vs time generated by <i>tec_maps.create()</i>]] <br />
<br />
Low frequency observations are affected by the ionosphere. A delay in the signal path is introduced between the two polarization of light that varies both as a function of time and line of sight (direction dependent). The delay is proportional to the Total Electron Content (TEC) along the line of sight and is inversely proportional to the square of the frequency. GPS measurements at two different frequencies provides us with an estimate of the TEC per square meter. This correction has been implemented in CASA which we shall apply as a calibration table by means of the <i>gencal</i> task. The task requires a TEC map that we will generate utilizing CASA recipes. For more information, see the section on [https://casa.nrao.edu/casadocs/casa-5.0.0/synthesis-calibration/preparing-for-calibration/#title5 ionospheric corrections] in the CASA documentation archives. For some versions of CASA, the first plot to the right (Figure 2) may not be generated if you have already run plotants: to generate this plot, you will need to quit CASA and start it again. If the IONEX data file was not downloaded you might be missing the curl package in the environment you are running CASA in. In this case you can provide the required file manually by downloading: ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/2015/058/igsg0580.15i.Z, (for the tutorial it is also also available in an unzipped version [http://casa.nrao.edu/Data/EVLA/Pband/igsg0580.15i here]) unziping it, and placing it in the directory from which you started CASA. The current implementation in CASA is in the development stage, however since the CASA 5.0 release both the Faraday rotation correction and dispersive delay corrections for the ionosphere are enabled. In previous releases only Faraday rotation corrections are enabled. In this example we will generate the calibration table and apply it throughout calibration. <br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
#<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='3C129.ms',doplot=True)<br />
#[NOTE: If you are using CASA 5.0.0, or earlier this should be: tec_image, tec_rms_image = tec_maps.create(vis='3C129.ms', doplot=True)]<br />
#<br />
gencal(vis='3C129.ms',caltable='3C129_pband.tecim',caltype='tecim',infile=tec_image)<br />
# Spatial plots like shown in Figure 3 can be looked at using the CASA viewer for the TEC values<br />
viewer('3C129.ms.IGS_TEC.im')<br />
# and for TEC rms<br />
viewer('3C129.ms.IGS_RMS_TEC.im')<br />
</source><br />
<br />
[[File:Pband_tec_new.png|200px|thumb|right|Figure 3: TEC map movie generated by the TEC tasks.]] <br />
<br />
A word of caution regarding the TEC map generation. The IGS website updates measurements only two weeks after the date of observation.<br />
<br />
=== Requantizer Gains ===<br />
<br />
The next step will correct the visibility amplitudes for the signal leveling (requantizer gains) that occurs at the inputs of the WIDAR correlator. These levels (per antenna, per polarization, per spectral window) are stored with the measurement set (in the <i>SYSPOWER</i> sub-table). This step is currently not essential, since the levels get set only once at the start of an observation and bandpass calibration will correct for this. But it will make your bandpass plots look better if you have multiple spectral windows. And, more importantly, in general it is possible to trigger the setting of requantizers throughout an observation. This is the case for new observations, when a P-band scan is preceded by a P-band setup scan, which itself is preceded by a scan with a different telescope configuration. <br />
<br />
The correction is done by means of the <i>gencal</i> task, where information in the <i>SYSPOWER</i> sub-table gets translated into a gain table. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltype='rq',caltable='3C129_pband.rq')<br />
</source><br />
<br />
== Dead antennas ==<br />
<br />
Now it is time to have a first visual look at the uncalibrated visibility data. It is important to identify dead antennas / polarizations early on, so we can exclude them from further data processing (which is the most efficient data reduction). A convenient way of doing this is through the <i>plotms</i> task on bright calibrator(s) (see the <i>listobs()</i> output), in our case 3C147 (see Fig. 4a).[[File:3C129_pband_3C147_prebp.png|200px|thumb|right|Figure 4a: <i>plotms</i> output before RFI flagging]]<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY', field='3C147', <br />
plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',iteraxis='baseline', <br />
plotfile='3C129_pband_3C147_prebp.png')<br />
</source><br />
<br />
This will load the <i>plotms</i> window that has queued up all the baselines of antenna ea01 and is currently displaying the first amplitude vs frequency plot with the colors representing the different spectral windows. Note that we are only plotting the XX and YY correlation of the antennas. This is because we expect most of the power in the linear cross correlation products to be in these two correlations. Notice that some spectral windows in particular are badly affected due to radio frequency interference (RFI). This is the issue that we will deal with after identifying dead antennas. If you scroll through the baselines using the green forward button in the <i>plotms</i> viewer, you will notice that the amplitude is particularly low when you encounter ea19. If we make a quick plot of the baselines made by ea19 (put <i>ea19</i> in the antenna field) we can see that there is no power in any of the polarizations indicating this antenna was dead and can be flagged out. It is also good to check for high power levels in cross-hand correlations in comparison to the parallel-hand correlations, which would uncover swapped polarization labels and severely mis-aligned dipoles. At the end of this tutorial some hints on solving for swapped or mislabeled dipole polarizations are given. In addition to the dead antenna, we also make sure that the first two scans are flagged, which were not labelled as setup scans but were used to set the attenuators and requantizers.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms',mode='manual',antenna='ea19')<br />
#<br />
flagdata(vis='3C129.ms',mode='manual',scan='1~2')<br />
</source><br />
<br />
We also noticed sharp RFI peaks during our data examination. To prevent Gibbs ringing, it is best to hanning smooth the data at this juncture before we proceed further with automatic flagging. Now is a good time to get a cup of coffee while hanning smoothing the data. [[File:3C129_pband_3C147_prebp_hanning.png|200px|thumb|right|Figure 4b: <i>plotms</i> output after Hanning smooth and first flagging.]]<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='3C129.ms',outputvis='3C129_pband.ms',datacolumn='data')<br />
</source><br />
<br />
We should replot the primary calibrator to see the effect of hanning smoothing on the data and the RFI (Fig. 4b). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning.png')<br />
<br />
</source><br />
<br />
== Automatic flagging ==<br />
<br />
<br />
It is worthwhile noting at this juncture that the standard P-band setup will ensure that the worst of the RFI is restricted to four of the spectral windows. This particular dataset is an example of higher than average levels of RFI across the band. Before throwing away bad spectral windows by hand, we'll let the automated flaggers in CASA have a go at it. The main task for this is <i>flagdata()</i>. While working on your data, <i>flagdata()</i> produces an abundance of output, not all of which is easy to understand. To help with this, we can get the flag status of our data before and after auto-flagging by running <i>flagdata()</i> in the <i>summary</i> mode. This first call provides the flag statistics of the flagging up to now, before any auto-flagging:<br />
<source lang='python'><br />
summary_1 = flagdata(vis='3C129_pband.ms', mode='summary')<br />
</source><br />
<br />
This returns a python dictionary with flagged versus total visibilities along various axes (antenna, scan, spw, field, correlation, etc.). For example, if we want to know the percentage flagged per scan, run the following (note that the scans may not appear in sorted order):<br />
<source lang='python'><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for id, stats in summary_1[ axis ].iteritems():<br />
print '%s %s: %5.1f percent flagged' % ( axis, id, 100. * stats[ 'flagged' ] / stats[ 'total' ] )<br />
--<br />
</source><br />
<br />
For this example, you will notice that scans 1 and 2 (the dummy scans on 3C147) are 100 percent flagged, which is what we did during the initial flagging.<br />
<br />
We will run <i>flagdata()</i> in the <i>tfcrop</i> mode, which will (per scan, per baseline, per spectral window, per polarization) look for visibility amplitude outliers. It uses a 5-piece polynomial in an attempt to remove any intrinsic bandpass (amplitude) structure (we did not calibrate for bandpass yet). We run the task twice to allow for slightly deeper flagging and separate treatment of the parallel and cross-hand correlations. For the first run, we tell CASA to make a backup of our visibility flag status up to here, giving us an option to restore them if we choose over-aggresive flagging parameters and consequently over-flag our data. The flag backup file name is reported in the log window, and can be found in the <i>3C129_pband.ms.flagversions</i> directory.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=4., freqcutoff=3., maxnpieces=5,<br />
action='apply', display='report', flagbackup=True, combinescans=True, ntime='3600s', correlation='ABS_XY,ABS_YX')<br />
#<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=3., freqcutoff=3., maxnpieces=2,<br />
action='apply', display='report', flagbackup=False, combinescans=True, ntime='3600s', correlation='ABS_XX,ABS_YY')<br />
#<br />
flagdata(vis='3C129_pband.ms', mode='extend')<br />
</source><br />
<br />
As a note of caution: When executing flagdata with <i>mode='extend'</i>, all flags that were ever applied to the dataset will get extended. Repeat execution of this command will incrementally flag more and more around already flagged data.<br />
<br />
Let's get another flag summary to see how much extra data was flagged: <br />
<source lang='python'><br />
# In CASA<br />
summary_2 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
# <br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_2[ axis ].iteritems():<br />
old_stats = summary_1[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning_automatic.png')<br />
<br />
</source><br />
[[File:3C129_pband_3C147_prbp_tfcropped.png|200px|thumb|right|Figure 4c: <i>plotms</i> output after RFI flagging with tfcrop]]<br />
There is quite a bit of RFI that gets missed in RFI-rich spectral windows as can be seen in Fig. 4c. The solution to this is to provide the flagging routines with more contrast between healthy and affected data. For this, we will perform a preliminary bandpass calibration to take out the bandpass shape. We will do some coarse preliminary calibration and apply it to the calibrator before flagging for RFI once more.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.G0', gaintype='G', calmode='p', solint='int', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim'])<br />
#<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K0', gaintype='K', solint='inf', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0'])<br />
#<br />
bandpass(vis='3C129_pband.ms', caltable='3C129_pband.B0', solint='inf', field='3C147',refant='ea09', minsnr=2.0,<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0'])<br />
#<br />
applycal(vis='3C129_pband.ms', field='3C147', applymode='calflagstrict',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0','3C129_pband.B0'] )<br />
<br />
</source><br />
<br />
On the command line you should see a notification stating: "FJones (iononsphere) apply now provisionally includes disp. delay." which indicates that both Faraday rotation as well as dispersive delay corrections for the ionosphere are applied.<br />
<br />
[[File:3c147_postbandpass_rflag.png|200px|thumb|right|Figure 4d: <i>plotms</i> output after RFI flagging with <i>mode='rflag'</i>]]<br />
<br />
We will now flag the corrected data column that contains the coarsely calibrated visibilities which provide better contrast to the flagging algorithms to remove the RFI present. We will begin by running the <i>RFLAG</i> algorithm in the task <i>flagdata</i>. We will again use summary mode to see how much more of the observed data was flagged.<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
summary_3 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
#<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_3[ axis ].iteritems():<br />
old_stats = summary_2[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C147_postbandpass_rflag.png', ydatacolumn='corrected')<br />
</source><br />
<br />
Taking another visual look at the data using <i>plotms</i>, we examine the effect of the automated flagging on the calibrator (Fig. 4d). We find that most spectral windows are RFI free and that about 60% of the data was flagged. We typically expect between 40-60% of flagging due to interference in a significant fraction of the band. In this example, a more careful RFI flagging approach is likely to recover more usable spectrum.<br />
<br />
==Flux Density Calibration==<br />
The initial flagging has cleaned up most of the stray RFI across the band. This allows us to proceed with the actual calibration of the data. Before we get to the calibration tables, it's essential to do the flux density calibration of our calibrator 3C147. This is done by the <i>setjy</i> task. Before we run the task we first clear the preliminary calibration that was carried out to enable better flagging. We do that by running the <i>clearcal</i> task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
clearcal(vis='3C129_pband.ms')<br />
#<br />
setjy(vis='3C129_pband.ms', standard='Scaife-Heald 2012', field='3C147')<br />
</source><br />
<br />
== Delay and bandpass calibration ==<br />
<br />
First, we use the full bandwidth on a single scan on the primary calibrator to determine a single delay per antenna, per polarization. This will determine a single (approximate) phase slope across frequency, mainly caused by propagation effects in the (time-variable) ionosphere, cable length differences, and electronics in the signal paths from antenna feeds to correlator. Note that for a single short (5-10 minutes) scan on the calibrator, we can get away with solving for a time-invariant delay per antenna, per polarization. We will use scan 3 on 3C147.<br />
[[File:3C147_init_delay.png|200px|thumb|right|Figure 5: Initial per antenna delays derived by <i>gaincal</i>.]]<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K1', field='3C147', solint='inf', refant='ea09', gaintype='K',<br />
gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq'],parang=True)<br />
#<br />
plotcal(caltable='3C129_pband.K1', xaxis='antenna', yaxis='delay', markersize=2.0, <br />
plotrange=[0,30,-50.,50.],figfile='3C147_delay.png')<br />
</source><br />
<br />
Since we picked ea09 as our reference antenna, the delays for this antenna ID (=0) are arbitrarily set to zero. All other delays should be within 30 nanosec or so (which they should be for this data set). Larger values should be treated with suspicion, possibly indicating a problem with the antenna or polarization. We will use the bandpass calibration to verify this in the next step.<br />
<br />
To determine the bandpass calibration, we use the same source and scan and apply the delay calibration before solving for the bandpass. Note that we request a minimum SNR of 3, which will make the solve fail for the worst, but not all, channels.<br />
<source lang='python'><br />
# In CASA<br />
bandpass( vis='3C129_pband.ms', caltable='3C129_pband.B1', field='3C147', solint='inf', refant='ea09', minsnr=3.0,<br />
parang = True, gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq','3C129_pband.K1'],<br />
interp=['','','','nearest,nearestflag'])<br />
<br />
</source><br />
<br />
After solving, we inspect the bandpass calibration amplitudes and phase, and flag any obvious residual outliers. Outliers may be found above or below the average bandpass curves, and tend to arise in the same channel (frequency) ranges for all antennas. Don't spend more than 1 minute per antenna for this tutorial and don't worry about missing some bad points; these bad channels will likely get flagged in a later stage anyway. Note that the bandpass amplitudes (Fig. 6) and phases may differ between polarizations of the same antenna. Also note that the bandpass phases across frequency should not have an overall gradient (this should be removed by the delay calibration). Also note that bandpass phases may wrap around from +/-180 to -/+180 degrees. If that is problematic, re-run the phase <i>plotcal()</i> with <i>plotrange = [ 200.,500.,0.,360. ]</i>. Also note that there are no bandpass solutions for antennas ea19 and ea28, and only a few solutions for ea14.<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='amp', iteration='antenna', markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='phase', iteration='antenna', plotrange=[ 200.,500.,-180.,180. ], markersize = 2.0)<br />
</source><br />
{|<br />
| [[File:3C147_bp0_ea01.png|200px|thumb|right|Figure 6a: Initial bandpass amplitudes for antenna ea01.]]<br />
| [[File:3C147_bp0_phase_ea01.png|200px|thumb|right|Figure 6b: Initial bandpass phase for antenna ea01.]]<br />
|}<br />
<br />
Flagging is made easier by maximizing the plotcal() window. We don't set the scale for plotting the bandpass amplitudes, which is convenient when flagging outliers as it will auto-rescale. While plotting the bandpass phases we do fix the scale, as it provides a better feeling for the relevant magnitude of phase outliers. Flagging of outliers is done using the <i>Mark Region</i> and <i>Flag</i> buttons. For more information on interactive flagging, see the topical guide [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Flagging VLA Data in CASA]. We are using this mechanism to efficiently flag data manually, since it is done per antenna rather than per baseline. These 'flags' will not be permanent until we call <i>applycal()</i> later on, but are effective while applying the calibration on the fly, as we will do in subsequent calibration steps.<br />
<br />
== Gain calibration ==<br />
<br />
With the delay and bandpass calibration in place, we will now look more closely at the time-variable behaviour of the VLA. We use the same scan on our primary calibrator, 3C147, over the full effective bandwidth to determine gain calibrations: one complex value per antenna, per polarization, per integration time. The delay and bandpass tables are applied on the fly.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal( vis='3C129_pband.ms', caltable='3C129_pband.G1', field='3C147', solint = 'int',<br />
refant = 'ea09', minsnr = 3.0, gaintype = 'G', calmode = 'ap',<br />
gaintable = ['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim', '3C129_pband.K1', '3C129_pband.B1' ],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag' ], parang = True )<br />
<br />
</source><br />
{|<br />
| [[File:3C147_Gamp_ea01.png|200px|thumb|right|Figure 7a: Initial gain amplitude solutions for antenna ea01.]]<br />
| [[File:3C147_Gphase_ea01.png|200px|thumb|right|Figure 7b: Initial gain phase solutions for antenna ea01.]]<br />
|}<br />
<br />
Similar to the bandpass flagging, we will flag outlier gain solutions in both amplitude and phase. For the amplitudes, the final scatter around an average close to one should be a few percent (see Fig. 7). Note antennas ea08, ea09 have particularly noisy solutions. Remember that the gain solutions for both polarizations can be flagged together, as we will lose the equivalent data of the surviving polarization anyway.<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='amp', iteration='antenna', markersize=2.0 )<br />
#<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='phase', iteration='antenna', <br />
plotrange=[ None,None,-180.,180.], markersize=2.0)<br />
</source><br />
<br />
Note that when plotting gains against time, it is convenient to use auto-scaling of the time axis by putting <i>None</i> in the first two fields of <i>plotrange</i>.<br />
<br />
Next, we will prepare a smoothed and interpolated version of the gain calibration table, which will be applied later to the target field data. This prevents flagging of target field data when one of the edges of a calibrator scan is flagged for one or more antennas. The result can be inspected with <i>plotcal</i>.<br />
<source lang='python'><br />
# In CASA<br />
smoothcal(vis='3C129_pband.ms', tablein='3C129_pband.G1', caltable='3C129_pband.Gs1', smoothtype='median', smoothtime = 60.*60.)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='amp', iteration='antenna', plotrange=[ None,None,0.8,1.2 ], markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='phase', iteration='antenna', plotrange=[ None,None,-10.,10. ], markersize=2.0)<br />
<br />
</source><br />
<br />
== Transfer of calibrations to the target field ==<br />
<br />
So far we have used one scan on the primary calibrator 3C147 to derive various calibration tables. These calibrations will now be applied to all scans of all sources in our measurement set, which includes 3C147 itself and our target field 3C129. As described above, the task <i>applycal()</i> creates a <i>CORRECTED_DATA</i> column in which the calibrated visibilities get stored. This may take a while.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='3C129_pband.ms', parang=True, applymode='calflagstrict', flagbackup=True, gaintable=['3C129_pband.antpos','3C129_pband.rq',<br />
'3C129_pband.tecim','3C129_pband.K1','3C129_pband.B1','3C129_pband.Gs1'],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag', 'nearest,nearestflag'])<br />
</source><br />
<br />
Now we split off the calibrated target field data, meaning that the visibilities of source 3C129 get copied from the <i>CORRECTED_DATA</i> column to the <i>DATA</i> column of a new measurement set. This is convenient for further processing. <br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='3C129_pband.ms', outputvis='3C129_pband_target.ms', datacolumn='corrected', field='3C129')<br />
</source><br />
<br />
It is also helpful to perform additional flagging for any residual RFI left behind in our target after all our calibration steps have been completed and we begin the imaging of our data.<br />
<br />
<source land='python'><br />
flagdata(vis='3C129_pband_target.ms', mode='rflag')<br />
flagdata(vis='3C129_pband_target.ms', mode='extend')<br />
</source><br />
<br />
<br />
== Statwt ==<br />
Before we go ahead and image our target field. We would like to weight any residual low level RFI down. This is done using the task <i>statwt</i>. The task runs through the data and computes the rms of the data and weighting down outliers. This makes a big impact for imaging of a target in the presence of low level residual RFI. For our purposes we will bin the data in a time interval larger than the integration time and run statwt as follows.<br />
<br />
<source lang='python'> <br />
statwt(vis='3C129_pband_target.ms', datacolumn='data', timebin=30.)<br />
</source><br />
<br />
With data weights for outliers being set we will now proceed with the imaging and self-calibration of our data.<br />
==Imaging==<br />
<br />
At this point, we're ready to make a first image of our target field. Imaging in CASA is done by means of the <i>tCLEAN</i> task. The task implements algorithms for wide field imaging, such as W-Projection (Cornwell et al. 2008 http://arxiv.org/abs/0807.4161) and Multi Term Multi Frequency Synthesis ( Rau et al. 2011 http://arxiv.org/abs/1106.2745) both of which we will utilize to make our initial image. Even though our source of concern, the radio galaxy, lies in the centre of our field. The observations were carried out in B configuration resulting in an effective maximum resolution of approximately 5x5 arcsecs. To effectively model the source spectral index in the sky, we will utilize the MTMFS algorithm and use the W-Projection algorithm to make a wide field image (The full beam at P Band is about 3 degrees in diameter). So, going by these requirements, we can now compute the required cell and image size. Consulting the resolution guide of NRAO science page, we can see that the expected HPBW for B - Configuration, P -Band Observation is 18.5 arcsec. So, we can sample it well by using a 5 arcsec cellsize. Having done that, if we decide to make a wide field image to account for all the point sources we will set the image size parameter to 4096 pixels. To enable the wide-field algorithm, we set the <i>gridder='wproject'</i>: this invokes the W-Projection algorithm, upon which we set the number of W-Projection planes to be 128. We set the deconvolution mode as Multi Term Multi Scale Multi Frequency Synthesis using the <i>deconvolver='mtmfs'</i> parameter along with the number of Taylor terms to be considered during imaging to be 2 and choosing three different scales to model the point source and extended emission in our field well, we do this by setting <i> scales=[0,20,30]</i>. This allows for the source spectral variation to be modeled by a second order polynomial. We launch the interactive tclean process, but only for <i>spw='3~8'</i> to begin with. Note, running the first clean call will take at least ~10 min to process.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_initial_clean', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=True, scales=[0,20,30]<br />
minpb=0.01, savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
</source><br />
<br />
{|<br />
| [[File:3C129_radio_gal_residual.png|200px|thumb|right|Figure 10a: Resulting residuals of 3C129 after first call of interactive <i>clean</i>.]]<br />
| [[File:3C129_radio_gal_clean.png|200px|thumb|right|Figure 10b: Resulting image of 3C129 after completing the first <i>clean</i>.]]<br />
|}<br />
The <i>tCLEAN</i> command launches an interactive session after a 100 iterations of clean and produces a wide field map with two sources at the center and a lot of bright sources far out in the field. As it is a snapshot image, the bright sources have significant side lobes and so tight clean boxing can help. This can done in the interactive viewer interface that pops up. This topic has been covered extensively in the 3C391 imaging tutorial, which can be found here (https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391#Initial_Imaging). If we proceed with interactive clean with subsequent steps to keep boxing out the strong sources that pop up in the image, we finally see the extended emission from the target radio galaxy begin to emerge. Continue boxing and cleaning to ensure that the residuals of the boxed cleaning look noise like (~10000 clean iterations). We now stop the interactive task and look at the final image it produced.<br />
<br />
<source lang='python'><br />
# In CASA<br />
viewer('3C129_initial_clean.image.tt0')<br />
</source><br />
<br />
The recovered structure at the center of the map should resemble what is shown in Fig. 10b. If the image looks worse then boxing of bright features in the map needs to be done more carefully. Note the image has some sources still showing strong side lobes and imaging artifacts around them. We expected this as we have carried out phase calibration only on our flux calibrator and have just transferred the solutions over to our target field. Since we used the <i>usescratch=True</i>, the MODEL_DATA column in the measurement set now contains the initial image model, which we will self calibrate against to produce a better image. Also in clean, do notice that the spw's utilized are the cleanest spectral windows that are totally RFI free.<br />
<br />
==Self Calibration ==<br />
<br />
We now proceed to compute gain phase solutions for our target field using the <i>gaincal</i> task as the first step in self-calibration.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScG0', field='3C129', solint='inf', refant='ea09', <br />
spw='3~8',minsnr=3.0, gaintype='G', parang=True, calmode='p')<br />
<br />
bandpass( vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScB0', field='3C129', solint='inf', refant='ea09', minsnr=3.0, spw='3~8',<br />
parang = True, gaintable=['3C129_pband_target.ScG0'],<br />
interp=['','','','nearest,nearestflag'])<br />
applycal(vis='3C129_pband_target.ms', gaintable=['3C129_pband_target.ScG0','3C129_pband_target.ScB0'], spw='3~8', applymode='calflagstrict')<br />
</source><br />
<br />
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the <i>tCLEAN</i> command once again.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_clean_sc0', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=False, scales=[0,20,30]<br />
minpb=0.01,savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
<br />
viewer('3C129_clean_sc0.image.tt0')<br />
</source><br />
[[File:3C129_radio_gal_cselfcal0.png|200px|thumb|right|Figure 11: Resulting image of 3C129 after completing a first selfcal.]]<br />
<br />
On boxing and cleaning, we already notice that the imaging artifacts have reduced significantly. We also see that the target source appears to contain more structure and a greater amount of flux. Further self calibration iterations involving an amplitude & phase gain calibration, and target bandpass calibration, are all possible steps that can be explored. <br />
<br />
== Appendix: Some P-band data issues you may want to know about ==<br />
<br />
Unfortunately, there are a few known issues with JVLA P-band observing. Most problems were discovered during the P-band commissioning period and have been fixed for newer data sets, but the archive keeps part of this history alive. Here we will go over some of those issues, and (if possible) provide ways of fixing them.<br />
<br />
=== Polarization labeling ===<br />
<br />
For a long period, P-band feeds have been labelled as being circular (R and L), while they are linear (X and Y). It is still possible to generate data with incorrect polarization labeling, however for standard observations this should not be the case anymore. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem. This may seem mostly harmless, but does make a difference for polarization calibration. The following (non-standard) task will check and fix this (and another related problem of swapped polarizations):<br />
<br />
Download [http://casa.nrao.edu/Data/EVLA/Pband/casa_vla_lowband.tar.gz this tarball] in your CASA working directory, open a shell there, and type the following commands:<br />
<br />
<source lang='bash'><br />
tar -xzvf casa_vla_lowband.tar.gz<br />
<br />
buildmytasks<br />
</source><br />
<br />
Now go back to your CASA session and type:<br />
<br />
<source lang='python'><br />
# In CASA<br />
execfile('mytasks.py')<br />
#<br />
fixlowband(vis='TSUB0001_pband_prehs.ms')<br />
</source><br />
<br />
Re-run the <i>listobs()</i> task to check if the fix was correctly applied.<br />
<br />
=== Swapped polarizations ===<br />
<br />
Throughout the whole history of the new VLA low-band system, even up to this day, there have been mistakes in the cabling of the full signal chain. This results in that some antennas have the X-polarization signal come in as Y and vice versa. The way to notice this in visibility data is that for baselines with one swapped antenna most power will be in the cross-hand correlations (XY and YX) rather than the parallel-hand correlations (XX and YY). Once antennas with this feature are manually identified, a contributed task <i>swappol()</i> is available to fix this problem (see under polarization labelling).<br />
<br />
=== Double data descriptor entries ===<br />
<br />
The data description table is part of the measurement set, and provides a link between the recorded visibilities, the spectral window information and the polarization information. For some data sets, the data description table contains two entries for each P-band spectral window, one pointing towards a circular (RL) polarization definition, and one pointing towards a linear (XY) polarization definition. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem.<br />
<br />
=== Continuous Radio Frequency Interference (RFI) ===<br />
<br />
The wide bandwidth of the low-band receiver is (unfortunately) guaranteed to contain significant amounts of RFI. There are a few RFI sources that are active all the time, and are visible in all array configurations. The default P-band observing setup of 16 x 16 MHz tries to capture as much of the continuous RFI in as few spectral windows as possible, allowing for a simple RFI mitigation strategy in which these spectral windows (spws 1,2 and 9,10, but possibly more) can be immediately flagged.<br />
<br />
=== Two spectral window setup ===<br />
<br />
In the early commissioning period, the default setup for P-band observing was to use 2 spectral windows of 1024 channels each to cover 256 MHz of bandwidth from 230 - 486 MHz. However, it was noticed that strong, narrow-band RFI events were causing data to be lost for the whole spectral window in which they occured. To make the system more robust against data loss from these events, the frequency range was divided into 16 x 16 MHz, and shifted downwards by 6 MHz to capture ever-present RFI into as few spectral windows as possible. If your data has only two P-band spectral windows, please be aware that a higher data loss due to RFI is possible. There is no way to repair this.<br />
<br />
=== Bandpass ripples ===<br />
<br />
Due to signal reflections in cables and within the VLA dish, sinusoidal amplitude and phase modulations are always present in P-band data. This is most easily seen in bandpass calibration plots of amplitude versus frequency. In some cases, mostly due to cable & connector problems, these modulations can be very strong (~up to 50 percent of the average amplitude level). These modulations tend not to vary over the duration of an observation and can therefore be removed through bandpass and polarization calibration. If they are found to be variable (e.g., by inspecting bandpass solutions for separate calibrator scans), the offending antenna / polarization should be flagged.<br />
<br />
=== High cross-polarization ===<br />
<br />
On each antenna, the P-band feed (dipole) is visually aligned with respect to the primary focus support legs. This is normally done within 5-10 degrees accuracy. On rare occasions the dipole on one antenna has accidentally rotated to much larger angles because the locking bolt on the back of the feed was not completely tightened. The result is a high cross-polarization in the baseline visibilities that include this antenna. If this is the case, it is safest to flag this data, since there is a possibility that the dipole has rotated during the observations.<br />
<br />
=== Instrumental Polarization Calibration ===<br />
<br />
At present polarization P-band observations are in RSRO mode only. While polarization leakage calibration has been highlighted in the past reports it does not directly impact imaging in Stokes I and hence has been dropped from this casaguide.<br />
<br />
== Useful links ==<br />
<br />
* [http://www.aoc.nrao.edu/lbr The EVLA Low-Band Receiver System Status & User Pages]<br />
* [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/topical-guides/vlofreq VLA Very Low Frequency Observing]<br />
<br />
<br />
{{Checked 5.4.1}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.4.0&diff=26241VLA Radio galaxy 3C 129: P-band continuum tutorial-CASA5.4.02018-12-01T22:26:39Z<p>Emomjian: </p>
<hr />
<div><!--By [http://www.aoc.nrao.edu/~pjaganna/Site/Home.html Preshanth Jagannathan] & [https://safe.nrao.edu/wiki/bin/view/Main/HuibIntemaWebHome Huib Intema] & Frank Schinzel--><br />
<br />
<br />
<b>This CASA Guide is for Version 5.4.0 of CASA.</b><p></p> <br />
<b> Note: This guide may take up to 2-days to complete </b><p></p> <br />
If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
<br />
This webpage provides a basic description on how to use CASA to reduce data from the upper part of the new VLA low-band system, also known as P-band, covering roughly 220-480 MHz. The goal is to make a wide-field continuum stokes I image of a typical blank field using the full effective bandwidth. The dataset that will be downloaded in this guide has a size of 26.46 GB and will require a total of ~140 GB of disk space if all generated files are kept. <br />
<br />
''Note: Starting with CASA 5.0 IPython 5.1.0 is used which for interactive CASA sessions. This does not allow to readily paste and execute multiple lines of code like in previous versions of the interpreter. If this is needed at any point in this tutorial, we use "%cpaste" to allow the copy/paste of loops, conditions, etc.. Press "Enter" (or "Return") after "--" at the end, this stops (or exits) the copy/paste prompt and will return you to the CASA prompt.''<br />
<br />
== Obtaining the raw data ==<br />
<br />
For this guide, we'll use some test data that was taken in B-configuration. To get a copy of the raw data, download [https://casa.nrao.edu/Data/EVLA/Pband/P_band_3C129.tgz P_band_3C129.tgz] onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
<br />
Alternatively, the raw data can also be obtained from the NRAO archive, go to the [https://archive.nrao.edu/archive/advquery.jsp NRAO advance archive query page], and enter the following search parameter in the field <i>Archive File ID</i> (leave the rest on default):<br />
<br />
<div style="text-align:justify;"><br />
<pre style="background-color: #gray;"><br />
imag-test-copy.57080.956837025464<br />
</pre><br />
</div><br />
<br />
The search should return a single row for Archive File: imag-test-copy.57080.956837025464, taken on Feb. 27, 2015 (or "15-Feb-27 22:57:52"). Check the box next to it. Next, fill in your email and select the "SDM-BDF dataset (all files)" option. Note, you may find it easier to download a tar file, in which case, you'll want to check the box for Create MS or SDM tar file. Click "Get My Data" and follow the on-screen prompts to reach the next page, where you should click "Retrieve over internet". The next page should report that the data staging is in progress. Wait until you receive an e-mail reporting that your archive data is copied, which should take a few minutes. Download the data onto your computer running CASA and untar it (i.e. <i>tar -xvfz <filename></i>).<br />
<br />
== Starting CASA ==<br />
<br />
Start CASA by typing <br />
<source lang='bash'><br />
casa<br />
</source><br />
on the (linux) command line. This should start a CASA interactive python (iPython) session, and open a separate log window. The CASA version is reported at startup, both in the python session and the log window.<br />
<br />
== Importing the raw data into CASA ==<br />
<br />
We will begin by importing our data into the measurement set format (CASA standard) from the binary format (SDM-BDF) as downloaded from the archive. We do this by means of the importasdm task. It is important to note that the importevla task has been deprecated.<br />
<br />
<source lang='python'><br />
# In CASA<br />
importasdm(asdm='imag-test-copy.57080.956837025464', vis ='3C129.ms', savecmds=True, outfile='importflags.txt')<br />
<br />
</source><br />
<br />
This task calls the external tool <i>asdm2MS</i> to perform the conversion. Together with the visibility data, also the flag commands from the VLA online system are imported. These flags are not directly applied, but stored in the <i>importflags.txt</i> file and applied later.<br />
<br />
== Preliminary data inspection ==<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms', verbose = True, listfile = '3C129.listobs')<br />
</source><br />
<br />
Alternatively, you can also output to the logger by running the command without the listfile argument. The sample listobs output shown has flag <i>verbose=False</i> for the sake of brevity.<br />
<br />
<source lang='python'><br />
# In CASA<br />
listobs(vis='3C129.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/fschinze/P_band_tutorial/3C129.ms MS Version 2<br />
================================================================================<br />
Observer: Project: uid://evla/sdm/X2<br />
Observation: EVLA(27 antennas)<br />
Data records: 4714983 Total elapsed time = 1422 seconds<br />
Observed from 27-Feb-2015/22:57:54.0 to 27-Feb-2015/23:21:36.0 (UTC)<br />
<br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 NONE 3C147 05:42:36.127300 +49.51.07.17800 J2000 0 1307124<br />
1 NONE 3C129 04:48:58.200000 +45.02.01.00000 J2000 1 3407859<br />
Spectral Windows: (19 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 EVLA_P#A0C0#0 128 TOPO 224.000 125.000 16000.0 231.9375 12 XX XY YX YY<br />
1 EVLA_P#A0C0#1 128 TOPO 240.000 125.000 16000.0 247.9375 12 XX XY YX YY<br />
2 EVLA_P#A0C0#2 128 TOPO 256.000 125.000 16000.0 263.9375 12 XX XY YX YY<br />
3 EVLA_P#A0C0#3 128 TOPO 272.000 125.000 16000.0 279.9375 12 XX XY YX YY<br />
4 EVLA_P#A0C0#4 128 TOPO 288.000 125.000 16000.0 295.9375 12 XX XY YX YY<br />
5 EVLA_P#A0C0#5 128 TOPO 304.000 125.000 16000.0 311.9375 12 XX XY YX YY<br />
6 EVLA_P#A0C0#6 128 TOPO 320.000 125.000 16000.0 327.9375 12 XX XY YX YY<br />
7 EVLA_P#A0C0#7 128 TOPO 336.000 125.000 16000.0 343.9375 12 XX XY YX YY<br />
8 EVLA_P#A0C0#8 128 TOPO 352.000 125.000 16000.0 359.9375 12 XX XY YX YY<br />
9 EVLA_P#A0C0#9 128 TOPO 368.000 125.000 16000.0 375.9375 12 XX XY YX YY<br />
10 EVLA_P#A0C0#10 128 TOPO 384.000 125.000 16000.0 391.9375 12 XX XY YX YY<br />
11 EVLA_P#A0C0#11 128 TOPO 400.000 125.000 16000.0 407.9375 12 XX XY YX YY<br />
12 EVLA_P#A0C0#12 128 TOPO 416.000 125.000 16000.0 423.9375 12 XX XY YX YY<br />
13 EVLA_P#A0C0#13 128 TOPO 432.000 125.000 16000.0 439.9375 12 XX XY YX YY<br />
14 EVLA_P#A0C0#14 128 TOPO 448.000 125.000 16000.0 455.9375 12 XX XY YX YY<br />
15 EVLA_P#A0C0#15 128 TOPO 464.000 125.000 16000.0 471.9375 12 XX XY YX YY<br />
16 EVLA_P#B0D0#16 64 TOPO 310.000 500.000 32000.0 325.7500 15 XX XY YX YY<br />
17 EVLA_P#B0D0#17 1024 TOPO 310.000 62.500 64000.0 341.9688 15 XX XY YX YY<br />
18 EVLA_P#B0D0#18 64 TOPO 342.000 500.000 32000.0 357.7500 15 XX XY YX YY<br />
Antennas: 27 'name'='station'<br />
ID= 0-3: 'ea01'='N12', 'ea02'='N16', 'ea03'='W28', 'ea05'='E20',<br />
ID= 4-7: 'ea06'='N20', 'ea07'='N32', 'ea08'='E28', 'ea09'='W08',<br />
ID= 8-11: 'ea10'='E36', 'ea11'='W12', 'ea12'='N36', 'ea13'='W04',<br />
ID= 12-15: 'ea14'='E08', 'ea15'='E24', 'ea16'='W24', 'ea17'='E04',<br />
ID= 16-19: 'ea18'='W36', 'ea19'='MAS', 'ea20'='N04', 'ea21'='E32',<br />
ID= 20-23: 'ea22'='N24', 'ea23'='E16', 'ea24'='W32', 'ea25'='W20',<br />
ID= 24-26: 'ea26'='E12', 'ea27'='N08', 'ea28'='N28'<br />
<br />
</pre><br />
<br />
You can inspect the output text file <i>3C129.listobs</i> in your favorite text editor (e.g., <i>gedit</i>). When taking some time to familiarize yourself with the output format, you will see that:<br />
* The observation consists of 5 scans, namely:<br />
# 3C147 (hardware setup)<br />
# 3C147 (setting requantizers)<br />
# 3C147 (primary calibrator)<br />
# 3C129 (target)<br />
# 3C129 (target)<br />
* The frequency coverage is 222 - 478 MHz, divided into 16 x 16 MHz spectral windows, each having 128 x 0.125 MHz channels. There are also two spectral windows with 64x0.5 MHz and one spectral window with 1024x0.0625 MHz. <br />
* Visibilities are recorded every 2 seconds in full polarization (4 polarization products labelled XX,XY,YX,YY).<br />
* Of the 28 VLA antennas labelled ea01 - ea028, antenna ea04 is not participating in this observation.<br />
* Source, antenna and spectral window IDs start at zero, but scan IDs start at one.<br />
<br />
<br />
[[File:3C129_pband_plotants.png|200px|thumb|right|Figure 1: plotants figure]] <br />
<br />
The array configuration can be inspected using:<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotants(vis='3C129.ms', figfile='3C129_pband_plotants.png')<br />
</source><br />
<br />
== Initial processing steps - Antenna Position, Requantizer Gain, Ionospheric Correction ==<br />
<br />
When importing the data, we saved the flags from the online system to an ASCII text file. This gives us the opportunity to review the flag commands before applying them. For our data set, the flag file is called <i>importflags.txt</i>. Please load this file into your favorite text editor. The bulk of the flag commands refer to times when the VLA is slewing (<i>ANTENNA_NOT_ON_SOURCE</i>) or when the movable secondary reflector of the VLA's Cassegrain system is not in place (<i>SUBREFLECTOR_ERROR</i>). The latter can cause antenna gain variations (amplitude and phase), so it is safest to apply all the flags. The final two rows in the flag file will remove visibilities that are pure zero, and flag antennas that are partly blocked by other antennas (shadowing) which occurs mostly in compact configurations when observing along a VLA arm).<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms', mode='list', inpfile='importflags.txt', action='apply', reason='any', flagbackup=True)<br />
</source><br />
This call, before applying the new flags to the data, created a new flagbackup file called flagdata_1 which can be used by the task <i>flagmanager</i> to restore the state of the measurement set prior applying the flags from <i>importflags.txt</i>.<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
Now that we have applied the import flags, we will correct for the antenna position offsets. Antenna positional errors translates to an error in the measured visibilities and need to be accounted for before we proceed with any of the other calibration steps. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltable='3C129_pband.antpos',caltype='antpos')<br />
</source><br />
<br />
The logger output shows that there are 13 antennas with positional offsets: <br />
<pre><br />
<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea03 : 0.00290 0.00000 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea08 : 0.00000 0.00390 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea10 : -0.00170 0.00000 0.00400<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea11 : 0.00000 0.00160 -0.00170<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea13 : 0.00000 0.00050 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea16 : -0.00180 0.00680 0.00230<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea18 : 0.00470 0.00350 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea21 : -0.00160 0.00000 0.00210<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea23 : 0.00140 0.00200 -0.00150<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea24 : 0.00380 0.00000 -0.00340<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea25 : 0.00330 0.00000 -0.00110<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea26 : 0.00000 0.00190 0.00000<br />
2016-11-28 22:51:37 INFO gencal offsets for antenna ea27 : -0.00120 0.00070 0.00000<br />
<br />
<br />
</pre><br />
<br />
=== Ionospheric TEC Corrections ===<br />
[[File:Pband_vla_tecu_new.png|200px|thumb|right|Figure 2: TEC value vs time generated by <i>tec_maps.create()</i>]] <br />
<br />
Low frequency observations are affected by the ionosphere. A delay in the signal path is introduced between the two polarization of light that varies both as a function of time and line of sight (direction dependent). The delay is proportional to the Total Electron Content (TEC) along the line of sight and is inversely proportional to the square of the frequency. GPS measurements at two different frequencies provides us with an estimate of the TEC per square meter. This correction has been implemented in CASA which we shall apply as a calibration table by means of the <i>gencal</i> task. The task requires a TEC map that we will generate utilizing CASA recipes. For more information, see the section on [https://casa.nrao.edu/casadocs/casa-5.0.0/synthesis-calibration/preparing-for-calibration/#title5 ionospheric corrections] in the CASA documentation archives. For some versions of CASA, the first plot to the right (Figure 2) may not be generated if you have already run plotants: to generate this plot, you will need to quit CASA and start it again. If the IONEX data file was not downloaded you might be missing the curl package in the environment you are running CASA in. In this case you can provide the required file manually by downloading: ftp://cddis.gsfc.nasa.gov/gnss/products/ionex/2015/058/igsg0580.15i.Z, (for the tutorial it is also also available in an unzipped version [http://casa.nrao.edu/Data/EVLA/Pband/igsg0580.15i here]) unziping it, and placing it in the directory from which you started CASA. The current implementation in CASA is in the development stage, however since the CASA 5.0 release both the Faraday rotation correction and dispersive delay corrections for the ionosphere are enabled. In previous releases only Faraday rotation corrections are enabled. In this example we will generate the calibration table and apply it throughout calibration. <br />
<br />
<source lang='python'><br />
# In CASA<br />
from recipes import tec_maps<br />
#<br />
tec_image, tec_rms_image, plotname = tec_maps.create(vis='3C129.ms',doplot=True)<br />
#[NOTE: If you are using CASA 5.0.0, or earlier this should be: tec_image, tec_rms_image = tec_maps.create(vis='3C129.ms', doplot=True)]<br />
#<br />
gencal(vis='3C129.ms',caltable='3C129_pband.tecim',caltype='tecim',infile=tec_image)<br />
# Spatial plots like shown in Figure 3 can be looked at using the CASA viewer for the TEC values<br />
viewer('3C129.ms.IGS_TEC.im')<br />
# and for TEC rms<br />
viewer('3C129.ms.IGS_RMS_TEC.im')<br />
</source><br />
<br />
[[File:Pband_tec_new.png|200px|thumb|right|Figure 3: TEC map movie generated by the TEC tasks.]] <br />
<br />
A word of caution regarding the TEC map generation. The IGS website updates measurements only two weeks after the date of observation.<br />
<br />
=== Requantizer Gains ===<br />
<br />
The next step will correct the visibility amplitudes for the signal leveling (requantizer gains) that occurs at the inputs of the WIDAR correlator. These levels (per antenna, per polarization, per spectral window) are stored with the measurement set (in the <i>SYSPOWER</i> sub-table). This step is currently not essential, since the levels get set only once at the start of an observation and bandpass calibration will correct for this. But it will make your bandpass plots look better if you have multiple spectral windows. And, more importantly, in general it is possible to trigger the setting of requantizers throughout an observation. This is the case for new observations, when a P-band scan is preceded by a P-band setup scan, which itself is preceded by a scan with a different telescope configuration. <br />
<br />
The correction is done by means of the <i>gencal</i> task, where information in the <i>SYSPOWER</i> sub-table gets translated into a gain table. <br />
<br />
<source lang='python'><br />
# In CASA<br />
gencal(vis='3C129.ms',caltype='rq',caltable='3C129_pband.rq')<br />
</source><br />
<br />
== Dead antennas ==<br />
<br />
Now it is time to have a first visual look at the uncalibrated visibility data. It is important to identify dead antennas / polarizations early on, so we can exclude them from further data processing (which is the most efficient data reduction). A convenient way of doing this is through the <i>plotms</i> task on bright calibrator(s) (see the <i>listobs()</i> output), in our case 3C147 (see Fig. 4a).[[File:3C129_pband_3C147_prebp.png|200px|thumb|right|Figure 4a: <i>plotms</i> output before RFI flagging]]<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY', field='3C147', <br />
plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',iteraxis='baseline', <br />
plotfile='3C129_pband_3C147_prebp.png')<br />
</source><br />
<br />
This will load the <i>plotms</i> window that has queued up all the baselines of antenna ea01 and is currently displaying the first amplitude vs frequency plot with the colors representing the different spectral windows. Note that we are only plotting the XX and YY correlation of the antennas. This is because we expect most of the power in the linear cross correlation products to be in these two correlations. Notice that some spectral windows in particular are badly affected due to radio frequency interference (RFI). This is the issue that we will deal with after identifying dead antennas. If you scroll through the baselines using the green forward button in the <i>plotms</i> viewer, you will notice that the amplitude is particularly low when you encounter ea19. If we make a quick plot of the baselines made by ea19 (put <i>ea19</i> in the antenna field) we can see that there is no power in any of the polarizations indicating this antenna was dead and can be flagged out. It is also good to check for high power levels in cross-hand correlations in comparison to the parallel-hand correlations, which would uncover swapped polarization labels and severely mis-aligned dipoles. At the end of this tutorial some hints on solving for swapped or mislabeled dipole polarizations are given. In addition to the dead antenna, we also make sure that the first two scans are flagged, which were not labelled as setup scans but were used to set the attenuators and requantizers.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129.ms',mode='manual',antenna='ea19')<br />
#<br />
flagdata(vis='3C129.ms',mode='manual',scan='1~2')<br />
</source><br />
<br />
We also noticed sharp RFI peaks during our data examination. To prevent Gibbs ringing, it is best to hanning smooth the data at this juncture before we proceed further with automatic flagging. Now is a good time to get a cup of coffee while hanning smoothing the data. [[File:3C129_pband_3C147_prebp_hanning.png|200px|thumb|right|Figure 4b: <i>plotms</i> output after Hanning smooth and first flagging.]]<br />
<source lang='python'><br />
# In CASA<br />
hanningsmooth(vis='3C129.ms',outputvis='3C129_pband.ms',datacolumn='data')<br />
</source><br />
<br />
We should replot the primary calibrator to see the effect of hanning smoothing on the data and the RFI (Fig. 4b). <br />
<br />
<source lang='python'><br />
# In CASA<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning.png')<br />
<br />
</source><br />
<br />
== Automatic flagging ==<br />
<br />
<br />
It is worthwhile noting at this juncture that the standard P-band setup will ensure that the worst of the RFI is restricted to four of the spectral windows. This particular dataset is an example of higher than average levels of RFI across the band. Before throwing away bad spectral windows by hand, we'll let the automated flaggers in CASA have a go at it. The main task for this is <i>flagdata()</i>. While working on your data, <i>flagdata()</i> produces an abundance of output, not all of which is easy to understand. To help with this, we can get the flag status of our data before and after auto-flagging by running <i>flagdata()</i> in the <i>summary</i> mode. This first call provides the flag statistics of the flagging up to now, before any auto-flagging:<br />
<source lang='python'><br />
summary_1 = flagdata(vis='3C129_pband.ms', mode='summary')<br />
</source><br />
<br />
This returns a python dictionary with flagged versus total visibilities along various axes (antenna, scan, spw, field, correlation, etc.). For example, if we want to know the percentage flagged per scan, run the following (note that the scans may not appear in sorted order):<br />
<source lang='python'><br />
# In CASA<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for id, stats in summary_1[ axis ].iteritems():<br />
print '%s %s: %5.1f percent flagged' % ( axis, id, 100. * stats[ 'flagged' ] / stats[ 'total' ] )<br />
--<br />
</source><br />
<br />
For this example, you will notice that scans 1 and 2 (the dummy scans on 3C147) are 100 percent flagged, which is what we did during the initial flagging.<br />
<br />
We will run <i>flagdata()</i> in the <i>tfcrop</i> mode, which will (per scan, per baseline, per spectral window, per polarization) look for visibility amplitude outliers. It uses a 5-piece polynomial in an attempt to remove any intrinsic bandpass (amplitude) structure (we did not calibrate for bandpass yet). We run the task twice to allow for slightly deeper flagging and separate treatment of the parallel and cross-hand correlations. For the first run, we tell CASA to make a backup of our visibility flag status up to here, giving us an option to restore them if we choose over-aggresive flagging parameters and consequently over-flag our data. The flag backup file name is reported in the log window, and can be found in the <i>3C129_pband.ms.flagversions</i> directory.<br />
<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=4., freqcutoff=3., maxnpieces=5,<br />
action='apply', display='report', flagbackup=True, combinescans=True, ntime='3600s', correlation='ABS_XY,ABS_YX')<br />
#<br />
flagdata(vis='3C129_pband.ms', field='*', mode='tfcrop', datacolumn='data', timecutoff=3., freqcutoff=3., maxnpieces=2,<br />
action='apply', display='report', flagbackup=False, combinescans=True, ntime='3600s', correlation='ABS_XX,ABS_YY')<br />
#<br />
flagdata(vis='3C129_pband.ms', mode='extend')<br />
</source><br />
<br />
As a note of caution: When executing flagdata with <i>mode='extend'</i>, all flags that were ever applied to the dataset will get extended. Repeat execution of this command will incrementally flag more and more around already flagged data.<br />
<br />
Let's get another flag summary to see how much extra data was flagged: <br />
<source lang='python'><br />
# In CASA<br />
summary_2 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
# <br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_2[ axis ].iteritems():<br />
old_stats = summary_1[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C129_pband_3C147_prebp_hanning_automatic.png')<br />
<br />
</source><br />
[[File:3C129_pband_3C147_prbp_tfcropped.png|200px|thumb|right|Figure 4c: <i>plotms</i> output after RFI flagging with tfcrop]]<br />
There is quite a bit of RFI that gets missed in RFI-rich spectral windows as can be seen in Fig. 4c. The solution to this is to provide the flagging routines with more contrast between healthy and affected data. For this, we will perform a preliminary bandpass calibration to take out the bandpass shape. We will do some coarse preliminary calibration and apply it to the calibrator before flagging for RFI once more.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.G0', gaintype='G', calmode='p', solint='int', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim'])<br />
#<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K0', gaintype='K', solint='inf', field='3C147',refant='ea09',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0'])<br />
#<br />
bandpass(vis='3C129_pband.ms', caltable='3C129_pband.B0', solint='inf', field='3C147',refant='ea09', minsnr=2.0,<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0'])<br />
#<br />
applycal(vis='3C129_pband.ms', field='3C147', applymode='calflagstrict',<br />
gaintable=['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim','3C129_pband.G0','3C129_pband.K0','3C129_pband.B0'] )<br />
<br />
</source><br />
<br />
On the command line you should see a notification stating: "FJones (iononsphere) apply now provisionally includes disp. delay." which indicates that both Faraday rotation as well as dispersive delay corrections for the ionosphere are applied.<br />
<br />
[[File:3c147_postbandpass_rflag.png|200px|thumb|right|Figure 4d: <i>plotms</i> output after RFI flagging with <i>mode='rflag'</i>]]<br />
<br />
We will now flag the corrected data column that contains the coarsely calibrated visibilities which provide better contrast to the flagging algorithms to remove the RFI present. We will begin by running the <i>RFLAG</i> algorithm in the task <i>flagdata</i>. We will again use summary mode to see how much more of the observed data was flagged.<br />
<source lang='python'><br />
# In CASA<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
flagdata(vis='3C129_pband.ms', field='3C147', mode='rflag', datacolumn='corrected', timedevscale=4., freqdevscale=3.,<br />
action='apply', flagbackup=True, combinescans=True, ntime='3600s', maxnpieces=5)<br />
#<br />
summary_3 = flagdata(vis='3C129_pband.ms' , mode='summary')<br />
#<br />
%cpaste<br />
<br />
# Press Enter or Return, then copy/paste the following:<br />
axis = 'scan'<br />
for value, stats in summary_3[ axis ].iteritems():<br />
old_stats = summary_2[ axis ][ value ]<br />
print '%s %s: %5.1f percent flagged additionally' % ( axis, value, 100. * ( stats[ 'flagged' ] - old_stats[ 'flagged' ] ) / stats[ 'total' ] )<br />
--<br />
#<br />
plotms(vis='3C129_pband.ms',xaxis='freq',yaxis='amp',antenna='ea01',correlation='XX,YY',<br />
field='3C147', plotrange=[0.2,0.5,0.0,100.0], coloraxis='spw',xlabel='Frequency',ylabel='Amplitude',<br />
iteraxis='baseline', plotfile='3C147_postbandpass_rflag.png', ydatacolumn='corrected')<br />
</source><br />
<br />
Taking another visual look at the data using <i>plotms</i>, we examine the effect of the automated flagging on the calibrator (Fig. 4d). We find that most spectral windows are RFI free and that about 60% of the data was flagged. We typically expect between 40-60% of flagging due to interference in a significant fraction of the band. In this example, a more careful RFI flagging approach is likely to recover more usable spectrum.<br />
<br />
==Flux Density Calibration==<br />
The initial flagging has cleaned up most of the stray RFI across the band. This allows us to proceed with the actual calibration of the data. Before we get to the calibration tables, it's essential to do the flux density calibration of our calibrator 3C147. This is done by the <i>setjy</i> task. Before we run the task we first clear the preliminary calibration that was carried out to enable better flagging. We do that by running the <i>clearcal</i> task.<br />
<br />
<source lang='python'><br />
# In CASA<br />
clearcal(vis='3C129_pband.ms')<br />
#<br />
setjy(vis='3C129_pband.ms', standard='Scaife-Heald 2012', field='3C147')<br />
</source><br />
<br />
== Delay and bandpass calibration ==<br />
<br />
First, we use the full bandwidth on a single scan on the primary calibrator to determine a single delay per antenna, per polarization. This will determine a single (approximate) phase slope across frequency, mainly caused by propagation effects in the (time-variable) ionosphere, cable length differences, and electronics in the signal paths from antenna feeds to correlator. Note that for a single short (5-10 minutes) scan on the calibrator, we can get away with solving for a time-invariant delay per antenna, per polarization. We will use scan 3 on 3C147.<br />
[[File:3C147_init_delay.png|200px|thumb|right|Figure 5: Initial per antenna delays derived by <i>gaincal</i>.]]<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband.ms', caltable='3C129_pband.K1', field='3C147', solint='inf', refant='ea09', gaintype='K',<br />
gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq'],parang=True)<br />
#<br />
plotcal(caltable='3C129_pband.K1', xaxis='antenna', yaxis='delay', markersize=2.0, <br />
plotrange=[0,30,-50.,50.],figfile='3C147_delay.png')<br />
</source><br />
<br />
Since we picked ea09 as our reference antenna, the delays for this antenna ID (=0) are arbitrarily set to zero. All other delays should be within 30 nanosec or so (which they should be for this data set). Larger values should be treated with suspicion, possibly indicating a problem with the antenna or polarization. We will use the bandpass calibration to verify this in the next step.<br />
<br />
To determine the bandpass calibration, we use the same source and scan and apply the delay calibration before solving for the bandpass. Note that we request a minimum SNR of 3, which will make the solve fail for the worst, but not all, channels.<br />
<source lang='python'><br />
# In CASA<br />
bandpass( vis='3C129_pband.ms', caltable='3C129_pband.B1', field='3C147', solint='inf', refant='ea09', minsnr=3.0,<br />
parang = True, gaintable=['3C129_pband.antpos','3C129_pband.tecim','3C129_pband.rq','3C129_pband.K1'],<br />
interp=['','','','nearest,nearestflag'])<br />
<br />
</source><br />
<br />
After solving, we inspect the bandpass calibration amplitudes and phase, and flag any obvious residual outliers. Outliers may be found above or below the average bandpass curves, and tend to arise in the same channel (frequency) ranges for all antennas. Don't spend more than 1 minute per antenna for this tutorial and don't worry about missing some bad points; these bad channels will likely get flagged in a later stage anyway. Note that the bandpass amplitudes (Fig. 6) and phases may differ between polarizations of the same antenna. Also note that the bandpass phases across frequency should not have an overall gradient (this should be removed by the delay calibration). Also note that bandpass phases may wrap around from +/-180 to -/+180 degrees. If that is problematic, re-run the phase <i>plotcal()</i> with <i>plotrange = [ 200.,500.,0.,360. ]</i>. Also note that there are no bandpass solutions for antennas ea19 and ea28, and only a few solutions for ea14.<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='amp', iteration='antenna', markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.B1', xaxis='freq', yaxis='phase', iteration='antenna', plotrange=[ 200.,500.,-180.,180. ], markersize = 2.0)<br />
</source><br />
{|<br />
| [[File:3C147_bp0_ea01.png|200px|thumb|right|Figure 6a: Initial bandpass amplitudes for antenna ea01.]]<br />
| [[File:3C147_bp0_phase_ea01.png|200px|thumb|right|Figure 6b: Initial bandpass phase for antenna ea01.]]<br />
|}<br />
<br />
Flagging is made easier by maximizing the plotcal() window. We don't set the scale for plotting the bandpass amplitudes, which is convenient when flagging outliers as it will auto-rescale. While plotting the bandpass phases we do fix the scale, as it provides a better feeling for the relevant magnitude of phase outliers. Flagging of outliers is done using the <i>Mark Region</i> and <i>Flag</i> buttons. For more information on interactive flagging, see the topical guide [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Flagging VLA Data in CASA]. We are using this mechanism to efficiently flag data manually, since it is done per antenna rather than per baseline. These 'flags' will not be permanent until we call <i>applycal()</i> later on, but are effective while applying the calibration on the fly, as we will do in subsequent calibration steps.<br />
<br />
== Gain calibration ==<br />
<br />
With the delay and bandpass calibration in place, we will now look more closely at the time-variable behaviour of the VLA. We use the same scan on our primary calibrator, 3C147, over the full effective bandwidth to determine gain calibrations: one complex value per antenna, per polarization, per integration time. The delay and bandpass tables are applied on the fly.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal( vis='3C129_pband.ms', caltable='3C129_pband.G1', field='3C147', solint = 'int',<br />
refant = 'ea09', minsnr = 3.0, gaintype = 'G', calmode = 'ap',<br />
gaintable = ['3C129_pband.antpos','3C129_pband.rq','3C129_pband.tecim', '3C129_pband.K1', '3C129_pband.B1' ],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag' ], parang = True )<br />
<br />
</source><br />
{|<br />
| [[File:3C147_Gamp_ea01.png|200px|thumb|right|Figure 7a: Initial gain amplitude solutions for antenna ea01.]]<br />
| [[File:3C147_Gphase_ea01.png|200px|thumb|right|Figure 7b: Initial gain phase solutions for antenna ea01.]]<br />
|}<br />
<br />
Similar to the bandpass flagging, we will flag outlier gain solutions in both amplitude and phase. For the amplitudes, the final scatter around an average close to one should be a few percent (see Fig. 7). Note antennas ea08, ea09 have particularly noisy solutions. Remember that the gain solutions for both polarizations can be flagged together, as we will lose the equivalent data of the surviving polarization anyway.<br />
<br />
<source lang='python'><br />
# In CASA<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='amp', iteration='antenna', markersize=2.0 )<br />
#<br />
plotcal(caltable='3C129_pband.G1', xaxis='time', yaxis='phase', iteration='antenna', <br />
plotrange=[ None,None,-180.,180.], markersize=2.0)<br />
</source><br />
<br />
Note that when plotting gains against time, it is convenient to use auto-scaling of the time axis by putting <i>None</i> in the first two fields of <i>plotrange</i>.<br />
<br />
Next, we will prepare a smoothed and interpolated version of the gain calibration table, which will be applied later to the target field data. This prevents flagging of target field data when one of the edges of a calibrator scan is flagged for one or more antennas. The result can be inspected with <i>plotcal</i>.<br />
<source lang='python'><br />
# In CASA<br />
smoothcal(vis='3C129_pband.ms', tablein='3C129_pband.G1', caltable='3C129_pband.Gs1', smoothtype='median', smoothtime = 60.*60.)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='amp', iteration='antenna', plotrange=[ None,None,0.8,1.2 ], markersize=2.0)<br />
#<br />
plotcal(caltable='3C129_pband.Gs1', xaxis='time', yaxis='phase', iteration='antenna', plotrange=[ None,None,-10.,10. ], markersize=2.0)<br />
<br />
</source><br />
<br />
== Transfer of calibrations to the target field ==<br />
<br />
So far we have used one scan on the primary calibrator 3C147 to derive various calibration tables. These calibrations will now be applied to all scans of all sources in our measurement set, which includes 3C147 itself and our target field 3C129. As described above, the task <i>applycal()</i> creates a <i>CORRECTED_DATA</i> column in which the calibrated visibilities get stored. This may take a while.<br />
<br />
<source lang='python'><br />
# In CASA<br />
applycal(vis='3C129_pband.ms', parang=True, applymode='calflagstrict', flagbackup=True, gaintable=['3C129_pband.antpos','3C129_pband.rq',<br />
'3C129_pband.tecim','3C129_pband.K1','3C129_pband.B1','3C129_pband.Gs1'],<br />
interp = ['','','','nearest,nearestflag', 'nearest,nearestflag', 'nearest,nearestflag'])<br />
</source><br />
<br />
Now we split off the calibrated target field data, meaning that the visibilities of source 3C129 get copied from the <i>CORRECTED_DATA</i> column to the <i>DATA</i> column of a new measurement set. This is convenient for further processing. <br />
<br />
<source lang='python'><br />
# In CASA<br />
split(vis='3C129_pband.ms', outputvis='3C129_pband_target.ms', datacolumn='corrected', field='3C129')<br />
</source><br />
<br />
It is also helpful to perform additional flagging for any residual RFI left behind in our target after all our calibration steps have been completed and we begin the imaging of our data.<br />
<br />
<source land='python'><br />
flagdata(vis='3C129_pband_target.ms', mode='rflag')<br />
flagdata(vis='3C129_pband_target.ms', mode='extend')<br />
</source><br />
<br />
<br />
== Statwt ==<br />
Before we go ahead and image our target field. We would like to weight any residual low level RFI down. This is done using the task <i>statwt</i>. The task runs through the data and computes the rms of the data and weighting down outliers. This makes a big impact for imaging of a target in the presence of low level residual RFI. For our purposes we will bin the data in a time interval larger than the integration time and run statwt as follows.<br />
<br />
<source lang='python'> <br />
statwt(vis='3C129_pband_target.ms', datacolumn='data', timebin=30.)<br />
</source><br />
<br />
With data weights for outliers being set we will now proceed with the imaging and self-calibration of our data.<br />
==Imaging==<br />
<br />
At this point, we're ready to make a first image of our target field. Imaging in CASA is done by means of the <i>tCLEAN</i> task. The task implements algorithms for wide field imaging, such as W-Projection (Cornwell et al. 2008 http://arxiv.org/abs/0807.4161) and Multi Term Multi Frequency Synthesis ( Rau et al. 2011 http://arxiv.org/abs/1106.2745) both of which we will utilize to make our initial image. Even though our source of concern, the radio galaxy, lies in the centre of our field. The observations were carried out in B configuration resulting in an effective maximum resolution of approximately 5x5 arcsecs. To effectively model the source spectral index in the sky, we will utilize the MTMFS algorithm and use the W-Projection algorithm to make a wide field image (The full beam at P Band is about 3 degrees in diameter). So, going by these requirements, we can now compute the required cell and image size. Consulting the resolution guide of NRAO science page, we can see that the expected HPBW for B - Configuration, P -Band Observation is 18.5 arcsec. So, we can sample it well by using a 5 arcsec cellsize. Having done that, if we decide to make a wide field image to account for all the point sources we will set the image size parameter to 4096 pixels. To enable the wide-field algorithm, we set the <i>gridder='wproject'</i>: this invokes the W-Projection algorithm, upon which we set the number of W-Projection planes to be 128. We set the deconvolution mode as Multi Term Multi Scale Multi Frequency Synthesis using the <i>deconvolver='mtmfs'</i> parameter along with the number of Taylor terms to be considered during imaging to be 2 and choosing three different scales to model the point source and extended emission in our field well, we do this by setting <i> scales=[0,20,30]</i>. This allows for the source spectral variation to be modeled by a second order polynomial. We launch the interactive tclean process, but only for <i>spw='3~8'</i> to begin with. Note, running the first clean call will take at least ~10 min to process.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_initial_clean', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=False, scales=[0,20,30]<br />
minpb=0.01, savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
</source><br />
<br />
{|<br />
| [[File:3C129_radio_gal_residual.png|200px|thumb|right|Figure 10a: Resulting residuals of 3C129 after first call of interactive <i>clean</i>.]]<br />
| [[File:3C129_radio_gal_clean.png|200px|thumb|right|Figure 10b: Resulting image of 3C129 after completing the first <i>clean</i>.]]<br />
|}<br />
The <i>tCLEAN</i> command launches an interactive session after a 100 iterations of clean and produces a wide field map with two sources at the center and a lot of bright sources far out in the field. As it is a snapshot image, the bright sources have significant side lobes and so tight clean boxing can help. This can done in the interactive viewer interface that pops up. This topic has been covered extensively in the 3C391 imaging tutorial, which can be found here (https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391#Initial_Imaging). If we proceed with interactive clean with subsequent steps to keep boxing out the strong sources that pop up in the image, we finally see the extended emission from the target radio galaxy begin to emerge. Continue boxing and cleaning to ensure that the residuals of the boxed cleaning look noise like (~10000 clean iterations). We now stop the interactive task and look at the final image it produced.<br />
<br />
<source lang='python'><br />
# In CASA<br />
viewer('3C129_initial_clean.image.tt0')<br />
</source><br />
<br />
The recovered structure at the center of the map should resemble what is shown in Fig. 10b. If the image looks worse then boxing of bright features in the map needs to be done more carefully. Note the image has some sources still showing strong side lobes and imaging artifacts around them. We expected this as we have carried out phase calibration only on our flux calibrator and have just transferred the solutions over to our target field. Since we used the <i>usescratch=True</i>, the MODEL_DATA column in the measurement set now contains the initial image model, which we will self calibrate against to produce a better image. Also in clean, do notice that the spw's utilized are the cleanest spectral windows that are totally RFI free.<br />
<br />
==Self Calibration ==<br />
<br />
We now proceed to compute gain phase solutions for our target field using the <i>gaincal</i> task as the first step in self-calibration.<br />
<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScG0', field='3C129', solint='inf', refant='ea09', <br />
spw='3~8',minsnr=3.0, gaintype='G', parang=True, calmode='p')<br />
<br />
bandpass( vis='3C129_pband_target.ms', caltable='3C129_pband_target.ScB0', field='3C129', solint='inf', refant='ea09', minsnr=3.0, spw='3~8',<br />
parang = True, gaintable=['3C129_pband_target.ScG0'],<br />
interp=['','','','nearest,nearestflag'])<br />
applycal(vis='3C129_pband_target.ms', gaintable=['3C129_pband_target.ScG0','3C129_pband_target.ScB0'], spw='3~8', applymode='calflagstrict')<br />
</source><br />
<br />
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the <i>tCLEAN</i> command once again.<br />
<br />
<source lang='python'><br />
# In CASA<br />
tclean(vis='3C129_pband_target.ms', imagename='3C129_clean_sc0', cell=['5.0arcsec','5.0arcsec'], imsize=[4860,4860], deconvolver='mtmfs',<br />
nterms=2, gridder='wproject', wprojplanes=128, stokes='I', niter=2000, spw='3~8', interactive=False, scales=[0,20,30]<br />
minpb=0.01,savemodel='modelcolumn', weighting='briggs', robust=0.0)<br />
<br />
viewer('3C129_clean_sc0.image.tt0')<br />
</source><br />
[[File:3C129_radio_gal_cselfcal0.png|200px|thumb|right|Figure 11: Resulting image of 3C129 after completing a first selfcal.]]<br />
<br />
On boxing and cleaning, we already notice that the imaging artifacts have reduced significantly. We also see that the target source appears to contain more structure and a greater amount of flux. Further self calibration iterations involving an amplitude & phase gain calibration, and target bandpass calibration, are all possible steps that can be explored. <br />
<br />
== Appendix: Some P-band data issues you may want to know about ==<br />
<br />
Unfortunately, there are a few known issues with JVLA P-band observing. Most problems were discovered during the P-band commissioning period and have been fixed for newer data sets, but the archive keeps part of this history alive. Here we will go over some of those issues, and (if possible) provide ways of fixing them.<br />
<br />
=== Polarization labeling ===<br />
<br />
For a long period, P-band feeds have been labelled as being circular (R and L), while they are linear (X and Y). It is still possible to generate data with incorrect polarization labeling, however for standard observations this should not be the case anymore. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem. This may seem mostly harmless, but does make a difference for polarization calibration. The following (non-standard) task will check and fix this (and another related problem of swapped polarizations):<br />
<br />
Download [http://casa.nrao.edu/Data/EVLA/Pband/casa_vla_lowband.tar.gz this tarball] in your CASA working directory, open a shell there, and type the following commands:<br />
<br />
<source lang='bash'><br />
tar -xzvf casa_vla_lowband.tar.gz<br />
<br />
buildmytasks<br />
</source><br />
<br />
Now go back to your CASA session and type:<br />
<br />
<source lang='python'><br />
# In CASA<br />
execfile('mytasks.py')<br />
#<br />
fixlowband(vis='TSUB0001_pband_prehs.ms')<br />
</source><br />
<br />
Re-run the <i>listobs()</i> task to check if the fix was correctly applied.<br />
<br />
=== Swapped polarizations ===<br />
<br />
Throughout the whole history of the new VLA low-band system, even up to this day, there have been mistakes in the cabling of the full signal chain. This results in that some antennas have the X-polarization signal come in as Y and vice versa. The way to notice this in visibility data is that for baselines with one swapped antenna most power will be in the cross-hand correlations (XY and YX) rather than the parallel-hand correlations (XX and YY). Once antennas with this feature are manually identified, a contributed task <i>swappol()</i> is available to fix this problem (see under polarization labelling).<br />
<br />
=== Double data descriptor entries ===<br />
<br />
The data description table is part of the measurement set, and provides a link between the recorded visibilities, the spectral window information and the polarization information. For some data sets, the data description table contains two entries for each P-band spectral window, one pointing towards a circular (RL) polarization definition, and one pointing towards a linear (XY) polarization definition. A contributed task <i>fixlowband()</i> is available to recognize and fix this problem.<br />
<br />
=== Continuous Radio Frequency Interference (RFI) ===<br />
<br />
The wide bandwidth of the low-band receiver is (unfortunately) guaranteed to contain significant amounts of RFI. There are a few RFI sources that are active all the time, and are visible in all array configurations. The default P-band observing setup of 16 x 16 MHz tries to capture as much of the continuous RFI in as few spectral windows as possible, allowing for a simple RFI mitigation strategy in which these spectral windows (spws 1,2 and 9,10, but possibly more) can be immediately flagged.<br />
<br />
=== Two spectral window setup ===<br />
<br />
In the early commissioning period, the default setup for P-band observing was to use 2 spectral windows of 1024 channels each to cover 256 MHz of bandwidth from 230 - 486 MHz. However, it was noticed that strong, narrow-band RFI events were causing data to be lost for the whole spectral window in which they occured. To make the system more robust against data loss from these events, the frequency range was divided into 16 x 16 MHz, and shifted downwards by 6 MHz to capture ever-present RFI into as few spectral windows as possible. If your data has only two P-band spectral windows, please be aware that a higher data loss due to RFI is possible. There is no way to repair this.<br />
<br />
=== Bandpass ripples ===<br />
<br />
Due to signal reflections in cables and within the VLA dish, sinusoidal amplitude and phase modulations are always present in P-band data. This is most easily seen in bandpass calibration plots of amplitude versus frequency. In some cases, mostly due to cable & connector problems, these modulations can be very strong (~up to 50 percent of the average amplitude level). These modulations tend not to vary over the duration of an observation and can therefore be removed through bandpass and polarization calibration. If they are found to be variable (e.g., by inspecting bandpass solutions for separate calibrator scans), the offending antenna / polarization should be flagged.<br />
<br />
=== High cross-polarization ===<br />
<br />
On each antenna, the P-band feed (dipole) is visually aligned with respect to the primary focus support legs. This is normally done within 5-10 degrees accuracy. On rare occasions the dipole on one antenna has accidentally rotated to much larger angles because the locking bolt on the back of the feed was not completely tightened. The result is a high cross-polarization in the baseline visibilities that include this antenna. If this is the case, it is safest to flag this data, since there is a possibility that the dipole has rotated during the observations.<br />
<br />
=== Instrumental Polarization Calibration ===<br />
<br />
At present polarization P-band observations are in RSRO mode only. While polarization leakage calibration has been highlighted in the past reports it does not directly impact imaging in Stokes I and hence has been dropped from this casaguide.<br />
<br />
== Useful links ==<br />
<br />
* [http://www.aoc.nrao.edu/lbr The EVLA Low-Band Receiver System Status & User Pages]<br />
* [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/topical-guides/vlofreq VLA Very Low Frequency Observing]<br />
<br />
<br />
{{Checked 5.4.1}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=File:3C129_pband_plotants.png&diff=26240File:3C129 pband plotants.png2018-12-01T19:23:31Z<p>Emomjian: Emomjian uploaded a new version of File:3C129 pband plotants.png</p>
<hr />
<div></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Pre-upgrade_VLA_Tutorials&diff=24758Pre-upgrade VLA Tutorials2018-07-27T17:15:31Z<p>Emomjian: </p>
<hr />
<div><!--<br />
* [[Data Editing]]<br />
** [[Data flagging with viewer | Data flagging with '''viewer''']]<br />
** [[Data flagging with plotms | Data flagging with '''plotms''']]<br />
--><br />
* Continuum calibration and imaging<br />
** [[VLA 5 GHz continuum survey of Seyfert galaxies]]<br />
* Spectral Line calibration and imaging<br />
** [[NGC 5921: red-shifted HI emission]]. <br />
** [[MRK 6: red-shifted HI absorption]]. This tutorial illustrates calibration and imaging of a spectral cube spread across two spectral windows (VLA 4IF mode).<br />
<!--<br />
* [[Imaging Flanking Fields]]<br />
--></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22959Karl G. Jansky VLA Tutorials2017-12-02T01:50:35Z<p>Emomjian: </p>
<hr />
<div><br />
'''Disclaimer: The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].'''<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22958Karl G. Jansky VLA Tutorials2017-12-02T01:50:20Z<p>Emomjian: </p>
<hr />
<div><br />
[['''Disclaimer:]] The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].'''<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22957Karl G. Jansky VLA Tutorials2017-12-02T01:48:39Z<p>Emomjian: </p>
<hr />
<div><br />
'''Disclaimer: The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].'''<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22956Karl G. Jansky VLA Tutorials2017-12-02T01:48:21Z<p>Emomjian: </p>
<hr />
<div><br />
'''Disclaimer:''' The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22955Karl G. Jansky VLA Tutorials2017-12-02T01:47:41Z<p>Emomjian: Undo revision 22954 by Emomjian (talk)</p>
<hr />
<div><br />
[['''Disclaimer:''']] The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22954Karl G. Jansky VLA Tutorials2017-12-02T01:45:24Z<p>Emomjian: </p>
<hr />
<div>The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
[['''Disclaimer:''']] The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22953Karl G. Jansky VLA Tutorials2017-12-02T01:44:41Z<p>Emomjian: </p>
<hr />
<div><br />
[['''Disclaimer:''']] The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=22952Karl G. Jansky VLA Tutorials2017-12-02T01:43:19Z<p>Emomjian: </p>
<hr />
<div><br />
== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
'''Disclaimer:''' The tutorials presented here are provided as guidelines to assist users in familiarizing themselves with VLA data reduction in [https://casa.nrao.edu/ CASA]. The tutorials offer examples for common general-use cases such as spectral line, continuum, linear polarization, and mosaicking. However, the results achieved in these tutorials are specific to the data sets herein. The flexibility of the VLA is such that the techniques presented here may not apply equally to every individual VLA data set, especially if the type of observations, quality of data, or imaging analysis are particularly different or complicated. More details about CASA and its tasks and tools are available in the [https://casa.nrao.edu/casadocs Online CASA Documentation]. Further assistance with VLA data reduction is available through the [https://help.nrao.edu/ NRAO HelpDesk].<br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 5.0.0) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
=== P-band spectroscopy tutorial: HI 21cm absorption in MG0414+0534 at z=2.64 ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial MG0414+0534 P-band Spectral Line Tutorial] (CASA 5.0.0)<br />
<br />
* This tutorial is based on an archival data-set of 142 Gb in size, but we will extract only a <font color=red>15Gb</font> portion of this data-set to work on. <br />
<br />
This VLA P-band tutorial is aimed at calibrating and making an image cube of line-emission centered on 390.6 MHz. It includes:<br />
* P-band spectral-line observing strategies<br />
* Basic flagging options<br />
* Low-frequency calibration and self-calibration<br />
* Spectral-line imaging<br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 5.0.0)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== VLA Multi-Configuration Data Combination ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_Data_Combination Topical Guide: VLA Data Combination Guide] (CASA 5.0.0) <br />
* This tutorial requires to download a dataset <font color=green>1.4GB</font> in size<br />
<br />
The VLA Data Combination Tutorial provides guidance on the combination of multi-configuration VLA data sets (in this case VLA A, B and C configurations). The data are re-weighted in two different ways, combined, and imaged. The guide also explains how to remove a time variable source from the data before combination. <br />
</blockquote><br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 5.0.0)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.7.2)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a VLA CASA pipeline run. Whereas the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA Pipeline Web Page] describes the pipeline structure, requirements, execution options including restoration, pipeline output products, and pipeline modifications, this CASA guide is a detailed walkthrough of the weblog and provides interpretations of pipeline plots and tables. <br />
</blockquote><br />
<br />
<br />
<br />
== CASA Guide Versions ==<br />
<br />
In addition to the most recent VLA casa guides, below we also provide links to older versions of the VLA CASA guides. Note that some graphs and links may be outdated, or, in some cases, actually point to the current CASA version as content was refreshed. For older CASA documentation, we refer to the links on [https://casa.nrao.edu/previous_casa.shtml Previous CASA Versions] on the [https://casa.nrao.edu CASA home page].<br />
<br />
=== CASA 5.0 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0 IRC+10216 Tutorial (CASA5.0.0)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA5.0.0 3C391 Continuum Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA5.0.0 3C129 P-band Tutorial (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/MG0414+0534_P-band_Spectral_Line_Tutorial_-_CASA_5.0.0 MG0414+0534 P-band Spectral Line Tutorial (CASA 5.0.0)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_spectral_line_tutorial-CASA5.0.0 P-band spectral line (CASA 5.0.0)]<br />
--><br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA5.0.0 Topical Guide: Flagging VLA Data (CASA 5.0.0)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA5.0.0 Topical Guide: Imaging VLA Data (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Data_Combination-CASA5.0.0 Topical Guide: VLA Multi-Configuration Data Combination (CASA 5.0.0)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA5.0.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 5.0.0)]<br />
<br />
=== CASA 4.7 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.2 VLA CASA Pipeline Guide (CASA 4.7.2)]<br />
<br />
=== CASA 4.6 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
=== CASA 4.5 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
=== CASA 4.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
=== CASA 4.3 versions ===<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
=== CASA 4.2 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 4.1 versions ===<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
=== CASA 3.4 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
=== CASA 3.3 versions ===<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA5.0.0&diff=22936VLA high frequency Spectral Line tutorial - IRC+10216-CASA5.0.02017-12-01T10:11:39Z<p>Emomjian: </p>
<hr />
<div>[[Category:EVLA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
'''Please use CASA 5.0.0 for this tutorial''' <br />
<br />
== Overview ==<br />
[[Image:irc10216_dust.jpg|200px|thumb|Figure 1: VLT V-band image of IRC+10216 showing dust rings out to a radius of 90" by Le&atilde;o et al. (2006, A&A, 455, 187).]]<br />
<br />
This tutorial describes the data reduction for two spectral lines observed toward the Asymptotic Giant Branch (AGB) star [https://en.wikipedia.org/wiki/CW_Leonis IRC+10216]. <br />
In this 36 GHz VLA observation, an 8 MHz wide subband (spectral window, abbreviated as ''spw'' in CASA) using 64 channels was observed in each of two 1 GHz wide basebands, with one of the subbands centered on the HC3N line and the other on the SiS line. The raw data were loaded into CASA with '''{{importevla}}''', where zero visibility values and shadowed data were flagged/removed (note that this already may be done by the [https://archive.nrao.edu/archive/advquery.jsp NRAO Data Archive]). The data were then '''{{split}}''' to only keep the antennas delivering good quality Ka-band data (i.e, antennas without a Ka-band receiver and X-band pointing data was discarded). Furthermore, the '''{{split}}''' operation was used to average the original 1-second visibility integrations to 10 seconds. The latter produces a significantly smaller data set for processing of this tutorial, though should not be done in general on any other data set.<br />
<br />
IRC+10216 (CW Leo, see Figure 1) is a 16th magnitude star in the optical, but the brightest star in the sky at a wavelength of 5 micrometer (60 THz). It was discovered during the first survey of the infrared sky, carried out by Bob Leighton and Gerry Neugebauer in 1965. This Asymptotic Giant Branch star is a Mira-type variable going through prodigious episodic mass loss. The dust condensed from the atmosphere during the mass loss is responsible for the millimeter and infrared emission; the radio continuum emission emerges from the stellar photosphere. Molecules form along with the dust, and a steady state chemistry occurs in the dense inner regions (Tsuji 1973 A&A 23, 411). As the density of the circumstellar material drops during the expansion of the outflow, the chemistry freezes while the molecules continue their long coast outward into the Galaxy. As the shell thins, ultraviolet light from the ambient Galactic radiation field penetrates the dust and initiates new chemistry in the gas.<br />
<br />
SiS, a simple molecule created in the dense inner envelope chemistry, is photodissociated as it coasts out into the shell. It is observed as centrally concentrated emission in this tutorial. Interestingly, in the transition at 18 GHz, the SiS line shape is much different from what is seen here. At the extreme velocities in the profile, very bright narrow emission is seen which has been interpreted as maser emission. More VLA observations of this line can be found in the NRAO Data Archive.<br />
<br />
HC3N, a more complex molecule, is created by photochemistry that is triggered when atoms and pieces of molecules, destroyed by ultraviolet radiation, undergo the next phase of chemistry in the shell. In addition to its rotational modes, HC3N has many vibrational modes which may be excited. Owing to this, it can re-radiate energy absorbed from ultraviolet radiation more effectively than some molecules with a single bond. Eventually, it too is destroyed. However, during its brief existence, its rise to abundance in the envelope results in a ring of emission, which is what is observed in this VLA tutorial. Cordiner & Millar (2009, ApJ, 697, 68) describe a new chemical model for the shell, which also takes into account the variation of mass loss by the star. These authors show that in addition to purely chemical effects, local gas and dust density peaks play a role in shaping the observed emission.<br />
<br />
==Obtaining the Data==<br />
<br />
The post-split averaged data can be downloaded from [http://casa.nrao.edu/Data/EVLA/IRC10216/day2_TDEM0003_10s_norx.tar.gz http://casa.nrao.edu/Data/EVLA/IRC10216/day2_TDEM0003_10s_norx.tar.gz] (download data size is 1.1GB, extracted it will be about 2.0 GB)<br />
<br />
We are providing this starting data set, rather than the original data set for two reasons. First, many of these initial processing steps can be rather time consuming (> 1 hr). Second, while necessary, many of these steps are not fundamental to the calibration and imaging process, which is the focus of this tutorial. For completeness, however, here are the steps that were taken from the original data set to produce the data set above.<br />
<br />
If you want to start the data reduction from the beginning, from the NRAO Archive select the TDEM0003_sb1345754_1.55312.131578217595 data set.<br />
<br />
* The initial Science Data Model (SDM) file was converted into a measurement set (MS).<br />
* Basic data flagging was applied to account for shadowing of the antennas. In the D array configuration antennas may be blocked (shadowed) by other antennas in the array, depending upon the direction and elevation of the source.<br />
* The data were averaged from the original 1-second correlator visibility integration time to 10-second averages. In the D configuration, the fringe rate is relatively slow and time-average smearing is less of a concern.<br />
<br />
All of these steps can be done directly from the NRAO Archive's Download page, by selecting '''CASA MS''' as the download format (it's a good idea to also check the '''Create MS or SDM tar file''' box), checking the '''Apply flags generated during observing''' box, and setting '''Time Averaging''' to 10s. Then, as noted above in the Overview, the Ka-band data for this source was split from the larger TDEM0003 MS to create the smaller data set that this tutorial covers. The last step, splitting out the Ka-band data, must be done by the user within CASA; it cannot be done using the NRAO archive interface.<br />
<br />
Once the download is complete, unzip and unpack the file (within its own working directory, in which then CASA is run):<br />
<br />
<pre style="background-color:lightgrey;"><br />
# in a terminal, outside of CASA:<br />
tar -xzf day2_TDEM0003_10s_norx.tar.gz<br />
</pre><br />
<br />
== How to Use This CASA Guide==<br />
<br />
[[Image:clean.png|200px|thumb|Figure 2: Inputs from one of the clean commands from this tutorial]]<br />
<br />
Please use CASA 5.0.0 for this tutorial (typing ''casa -ls'' in a linux window shows the available versions and the current version; to explicitly change the current version type e.g. ''casa -r 5.0.0-218'')<br />
<br />
There are at least three different ways to interact with CASA, described in more detail in [[Getting Started in CASA]]. <br />
<br />
* Interactively examining task inputs. In this mode, one types '''taskname''' to load the task, '''inp''' to examine the inputs (see Figure 2), and '''go''' once those inputs have been set to your satisfaction. Allowed inputs are shown in blue and bad inputs are colored red. The input parameters themselves are changed one by one, e.g., ''selectdata=True''. Screenshots of the inputs to various tasks used in the data reduction below are provided, to illustrate which parameters need to be set. More detailed help can be obtained on any task by typing '''help ''taskname'''''. Once a task is run, the set of inputs are stored and can be retrieved via '''tget ''taskname'''''; subsequent runs will overwrite the previous '''tget''' file.<br />
<br />
* Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set.<br />
<br />
* Non-interactively via a script. A series of task function calls can be combined together into a script, and run from within CASA via '''execfile('scriptname.py')'''. This and other CASA Tutorial Guides have been designed to be extracted into a script via the script extractor by using the method described within the [[Extracting scripts from these tutorials]] page. Should you use the script generated by the script extractor for this CASA Guide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is in fact unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered).<br />
<br />
If you are a relative novice or just new to CASA, it is strongly recommended to work through this tutorial by cutting and pasting the task function calls provided below after you have read all the associated explanations. Work at your own pace, look at the inputs to the tasks to see what other options exist, and read the help files. Later, when you are more comfortable, you might try to extract the script, modify it for your purposes, and begin to reduce other data.<br />
<br />
==Initial Inspection and Flagging==<br />
<br />
=== Observing Logs ===<br />
<br />
For all VLA observations, the operators keep an observing log. To can access the operator logs, go to the [http://www.vla.nrao.edu/cgi-bin/oplogs.cgi observing log website] and enter a date range covering only 26 Apr 2010, then select TDEM0003. Pertinent information from this observation is repeated below:<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
INFORMATION FROM OBSERVING LOG:<br />
Top left:<br />
Date of the observation: 26-April-2010<br />
Antenna configuration is "D"<br />
Antenna 10 (ea10) is not part of the 27 element array (it is the 28th antenna undergoing routine maintenance)<br />
Operator comments:<br />
There are no Ka-band receivers on ea11, ea13, ea14, ea16, ea17, ea18, ea26 (those still had to be commissioned)<br />
Antenna ea06 is also taken out of the array (for major maintenance)<br />
Antenna 15 has a focus timeout in C band (used for pointing - be careful when these solutions are applied to Ka observations for this antenna)<br />
Antennas ea10, ea12, ea22 do not have good baseline positions at the time of the observations (can be corrected later)<br />
</pre><br />
<br />
All of that information should be kept in mind during the calibration, e.g., if the mentioned list of unavailable receivers do appear in the data, the data is just noise and should be flagged. For the data set that you have downloaded, these antennas have already been removed. Also, keep an eye on other messages such as loss of data, subreflector problems, or any other such issues. If they were not captured in the online flagging, they should be carefully inspected and treated appropriately during calibration or flagged. <br />
<br />
To start with, look at the content of the raw data. Task '''{{listobs}}''' provides almost all relevant observational parameters such as correlator setup (frequencies, bandwidths, channel number and widths, polarization products), sources, scans, scan intents, and antenna locations:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='day2_TDEM0003_10s_norx')<br />
</source><br />
<br />
Below we have cut and pasted the output from the logger that is listed after the observed scan listing. Note that ''Field'' refers to an observed phase center, i.e., a position in the sky. A ''Source'' refers to a Field using a specific frequency setting.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 4<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
2 D J0954+1743 09:54:56.823626 +17.43.31.22243 J2000 2 65326<br />
3 NONE IRC+10216 09:47:57.382000 +13.16.40.65999 J2000 3 208242<br />
5 F J1229+0203 12:29:06.699729 +02.03.08.59820 J2000 5 10836<br />
7 E J1331+3030 13:31:08.287984 +30.30.32.95886 J2000 7 5814<br />
Spectral Windows: (2 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) Corrs <br />
0 Subband:0 64 TOPO 36387.229 125.000 8000.0 36391.1670 RR RL LR LL<br />
1 Subband:0 64 TOPO 36304.542 125.000 8000.0 36308.4795 RR RL LR LL<br />
Sources: 10<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1008+0730 0 0.03639232 -0.026 <br />
0 J1008+0730 1 0.03639232 -0.026 <br />
2 J0954+1743 0 0.03639232 -0.026 <br />
2 J0954+1743 1 0.03639232 -0.026 <br />
3 IRC+10216 0 0.03639232 -0.026 <br />
3 IRC+10216 1 0.03639232 -0.026 <br />
5 J1229+0203 0 0.03639232 -0.026 <br />
5 J1229+0203 1 0.03639232 -0.026 <br />
7 J1331+3030 0 0.03639232 -0.026 <br />
7 J1331+3030 1 0.03639232 -0.026 <br />
Antennas: 19:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 ea01 W09 25.0 m -107.37.25.2 +33.53.51.0 -521.9407 -332.7782 -1.1977 -1601710.017000 -5042006.928200 3554602.355600<br />
1 ea02 E02 25.0 m -107.37.04.4 +33.54.01.1 9.8247 -20.4292 -2.7808 -1601150.059500 -5042000.619800 3554860.729400<br />
2 ea03 E09 25.0 m -107.36.45.1 +33.53.53.6 506.0591 -251.8666 -3.5832 -1600715.948000 -5042273.187000 3554668.184500<br />
3 ea04 W01 25.0 m -107.37.05.9 +33.54.00.5 -27.3562 -41.3030 -2.7418 -1601189.030140 -5042000.493300 3554843.425700<br />
4 ea05 W08 25.0 m -107.37.21.6 +33.53.53.0 -432.1158 -272.1493 -1.5032 -1601614.091000 -5042001.655700 3554652.509300<br />
5 ea07 N06 25.0 m -107.37.06.9 +33.54.10.3 -54.0667 263.8720 -4.2292 -1601162.593200 -5041829.000000 3555095.890500<br />
6 ea08 N01 25.0 m -107.37.06.0 +33.54.01.8 -30.8810 -1.4664 -2.8597 -1601185.634945 -5041978.156586 3554876.424700<br />
7 ea09 E06 25.0 m -107.36.55.6 +33.53.57.7 236.9058 -126.3369 -2.4443 -1600951.588000 -5042125.911000 3554773.012300<br />
8 ea12 E08 25.0 m -107.36.48.9 +33.53.55.1 407.8394 -206.0057 -3.2252 -1600801.916000 -5042219.371000 3554706.449900<br />
9 ea15 W06 25.0 m -107.37.15.6 +33.53.56.4 -275.8288 -166.7451 -2.0590 -1601447.198000 -5041992.502500 3554739.687600<br />
10 ea19 W04 25.0 m -107.37.10.8 +33.53.59.1 -152.8599 -83.8054 -2.4614 -1601315.893000 -5041985.320170 3554808.304600<br />
11 ea20 N05 25.0 m -107.37.06.7 +33.54.08.0 -47.8454 192.6015 -3.8723 -1601168.786100 -5041869.054000 3555036.936000<br />
12 ea21 E01 25.0 m -107.37.05.7 +33.53.59.2 -23.8638 -81.1510 -2.5851 -1601192.467800 -5042022.856800 3554810.438800<br />
13 ea22 N04 25.0 m -107.37.06.5 +33.54.06.1 -42.5986 132.8623 -3.5431 -1601173.953700 -5041902.660400 3554987.536500<br />
14 ea23 E07 25.0 m -107.36.52.4 +33.53.56.5 318.0523 -164.1848 -2.6960 -1600880.570000 -5042170.388000 3554741.457400<br />
15 ea24 W05 25.0 m -107.37.13.0 +33.53.57.8 -210.0944 -122.3885 -2.2581 -1601377.008000 -5041988.665500 3554776.393400<br />
16 ea25 N02 25.0 m -107.37.06.2 +33.54.03.5 -35.6245 53.1806 -3.1345 -1601180.861480 -5041947.453400 3554921.628700<br />
17 ea27 E03 25.0 m -107.37.02.8 +33.54.00.5 50.6647 -39.4832 -2.7249 -1601114.365500 -5042023.153700 3554844.945600<br />
18 ea28 N08 25.0 m -107.37.07.5 +33.54.15.8 -68.9057 433.1889 -5.0602 -1601147.940400 -5041733.837000 3555235.956000<br />
</pre><br />
<br />
Source J1008+0730, with ID 0, does not show up in the Field listing as this source did not generate any unflagged visibilities in the data set for these Ka-band observations. It was used for pointing in C-band only.<br />
<br />
In addition to source names, antenna names, etc., CASA indexes each of these parameters starting from 0. To specify, e.g., a source field, one can use the index or the name (both are accepted). <!-- Note: &#151; is HTML code for em dash LOS:2017 but does not work in my version. ALSO m-dashes should be avoided in favor of using proper running sentences so I am removing them all --><br />
<br />
'''Note:''' the Rest Frequency and Systemic Velocity are wrong in the listobs log by a factor 10^6 and 1000, respectively, given the quoted units (MHz) and (km/s). This was due to a temporary error in the VLA Observation Preparation Tool (OPT) that has subsequently been fixed. Because the sky frequencies are correct, and we set the rest frequency explicitly later in the deconvolution stage, this does not present a problem for the data reduction. <br />
<br />
<pre style="background-color: #E0FFFF;"><br />
Summary of Observing Strategy<br />
Gain Calibrator: J0954+1743, field id=2<br />
Bandpass Calibrator: J1229+0203, field id=5<br />
Flux Calibrator: J1331+3030 (3C286), field id=7<br />
Target: IRC+10216, field id=3<br />
Ka-band spws = 0,1<br />
</pre> <br />
<br />
[[Image:Ant_locations.png|200px|thumb|Figure 3: Antenna locations from running plotants ]]<br />
Next, look at a graphical plot of the antenna locations and save a hard copy<br />
in case you want it later using '''{{plotants}}''' (see Figure 3). This will be useful for picking a reference antenna. Typically a good choice is an antenna close to the center of the array, but not too close to the center that it suffers from shadowing (as could be in this D-array configuration). Unless it shows problems after inspection of the data, we provisionally choose ea02.<br />
<br />
[[Image:elevationvstime.png|200px|thumb|Figure 4: Elevation as a function of time (after selecting colorize by field).]]<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis='day2_TDEM0003_10s_norx',figfile='ant_locations.png')<br />
</source><br />
<br />
Now look at the elevation as a function of time for all sources (see Figure 4). While it's not the case for these data, you may want to flag if the elevation is very low (usually at the start or end of an observation). Also, how near in elevation your flux calibrator is to your target will impact your ultimate absolute flux density calibration accuracy. <br />
<br />
Unfortunately, the target and flux density calibrator are not particularly well-matched in elevation for this observation, as you can see by plotting the elevation for each source (each source has a different color). We will be using data in spw 0, channels 4 to 60 for this plot as given by the parameter ''spw = '0:4~60' ''; the general CASA selection syntax is described in the [https://casa.nrao.edu/casadocs/casa-5.0.0/data-selection/data-selection-in-a-measurementset Measurement Selection Syntax]. Note that because we have set the channel averaging parameter to 64, but only select 57 channels (4 to 60), CASA will issue a warning about this in the terminal. However, as the extra channels are not selected to contain additional data, this is nothing to be concerned about:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',<br />
xaxis='time',yaxis='elevation',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60', coloraxis='field')<br />
</source><br />
<br />
Thus, we are strongly dependent on the opacity and gaincurve corrections to get the flux scale right for these data. (This is something to keep in mind when planning observations.) We will deal with this after the initial inspection.<br />
<br />
[[Image:plotallfields.png|200px|thumb|Figure 5: Result of plotms]]<br />
[[Image:Zoom1_mark.png|200px|thumb|Figure 6: Zooming in and marking region (hatched box)]]<br />
<br />
Next, look at all the source amplitudes as a function of time (see Figure 5):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60', coloraxis='field')<br />
</source><br />
<br />
Now zoom in on the region very near zero amplitude for sources J0954+1743 and IRC+10216 between 3h30 and 4h30 (see Figure 6). To zoom, select the <br />
Zoom tool [[File:casaplotms-zoom.png]] in the lower left corner of the '''{{plotms}}''' GUI, then you can left click to draw a box. <br />
Look for the low values (you may want to zoom in a few times to really see the suspect points clearly). Now use the Mark Region [[Image:MarkRegionsButton.png]] and Locate [[File:casaplotms-locate-tool.png]] buttons (positioned along the bottom of the GUI) to see which antenna is causing problems. The output will be shown in the logger. Since all the located baselines include ea12, this is likely the responsible antenna.<br />
<br />
<pre style="background-color: #98FB98;"><br />
IMPORTANT NOTES ON PLOTMS:<br />
<br />
* When using the locate button, it is important to have only selected a modest number <br />
of points with the mark region tool (see example of marked region in the thumbnail), <br />
otherwise the response will be very slow and possibly hang the tool <br />
(all of the information will be output to your terminal window, not the logger). <br />
<br />
* Throughout the tutorial, when you are done marking/locate, use the Clear Regions <br />
tool to get rid of the marked box before plotting other things. <br />
<br />
* After flagdata command flagging, you have to force a complete reload of the cache <br />
to look at the same plot again with the new flags applied. To do this, either check <br />
the "reload" box in the lower left before clicking Plot, or do Shift+Plot.<br />
<br />
* Fields in the plotms GUI tabs can be altered and the data replotted<br />
without having to close and restart plotms. After making changes, check the<br />
Reload box and then click the Plot button adjacent to the Reload box.<br />
<br />
<br />
</pre> <br />
<br />
Now click the clear region [[File:casaplotms-clear-region.png]] button, then go back to the zoom button to zoom in further to note exactly what the time range is: 03:41:00~04:10:00.<br />
<br />
Check the other sideband by changing spw to 1:4~60. You will have to<br />
re-zoom. If you have trouble, click on the Mark icon and then back to<br />
zoom. In spw 1, ea07 is bad from the beginning until after the next<br />
pointing run: within the Data tab of plotms, set the timerange parameter to 03:21:40~04:10:00. To see this, compare the amplitudes when antenna is set to 'ea07' and when it is set to one of the other antennas, such as 'ea08'.<br />
<br />
If you set antenna to 'ea12' and zoom in on this initial time range, you can also see that ea12 is bad during the same time range as for spw 0. You can also see this by entering '!ea07' in the antenna parameter, which removes ea07 from the plot (in CASA, ! deselects). Actually, one should also flag some bad data from antenna ea08..<br />
<br />
We can set up a flagging command to get both bad antennas for the<br />
appropriate time and spw:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='day2_TDEM0003_10s_norx',<br />
mode='list', <br />
inpfile=["field='2,3' antenna='ea12' timerange='03:41:00~04:10:00'",<br />
"field='2,3' antenna='ea07,ea08' timerange='03:21:40~04:10:00' spw='1'"])<br />
</source><br />
<br />
<br />
Task '''{{flagdata}}''' in ''mode='list' ''can take several commands separated by commas. The data selection will be connected by AND for each string. <br />
<br />
Note that because the chosen time range is limited to fields 2 and 3, the field parameter is not really<br />
needed but the above is a nice example of the task syntax.<br />
<br />
Now remove the !ea07 from the antenna parameter and replot both spw, zooming in to<br />
be sure that all obviously low points are gone. Also zoom in and<br />
check 3C286 (J1229+0203 is already obvious because it is so bright!). <br />
<br />
[[Image:IRC10216_uvdist1.png|200px|thumb|Figure 7: Amplitude vs. uv-distance for IRC+10216, both spw]]<br />
<br />
Next, look more closely at IRC+10216:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='3',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0~1:4~60', coloraxis='spw')<br />
</source><br />
<br />
You can see that there are some noisy high points. So now try:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='3',<br />
xaxis='uvdist',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0~1:4~60', coloraxis='spw')<br />
</source><br />
<br />
Now we can see that most of the high points on IRC+10216 are due to large scale emission on short baselines (see Figure 7), but there is still some noisy stuff. For a target like this with extended emission it's best to wait until later to decide what to do about it. We will not be able to get adequate calibration for antennas that are truly bad (even if they don't stand out here), so these will be obvious later.<br />
<br />
== The Observing Log, Antenna Position Corrections, Opacities, Gaincurves, Antenna Efficiencies and other Calibration "Priors"==<br />
<br />
=== Antenna Position Corrections ===<br />
<br />
As mentioned in the observing log above, antennas ea10, ea12, and ea22 do not have good baseline positions. <br />
Antenna ea10 was not in the array, but for the other two antennas we need to check for<br />
any improved baseline positions that were derived after the observations were taken.<br />
In CASA, these corrections can be derived by '''{{gencal}}''' for the VLA. The parameter ''caltype='antpos' ''will use antenna position offset values and create a calibration table for the delay changes that this correction involves. If the 'antenna' parameter is not specified, '''{{gencal}}''' will query the VLA webpages for the offsets ([http://www.vla.nrao.edu/astro/archive/baselines/ VLA Baseline Corrections page]).<br />
<br />
<br />
<source lang="python" style="color: red"><br />
# In CASA<br />
gencal(vis='day2_TDEM0003_10s_norx',caltable='antpos.cal',<br />
caltype='antpos',<br />
antenna='')<br />
</source><br />
<br />
which return the antenna offsets in the logger:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Determine antenna position offests from the baseline correction database<br />
offsets for antenna ea01 : 0.00000 0.00300 0.00000<br />
offsets for antenna ea02 : -0.00080 0.00000 0.00000<br />
offsets for antenna ea03 : -0.00280 0.00000 0.00000<br />
offsets for antenna ea05 : 0.00000 0.00280 0.00000<br />
offsets for antenna ea12 : -0.01000 0.00450 -0.00170<br />
offsets for antenna ea22 : -0.02570 0.00270 -0.01900<br />
offsets for antenna ea23 : -0.00140 0.00000 0.00000<br />
offsets for antenna ea24 : -0.00150 0.00000 0.00000<br />
offsets for antenna ea27 : 0.00000 0.00190 -0.00160<br />
</pre><br />
<br />
Apparently more antennas than the ones noted already by the operator in the log needed, and now benefit from updated positions although ea12 and ea22 clearly have the largest corrections.<br />
<br />
'''Please note: '''if you are reducing VLA data taken before March 1, 2010, you need to set '' caltype='antposvla' ''. Before this date the automated lookup will not work and one has to specify the the antenna positions explicitly.<br />
<br />
=== Gaincurves and Antenna Efficiencies ===<br />
<br />
<br />
The gaincurve describes how each antenna behaves as a function of elevation, for each receiver band. Currently, only gaincurves for the VLA are available<!-- BAD LINK - HIDDEN FOR TIME BEING (see [http://www.vla.nrao.edu/astro/guides/vlbivla/current/node18.html] for the incorporated models)-->. This option should not be used with any other telescopes. <br />
<br />
Gaincurves are generated into a separate calibration table using the task '''{{gencal}}''' with the '' caltype='gceff' ''. This option will also include antenna efficiencies in the resulting calibration table:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gencal(vis='day2_TDEM0003_10s_norx',caltable='gaincurve.cal',<br />
caltype='gceff')<br />
</source><br />
<br />
<!-- ''' Note that this method replaces the ''gaincurve=True'' option in the calibrations tasks in CASA 3.4 and earlier ''' --><br />
<br />
=== Opacities ===<br />
<br />
Opacity corrections are typically only applied for higher frequency observations, where the atmospheric opacity affects the flux density scale if the sources (including calibrators) are observed at different elevations. For lower frequencies the effect is marginal, and opacity correction is usually skipped.<br />
The opacity of the observation can be computed from a seasonal model and/or weather station information. After the zenith opacities are derived, they will be recomputed for the correct elevation of the data automatically using <math>e^{(-\csc[el]\tau_z)}</math> in '''{{gaincal}}''', '''{{applycal}}''', '''{{bandpass}}''', etc. <br />
<br />
To start, we want to plot the opacity of the atmosphere at 36 GHz at the time this observation was taken so it can be corrected for in<br />
subsequent calibrations. The task '''{{plotweather}}''' takes care of that. The task plots the weather conditions at the time of the observations and calculate the atmospheric opacities based on these data in combination with a seasonal model that contains long-term statistics at the VLA site (see Figure 8). The parameter'' 'seasonal_weight=0.5' ''gives both equal weights: <br />
<br />
[[Image:day2_TDEM0003_10s_norx.plotWX.png|200px|thumb|right|Figure 8: plotweather output]]<br />
<br />
We will be running '''{{plotweather}}''' in a way that will assign the opacity list (one entry for each spw in ascending order) to the variable myTau:<br />
<br />
<source lang="python"><br />
# In CASA<br />
myTau = plotweather(vis='day2_TDEM0003_10s_norx', doPlot=True)<br />
</source><br />
<br />
With this input, the task <br />
<br />
* returns the opacity values to the logger:<br />
<br />
<pre style="background-color: #fffacd;"><br />
SPW : Frequency (GHz) : Zenith opacity (nepers)<br />
0 : 36.387 : 0.039<br />
1 : 36.305 : 0.039<br />
wrote weather figure: day2_TDEM0003_10s_norx.plotweather.png<br />
</pre> <br />
<br />
* creates a file '''"day2_TDEM0003_10s_norx.plotweather.png"''' (in the current working directory) with the elevation of the sun, wind speed and direction, temperature, and precipitable water vapor (PWV) as functions of time over the observation. View this file with your preferred image viewer like gthumb, xv, or Preview;<br />
<br />
* and assigns the myTau variable to the list of opacities per spectral window: <br />
<br />
<source lang="python"><br />
# In CASA typing the variable by itself<br />
myTau<br />
</source><br />
It echoes the following output, the values set per spectral window:<br />
<pre style="background-color: #fffacd;"><br />
Out[..]: [0.039224935645330521, 0.039003557322827495]<br />
</pre> <br />
<br />
We can now create a calibration table for the opacities via '''{{gencal}}''' with the'' calmode='opac' ''parameter; we can either input the opacities directly or use the myTau variable:<br />
<br />
<source lang="python"><br />
# In CASA<br />
gencal(vis='day2_TDEM0003_10s_norx',caltable='opacity.cal',<br />
caltype='opac',<br />
spw='0,1',<br />
parameter=myTau)<br />
</source><br />
<!-- ''' Note that this method replaces the ''opacity'' option in the calibrations tasks in CASA 3.4 and earlier ''' --><br />
<br />
==Set Up the Model for the Flux Calibrator==<br />
<br />
Next, we set the model for the flux calibrator. Depending on your observing frequency and angular resolution you can do this several ways. In the past, one typically used a point source (constant flux) model for <br />
the flux calibrator, possibly with a uvrange cutoff if necessary. More recently, model images for the most common flux calibrators have been made available for use in cases where the sources are somewhat resolved. This is most likely to be true at higher frequencies and at higher resolutions (more extended arrays). <br />
<br />
The VLA flux calibrator models that are available in CASA can be checked through '''{{setjy}}''':<br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis='day2_TDEM0003_10s_norx',listmodels=True)<br />
</source><br />
<br />
The terminal (not the logger) will now show the models, e.g., 3C286_C.im, 3C48_K.im, etc. Task '''{{setjy}}''' will search in the current working directory for images that may contain models, as well as in a CASA directory where known calibrator models are stored.<br />
<br />
We will pick the Ka ('A') band model of 3C286: 3C286_A.im.<br />
Task '''{{setjy}}''' scales the total flux in the model image to that appropriate for your individual spectral window frequencies according to the calibrator's flux and reports this number to the logger. It is a good idea to save this information for your records.<br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis='day2_TDEM0003_10s_norx',field='7',spw='0~1',<br />
scalebychan=True,<br />
model='3C286_A.im')<br />
</source><br />
<br />
The logger output for each spw is:<br />
<pre style="background-color: #fffacd;"><br />
<br />
J1331+3030 (fld ind 7) spw 0 [I=1.7483, Q=0, U=0, V=0] Jy @ 3.6387e+10Hz, (Perley-Butler 2013)<br />
J1331+3030 (fld ind 7) spw 1 [I=1.7512, Q=0, U=0, V=0] Jy @ 3.6305e+10Hz, (Perley-Butler 2013)<br />
Using model image /home/casa/data/distro/nrao/VLA/CalModels/3C286_A.im<br />
Scaling spw(s) [0, 1]'s model image by channel to I = 1.75124, 1.7496, 1.74797 Jy @(3.63045e+10, 3.63499e+10, 3.63951e+10)Hz (LSRK) for visibility prediction (a few representative values are shown).<br />
</pre><br />
<br />
The absolute fluxes for the frequencies of all spectral window channels have now been determined, and one can proceed to the bandpass and complex gain calibrations.<br />
<br />
==Bandpass and Delay==<br />
<br />
Before determining the bandpass solution, we need to inspect phase and amplitude<br />
variations with time and frequency on the bandpass calibrator to<br />
decide how best to proceed. We limit the number of antennas to make<br />
the plot easier to see. We chose ea02 as it seems like a good<br />
candidate for the reference antenna. Let's try a single baseline to antenna ea23:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='5',<br />
xaxis='channel',yaxis='phase',correlation='RR',<br />
avgtime='1e8',spw='0:4~60',antenna='ea02&ea23')<br />
</source><br />
[[Image:Nobandpass_phase.png|200px|thumb|Figure 9: Phase as a function of channel for ea02.]]<br />
<br />
The phase variation is modest ~10 degrees. Now expand to all baselines that include ea02, and add an extra dimension of color (see Figure 9):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='5',<br />
xaxis='channel',yaxis='phase',correlation='RR',<br />
avgtime='1e8',spw='0:4~60',antenna='ea02', coloraxis='antenna2')<br />
</source><br />
[[Image:Nobandpass_phasetime.png|200px|thumb|Figure 10: Phase as a function of time for all baselines with antenna ea02.]]<br />
From this<br />
you can see that the phase variation across the bandpass for each baseline to antenna ea02 is<br />
modest. Next check LL, and then spw=1 for both correlations. Also check<br />
other antennas if you like.<br />
<br />
Now look at the phase as a function of ''time'' (note that there is a 360 degree wrap ambiguity on this baseline):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='5',<br />
xaxis='time',yaxis='phase',correlation='RR',<br />
avgchannel='64',spw='0:4~60',antenna='ea02&ea23')<br />
</source><br />
<br />
Expand to all antennas with ea02:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='5',<br />
xaxis='time',yaxis='phase',correlation='RR',<br />
avgchannel='64',spw='0:4~60',antenna='ea02', coloraxis='antenna2')<br />
</source><br />
<br />
You can see that the phase variations are smooth (see Figure 10), but do vary<br />
significantly over the 5 minutes of observation and in most cases by<br />
a few 10s of degrees. If you want, zoom in to see this better.<br />
<br />
The conclusion from this investigation is that, especially for high frequency observations, you need to correct<br />
the phase variations with time before solving for the bandpass to<br />
prevent decorrelation of the vector averaged bandpass<br />
solution. Since the phase variation as a function of channel is<br />
modest (as we noted in Figure 9), you can average over several (or all) channels to increase the signal<br />
to noise of the phase vs. time solution. If the phase variation, as a<br />
function of channel is larger, you may need to restrict averaging to only a few<br />
channels to prevent introducing delay-based closure errors as can happen from averaging over<br />
non-bandpass corrected channels with large phase variations.<br />
<br />
First, let's work on the delay calibration. The delay is the linear slope of phase across frequency. From the plot we saw that it was rather modest (the phases over frequency per baseline were aligned almost horizontal) and the bandpass calibration will certainly take care of it. Nevertheless, it is best to derive a delay calibration first and then calculate the bandpass. The delay calibration is an antenna based calibration solution and it can be derived in '''{{gaincal}}''' with parameter'' gaintype='K' ''. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='day2_TDEM0003_10s_norx', caltable='delays.cal', field='5', <br />
refant='ea02', gaintype='K', gaintable=['antpos.cal','gaincurve.cal','opacity.cal'])<br />
</source><br />
<br />
<br />
We use the strong bandpass calibrator for the solution. It will be extrapolated in time to all observations.<br />
<br />
This call also exemplifies how the CASA calibration table system works. Calibration tables are incremental. So we supply the previous 'antpos.cal', 'gaincurve.cal', and 'opacity.cal' tables as input to '''{{gaincal}}'''. The following calibration step will require the parameter'' gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delay.cal'] ''and so on. <br />
<br />
<br />
Now we proceed to the actual bandpass calibration. <br />
Since the bandpass calibrator is quite strong we do the phase-only<br />
solution on the integration time of 10 seconds (parameter ''solint='int' '').<br />
<br />
<br />
[[Image:Prebp_phasecal2.png|200px|thumb|Figure 11: Phase only calibration before bandpass. The four lines are both polarizations in both spw, unfortunately two of them get the same color green.]]<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='bpphase.gcal',<br />
field='5',spw='0~1:20~40',<br />
refant='ea02',calmode='p',solint='int',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal'])<br />
</source><br />
<br />
Plot the solutions (see Figure 11) (note that '''{{plotms}}''' can also be used for plotting calibration tables):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='bpphase.gcal',xaxis='time',yaxis='phase',<br />
iteration='antenna',subplot=331,plotrange=[0,0,-180,180])<br />
</source><br />
<br />
These solutions will appear in the CASA plotter GUI. If you closed it after plotting the antennas positions above, it should reopen. If it is still open from before, the new plots should just appear. After you are done looking at the first set of plots, push the Next button on the GUI to see the next set of antennas.<br />
<br />
Now we can apply this phase solution on-the-fly while determining<br />
the bandpass solutions on the timescale of the bandpass calibrator scan (parameter '' solint='inf' ''). <br />
<br />
<source lang="python"><br />
# In CASA<br />
bandpass(vis='day2_TDEM0003_10s_norx',caltable='bandpass.bcal',field='5',<br />
refant='ea02',solint='inf',solnorm=True,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bpphase.gcal'])<br />
</source><br />
<br />
'''A few words about solint and combine:'''<br />
<br />
The use of parameter'' solint='inf' ''in '''{{bandpass}}''' will derive one bandpass<br />
solution for the whole J1229+0203 scan. Note that if there had been two observations of the bandpass calibrator, this command would have combined the data from both scans to form one bandpass solution, because the default of the combine parameter for '''{{bandpass}}''' is ''combine='scan' ''. To solve for one bandpass for each bandpass calibrator scan you would need to use ''combine=' ' ''in the bandpass call. In all calibration tasks, regardless of solint, scan boundaries are only crossed when ''combine='scan' ''. Likewise, field (spw) boundaries are only crossed if parameter ''combine='field' '' or parameter ''combine='spw' '', the latter two are not generally good ideas for bandpass solutions. <br />
<br />
Plot the solutions, amplitude (see Figure 12) and phase (see Figure 13):<br />
[[Image:Bandpass_amp.png|200px|thumb|Figure 12: Amplitude Bandpass solutions]]<br />
[[Image:Bandpass-afterdelays.png|200px|thumb|Figure 13: Phase Bandpass solutions]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='bandpass.bcal',xaxis='chan',yaxis='amp',<br />
iteration='antenna',subplot=331)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='bandpass.bcal',xaxis='chan',yaxis='phase',<br />
iteration='antenna',subplot=331)<br />
</source><br />
<br />
Note the scale for the phases, they are very small since the delays were already taken out in a previous setup.<br />
<br />
This step isn't necessary from a calibration perspective, but if you<br />
want to go ahead and check the bandpass calibration on the bandpass<br />
calibrator you can run '''{{applycal}}''' here. In the future we hope to plot<br />
corrected data on-the-fly without this '''{{applycal}}''' step. Subsequent applycals<br />
will overwrite this one, so no need to worry.<br />
<br />
[[Image:Applybandpass_phase.png|200px|thumb|Figure 14: Phase as a function of channel, plotting the corrected data (after Custom and adjusting "Style" to 3.)]]<br />
<br />
<source lang="python"><br />
applycal(vis='day2_TDEM0003_10s_norx',field='5',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal'],<br />
gainfield=['','','','5','5'],<br />
calwt=False)<br />
</source><br />
<br />
Task '''{{applycal}}''' works like a matrix. The first entries of the ''gaintable'' and ''gainfield'' lists are to be used together, so are the second entries etc. (except for the opacity list, which is referring to spws). All will be applied to the 'field' selection (field='5' in the first line). In the above example, 'antpos.cal' (gaintable='antpos'), which is a field independent solution (gainfield=''), is applied to field '5'. The 'bandpass.bcal' (gaintable='bandpass.bcal') that was obtained for field '5' (gainfield='5', the bandpass observation) is also applied to field '5', etc. Again, lists within the lists are fine. <br />
<br />
<source lang="python"><br />
plotms(vis='day2_TDEM0003_10s_norx',field='5',<br />
xaxis='channel',yaxis='phase',ydatacolumn='corrected',<br />
correlation='RR',<br />
avgtime='1e8',spw='0:4~60',antenna='ea02', coloraxis='antenna2')<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='day2_TDEM0003_10s_norx',field='5',<br />
xaxis='channel',yaxis='amp',ydatacolumn='corrected',<br />
correlation='RR',<br />
avgtime='1e8',spw='0:4~60',antenna='ea02', coloraxis='antenna2')<br />
</source><br />
<br />
Note that the phase and amplitude as a function of channel are very flat now (see Figure 14).<br />
<br />
==Gain Calibration==<br />
<br />
Now that we have a bandpass solution to apply, we can solve for the antenna-based phase and amplitude gain calibration. Since the phase changes on a much shorter timescale than the amplitude, at least for the higher frequencies and/or longer baselines, we will solve for them separately. If the phase changes significantly over time and if the un-corrected phase were averaged over this timescale, the amplitude would decorrelate. Note that we now also re-solve for the gain solutions of the bandpass calibrator to derive new solutions that are corrected for the bandpass shape. Since the bandpass calibrator will not be used again, while not strictly necessary, it is useful to check its calibrated flux density. We use a minimum signal-to-noise of 2 here as it seems to be a good compromise for using good data without rejecting too many solutions (parameter ''minsnr = 2 ''). <br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='intphase.gcal',<br />
field='2,5,7',spw='0~1:4~60',<br />
refant='ea02',calmode='p',solint='int',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal'])<br />
</source><br />
[[Image:allcal_phaseint2.png|200px|thumb|Figure 15: Plot of phase solutions on an integration time.]]<br />
<br />
Here parameter ''solint='int' ''coupled with parameter ''calmode='p' ''will derive a single phase solution for each integration (recall, we averaged the data for every 10 seconds). Note that the bandpass table is applied on-the-fly before solving for the phase solutions, however the bandpass is NOT applied to the data permanently until applycal is run later on.<br />
<br />
Notice that quite a few solutions are rejected due to SNR < 2 (printed to terminal). For the most part it <br />
is only one or two solutions out of >30 so this isn't too worrying. Take note if you see large numbers of rejected solutions per integration; this is likely an indication that solint is too short for the S/N of the data.<br />
<br />
Now look at the phase solution, and note the scatter within a scan time (see Figure 15):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='intphase.gcal',xaxis='time',yaxis='phase',<br />
iteration='antenna',subplot=331,plotrange=[0,0,-180,180])<br />
</source><br />
<br />
Although ''solint='int' ''(i.e. the integration time of 10 seconds) is the best choice to apply before for solving for the amplitude solutions, it is not a good idea to use this to apply to the target. This is because the phase-scatter within a scan can dominate the interpolation between calibrator scans. Instead, we also solve for the phase on the scan time, ''solint='inf' ''(but ''combine=' ' '', since we want one solution per scan) for application to the target later on. '''Unlike the bandpass task,''' the default of the combine parameter in the gaincal task is ''combine=' ' ''.<br />
[[Image:allcal_phaseinf2.png|200px|thumb|Figure 16: Plot of phase solutions on a scan time.]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='scanphase.gcal',<br />
field='2,5,7',spw='0~1:4~60',<br />
refant='ea02',calmode='p',solint='inf',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal'])<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='scanphase.gcal',xaxis='time',yaxis='phase',<br />
iteration='antenna',subplot=331,plotrange=[0,0,-180,180])<br />
</source><br />
<br />
Note that there are no failed solutions here because of the added S/N acquired by the longer solint (see Figure 16).<br />
Alternatively, instead of making a separate phase solution for application to the target, one can also run '''{{smoothcal}}''' to smooth the solutions derived on the integration time.<br />
<br />
Next we apply the bandpass and parameter ''solint='int' ''phase-only calibration solutions on-the-fly to derive amplitude solutions. <br />
Here the use of ''solint='inf' '', not ''combine=' ' '',will result in one solution per scan interval.<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='amp.gcal',<br />
field='2,5,7',spw='0~1:4~60',<br />
refant='ea02',calmode='ap',solint='inf',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal','intphase.gcal'])<br />
</source><br />
[[Image:allcal_ampphase.png|200px|thumb|Figure 17: Plot of residual phase solutions on a scan time]]<br />
<br />
Now let's look at the resulting phase solutions (see Figure 17). Since we have taken out the phase as best we can by applying the ''solint='int' ''phase-only solution, this plot will give a good idea of the residual phase error. If you see scatter of more than a few degrees here, you should consider going back and looking for more data to flag, in particular for the times where these errors are large, or for other effects that may explain this behavior (and flag as necessary).<br />
<br />
<source lang="python"><br />
# In CASA <br />
plotcal(caltable='amp.gcal',xaxis='time',yaxis='phase',<br />
iteration='antenna',subplot=331,plotrange=[-1,-1,-20,20])<br />
</source><br />
<br />
Note that we have restricted the plot range so that all antennas are plotted with the same scale (instead of autoscaled; for the X-axis the ''-1,-1'' values still allow autoscaling) (see Figure 18). We can see that both antenna ea12 (all times) and ea23 (first 1/3 of observation) show particularly large residual phase noise.<br />
[[Image:allcal_amp.png|200px|thumb|Figure 18: Plot of amplitude solutions on a scan time]]<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='amp.gcal',xaxis='time',yaxis='amp',<br />
iteration='antenna',subplot=331,plotrange=[-1,-1,0,1])<br />
</source><br />
<br />
Note that the amplitude solutions for ea12 are very low, in particular when you set ''plotrange=[-1,-1,0,1]''; this is another indication that this antenna is dubious.<br />
<br />
Next we use the flux calibrator (whose flux density was set in '''{{setjy}}''' above) to derive the flux of the other calibrators (see Figure 19). Note as of CASA 4.0, the flux table can be written as an incremental table, just like all other calibration tables. In that case it won't replace the amp.gcal but both tables, amp.gcal as well as flux.gcal need to be carried onward. <br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='day2_TDEM0003_10s_norx',caltable='amp.gcal',<br />
fluxtable='flux.cal',reference='7',incremental=True)<br />
</source><br />
<br />
[[Image:allcal_flux.png|200px|thumb|Figure 19: Plot of flux corrected amplitude solutions.]]<br />
It is a good idea to note down for your records the derived flux densities:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): J1331+3030<br />
Found transfer field(s): J0954+1743 J1229+0203<br />
<br />
Flux density for J0954+1743 in SpW=0 (freq=3.63912e+10 Hz) is: 0.221046 +/- 0.0308169 (SNR = 7.17288, N = 38)<br />
Flux density for J0954+1743 in SpW=1 (freq=3.63085e+10 Hz) is: 0.227485 +/- 0.0298859 (SNR = 7.61178, N = 38)<br />
Flux density for J1229+0203 in SpW=0 (freq=3.63912e+10 Hz) is: 27.2673 +/- 0.245114 (SNR = 111.243, N = 38)<br />
Flux density for J1229+0203 in SpW=1 (freq=3.63085e+10 Hz) is: 26.555 +/- 0.256801 (SNR = 103.407, N = 38)<br />
20</pre><br />
<br />
==Applycal and Inspect==<br />
<br />
Now we apply the calibration to each source according to which tables are appropriate and which source should be used to do that particular calibration. For the calibrators, all bandpass solutions come from the bandpass calibrator (id=5), and the phase and amplitude calibration comes from their own solutions. <br />
<br />
'''Note:''' In all '''{{applycal}}''' steps we set ''calwt=False''. It is very important to turn off this parameter which determines if the weights are calibrated along with the data. Data from antennas with better receiver performance and/or longer integration times should have higher weights, and it can be advantageous to factor this information into the calibration. Before the VLA upgrade, meaningful weights were available for each visibility. However, at the time of this observation during commissioning, the VLA was not yet recording the information necessary to calculate meaningful weights. Since these data weights are used at the imaging stage you can get strange results from having ''calwt=True'' when the input weights are themselves not meaningful, especially for self-calibration on resolved sources (for example, flux calibrator and target). <br />
<br />
For more recent data, the switched power information is recorded, but we currently do not recommend using this information to calculate data weights without exercising considerable caution.<br />
<br />
<source lang="python"><br />
# In CASA<br />
# for the gain/phase calibrator (field '2'):<br />
applycal(vis='day2_TDEM0003_10s_norx',field='2',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal','intphase.gcal','amp.gcal','flux.cal'],<br />
gainfield=['','','','5','5','2','2','2'],<br />
calwt=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
# for the bandpass calibrator (field '5'):<br />
applycal(vis='day2_TDEM0003_10s_norx',field='5',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal','intphase.gcal','amp.gcal','flux.cal'],<br />
gainfield=['','','','5','5','5','5','5'],<br />
calwt=False)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
# for the flux calibrator (field '7'):<br />
applycal(vis='day2_TDEM0003_10s_norx',field='7',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal','intphase.gcal','amp.gcal','flux.cal'],<br />
gainfield=['','','','5','5','7','7','7'],<br />
calwt=False)<br />
</source><br />
<br />
For the target we apply the bandpass (id=5), and the calibration from the gain calibrator (id=2):<br />
<br />
<source lang="python"><br />
# In CASA<br />
# for the target source IRC10216 (field '3'):<br />
applycal(vis='day2_TDEM0003_10s_norx',field='3',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass.bcal','scanphase.gcal','amp.gcal','flux.cal'],<br />
gainfield=['','','','5','5','2','2','2'],<br />
calwt=False)<br />
</source><br />
<br />
Now inspect the corrected data:<br />
[[Image:applycal_inspect.png|200px|thumb|Figure 20: Plot of calibrated amplitudes over time.]]<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='5',ydatacolumn='corrected',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='', coloraxis='antenna1')<br />
</source><br />
<br />
This plot shows some data deviating from the average amplitudes (see Figure 20). Use methods previously described to <br />
mark a region for a small number of deviant data points and click Locate. You will find that ea12 is responsible.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='2',ydatacolumn='corrected',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='', coloraxis='antenna2')<br />
</source><br />
<br />
Here we see some problems, with high points. Mark some regions<br />
and locate in '''{{plotms}}''' to find out which antennas and in which spws. Pay special<br />
attention to antennas that have been called out already as showing some dubious behavior.<br />
<br />
What you find is that ea07, which we flagged spw=1 above, is also bad for the same timerange in spw=0. This was not obvious in the raw data because spw=0 was adjusted in the on-line system by a gain attenuator, while spw=1 wasn't. So a lack of power on this antenna can look like very low (and obvious) amplitudes in spw=1 but not for spw=0. Looking carefully you'll see that ea07 is actually pretty noisy throughout.<br />
[[Image:ea12.png|200px|thumb|Figure 21: Plot of antenna ea12 by itself]]<br />
[[Image:ea23.png|200px|thumb|Figure 22: Plot of antenna ea23 by itself]]<br />
<br />
From the locate we also find that ea12 and ea23 show some high points; to see this, replot baselines using each of them alone (see Figures 21 and 22):<br />
<br />
<source lang="python"><br />
plotms(vis='day2_TDEM0003_10s_norx',field='2',ydatacolumn='corrected',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='ea12', coloraxis='antenna2')<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='day2_TDEM0003_10s_norx',field='2',ydatacolumn='corrected',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='ea23', coloraxis='antenna2')<br />
</source><br />
<br />
It may be a good idea to flag ea12 completely. It's just a bit noisy all around and ea23 is pretty noisy during the first scans between the initial and second pointing. Recall that these antennas had more noisy calibration solutions.<br />
<br />
[[Image:target_uvdist.png|200px|thumb|Figure 23: IRC+10216 as a function of uv-distance.]]<br />
Now let's see how the target looks. Because the target has resolved structure, it's best to look at it as<br />
a function of uvdistance. We will exclude the three antennas we already know have problems.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='3',ydatacolumn='corrected',<br />
xaxis='uvdist',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='!ea07;!ea12;!ea23', coloraxis='antenna2')<br />
</source><br />
<br />
The color indicates that the spikes are caused by a single antenna (see Figure 23). Use zoom, mark, and locate to see which one.<br />
Also look at spw=1.<br />
<br />
The bad antenna turns out to be ea28; to confirm, replot with ''antenna=!ea28'':<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='3',ydatacolumn='corrected',<br />
xaxis='uvdist',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='!ea07;!ea12;!ea23;!ea28', coloraxis='antenna2')<br />
</source><br />
<br />
Looks much better. To see if it's restricted to a certain time, do:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='3',ydatacolumn='corrected',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='ea28', coloraxis='antenna1')<br />
</source><br />
<br />
Baselines with ea28 clearly show issues until about two-thirds of the way through the observation. <br />
Plot another distant antenna to compare. We will flag it all, since it's hanging far out on the north<br />
arm by itself.<br />
<br />
The additional data we've identified as bad need to be flagged, and then all the calibration steps will need to be run again.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='day2_TDEM0003_10s_norx',<br />
mode='list',<br />
inpfile=["antenna='ea07,ea12,ea28'",<br />
"antenna='ea07,ea23' timerange='03:21:40~04:10:00'"])<br />
</source><br />
<br />
==Redo Calibration after more Flagging==<br />
<br />
After flagging, you'll need to repeat the calibration steps above. Here, we append '''_redo''' to the table names to distinguish them from the first round, in case we want to compare with previous versions. <br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='bpphase_redo.gcal',<br />
field='5',spw='0~1:20~40',<br />
refant='ea02',calmode='p',solint='int',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal'])<br />
<br />
bandpass(vis='day2_TDEM0003_10s_norx',caltable='bandpass_redo.bcal',<br />
field='5',<br />
refant='ea02',solint='inf',solnorm=True,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bpphase_redo.gcal'])<br />
<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='intphase_redo.gcal',<br />
field='2,5,7',spw='0~1:4~60',<br />
refant='ea02',calmode='p',solint='int',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass_redo.bcal'])<br />
<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='scanphase_redo.gcal',<br />
field='2,5,7',spw='0~1:4~60',<br />
refant='ea02',calmode='p',solint='inf',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass_redo.bcal'])<br />
<br />
gaincal(vis='day2_TDEM0003_10s_norx',caltable='amp_redo.gcal',<br />
field='2,5,7',spw='0~1:4~60',<br />
refant='ea02',calmode='ap',solint='inf',minsnr=2.0,<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass_redo.bcal','intphase_redo.gcal'])<br />
<br />
fluxscale(vis='day2_TDEM0003_10s_norx',caltable='amp_redo.gcal',<br />
fluxtable='flux_redo.cal',reference='7',incremental=True)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
Flux density for J0954+1743 in SpW=0 (freq=3.63912e+10 Hz) is: 0.238671 +/- 0.0285271 (SNR = 8.36646, N = 32)<br />
Flux density for J0954+1743 in SpW=1 (freq=3.63085e+10 Hz) is: 0.230055 +/- 0.0290958 (SNR = 7.90682, N = 32)<br />
Flux density for J1229+0203 in SpW=0 (freq=3.63912e+10 Hz) is: 26.3266 +/- 0.285249 (SNR = 92.2933, N = 32)<br />
Flux density for J1229+0203 in SpW=1 (freq=3.63085e+10 Hz) is: 25.7897 +/- 0.292791 (SNR = 88.0825, N = 32)<br />
</pre><br />
<br />
Feel free to pause here and remake the calibration solution plots from above, just be sure to put in the revised table names.<br />
<br />
==Redo Applycal and Inspect==<br />
<br />
Now apply all of the new calibrations, which will overwrite the old ones. These commands are identical to those above, with the exception of the '''_redo''' part of each calibration filename.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='day2_TDEM0003_10s_norx',field='2',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass_redo.bcal','intphase_redo.gcal','amp_redo.gcal','flux_redo.cal'],<br />
gainfield=['','','','5','5','2','2','2'],<br />
calwt=False)<br />
<br />
applycal(vis='day2_TDEM0003_10s_norx',field='5',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass_redo.bcal','intphase_redo.gcal','amp_redo.gcal','flux_redo.cal'],<br />
gainfield=['','','','5','5','5','5','5'],<br />
calwt=False)<br />
<br />
applycal(vis='day2_TDEM0003_10s_norx',field='7',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass_redo.bcal','intphase_redo.gcal','amp_redo.gcal','flux_redo.cal'],<br />
gainfield=['','','','5','5','7','7','7'],<br />
calwt=False)<br />
<br />
applycal(vis='day2_TDEM0003_10s_norx',field='3',<br />
gaintable=['antpos.cal','gaincurve.cal','opacity.cal','delays.cal','bandpass_redo.bcal','scanphase_redo.gcal','amp_redo.gcal','flux_redo.cal'],<br />
gainfield=['','','','5','5','2','2','2'],<br />
calwt=False)<br />
</source><br />
<br />
[[Image:gaincal_corrflag.png|200px|thumb|Figure 24: Gain calibrator after further flagging and recalibration]]<br />
[[Image:target_corrflag.png|200px|thumb|Figure 25: IRC+10216 after further flagging and recalibration (after selecting colorize by spw).]]<br />
<br />
Now you can inspect the calibrated data again (see Figure 24). Except for random scatter, things look pretty good.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='2',ydatacolumn='corrected',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0:4~60',antenna='', coloraxis='antenna2')<br />
</source><br />
<br />
You can use the Mark and Locate buttons to assess that the remaining scatter seems random, i.e., no particular antenna or time range appears to be responsible (see Figure 25).<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='3',ydatacolumn='corrected',<br />
xaxis='uvdist',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='0~1:4~60',antenna='', coloraxis='spw')<br />
</source><br />
<br />
==Split==<br />
<br />
Now we split the data into individual files. This is not strictly necessary, as you can select the appropriate fields in later clean stages, but it is safer in case for example you get confused with later processing and want to fall back to this point (this is especially a good idea if you plan to do continuum subtraction or self calibration later on). It also makes smaller individual files in case you want to copy to another machine or want to share the data with a colleague.<br />
<br />
Here, we split off the data for the phase calibrator and the target:<br />
<br />
<source lang="python"><br />
# In CASA<br />
split(vis='day2_TDEM0003_10s_norx',outputvis='J0954',<br />
field='2')<br />
<br />
split(vis='day2_TDEM0003_10s_norx',outputvis='IRC10216',<br />
field='3')<br />
</source><br />
<br />
<br />
<!--<br />
<br />
<span style="color:red"><br />
<br />
</span><br />
<br />
To reinitialize the scratch columns for use by later tasks, we need to run clearcal for both new datasets<br />
<br />
<source lang="python"><br />
# In CASA<br />
clearcal(vis='J0954')<br />
#<br />
clearcal(vis='IRC10216')<br />
</source><br />
--><br />
<br />
This concludes the calibration phase of the data reductions.<br />
<br />
== UV Continuum Subtraction and Setting Up for Self-Calibration==<br />
<br />
[[Image:irc10216_uvspec.png|200px|thumb|Figure 26: UV-plot of the spectral line signal in both spw for IRC+10216.]]<br />
<br />
Now we can make a vector averaged uv-plot of the calibrated target spectral line data. It is important to note that you will only see signal in such a plot if (1) the data are well calibrated, and (2) there is significant signal near the phase center of the observations or if the line emission (or absorption) is weak but extended. If this isn't true for your data, you won't be able to see the line signal in such a plot and will need to make an initial (dirty or lightly cleaned) line+continuum cube to determine the line-free channels. Generally, this is the recommended course for finding the line-free channels more precisely than is being done here, as weak line signal would not be obvious in this plot. <br />
<br />
<source lang="python"><br />
plotms(vis='IRC10216',field='',ydatacolumn='data',<br />
xaxis='channel',yaxis='amp',correlation='RR',<br />
avgtime='1e8',avgscan=True,spw='0~1:4~60',antenna='', coloraxis='spw')<br />
</source><br />
<br />
In the Display tab, change the Unflagged Points Symbol to Custom and Style to circle or diamond and 3 pixels. <br />
<br />
You should see the horned profile typical of an expanding shell (see Figure 26). From this plot, you can guess that strong <br />
line emission is restricted to channels 18 to 47 (zoom in if necessary to see exactly what the channel numbers are). <br />
<br />
In the Data tab, under Averaging, you can also click on All Baselines to average all baselines, but this is a little harder to see.<br />
<br />
Now we want to use the line-free channels to create a model of the continuum emission that can be subtracted to create a line-only data set. We want to refrain from going to close to the edges of the band. These channels are typically noisy, and we don't want to get too close to the line channels because we could only see strong line emission in the vector averaged uv-plot and there may be weaker line emission in the channels next to the channel interval with strong lines.<br />
<br />
We can now either specify the line free channels via parameters ''fitspw='0~1:4~13;52~63' ''and ''excludechans=False ''in '''{{uvcontsub}}''', or do the inverse selection via:<br />
<br />
<source lang="python"><br />
uvcontsub(vis='IRC10216',fitspw='0~1:14~51',excludechans=True,<br />
want_cont=True)<br />
</source><br />
<br />
which excludes the line channels from the continuum fit.<br />
<br />
The parameter ''want_cont=True'' will produce two new data sets; IRC10216.contsub is the continuum subtracted line data, and IRC10216.cont is the continuum estimate (note however, that it is still a multi-channel data set).<br />
<br />
==Velocity Systems and Doppler corrections==<br />
<br />
The VLA WIDAR correlator does not support Doppler ''tracking''. Doppler ''setting'' is possible which will use the calculated sky frequency based on the velocity of a source at the start of the observation. The sky frequency is then fixed throughout that observing run. Typically, a fixed frequency is better for the calibration of interferometric data. The downside, however, is that a spectral line may shift over one or more channels during an observation. Task '''{{clean}}''' takes care of such a shift when regridding the visibilities in velocity space (default is LSRK) to form an image. Sometimes, in particular when adding together different observing runs, it may be advisable to regrid all data sets to the same velocity grid, combine all data to a single file, then Fourier transform and deconvolve. The tasks '''{{cvel2}}''', '''{{concat}}''', and '''{{clean}}''' serve this purpose respectively. The following run of '''{{cvel2}}''' shows an example on how the parameters of '''{{cvel2}}''' may be set. <br />
<br />
The '''IRC10216.contsub''' visibility spans the following channel range (see also the listobs output in the first part of the tutorial): <br />
<br />
<source lang="python"><br />
# In CASA<br />
vishead(vis='IRC10216.contsub', mode='summary')<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) Corrs <br />
0 Subband:0 64 TOPO 36387.229 125.000 8000.0 36391.1670 RR RL LR LL<br />
1 Subband:0 64 TOPO 36304.542 125.000 8000.0 36308.4795 RR RL LR LL<br />
</pre><br />
<br />
For spw 0, this corresponds to about 1 km/s channel width. If we want to image the HC3N spectral line with a rest frequency of 36.39232 GHz over a velocity range of &minus;50km/s to 0km/s and a channel width of 5 km/s, we may decide to regrid the visibilities in '''{{cvel2}}''' as:<br />
<br />
'''Note that this step is not necessary for the processing further down in this tutorial. You may skip to the next section if you wish.'''<br />
<br />
<source lang="python"><br />
# In CASA<br />
cvel2(vis='IRC10216.contsub', outputvis='IRC10216.contsub-cveled', mode='velocity',<br />
interpolation='linear', nchan=10, start='-50km/s', width='5km/s',<br />
restfreq='36.39232GHz',outframe='LSRK', veltype='optical') <br />
</source><br />
<br />
This will create a new data set where the data is binned into the new grid. Since all data in measurement sets are stored in frequency space, an inspection with '''{{vishead}}''' now gives:<br />
<br />
<source lang="python"><br />
# In CASA<br />
vishead(vis='IRC10216.contsub-cveled', mode='summary')<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) Corrs <br />
0 Subband:0 10 LSRK 36392.927 606.974 6070.6 36395.6588 RR RL LR LL<br />
</pre><br />
<br />
After the '''{{cvel2}}''' step, the data can then be combined with other observations via '''{{concat}}''' and imaged in '''{{clean}}''' with parameter ''mode='channel' ''to conserve that velocity system and grid.<br />
<br />
Note that '''{{cvel2}}''' can also Hanning smooth the data, if needed (as an alternative to the stand-alone '''{{hanningsmooth}}''' task).<br />
<br />
==Image the Spectral Line Data==<br />
<br />
<br />
Before any imaging, it is good practice to remove the effects of relative noise scatter introduced by flagging uneven bits in the visibility data between the channels and times. Of course line emission affects this so the range of channels with line emission should be excluded in the noise determination. The task to weigh the data using noise statistics is '''{{statwt}}'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
statwt(vis='IRC10216.contsub', fitspw='0~1:5~14,0~1:48~59')<br />
</source><br />
<br />
<br />
First we make images from the continuum-subtracted, calibrated spectral line data. Because the spectral line emission from IRC+10216 has significant extended emission, it is very important to run clean interactively, and make a clean mask. To make the cube a bit smaller and stay away from noisy edge channels we restrict the <br />
channel range using the spw parameter.<br />
[[Image:viewer_interactive.png|200px|thumb|Figure 27: Channel 32 shown (#26 out of the 54 in the cube) for the HC3N cube shown in the interactive viewer with the white contour showing the mask contour drawn with the polygon tool.]]<br />
<br />
'''Note that interrupting {{clean}} by Ctrl+C may corrupt your visibilities so you may be better off choosing to let {{clean}} finish. We are currently implementing a command that will nicely exit to prevent this from happening, but for the moment try to avoid Ctrl+C.'''<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='IRC10216.contsub',imagename='IRC10216_HC3N.cube_r0.5',<br />
imagermode='csclean',<br />
imsize=300,cell=['0.4arcsec'],spw='0:5~58',<br />
mode='velocity',interpolation='linear',<br />
restfreq='36.39232GHz',outframe='LSRK',<br />
weighting='briggs',robust=0.5,<br />
interactive=True,<br />
threshold='3.0mJy',niter=100000)<br />
</source><br />
<br />
* ''imagermode = csclean ''will invoke the Cotton-Schwab cleaning algorithm and the data will be regridded into a new output velocity frame, correcting for Doppler shifts of the line during the run (VLA data for each observation is always at a fixed sky frequency). The iterations are chosen as a high value to allow many clean cycles when needed. Typically, however, the threshold will kick in earlier and stop the cleaning process. <br />
<br />
It will take a little while to grid the data, but the '''{{viewer}}''' will open when it's ready to start an interactive clean. Use the Channels tape deck buttons at the bottom (sometimes it will load on the side) of the Viewer display GUI to step through to the channel with the most extended (in angular size) emission. Select all channels for the clean mask, select the polygon tool [[File:Polygon_btn.png]], and make a single mask that applies to all channels by clicking on the image to drop anchor points (see Figure 27) and drawing lines between the points; double click for the last point after which you can adjust the anchor points (but be careful not to click outside as that removes the polygon). Once you make the polygon region, you need to '''double click inside it''' to save the mask region. If you see the polygon turn white you will know you succeeded. Note, that if you had the time and patience you could make a clean mask for each channel, and this would create a slightly better result. <br />
<br />
After making the mask, you should check that the emission in all the other channels fits within the mask you made using the tape deck controls to move back and forth. If you need to include more area in the mask, you can chose the erase toggle at the top, and then encircle your existing mask with a polygon and double click inside. Then go back to add toggle at top and make a new mask. Alternatively, you can erase a part of the mask, or you can add to the existing mask by drawing new polygons. Feel free to experiment with these tools.<br />
<br />
'''Note''': If you start an interactive clean, and then do not make a mask, clean will stop when you tell it to go on because it has nothing to clean. There is no default mask. <br />
<br />
To continue with '''{{clean}}''' use one of the Next action buttons in the green area on the Viewer Display GUI: The red X [[File:clean-stop.png]] will stop '''{{clean}}''' where you are; the blue arrow [[File:clean-continue.png]] will stop the interactive part of '''{{clean}}''', but continue to clean non-interactively until reaching the stopping niter ('''iterations x cycles''') or threshold, whichever comes first. The green arrow [[File:clean-redo.png]] will clean until it reaches the iterations parameter on the left side of the green area. When the interactive viewer comes back use the tape deck to recheck that your mask encompasses what you think is real emission. If your mouse has a middle button, then by default it controls the image stretch.<br />
<br />
For this example, the threshold has been set to ''threshold = '3mJy' ''to protect you from cleaning too deeply. With a careful clean mask you can clean too close to the thermal noise limit (the actual observed rms noise limit and not the theoretical one calculated for the proposal, as flagging, weather, etc., can affect what you actually get). It is ALWAYS best to clean each channel in a cube to a specific threshold than to stop by simply using the ''niter'' parameter, which can leave each channel cleaned to different levels. There are many ways to determine a suitable threshold. One way is to make a dirty image (''niter = 0''), open the cube using the viewer, go to a line free channel, select the box region tool, make a box near the field center about the size of your source, and double click inside. The rms noise of that channel will appear in the terminal window from which the '''{{viewer}}''' was launched. Try a few different boxes and average the results; this is a good estimate of the rms per channel assuming your data are not dynamic range limited (i.e., noise can be higher in channels with strong signal). This is the absolute minimum for threshold and with no mask you probably shouldn't clean deeper than 3x this rms. <br />
<br />
[[Image:SiS_interactive.png|200px|thumb|Figure 28: Channel 43 (#37 of 54 in the cube) shown for the SiS cube in the interactive viewer with the white contour showing the mask contour drawn with the polygon tool.]]<br />
<br />
Keep cleaning by using the green Next Action arrow [[File:clean-redo.png]] until the residual displayed in the viewer looks noise-like. To speed things up, you might change the iteration parameter in the viewer to something like 300. This parameter can also be set in the task command. You will notice that in this particular case there are residuals that cannot be cleaned. These residuals are due to the extended, resolved out structure on size scales larger than the array is sensitive to (Largest Angular Scale (LAS) that the array is sensitive to can be calculated from the shortest baseline length), and potential residual phase and amplitude calibration errors. We will explore this in a few sections with self-calibration. <br />
<br />
Repeat the process for the SiS line using the call below (see Figure 28). Note the emission for this line is less extended than the HC3N. It has to do with the different excitation requirements of the two different lines. The SiS is excited closer to the central star than the HC3N.<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='IRC10216.contsub',imagename='IRC10216_SiS.cube_r0.5',<br />
imagermode='csclean',<br />
imsize=300,cell=['0.4arcsec'],spw='1:5~58',<br />
mode='velocity',interpolation='linear',<br />
restfreq='36.30963GHz',outframe='LSRK',<br />
weighting='briggs',robust=0.5,<br />
interactive=True, <br />
threshold='3.0mJy',niter=100000)<br />
</source><br />
<br />
You can look at both cubes using the viewer, and use the tape deck controls to play the cube as a movie.<br />
<source lang="python"><br />
# In CASA<br />
viewer<br />
</source><br />
<br />
==Image the Continuum data==<br />
<br />
Below the use of parameter ''mode='mfs' ''will make a single multi-frequency synthesis image out of the specified spw/channels. Again, you should make an interactive clean mask. Since no threshold is set, you will need to stop cleaning when the residual looks noise-like using the red X Next Action button [[File:clean-stop.png]] (it will be done when the viewer comes back the second time). The continuum for IRC10216 is very weak but interesting, tracing the photosphere of the AGB star. <br />
<br />
The continuum data set produced with parameter ''wantcont=True'' in uvcontsub is the model fit. To image the continuum itself, use the line-free channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='IRC10216',imagename='IRC10216.36GHzcont',<br />
mode='mfs',imagermode='csclean',<br />
imsize=300,cell=['0.4arcsec'],spw='0~1:5~14,0~1:48~59',<br />
weighting='briggs',robust=0.5,<br />
interactive=True)<br />
</source><br />
<br />
Now look at the result in the viewer, if you like:<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer<br />
</source><br />
<br />
==Image Analysis and Viewing==<br />
<br />
Next, make integrated intensity maps (moment 0) and intensity-weighted velocity maps (moment 1). For HC3N, we also produce a velocity dispersion, peak flux, and median map. All are derived with '''{{immoments}}'''. In order to do this, we'll want to know what channels the line emission starts and ends on and the rms noise in a single channel.<br />
<br />
First, open with the viewer file IRC10216_HC3N.cube_r0.5.image:<br />
<br />
<source lang="python"><br />
# In CASA <br />
viewer('IRC10216_HC3N.cube_r0.5.image')<br />
</source><br />
<br />
Use the tape deck controls to see which channels have significant line emission. For HC3N, the line channel range in the cube is 10 to 39, which is 16 to 45 in the original 64-channel uv-data, but for imaging the five beginning and ending channels were discarded by selecting 0:5~58, which resulted in a 54 channel image cube (see Figure 29); this is also the same for SiS (see Figure 30). <br />
<br />
[[Image:irc10216.jpg|200px|thumb|Figure 29: HC3N moment 0 map with white continuum contours superposed.]]<br />
[[Image:irc10216_sismom0.jpg|200px|thumb|Figure 30: SiS moment 0 map with white continuum contours superposed.]]<br />
<br />
Then use the tape deck to go to a line free channel, select the box region tool and make a box. When you double click in the box, the image statistics for the channel you are on will print to the terminal. Move the box around a bit to see what the variation in rms noise is. You should get something like 2 mJy. Note that the rms is much worse in channels with strong emission because of the low dynamic range of these data. If you want the box tool to go away, i.e., if you want to make a new one, press the Escape key. <br />
<br />
Now let's make the moment 0 and moment 1 maps. For moment 0, it's best to limit the calculation to image channels with significant signal in them, but not to apply a flux cutoff, as this will bias the derived integrated intensities upward.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments(imagename='IRC10216_HC3N.cube_r0.5.image',moments=[0],<br />
axis='spectral',<br />
chans='10~39',<br />
outfile='IRC10216_HC3N.cube_r0.5.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments(imagename='IRC10216_SiS.cube_r0.5.image',moments=[0],<br />
axis='spectral',<br />
chans='10~39',<br />
outfile='IRC10216_SiS.cube_r0.5.image.mom0')<br />
</source><br />
<br />
To have a look at these, use the viewer (see Figures 29 & 30):<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('IRC10216_HC3N.cube_r0.5.image.mom0')<br />
<br />
viewer('IRC10216_SiS.cube_r0.5.image.mom0')<br />
</source><br />
<br />
For moment 1, it is essential to apply a conservative flux cutoff to limit the calculation to high signal-to-noise areas. Here we use about 5&sigma;:<br />
<br />
[[Image:irc10216_dust_HC3N_cont.png|200px|thumb|Figure 31: The VLT V-band image from Le&atilde;o et al. (2006, A&A, 455,187) showing dust rings, overlaid with white HC3N moment 0 contours and black 36 GHz continuum contours.]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments(imagename='IRC10216_HC3N.cube_r0.5.image',moments=[1],<br />
axis='spectral',<br />
chans='10~39',excludepix=[-100,0.01],<br />
outfile='IRC10216_HC3N.cube_r0.5.image.mom1')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments(imagename='IRC10216_SiS.cube_r0.5.image',moments=[1],<br />
axis='spectral',<br />
chans='10~39',excludepix=[-100,0.01],<br />
outfile='IRC10216_SiS.cube_r0.5.image.mom1')<br />
</source><br />
<br />
Finally, we will do velocity dispersion, peak flux, and median map in a single step for HC3N. Task '''{{immoments}}''' can perform these steps even though the latter two are not moments in a mathematical sense. Check the help file to find out the options. Peak flux and median are produced with the moment parameter set to 8 and 3:<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments(imagename='IRC10216_HC3N.cube_r0.5.image',moments=[2,8,3],<br />
axis='spectral',<br />
chans='10~39',excludepix=[-100,0.01],<br />
outfile='IRC10216_HC3N.cube_r0.5.image.extramoms')<br />
</source><br />
<br />
will create the files IRC10216_HC3N.cube_r0.5.image.extramoms.weighted_dispersion_coord, IRC10216_HC3N.cube_r0.5.image.extramoms.median, and IRC10216_HC3N.cube_r0.5.image.extramoms.maximum.<br />
<br />
Now use the viewer to further explore the images you've made.<br />
<br />
For fun you can download the VLT V-band image at http://casa.nrao.edu/Data/EVLA/IRC10216/irc_fors1_dec_header.fits kindly provided by <br />
Izan Le&atilde;o and overlay the moment images and 36 GHz continuum (see Figure 31). More information about the dust properties can be found in the Le&atilde;o et al. (2006) paper http://adsabs.harvard.edu/abs/2006A%26A...455..187L.<br />
<br />
Note that it should be possible to create position velocity cuts using the viewer (as [https://casa.nrao.edu/docs/cookbook/casa_cookbook008.html#section%3Adisplay.image.pvdiagram described] in the [https://casa.nrao.edu/docs/cookbook/index.html CASA Cookbook]). Masking the data cube to extract the emission is described in the [[Masking_images_for_analysis|Masking_images_for_analysis]] page.<br />
<br />
== Spectrum Fitting ==<br />
<br />
Frequently, one may like to fit Gaussians or polynomials to the spectral line in the data cube. This can be done with CASA's '''{{specfit}}''' task; '''{{specfit}}''' can fit those functions to an average spectrum defined by some area, or, alternatively, for each pixel. We will fit an average spectrum.<br />
<br />
=== Fitting an average spectrum ===<br />
<br />
[[Image:viewer-irc-specfit1.png|200px|thumb|Figure 32: The viewer showing the cube at plane 28 (#34 of 64). The magenta box marks the region that is averaged for the spectral profile. ]]<br />
<br />
[[Image:viewer-irc-specfit1-profile.png|200px|thumb|Figure 33: Spectral profile of the HC3N line within the magenta rectangle.]]<br />
<br />
First, we want to inspect the spectrum. Load the image into the viewer (here the HC3N image cube) and mark a region with the mouse button that is assigned to the rectangular '''R''' region in the tool bar. It is probably best to do this at a plane that shows the entire extent of the source. When done, select '''spectral profile''' from the '''Tools''' menu. The average spectrum will be displayed in a separate panel (see Figure 33).<br />
<br />
To fit this profile in '''{{specfit}}''', we need a region file outlining the 2-D region that is averaged (the magenta box in the viewer screenshot).<br />
In the following we use the new CASA region format (CASA 3.3 and higher) that is described at the [[CASA Region Format | CASA Region Format]] page. Following the guidelines on that page, we create a file named '''specfit.crtf''' that describes a box with its [[x1,y1],[x2,y2]] corners in J2000 RA DEC coordinates. Use any text editor (e.g., emacs, vi, etc.) to create the file '''specfit.crtf''' with the CRTF identifier line as well as the following box parameters:<br />
<br />
<pre style="background-color: #fffacd;"><br />
#CRTFv0<br />
box[[09:47:59.2, 13.16.24], [09:47:55.8, 13.17.09]]<br />
</pre><br />
<br />
We will fit two Gaussians to the two peaks of the spectrum (the scientific merit is debatable). A file with initial values for the fit can be provided via the parapmeter ''estimates'' (see '''{{specfit}}''' for more information). Here we will let CASA figure out the start values by itself: <br />
<br />
<source lang="python"><br />
# In CASA<br />
myfit = specfit(imagename='IRC10216_HC3N.cube_r0.5.image', region='specfit.crtf', multifit=False,<br />
estimates='', ngauss=2) <br />
<br />
</source><br />
<br />
Note that the output is stored in a Python dictionary called "myfit", as well as printed to the CASA logger. You should get something similar to the following output, the results will vary based on your cleaning regions, depths, etc.:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Results for component 0:<br />
Type : GAUSSIAN<br />
Peak : 5.78 +/- 0.44 mJy/beam<br />
Center : -16.99 +/- 0.41 km/s<br />
33.78 +/- 0.40 pixel<br />
FWHM : 9.9 +/- 1.0 km/s<br />
9.57 +/- 0.98 pixel<br />
Integral : 60.6 +/- 7.7 mJy/beam.km/s<br />
Results for component 1:<br />
Type : GAUSSIAN<br />
Peak : 5.11 +/- 0.40 mJy/beam<br />
Center : -34.29 +/- 0.52 km/s<br />
16.98 +/- 0.50 pixel<br />
FWHM : 12.5 +/- 1.3 km/s<br />
12.1 +/- 1.3 pixel<br />
Integral : 68.1 +/- 8.9 mJy/beam.km/s<br />
</pre><br />
<br />
which seems to have caught the two peaks pretty well.<br />
<br />
<!--<br />
=== Spectral Fitting pixel by pixel ===<br />
<br />
[[Image:irc-amp0.png|200px|thumb|Figure 34: Output image of '''{{specfit}}''' of the amplitude of the first Gaussian component.]]<br />
<br />
Task '''{{specfit}}''' can also fit Gaussians to every spectrum in each single spatial pixel. The following command will do this within the '''spectfit.crtf''' region defined above:<br />
<br />
<source lang="python"><br />
# In CASA<br />
pixfit = specfit(imagename='IRC10216_HC3N.cube_r0.5.image', region='specfit.crtf', ngauss=2,<br />
multifit=True, amp='fit-amp.image', center='fit-center.image', fwhm='fit-fwhm.image')<br />
</source><br />
<br />
In this example, '''{{specfit}}''' will produce three images per Gaussian, images that map the best fit values of the Gauss peaks (amplitudes), velocity centers, and full widths at half maximum. For the first Gaussian, the image to the right displays the amplitude image '''fit.amp.image_0'''.<br />
<br />
--><br />
<br />
== Self-Calibration ==<br />
<br />
The many different aspects of self-calibration could fill several casaguides. Here we describe a simple process for this particular relatively low S/N data (low S/N per channel, at least).<br />
<br />
While running '''{{clean}}''' above, the model column for each channel will have been filled with the clean model (if you made a Fourier transform of this model, you would see an image of the clean components). <br />
<br />
We choose to do the self cal on the spw=1 SiS line data because it has the strongest emission in a single channel and is a bit more compact than the HC3N data. We will run '''{{gaincal}}''' specifying the channel in the uv-data that has the brightest peak in the image (use the '''{{viewer}}''' to figure out which channel this is for spw=1), and note the peak flux. Since we started the image with a channel range, we need to account for the fact that the image channel numbers do not map exactly to the uv-data channel numbers (they are off by five, so that channel 13 in the image is roughly channel 19 in the uv-data). <br />
<br />
Next we need to understand the S/N of the data. In particular, to self-cal, you need enough signal on a single baseline within the chosen solution interval, ''solint'', to get a S/N of at least 3. We calculated above an average rms noise of about 2 mJy/beam/channel for the entire timerange (about 95 minutes of on-source time) and all antennas (16). We can use our knowledge of the radiometer equation (see [https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/sensitivity VLA Sensitivity]) where rms scales as 1/sqrt(time * #baselines), and the number of baselines= N(N-1)/2 and N=# of antennas. So the rms noise on one baseline, for one 10 second integration in this observation is given by:<br />
<br />
<math> {\rm RMS(baseline)} = {\rm 2\ mJy\ beam^{-1}\ channel^{-1}} \sqrt{ \frac{95\times 60\ {\rm sec}}{10\ {\rm sec}}\times\frac{16\times 15}{2\times 1}}\ {\sim}\ 500\ {\rm mJy\ beam^{-1}\ channel^{-1}}</math><br />
<br />
[[Image:timeonsource.png|200px|thumb|Figure 34: Plot to estimate the time on source.]]<br />
<br />
The 95 minutes of on-source time can be estimated from a plot where you can sum up the amount of time on a source (see Figure 34):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='day2_TDEM0003_10s_norx',field='3',ydatacolumn='corrected',<br />
xaxis='time',yaxis='amp',correlation='RR,LL',<br />
avgchannel='64',spw='1:4~60',antenna='')<br />
</source><br />
<br />
This analysis suggests that the rms noise on one baseline, for one 10 second integration, is about 500 mJy. In contrast, the peak flux density in the strongest SiS channel is only about 200 mJy (you can check using the '''{{viewer}}'''). Since the emission is fairly compact, most baselines will see about this peak flux; this is why we choose the more compact of the two possible lines. Thus, a 10 second solution interval is not enough to get a SNR of at least 3 on a 200 mJy peak. We need to use a solint large enough so that the rms noise is not worse than about 1/3 of 200 mJy. Thus, a solint of 10 minutes is about the shortest we can use and be reasonably confident of the solutions. <br />
<br />
Now we run '''{{gaincal}}''' with the solint we have determined. Note that because our desired solint is more <br />
than the scan time, we need to include parameter ''combine='scan' ''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='IRC10216.contsub',caltable='pcal_ch19one_10min',<br />
spw='1:19',calmode='p',solint='10min',combine='scan',<br />
refant='ea02',minsnr=3.0)<br />
</source><br />
<br />
[[Image:selfcalone.png|200px|thumb|Figure 35: Phase-only self-calibration solutions with 10 minute solint (second page of the display). ]]<br />
Next, look at the solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='pcal_ch19one_10min',xaxis='time',yaxis='phase',<br />
iteration='antenna',subplot=331,plotrange=[0,0,-50,50])<br />
</source><br />
<br />
For some antennas you can see clear global trends away from zero: ea08, ea21, and ea24 are examples, and you can also see some smaller variations with time (see Figure 35).<br />
<br />
Now explore whether applying this solution actually improves matters. To do this, run '''{{applycal}}''' to apply the solutions to the line data set for both spw. Use parameter ''spwmap'' to tell it that the solutions derived for spw=1 should be applied to both spw=0 and spw=1. Again, it's important to set parameter ''calwt=False'' here.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='IRC10216.contsub',field='',spw='0,1',<br />
gaintable=['pcal_ch19one_10min'],spwmap=[[1,1]],calwt=False)<br />
</source><br />
<br />
'''Note:''' In this example, we ran the self-cal steps on the full uv continuum subtracted spectral line data set. For a more complex iterative self-calibration procedure, you may find it easier to split off the channel/spw you want to experiment on with '''{{split}}''', and then do all the imaging ('''{{clean}}''') and '''{{gaincal}}''' steps with it. The '''{{gaincal}}''' tables created on the single channel can still be applied with '''{{applycal}}''' to the multi-channel/spw data set. If you do this, keep in mind that once split, the single-channel data will have its spw ID reset to 0 (you can check with '''{{listobs}}'''), no matter what spw it came from. Thus, in order to applycal with it, you would need parameter ''spwmap=<nowiki>[[0,0]]</nowiki>''.<br />
<br />
To save time, use the clean mask you made before and run in a non-interactive mode. You can use a mask over again as long as the number of channels in the '''{{clean}}''' call haven't changed. You can change cell or imsize and it will still do the right thing.<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='IRC10216.contsub',imagename='IRC10216_HC3N.cube_r0.5.pselfcal',<br />
imagermode='csclean',<br />
imsize=300,cell=['0.4arcsec'],spw='0:5~58',<br />
mode='velocity',interpolation='linear',<br />
restfreq='36.39232GHz',outframe='LSRK',<br />
weighting='briggs',robust=0.5,<br />
mask='IRC10216_HC3N.cube_r0.5.mask',<br />
interactive=False,threshold='3.0mJy',niter=100000)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='IRC10216.contsub',imagename='IRC10216_SiS.cube_r0.5.pselfcal',<br />
imagermode='csclean',<br />
imsize=300,cell=['0.4arcsec'],spw='1:5~58',<br />
mode='velocity',interpolation='linear',<br />
restfreq='36.30963GHz',outframe='LSRK',<br />
weighting='briggs',robust=0.5,<br />
mask='IRC10216_SiS.cube_r0.5.mask', <br />
interactive=False,threshold='3.0mJy',niter=100000)<br />
</source><br />
<br />
Now investigate the original and self-calibrated images in the viewer. You will find that even this single self-cal step significantly improves the images. Try opening both versions of the SiS image cubes. <br />
Select a bright channel via the tape deck controls, then use the wrench and pwrench tools to make a plot setting the same image range for both cubes, and two panels in x, then to see both images of that channel side-by-side click the blink toggle (see Figure 36 for more tips on setup parameters).<br />
<br />
[[Image:compareselfcal.png|600px|]]<br />
<br />
Figure 36: Original and self-cal SiS images for channel 37/54, notice the decrease in residuals.<br />
<br />
Repeat for HC3N (see Figure 37):<br />
<br />
[[Image:compareselfcal2.png|600px|]]<br />
<br />
Figure 37: Original and self-cal HC3N images for channel 34/54, notice the decrease in residuals.<br />
<br />
Now you can redo the moment images with the improved cubes. Be sure to change the output file names. <br />
<br />
<br />
Questions about this tutorial? Please contact the [https://science.nrao.edu/observing/helpdesk NRAO Helpdesk].<br />
<br />
[[Main Page | &#8629; '''CASAguides''']]<br />
<br />
<!-- --[[User:Cbrogan|Crystal Brogan]] <br />
--additions:[[User:Jott| Juergen Ott]], [[User:Mkrauss| Miriam Krauss]], [[User:Dshepher| Deb Shepherd]]<br />
--corrections: [[User:Lsjouwerman| Lorant Sjouwerman]] (4.4.0, 2015/06/24)<br />
-- 4.5 update: [[User:Jott| Juergen Ott]] (2015/11/3)<br />
--content update: [[User:Hmedlin| Heidi Medlin]] (4.5.2, 2016/02/26) <br />
--content update: [[User: Tperreau| Tony Perreault]] (4.5.2, 2016/03/01)<br />
-- 4.6 update and corrections: [[User:Lsjouwerman| Lorant Sjouwerman]] (2016/05/05)<br />
-- 5.0 update and corrections: [[User:Lsjouwerman| Lorant Sjouwerman]] (2017/09/20)<br />
--><br />
<br />
{{Checked 5.0.0}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Data_flagging_with_viewer&diff=21908Data flagging with viewer2017-06-23T21:10:58Z<p>Emomjian: </p>
<hr />
<div>[[Category: Interactive Tools]][[Category: VLA]][[Category: Flagging]]<br />
Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
<br />
== Overview ==<br />
<br />
Your data will inevitably have some bad data. Perhaps a receiver on a given antenna is acting up, or perhaps the pointing on a given antenna is poor, and these problematic data will (hopefully) appear obviously in your measurement set. It's a good idea to inspect your data carefully before calibration and imaging, and CASA offers several tools to flag bad data interactively. For loyal users of TVFLG and SPFLG in AIPS, you'll be happy to know that {{viewer}} reproduces and even enhances these functionalities.<br />
<br />
This tutorial illustrates how to use {{viewer}} to flag poor quality data. It will use the continuum dataset [https://archive.nrao.edu/archive/ArchiveQuery?PASSWD=&QUERYTYPE=ARCHIVE&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&MAX_ROWS=NO+LIMIT&SORT_PARM2=Starttime&SORT_ORDER2=Asc&QUERY_ID=9999&QUERY_MODE=Prepare+Download&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&WRITELOG=0&PROJECT_CODE=AU0079&SEGMENT=&OBSERVER=&ARCHIVE_VOLUME=&TIMERANGE1=&TIMERANGE2=&SOURCE_ID=&SRC_SEARCH_TYPE=SIMBAD+or+NED+Resolver&CALIB_TYPE=ALL+Srcs&CENTER_RA=&LONG_RANGE=&FRAME=Equatorial&CENTER_DEC=&LAT_RANGE=&EQUINOX=J2000&SRAD=1.0%27&MIN_EXPOSURE=&OBS_BANDS=L&TELESCOPE=VLA&OBS_MODE=ALL&CORR_MODE=ALL&TELESCOPE_CONFIG=A&OBS_POLAR=ALL&OBSFREQ1=&DATATYPE=ALL&OBSBW1=&ARCHFORMAT=ALL&SUBMIT=Submit+Query AU079] from the VLA archive; this dataset is also used in the [[Data flagging with casaplotms]] tutorial.<br />
<br />
== Loading the Measurement Set into Viewer ==<br />
<br />
The data may be loaded into CASA using the {{importvla}} command. The following commands import the data into the measurement set ''au079.ms'' and send them to the {{viewer}}.<br />
<br />
<source lang="python"><br />
# from loaddata.py<br />
from glob import glob<br />
<br />
# Define the list of files for reading. Use glob to perform wildcard matching with VLA archive filenames.<br />
fileList = glob('AU079_*.xp?')<br />
<br />
importvla(archivefiles=fileList,vis='au079.ms')<br />
viewer('au079.ms') # load the data into the viewer for editing<br />
</source><br />
<br />
The {{viewer}} command can also be invoked without argument, in which case a file selector GUI will appear.<br />
<br />
== Setting Up for Antenna-based Flagging ==<br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
The figure at right shows the default {{viewer}} display for a measurement set. The axes are (''y'') time and (''x'') baseline, or antenna pair, enumerated by the common antenna number of the pair (like TVFLG in AIPS). Flagged data are highlighted in pale blue. <br />
<br />
Another window, titled ''Data Display Options'', will also appear, allowing you to alter the display and flagging options. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:au079_ms_in_viewer_with_flags.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
The blue highlights can be initially distracting, and they can be optionally suppressed by altering the '''Flagging Options''' in the '''Data Display Options'''. The figure at right shows the options.<br />
<br />
* Set '''Show Flagged Regions...''' to '''Masked to Background'''. This operation will effectively set the flagged regions to black. <br />
<br />
In anticipation of the antenna-based editing to come, it's also worth setting the antenna flag.<br />
<br />
* Set '''Flag / Unflag Entire Antenna''' to '''Yes'''. <br />
<br />
Now, when multiple baselines are selected, all baselines to the common antenna will be flagged.<br />
<br />
There are also options to flag all '''correlations''' or '''channels''', the latter being particularly helpful for spectral-line experiments. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_fig2.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
<br />
There is a choice of data to display. Select the '''MS and Visibility Selection''' tab on the '''Data Display Options''' window; see the figure at right. The '''Visibility Component''' menu allows you to choose to display visibility data from the following list.<br />
* Amplitude<br />
* Phase<br />
* Real (the real part of the visibilities)<br />
* Imaginary (the imaginary part of the visibilities)<br />
* Amp Diff (amplitudes after subtracting a local, rolling time-average)<br />
* Amp RMS (based on a rolling time window)<br />
* Phase Diff (phases after subtracting a local, rolling time-average)<br />
* Phase RMS (based on a rolling time window)<br />
<br />
The rolling time windows needed to calculate the '''Diff''' and '''RMS''' options are tuned using the '''Moving Average Size''' slider. <br />
<br />
All of these options are useful to investigate aberrant data, but '''Amp Diff''' can be efficient to start with. Reasonable options are shown in the figure at right. Be sure to turn down the display maximum ('''Data Maximum''' under '''Basic Settings''') to bring out the fainter variations.<br />
<br />
You can also select a single source, or subset of sources, to display using the '''Field IDs''' blank. After you change the '''Field IDs''' box, you'll probably need to click on the '''Apply''' button. If you don't see the button, scroll all the way down in the '''Data Display Options''' window (Note that you don't see the '''Apply''' button in the figure to the immediate right because several submenus are open, but you can see it in the figure above).<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_ampdiffselect.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
<br />
The result of these operations is illustrated in the figure at right. Data from the RR correlation are shown, but you can easily select the other correlations (RL, LR, LL) by using the blue, DVD-style control buttons [[File:vcrNext.png]] below the main display.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' For spectral line datasets, you'll probably also want to use the DVD-style control buttons to step through spectral channels. You can change which data dimension is affected by these control buttons with the '''Animation Axis''' menu, located under '''Display Axes''' in the '''Data Display Options''' window. <br />
</div><br />
<br />
<br />
The RR data look overall pretty good. The brighter horizontal stripes mark calibrator observations; brighter sources show greater noise. Antenna 13 looks a little junky, however, especially between time marks 1000 and 2500. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:ampDiffExample.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' For spectral line datasets, you might also like to display (''y'') time against (''x'') spectral channel (as in AIPS SPFLG). You can do this by going to '''Display Axes''' in the '''Data Display Options''' and selecting '''X Axis''' as '''Channel'''. <br />
</div><br />
<br />
== Antenna-Based Flagging ==<br />
<br />
The following steps outline flagging with {{viewer}}.<br />
<br />
* Zoom in on junky-looking data using the zoom tool [[File:viewer_zoomselect.png]].<br />
* Use one of the drawing selectors [[File:drawingSelector.png]] to identify the bad data. <br />
* Zoom out---either incrementally with [[File:zoomOut.png]] or to show the entire image with [[File:zoomTool.png]]---and repeat.<br />
* '''Important:''' save your edits! Under '''Flagging Options''', use the '''Save Edits''' button.<br />
<br />
This section of the tutorial provides a more detailed walkthrough.<br />
<br />
{|<br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Time to zap those junky antenna 13 data. First, use the zoom tool [[File:viewer_zoomselect.png]], which by default is activated by the '''left mouse button''', to click and drag a box around the region of interest. '''Left double-click''' within the box to zoom. The figure at right shows the zoom selector box and the resulting zoom. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_zoom.png | 400px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Now, use the drawing rectangle selector tool [[File:drawingSelector.png]], which by default is activated by the '''right mouse button'''. Right-click and drag to draw a box around the data to be flagged. Since we're flagging all to a common antenna, it's better to make the box narrow in the baseline (''x'') direction to avoid accidentally flagging the neighboring antenna. Then, '''right double-click''' within the selector box to flag.<br />
<br />
The figure at right illustrates the data selection tool and the result of data flagging. Although only a subset of the Antenna 13 baselines were selected, all of the Antenna 13 baselines were flagged because of our choice of '''Flag / Unflag Entire Antenna''' in the '''Data Display Options - Flagging Options''' menu.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
<br />
[[File:viewerFlag_flags.png | 400px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
== Finishing up ==<br />
<div style="float: right; width: 50%; text-align: center;"><br />
<br />
[[File:viewerSaveFlags.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
To flag additional data, zoom out [[File: zoomOut.png]], select correlations [[File:vcrNext.png]] as necessary, and repeat the steps outlined above. <br />
<br />
It is absolutely vital to save your edits before leaving {{viewer}}. From the '''Data Display Options''' window, select '''Flagging Options''' and '''Save Edits''' (see figure at right). You are now ready for calibration and imaging!<br />
<br />
== Displaying Calibrated Data ==<br />
<br />
You might like to double-check the data after calibration, to ensure that the calibration looks good and no new bad data has popped up. You can do this easily by selecting 'corrected' as the '''Visibility Type''', which can be found in '''MS and Visibility Selection''' section of the '''Data Display Options'''.<br />
<br />
[[Pre-upgrade VLA Tutorials | &#8629; '''Pre-upgrade VLA Tutorials''']]<br><br />
[[Main Page | &#8629; '''CASA Guides''']]<br />
<br />
<br />
<br />
<br />
<br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=MRK_6:_red-shifted_HI_absorption_5.5.0&diff=21907MRK 6: red-shifted HI absorption 5.5.02017-06-23T21:05:30Z<p>Emomjian: </p>
<hr />
<div>Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
<br />
[[Category: VLA]][[Category: Calibration]] [[Category: Imaging]] [[Category: Spectral Line]]<br />
<br />
== Overview ==<br />
<br />
This example demonstrates a trickier spectral line data set. In this case, several radio bright active galaxies were observed to look for redshifted 21 cm absorption. The data were obtained using 4IF mode, which means that the full range of velocities were split into two spectral windows (in this example, CASA identifies them as spectral windows 14 and 15). Producing the data cube will require eventually stitching these windows back together. Happily, {{clean}} includes flexible regridding and interpolation in the spectral domain. {{clean|Clean}} further allows you to adjust the velocity reference frame.<br />
<br />
== Retrieve the data from the NRAO archive ==<br />
<br />
Download your data from the [http://archive.nrao.edu: VLA Archive]; in this example we'll use the publicly available survey [https://archive.nrao.edu/archive/ArchiveQuery?QUERY_ID=9999&QUERY_MODE=Prepare+Download&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&PROJECT_CODE=ab658&OBSERVER=&TELESCOPE=ALL&TIMERANGE1=&TIMERANGE2=&PASSWD=&QUERYTYPE=ARCHIVE&Submit=Submit+Query AB658]. These data were published in [http://adsabs.harvard.edu/abs/1999ApJ...524..684G Gallimore et al. (1999)]. <br />
<br />
Download all files associated with AB658. With the present archive defaults, you should have obtained the following VLA archive files:<br />
<br />
<pre><br />
'AB658_A921122.xp1'<br />
'AB658_A921122.xp2'<br />
'AB658_A921122.xp3'<br />
'AB658_A921122.xp4'<br />
</pre><br />
<br />
This tutorial broadly follows the techniques described in the [[Calibrating a VLA 5 GHz continuum survey|continuum survey tutorial]] and the [[NGC 5921: red-shifted HI emission|21 cm emission tutorial]], and the basic calibration steps are presented only in outline.<br />
<br />
== Loading the Data ==<br />
<br />
Recall that the python file globber [http://docs.python.org/library/glob.html glob] is your friend here!<br />
<br />
<source lang="python"><br />
<br />
measurementSet = "ab658.ms" # Store the desired name of the measurement set<br />
#<br />
from glob import glob<br />
fileList = sorted(glob("AB658*.xp?"))<br />
#<br />
# double check to see if the file already exists<br />
import os<br />
#<br />
if not os.path.isdir("./" + measurementSet): # measurement sets are directories<br />
importvla(archivefiles=fileList, vis=measurementSet) # only import if ab658.ms doesn't yet exist<br />
<br />
vis = measurementSet<br />
mode = "summary"<br />
listobs<br />
</source><br />
<br />
For the purposes of this tutorial, we are interested only in the source [http://en.wikipedia.org/wiki/B._E._Markarian Markarian] [http://nedwww.ipac.caltech.edu/cgi-bin/nph-objsearch?objname=mrk+6&extend=no&hconst=73&omegam=0.27&omegav=0.73&corr_z=1&out_csys=Equatorial&out_equinox=J2000.0&obj_sort=RA+or+Longitude&of=pre_text&zv_breaker=30000.0&list_limit=5&img_stamp=YES 6], but the reduction techniques could be applied to any of the sources in the measurement set. <br />
<br />
[[File:MRK6-shot3.png|thumb|{{listobs|Listobs}} output for AB658. The data for MRK 6 and its calibrators are highlighted.]]<br />
<br />
Using {{listobs}} and {{plotants}} we learn the source names, spectral windows, and viable reference antennas. A screenshot of the {{listobs}} output is given at right.<br />
<br />
{| border=1<br />
| '''Source''' || MKN6<br />
|-<br />
| '''Amp Cal''' || 0134+329 = 3C48<br />
|-<br />
| '''Phase Cal''' || 1003+830<br />
|- <br />
| '''Central Antennas''' || VA27, VA09, VA23<br />
|-<br />
| '''Spectral Window IDs''' || 10, 11<br />
|}<br />
<br />
It's just as well to put that information into python global variables.<br />
<br />
<source lang="python"><br />
sourceName = "MKN6"<br />
phaseCal = "1003+830"<br />
ampCal = "0134+329"<br />
refAnt = "VA27" # or perhaps "VA09" or "VA23"<br />
spwStr = '10, 11'<br />
</source><br />
<br />
The selection of the spectral windows, stored here in the string variable ''spwStr'', is key to getting the calibration right. Each science target in the multisource measurement set has a tuning respective of the source redshift, and the calibration observations are matched to those spectral windows. Were the spectral windows parameter (usually '''spw''') left to its default value (all available spectral windows), many calibration routines would necessarily fail.<br />
<br />
== Editing the Data ==<br />
<br />
[[File:MRK6-shot1.png|thumbnail|Plotms display of (uncalibrated) visibility amplitudes vs. time for the source MKN6.]]<br />
<br />
First inspect the data using {{plotms}}. <br />
<br />
<source lang="python"><br />
plotms(vis = measurementSet, field = sourceName)<br />
</source><br />
<br />
Be sure to select '''Axes &rarr; X Axis &rarr; Time''' and '''Axes &rarr; Y Axis &rarr; Amp'''. There is some obvious junk isolated in time; see the screenshot at right. Follow the [[Data flagging with plotms|tutorial]] to flag discrepant data for the source and calibrators.<br />
<br />
== Calibration ==<br />
<br />
Calibration of HI absorption spectra follows the basic techniques outlined in the [[Calibrating a VLA 5 GHz continuum survey|continuum tutorial]] and [[NGC 5921: red-shifted HI emission|HI emission tutorial]].<br />
<br />
First, set the flux of the amplitude calibrator. Notice that we are being careful to keep track of the spectral windows and not using the default ''spw''.<br />
<br />
<source lang="python"><br />
# set the flux standard<br />
default('setjy')<br />
vis = measurementSet<br />
field = ampCal<br />
spw = spwStr # this is key to getting the calibration right!<br />
modimage = ''<br />
setjy()<br />
</source><br />
<br />
Bandpass calibration comes next. <br />
<br />
<source lang="python"><br />
# set some default table names<br />
btable = 'ab658.bcal' # bandpass<br />
gtable = 'ab658.gcal' # gain<br />
ftable = sourceName + '.fluxscale' # flux<br />
#<br />
if os.path.exists(btable): rmtables(btable)<br />
#<br />
if os.path.exists(gtable): rmtables(gtable)<br />
#<br />
if os.path.exists(ftable): rmtables(ftable)<br />
#<br />
# bandpass calibration<br />
default('bandpass')<br />
vis = measurementSet<br />
spw = spwStr # this is key to getting the calibration right!<br />
caltable = btable<br />
gaintable = ''<br />
gainfield = ''<br />
interp = ''<br />
field = ampCal<br />
selectdata = False<br />
opacity = 0.0<br />
bandtype = 'B'<br />
solint = 'inf'<br />
combine = 'scan'<br />
refant = refAnt<br />
bandpass()<br />
</source><br />
<br />
[[File:MRK6-shot2.png|thumb|Bandpass calibration curves. Note that the response curve is flat between channels 2 and 27.]]<br />
<br />
In order to perform the amplitude and phase calibrations, we'll need to inspect the bandpass response curve to select appropriate channels to average. We want to choose those channels where the response is relatively flat. The result of plotcal is shown at right: the bandpass is flat over channels 2:27. <br />
<br />
<source lang="python"><br />
# plot the bandpass calibration<br />
default('plotcal')<br />
spw = spwStr<br />
caltable = btable<br />
field = ampCal<br />
subplot = 211<br />
yaxis = 'amp'<br />
showgui = True<br />
plotcal()<br />
subplot = 212<br />
yaxis = 'phase'<br />
plotcal()<br />
</source><br />
<br />
<br />
The following steps generate amplitude and phase solutions for the calibrators, and the fluxscale is bootstrapped from the amplitude calibrator onto the phase calibrator, effectively turning the phase calibrator into a local flux calibrator for the science target.<br />
<br />
<source lang="python"><br />
default('gaincal')<br />
vis = measurementSet<br />
caltable = gtable<br />
gaintable = btable<br />
gainfield = ''<br />
interp = 'nearest'<br />
field = ampCal + ',' + phaseCal<br />
# specify data only from the flat region of the bandpass response<br />
spw = '10:2~27,11:2~27'<br />
selectdata = False<br />
opacity = 0.0<br />
gaintype = 'G'<br />
solint = 'inf'<br />
combine = ''<br />
calmode = 'ap'<br />
minsnr = 1.0<br />
refant = refAnt<br />
gaincal()<br />
#=====================================================================<br />
#<br />
# Bootstrap flux scale<br />
#<br />
vis = measurementSet<br />
# set the name for the output rescaled caltable<br />
fluxtable = ftable<br />
caltable = gtable<br />
reference = ampCal<br />
transfer = phaseCal<br />
fluxscale()<br />
</source><br />
<br />
The task {{fluxscale}} will give the bootstrapped flux density of the phase calibrator. If all goes well, something like the following should appear in the log window.<br />
<br />
<pre><br />
Beginning fluxscale--(MSSelection version)-------<br />
Found reference field(s): 0134+329<br />
Found transfer field(s): 1003+830<br />
Flux density for 1003+830 in SpW=10 (freq=1.39215e+09 Hz) is: 0.492631 +/- 0.00341773 (SNR = 144.14, N = 52)<br />
Flux density for 1003+830 in SpW=11 (freq=1.3943e+09 Hz) is: 0.489476 +/- 0.00286404 (SNR = 170.904, N = 52)<br />
Storing result in ab658.fluxscale<br />
</pre><br />
<br />
We should inspect the calibration solutions to make sure they are reasonable, but we'll be [http://en.wikipedia.org/wiki/Virginia_Cavaliers cavalier] and tread forward. Apply the calibrations to the science target and split that target into its own measurement set.<br />
<br />
<source lang="python"><br />
default('applycal')<br />
vis = measurementSet<br />
gaintable = [ftable,btable]<br />
gainfield = [phaseCal,'*']<br />
interp = ['linear','nearest']<br />
field = sourceName<br />
# choose the appropriate spectral windows<br />
spw = spwStr<br />
# by default, calibrations for a spectral window will map to the same<br />
spwmap = []<br />
selectdata = False<br />
opacity = 0.0<br />
applycal()<br />
#<br />
default('split')<br />
splitms = 'ab658-MKN6.split.ms'<br />
if os.path.isdir(splitms): rmtables(splitms)<br />
#<br />
vis = measurementSet<br />
outputvis = splitms<br />
field = sourceName<br />
# weed out the end channels<br />
spw = '10:2~27,11:2~27'<br />
datacolumn = 'corrected'<br />
split()<br />
</source><br />
<br />
== Continuum Subtraction ==<br />
<br />
Ideally, we would use {{plotms}} to identify the channels where HI absorption is present (see [[NGC 5921: red-shifted HI emission#Plot the Spectrum|the HI emission tutorial.]]) The absorption line is however weak and isolated against one of a number of radio continuum sources in the data; as such, the absorption is not apparent in averaged visibilities. Instead, we'll have to produce a dirty cube using {{clean}} and inspect the spectrum in the image domain.<br />
<br />
=== Search for the Absorption Line ===<br />
<br />
{{uvcontsub|Uvcontsub}} currently identifies continuum ranges by channel rather than frequency or velocity. We'll make two cubes, one for each spectral window.<br />
<br />
<source lang="python"><br />
default(clean)<br />
vis = splitms<br />
imagename = 'temp0'<br />
spw = '0:0~31'<br />
mode = 'channel'<br />
nchan = -1<br />
start = 0<br />
width = 1<br />
interpolation = 'nearest'<br />
niter = 2000<br />
cell = ['0.4arcsec', '0.4arcsec']<br />
phasecenter = 'J2000 06h52m12.2 +74d25m37'<br />
clean()<br />
#<br />
imagename='temp1'<br />
spw='1:0~31'<br />
clean()<br />
</source><br />
<br />
It's not ordinarily necessary to give the phase center, but this is a good opportunity to use the regridding capabilities of {{clean}}. The data were originally taken in B1950 coordinates, and the observations followed the good practice of offsetting the pointing center slightly away from the source coordinates. Here, we specify the J2000 coordinates (looked up on [http://nedwww.ipac.caltech.edu NED]) and allow {{clean}} to handle the precession and centering that we require.<br />
<br />
The output cubes will go to '''temp0.image''' and '''temp1.image'''.<br />
<br />
Now we can inspect the cube in {{viewer}}.<br />
<br />
<source lang="python"><br />
infile = 'temp0.image'<br />
displaytype = 'raster'<br />
viewer()<br />
</source><br />
<br />
[[File:MRK6-shot4-annotated.png|thumb|Spectral profile for MRK 6 in the 2nd spectral window, '''spw = 1'''. The location of the weak absorption line is marked by green arrow.]]<br />
<br />
Now, use {{viewer}} to generate a spectrum. The source spectrum in '''spw = 1''' is shown in the figure at right.<br />
<br />
* If necessary. use the player buttons [[File:VcrNext.png]] to scan through the channels until you see the source. <br />
* (Tools)'''Spectral Profile'''<br />
* Right-click and drag a box around the source; the spectrum will appear in the '''Image Profile''' window.<br />
* On the '''Image Profile''' window, change the velocity convention to '''Channel''' using the pull-down menu.<br />
* ''Optional:'' Save the numerical data to a file. Click the save-as-text button [[File:Save-as-text-file.png]] and write the data to a file; for this example, we'll use '''temp0.txt'''.<br />
<br />
Repeat for '''spw = 1'''.<br />
<br />
<source lang="python"><br />
infile = 'temp1.image'<br />
displaytype = 'raster'<br />
viewer()<br />
</source><br />
<br />
=== Perform the Continuum Subtraction ===<br />
<br />
Looking over the plots and text files, only channels 0~25 in either spectral window contain usable data. The absorption line is located in 1:15 and surrounding channels. Just to play it safe, we'll use the channels outside of 1:10~20 (spectral window 1, channels 10-20) to define the continuum for subtraction.<br />
<br />
<source lang="python"><br />
default(uvcontsub)<br />
vis = 'ab658-MKN6.split.ms'<br />
# remove old versions of the continuum subtracted data<br />
if os.path.exists(vis + '.contsub'): rmtables(vis+'.contsub')<br />
#<br />
fitspw = '0:0~25,1:0~9;21~25' # define the continuum windows for each spectral window<br />
solint = 'int'<br />
fitorder = 1 # fit linear baselines<br />
uvcontsub()<br />
</source><br />
<br />
{{uvcontsub|Uvcontsub}} produces the uv subtracted measurement set:<br />
<!-- * ab658-MKN6.split.ms.cont (continuum based on fit to linefree channels), & --><br />
* ab658-MKN6.split.ms.contsub (continuum-subtracted visibilities).<br />
<br />
== Imaging the Continuum Subtracted Cube ==<br />
<br />
[[File:MRK6-shot6.png|thumb|Viewer windows displaying the HI (21cm) absorption line cube of MRK6. The viewer window shows the channel containing the strongest absorption (at 5456 km/s), and the spectral profile window shows the average spectrum around the position of the strongest absorption.]]<br />
<br />
Inspecting the preliminary (continuum-containing) cubes, temp0.image and temp1.image, that we generated [[#Search for the Absorption Line|above]], we can determine good parameters for the velocity axis. Here are the velocity options I came up with.<br />
<br />
<source lang="python"><br />
default(clean)<br />
vis = 'ab658-MKN6.split.ms.contsub'<br />
spw=''<br />
imagename = 'MKN6-HIABS'<br />
mode = 'velocity'<br />
nchan = 48<br />
start = '5380km/s'<br />
# This will set the channel width equal to the first channel in the first spectral window<br />
width = ''<br />
interpolation = 'linear'<br />
outframe = 'bary'<br />
veltype = 'optical'<br />
restfreq = "1420.405752MHz"<br />
</source><br />
<br />
It also appears that the channel rms is about 0.5 mJy/beam; to avoid over-cleaning, we'll set the threshold at 1 mJy/beam.<br />
<br />
<source lang="python"><br />
niter = 2000<br />
threshold = '1.0mJy'<br />
weighting = 'natural'<br />
</source><br />
<br />
Set the source coordinates. We'll use defaults for the rest of the parameters; with the continuum subtracted, the data have only simple structure with no confusing sources. <br />
<br />
<source lang="python"><br />
# VLA-A; expect 1.5--2 arcsec beams. 0.4arcsec pixels ensure sampling the beam by better than 3x<br />
cell = ['0.4arcsec', '0.4arcsec']<br />
phasecenter = 'J2000 06h52m12.2 +74d25m37'<br />
clean()<br />
</source><br />
<br />
The ({{viewer}}) results are illustrated in the figure at upper right.<br />
<br />
=== Imaging the Continuum Dataset ===<br />
<br />
[[File:MRK6-shot7.png|thumb|21cm continuum image of MRK 6. The image has been scaled, using (Data)(Adjust)(Basic Settings)'''Scaling Power Cycles''' = -1.9, to bring out artifacts apparent in the background. The 6-pointed star pattern around the source is symptomatic of residual phase-errors, which can be healed using [[Calibrating a VLA 5 GHz continuum survey#Self-calibration|self-calibration]], and the arcuate stripes across the field are sidelobes of neighboring sources that need to be cleaned using [[Imaging Flanking Fields|flanking fields]].]]<br />
<br />
We can similarly image the continuum data with just a few changes of the {{clean}} parameters. Here, we call clean by selecting the spectral windows and channels pertaining to only the continuum.<br />
<br />
<source lang="python"><br />
tget clean<br />
vis = 'ab658-MKN6.split.ms'<br />
mode = 'mfs'<br />
spw = '0:0~25,1:0~9;21~25'<br />
imagename = 'MKN6-CONT'<br />
clean()<br />
</source><br />
<br />
The result is shown at right. The continuum image needs some more help. Phase errors produce the 6-pointed star pattern surrounding the source and can be healed using [[Calibrating a VLA 5 GHz continuum survey#Self-calibration|self-calibration]]. The striping across the field is actually the combination of sidelobes from neighboring sources; these sidelobes can be suppressed by cleaning [[Imaging Flanking Fields|flanking fields]].<br />
<br />
== Extract the Absorption Spectrum ==<br />
<br />
{{viewer|Viewer}} provides a handy first-look at the spectrum, but the extracted spectrum is an average over a spatial box. What we really want is the ''integrated'' spectrum, not the average. To do this, we'll use CASA's {{imstat}}, {{imval}}, and {{imhead}} tools to extract cube information to python variables and then let python build up the spectrum.<br />
<br />
=== Extract the Spectrum: load the cube data into python arrays ===<br />
<br />
First, we need to determine where in the cube to extract the spectrum; equivalently, we need to find the location of the minimum value in the cube. {{imstat|Imstat}} is the tool for this.<br />
<br />
<source lang="python"><br />
cubeStat=imstat("MKN6-HIABS.image")<br />
</source><br />
<br />
The variable <tt>cubeStat</tt> is now a [http://docs.python.org/tutorial/datastructures.html#dictionaries python dictionary]. To see what it contains, just enter the variable name:<br />
<br />
<source lang="python"><br />
cubeStat<br />
</source><br />
<br />
and the following summary of <tt>cubeStat</tt> appears.<br />
<br />
<pre><br />
{'blc': array([0, 0, 0, 0], dtype=int32),<br />
'blcf': '06:52:24.903, +74.24.45.777, I, 1.39522e+09Hz',<br />
'flux': array([-0.05243921]),<br />
'max': array([ 0.00279196]),<br />
'maxpos': array([173, 82, 0, 33], dtype=int32),<br />
'maxposf': '06:52:07.732, +74.25.18.597, I, 1.39199e+09Hz',<br />
'mean': array([ -3.96712512e-07]),<br />
'medabsdevmed': array([ 0.00029433]),<br />
'median': array([ 0.]),<br />
'min': array([-0.01497807]),<br />
'minpos': array([127, 131, 0, 7], dtype=int32),<br />
'minposf': '06:52:12.299, +74.25.38.200, I, 1.39453e+09Hz',<br />
'npts': array([ 3145728.]),<br />
'quartile': array([ 0.00058867]),<br />
'rms': array([ 0.00047121]),<br />
'sigma': array([ 0.00047121]),<br />
'sum': array([-1.24794966]),<br />
'sumsq': array([ 0.69847239]),<br />
'trc': array([255, 255, 0, 47], dtype=int32),<br />
'trcf': '06:51:59.574, +74.26.27.778, I, 1.39063e+09Hz'}<br />
</pre><br />
<br />
These entries can be accessed individually like an array variable. For example, the following commands copy the ''x'' and ''y'' positions of the absorption line to scalar variables. <br />
<br />
<source lang=python><br />
x2extract = cubeStat['minpos'][0]<br />
y2extract = cubeStat['minpos'][1]<br />
</source><br />
<br />
Now, extract the spectrum using {{imval}}.<br />
<br />
<source lang=python><br />
extractBox = "%d,%d" % (x2extract, y2extract) # copy the position to a string<br />
cubeSpec = imval("MKN6-HIABS.image", box=extractBox, stokes='I')<br />
</source><br />
<br />
<tt>cubeSpec</tt> is another [http://docs.python.org/tutorial/datastructures.html#dictionaries python dictionary], and the spectrum is stored in <tt>cubeSpec['data']</tt>.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Comment:''' The absorption line source is unresolved by this observation, or, to put it another way, is entirely contained within a synthesized beam. The extraction at a single spatial pixel gives the integrated spectrum and therefore changes units from surface brightness (mJy/beam) to flux density (mJy). However, the data have been forced into a rectangular grid, and there's no guarantee that the absorption minimum centers within a pixel. Better to use {{imfit}} to determine more accurately the position of the minimum, and perhaps re-{{clean}} with a new phasecenter. To avoid complicating the present tutorial, we'll just stick with the minimum in the current {{clean}} grid.<br />
</div><br />
<br />
=== Generate the frequency axis ===<br />
<br />
We can derive the frequency axis from the header.<br />
<br />
<source lang="python"><br />
import numpy # make sure vector and array arithmetic options are loaded<br />
cubeHead = imhead("MKN6-HIABS.image", mode="list")<br />
nSpec = cubeStat['trc'][3] + 1 # get the number of frequency channels. <br />
f0 = float(cubeHead['crval4']) # reference freq in Hz<br />
df = float(cubeHead['cdelt4']) # channel width in Hz<br />
i0 = cubeHead['crpix4'] # reference pixel<br />
freqSpec = (numpy.arange(nSpec) - i0)*df + f0<br />
</source><br />
<br />
=== Plot the spectrum using matplotlib ===<br />
<br />
[[File:MRK6-shot8.png|thumb|Absorption spectrum of MRK 6 in frequency space.]]<br />
<br />
Now we have the spectrum stored in <tt>freqSpec</tt>, <tt>cubeSpec['data']</tt>. Use [http://matplotlib.sourceforge.net/api/pyplot_api.html matplotlib.pyplot] to plot the spectrum (see figure at right).<br />
<br />
<source lang="python"><br />
import matplotlib.pyplot as plt<br />
plt.clf() # clear the plot (figure)<br />
plt.plot(freqSpec/1.e9, cubeSpec['data'], 'k-')<br />
plt.xlabel("Freq (GHz)")<br />
plt.ylabel("Flux Density (mJy)")<br />
plt.show()<br />
</source><br />
<br />
=== A publication-quality figure: plot the spectrum against velocity ===<br />
<br />
Nice, now let's make a velocity axis instead. We'll use the optical convention to convert frequency to velocity; the radio convention is offered as an alternative but is commented out. <br />
<br />
<source lang="python"><br />
fRest = cubeHead['restfreq'][0]<br />
velocity = 299792.458 * (fRest/freqSpec - 1.0) # optical convention, km/s<br />
# velocity = 299792.458 * (1.0 - freqSpec/fRest) # radio convention, km/s<br />
</source><br />
<br />
[[File:MRK6-shot9.png|thumb|Publication quality spectrum of MRK 6.]]<br />
<br />
Plot the spectrum again, but this time try for publication-ready quality (see figure at right). This example is somewhat more clever to highlight the use of TeX typography in <tt>matplotlib</tt>. Note that you will have to have a compatible version of LaTeX installed for this to be successful.<br />
<br />
<source lang="python"><br />
# set up fonts<br />
from matplotlib import rc<br />
rc('font',**{'family':'serif','serif':['Times']})<br />
rc('text', usetex=True)<br />
plt.clf() # clear the plot (figure)<br />
plt.plot(velocity, cubeSpec['data'], 'k-')<br />
# the r prefix to the following format strings prevents \ from triggering an escape sequence, <br />
# like \n for line feed<br />
plt.xlabel(r"$cz$ (km s$^{-1}$)",fontsize=16) <br />
plt.ylabel(r"$S_{\nu}$ (mJy)",fontsize=16)<br />
plt.ylim(-0.023,0.005)<br />
plt.savefig("mkn6-abs.eps") # save the figure as encapsulated postscript, suitable for journals<br />
plt.savefig("mkn6-abs.png") # save the figure as a bitmap, better suited for astro-ph<br />
</source><br />
<br />
=== Save the spectrum to a text file ===<br />
<br />
It's handy to have a simple, columnar text file to inspect and analyze later. Or perhaps you might want to analyze the spectrum outside of CASA. Numpy makes it easy to save and restore text files.<br />
<br />
<source lang="python"><br />
# First, combine the spectrum into a single array<br />
spectrum = numpy.vstack((velocity, cubeSpec['data']))<br />
numpy.savetxt("mrk6.txt", numpy.transpose(spectrum))<br />
</source><br />
<br />
Here's a listing of the first few lines of ''mrk6.txt''. <br />
<br />
<pre><br />
5.379999999489542461e+03 -5.940478877164423466e-04<br />
5.399939886933506386e+03 -3.919676819350570440e-04<br />
5.419882380281690530e+03 -3.093948180321604013e-04<br />
5.439827480044931690e+03 -1.092202146537601948e-03<br />
5.459775186734200361e+03 -4.310846794396638870e-03<br />
5.479725500860732609e+03 -3.458642400801181793e-03<br />
5.499678422935699928e+03 -7.933050510473549366e-04<br />
5.519633953470603956e+03 -2.004600130021572113e-03<br />
5.539592092976883578e+03 -8.163747377693653107e-03<br />
5.559552841966238702e+03 -1.676503755152225494e-02<br />
</pre><br />
<br />
Here's how to restore that text file back into CASA (or python, for that matter).<br />
<br />
<source lang="python"><br />
# import numpy # if you haven't already<br />
spectrum = numpy.loadtxt("mrk6.txt",dtype="float")<br />
x = spectrum[:,0]<br />
y = spectrum[:,1]<br />
</source><br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=NGC_5921:_red-shifted_HI_emission_5.7.2&diff=21906NGC 5921: red-shifted HI emission 5.7.22017-06-23T21:04:55Z<p>Emomjian: </p>
<hr />
<div>Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
<br />
[[Category: VLA]] [[Category: Calibration]] [[Category: Imaging]] [[Category: Spectral Line]]<br />
<br />
== Overview ==<br />
<br />
The technique used to calibrate and image continuum datasets generally applies to spectral line observations, except that an additional calibration step is required. '''Bandpass calibration''' flattens the spectral response of the observations, ensuring that spectral channel images are properly calibrated in amplitude and phase.<br />
<br />
The following tutorial derives from an annotated script provided in the [http://casa.nrao.edu/ref_cookbook.shtml CASA Cookbook]. The script is largely reproduced and additionally annotated with figures and illustrations. It is assumed that this tutorial will be used interactively, and so interactive pauses in the original script have been removed.<br />
<br />
The data are included with the CASA installation.<br />
<br />
== Setting up the CASA Environment ==<br />
<br />
Start up CASA in the directory you want to use. <br />
<br />
<source lang="bash"><br />
# in bash<br />
mkdir NGC5921<br />
cd NGC5921<br />
casa<br />
</source><br />
<br />
Now, in CASA, set up paths and global variables to facilitate the data reduction. These operations can be performed on-the-fly if you are reducing data interactively, but it's better to have them prepared from the start in the scripting environment.<br />
<br />
<source lang="python"><br />
import os<br />
scriptmode = True<br />
prefix = 'ngc5921.demo' # The prefix to use for all output files<br />
# Set up some useful variables (these will be altered later on)<br />
msfile = prefix + '.ms'<br />
btable = prefix + '.bcal'<br />
gtable = prefix + '.gcal'<br />
ftable = prefix + '.fluxscale'<br />
splitms = prefix + '.src.split.ms'<br />
imname = prefix + '.cleanimg'<br />
</source><br />
<br />
We'll use a python ''os'' command to get the appropriate CASA path for your installation. The use of '''os.environ.get''' is explained in [[#os.environ.get | the Appendix]].<br />
<br />
<source lang="python"><br />
pathname=os.environ.get('CASAPATH').split()[0]<br />
fitsdata=pathname+'/data/demo/NGC5921.fits'<br />
</source><br />
<br />
Scripts are of course modified and repeated to the satisfaction of observer. To help clean up the bookkeeping and further avoid issues of write privileges, remove prior versions of the measurement set and calibration tables.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' The first command in the following code block removes the measurement set. Depending on the size of the source data, refilling a measurement set can be time-consuming, and so you may want to consider editing a separate script that preserves the measurement set but skips the '''importuvfits''' command [[#Import the Data | given below]]. This NGC5921 dataset, however, is not large and refilling goes quickly.<br />
</div><br />
<br />
<source lang="python"><br />
rmtables(msfile)<br />
rmtables(btable)<br />
rmtables(gtable)<br />
rmtables(ftable)<br />
rmtables(ftable)<br />
rmtables(splitms+'*')<br />
rmtables(imname+'.*')<br />
rmtables(prefix+'.moments*')<br />
# Final clean up of auxiliary files<br />
os.system('rm -rf '+prefix+'*')<br />
</source><br />
<br />
== Import the Data ==<br />
<br />
The next step is to import the multisource UVFITS data to a CASA measurement set via the {{importuvfits}} filler. <br />
<br />
<source lang="python"><br />
# Safest to start from task defaults<br />
default('importuvfits')<br />
# Set up the MS filename and save as new global variable<br />
msfile = prefix + '.ms'<br />
# Use task importuvfits<br />
fitsfile = fitsdata<br />
vis = msfile<br />
saveinputs('importuvfits',prefix+'.importuvfits.saved')<br />
importuvfits()<br />
</source><br />
<br />
[[saveinputs | Saveinputs]] saves the parameters of a given command to specified text file, handy to debug a script and see what actually was run. The parameters of {{importuvfits}} are saved to the file "ngc5921.demo.importuvfits.saved". A listing of this file follows. Notice that it is executable with '''execfile''' in CASA (remove the # commenting symbol before {{importuvfits}} to have the execfile run the command).<br />
<br />
<pre><br />
CASA <71>: os.system('cat ngc5921.demo.importuvfits.saved')<br />
taskname = "importuvfits"<br />
fitsfile = "/usr/lib64/casapy/30.0.9709test-001/data/demo/NGC5921.fits"<br />
vis = "ngc5921.demo.ms"<br />
antnamescheme = "old"<br />
#importuvfits(fitsfile="/usr/lib64/casapy/30.0.9709test-001/data/demo/NGC5921.fits",vis="ngc5921.demo.ms",antnamescheme="old")<br />
<br />
</pre><br />
<br />
== A Summary of the Data ==<br />
<br />
We'll need to have a look at the observing tables to learn the calibrator and source names. The relevant command is {{listobs}}. <br />
<br />
[[File:n5921-01.png | thumb | Logger output of {{listobs}}.]]<br />
<br />
<source lang="python"><br />
listobs(vis='ngc5921.demo.ms', verbose=True)<br />
</source><br />
<br />
The output goes to the logger window; see the screenshot at right.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' You can control the text size of the logger window using <ctrl>-A (smaller font) and <ctrl>-L (larger font) in Linux (<Command>-A and <Command>-L on MacOS X).<br />
</div><br />
<br />
A more complete listing of the {{listobs}} output follows.<br />
<br />
<pre><br />
2011-04-25 19:52:17 INFO listobs ##########################################<br />
2011-04-25 19:52:17 INFO listobs ##### Begin Task: listobs #####<br />
2011-04-25 19:52:17 INFO listobs::::casa<br />
2011-04-25 19:52:17 INFO listobs ================================================================================<br />
2011-04-25 19:52:17 INFO listobs MeasurementSet Name: /Science/VLA_tutorial_check/NGC_5921_red-shifted_HI/ngc5921.demo.ms MS Version 2<br />
2011-04-25 19:52:17 INFO listobs ================================================================================<br />
2011-04-25 19:52:17 INFO listobs Observer: TEST Project: <br />
2011-04-25 19:52:17 INFO listobs Observation: VLA<br />
2011-04-25 19:52:17 INFO listobs Data records: 22653 Total integration time = 5280 seconds<br />
2011-04-25 19:52:17 INFO listobs Observed from 13-Apr-1995/09:19:00.0 to 13-Apr-1995/10:47:00.0 (TAI)<br />
2011-04-25 19:52:17 INFO listobs::ms::summary<br />
2011-04-25 19:52:17 INFO listobs ObservationID = 0 ArrayID = 0<br />
2011-04-25 19:52:17 INFO listobs Date Timerange (TAI) Scan FldId FieldName nVis Int(s) SpwIds ScanIntent<br />
2011-04-25 19:52:17 INFO listobs 13-Apr-1995/09:19:00.0 - 09:24:30.0 1 0 1331+305000* 4509 30 [0] <br />
2011-04-25 19:52:17 INFO listobs 09:27:30.0 - 09:29:30.0 2 1 1445+099000* 1890 30 [0] <br />
2011-04-25 19:52:17 INFO listobs 09:33:00.0 - 09:48:00.0 3 2 N5921_2 6048 30 [0] <br />
2011-04-25 19:52:17 INFO listobs 09:50:30.0 - 09:51:00.0 4 1 1445+099000* 756 30 [0] <br />
2011-04-25 19:52:17 INFO listobs 10:22:00.0 - 10:23:00.0 5 1 1445+099000* 1134 30 [0] <br />
2011-04-25 19:52:17 INFO listobs 10:26:00.0 - 10:43:00.0 6 2 N5921_2 6804 30 [0] <br />
2011-04-25 19:52:17 INFO listobs 10:45:30.0 - 10:47:00.0 7 1 1445+099000* 1512 30 [0] <br />
2011-04-25 19:52:17 INFO listobs (nVis = Total number of time/baseline visibilities per scan) <br />
2011-04-25 19:52:17 INFO listobs Fields: 3<br />
2011-04-25 19:52:17 INFO listobs ID Code Name RA Decl Epoch SrcId nVis <br />
2011-04-25 19:52:17 INFO listobs 0 C 1331+305000* 13:31:08.2873 +30.30.32.9590 J2000 0 4509 <br />
2011-04-25 19:52:17 INFO listobs 1 A 1445+099000* 14:45:16.4656 +09.58.36.0730 J2000 1 5292 <br />
2011-04-25 19:52:17 INFO listobs 2 N5921_2 15:22:00.0000 +05.04.00.0000 J2000 2 12852 <br />
2011-04-25 19:52:17 INFO listobs (nVis = Total number of time/baseline visibilities per field) <br />
2011-04-25 19:52:17 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-04-25 19:52:17 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-04-25 19:52:17 INFO listobs 0 63 LSRK 1412.66507 24.4140625 1550.19688 1413.42801 RR LL <br />
2011-04-25 19:52:17 INFO listobs Sources: 3<br />
2011-04-25 19:52:17 INFO listobs ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
2011-04-25 19:52:17 INFO listobs 0 1331+305000* 0 1420.405752 0 <br />
2011-04-25 19:52:17 INFO listobs 1 1445+099000* 0 1420.405752 0 <br />
2011-04-25 19:52:17 INFO listobs 2 N5921_2 0 1420.405752 0 <br />
2011-04-25 19:52:17 INFO listobs Antennas: 27:<br />
2011-04-25 19:52:17 INFO listobs ID Name Station Diam. Long. Lat. <br />
2011-04-25 19:52:17 INFO listobs 0 1 VLA:N7 25.0 m -107.37.07.2 +33.54.12.9 <br />
2011-04-25 19:52:17 INFO listobs 1 2 VLA:W1 25.0 m -107.37.05.9 +33.54.00.5 <br />
2011-04-25 19:52:17 INFO listobs 2 3 VLA:W2 25.0 m -107.37.07.4 +33.54.00.9 <br />
2011-04-25 19:52:17 INFO listobs 3 4 VLA:E1 25.0 m -107.37.05.7 +33.53.59.2 <br />
2011-04-25 19:52:17 INFO listobs 4 5 VLA:E3 25.0 m -107.37.02.8 +33.54.00.5 <br />
2011-04-25 19:52:17 INFO listobs 5 6 VLA:E9 25.0 m -107.36.45.1 +33.53.53.6 <br />
2011-04-25 19:52:17 INFO listobs 6 7 VLA:E6 25.0 m -107.36.55.6 +33.53.57.7 <br />
2011-04-25 19:52:17 INFO listobs 7 8 VLA:W8 25.0 m -107.37.21.6 +33.53.53.0 <br />
2011-04-25 19:52:17 INFO listobs 8 9 VLA:N5 25.0 m -107.37.06.7 +33.54.08.0 <br />
2011-04-25 19:52:17 INFO listobs 9 10 VLA:W3 25.0 m -107.37.08.9 +33.54.00.1 <br />
2011-04-25 19:52:17 INFO listobs 10 11 VLA:N4 25.0 m -107.37.06.5 +33.54.06.1 <br />
2011-04-25 19:52:17 INFO listobs 11 12 VLA:W5 25.0 m -107.37.13.0 +33.53.57.8 <br />
2011-04-25 19:52:17 INFO listobs 12 13 VLA:N3 25.0 m -107.37.06.3 +33.54.04.8 <br />
2011-04-25 19:52:17 INFO listobs 13 14 VLA:N1 25.0 m -107.37.06.0 +33.54.01.8 <br />
2011-04-25 19:52:17 INFO listobs 14 15 VLA:N2 25.0 m -107.37.06.2 +33.54.03.5 <br />
2011-04-25 19:52:17 INFO listobs 15 16 VLA:E7 25.0 m -107.36.52.4 +33.53.56.5 <br />
2011-04-25 19:52:17 INFO listobs 16 17 VLA:E8 25.0 m -107.36.48.9 +33.53.55.1 <br />
2011-04-25 19:52:17 INFO listobs 17 18 VLA:W4 25.0 m -107.37.10.8 +33.53.59.1 <br />
2011-04-25 19:52:17 INFO listobs 18 19 VLA:E5 25.0 m -107.36.58.4 +33.53.58.8 <br />
2011-04-25 19:52:17 INFO listobs 19 20 VLA:W9 25.0 m -107.37.25.1 +33.53.51.0 <br />
2011-04-25 19:52:17 INFO listobs 20 21 VLA:W6 25.0 m -107.37.15.6 +33.53.56.4 <br />
2011-04-25 19:52:17 INFO listobs 21 22 VLA:E4 25.0 m -107.37.00.8 +33.53.59.7 <br />
2011-04-25 19:52:17 INFO listobs 23 24 VLA:E2 25.0 m -107.37.04.4 +33.54.01.1 <br />
2011-04-25 19:52:17 INFO listobs 24 25 VLA:N6 25.0 m -107.37.06.9 +33.54.10.3 <br />
2011-04-25 19:52:17 INFO listobs 25 26 VLA:N9 25.0 m -107.37.07.8 +33.54.19.0 <br />
2011-04-25 19:52:17 INFO listobs 26 27 VLA:N8 25.0 m -107.37.07.5 +33.54.15.8 <br />
2011-04-25 19:52:17 INFO listobs 27 28 VLA:W7 25.0 m -107.37.18.4 +33.53.54.8 <br />
2011-04-25 19:52:17 INFO listobs::::casa<br />
2011-04-25 19:52:17 INFO listobs ##### End Task: listobs #####<br />
2011-04-25 19:52:17 INFO listobs ##########################################<br />
</pre><br />
<br />
=== Key Information from {{listobs | Listobs}} ===<br />
<br />
Certainly the output of {{listobs}} is dense with information, but there are some particularly vital data that we'll need for the calibration.<br />
<br />
* The calibrators are '''1331+305*''' (3C286, the flux and bandpass calibrator) and '''1445+099*''' (the phase calibrator). We can use wild-cards '''1331*''' and '''1445*''' since they uniquely identify the sources.<br />
* The calibrator field indices are field='0' (1331+305) and field='1' (1445+099).<br />
* The name of the source in the observations list is '''N5921_2''', or field = '2'.<br />
* The data were taken in a single IF (a single spectral window, SpwID = 0), divided into 63 channels.<br />
* Only RR and LL correlations are present; cross-pols are absent.<br />
<br />
== Flagging ==<br />
<br />
=== Flag the autocorrelations ===<br />
<br />
We don't need the autocorrelation data, and we can use {{flagdata}} to get rid of them. You shouldn't have to specify the measurement set, because the variable ''vis'' is already set, but it never hurts to be cautious.<br />
<br />
<source lang="python"><br />
flagdata(vis="ngc5921.demo.ms", autocorr=True)<br />
</source><br />
<br />
=== Interactive Flagging ===<br />
<br />
[[file:spectrum-flagging.png|thumb|Plotms settings for flagging spectral line data. Click to enlarge.]]<br />
<br />
<br />
{{plotms | Plotms}} is a good tool for flagging spectral line data. Check out the [[Data flagging with casaplotms | tutorial]] that describes editing VLA continuum data. Spectral line data of course require some consideration of channels and channel averaging. <br />
<br />
<source lang="python"><br />
plotms()<br />
</source><br />
<br />
<br />
The figure at right highlights the settings needed for effective editing of a spectral line data set. The key settings are as follows.<br />
<br />
* Specify the measurement set in '''File Location'''; the '''Browse''' button allows you to hunt down the measurement set.<br />
* It's better to edit one source at a time. In the illustrated example, the flux / bandpass calibrator 1331+305* is displayed.<br />
* Average the channels. First, specify the central channels to remove band edge effects. Channels 6~56 in the first spectral window (IF) are appropriate (see [[#Inspect the Bandpass Response Curve]], below). In the '''Channel Averaging''' box, enter 51 channels to average over all channels in the given range.<br />
* Ideally you want the channels to have the same (''u'', ''v'') coverage (projected baseline spacings as viewed from the source); otherwise, the beam (point spread function) will be different for each channel. Therefore, if you flag data from a given channel it's usually a good idea to flag those data from all channels. Under the '''Flagging''' tab, specify '''Extend flags''' to '''Channel.'''<br />
<br />
<br />
With these settings, interactive flagging proceeds as for [[Data flagging with plotms | continuum data]]. When you're satisfied with the edits, '''File &rarr; Quit''' to return to the CASA prompt.<br />
<br />
== Calibration ==<br />
<br />
Calibration of spectral line data broadly follows the approach for [[Calibrating a VLA 5 GHz continuum survey | continuum data]], except that the amplitude and phase corrections are a function of frequency and so must be corrected by '''bandpass calibration.''' The basic calibration steps follow.<br />
* [[#Setting the Flux Scale | Set the flux scale]] of the primary calibrator, here, 1331+305 = 3C 286.<br />
* [[#Bandpass Calibration | Determine bandpass corrections]] based on the primary calibrator. In the script that follows, the bandpass calibration is stored in ''ngc5921.demo.bcal'', which is itself referenced by the python variable ''btable.''<br />
* [[#Inspect the Bandpass Response Curve | Inspect the bandpass correction]] to determine viable channels for averaging and imaging. We want to toss out end channels where the response is poor.<br />
* [[#Gain Calibration | Determine the gain calibrations]] on the bandpass-corrected and channel-averaged data. In this step, we effectively turn the spectral line data into a single-channel continuum data set and calibrate accordingly. The calibration is stored in ''ngc5921.demo.gcal'', which is itself referenced by the python variable ''gtable.''<br />
* [[#Inspect the Calibration Solutions | Inspect the gain calibration solutions]] to look for any aberrant solutions that hint at bad calibrator data.<br />
* [[#Apply the Solutions | Apply the calibration solutions]] to the source (N5921_2). This action literally adds a new column of data to the measurement set. This new column contains the data with the gain calibration and bandpass calibration applied, but it does not overwrite the raw data in case the calibration needs revision.<br />
<br />
<br />
<br />
=== Setting the Flux Scale ===<br />
<br />
{{setjy | Setjy}} generates a point source model for the primary calibrator, 1331+305 = 3C286. These data are of low angular resolution, and so the point source model is adequate for our purposes. For observations with higher angular resolution (longer baseline configurations), you can specify a model of the calibrator using the ''modimage'' parameter (see the tutorial [[Calibrating a VLA 5 GHz continuum survey#Set the Flux Scale]] for an example of how to use ''modimage''). <br />
<br />
{{setjy | Setjy}} also looks up the radio SED for common flux calibrators and automatically assigns the total flux density.<br />
<br />
<source lang="python"><br />
default('setjy')<br />
vis = msfile<br />
#<br />
# 1331+305 = 3C286 is our primary calibrator<br />
# Use the wildcard on the end of the source name<br />
# since the field names in the MS have inherited the<br />
# AIPS qualifiers<br />
field = '1331+305*'<br />
# This is 1.4GHz D-config and 1331+305 is sufficiently unresolved<br />
# that we dont need a model image. For higher frequencies<br />
# (particularly in A and B config) you would want to use one.<br />
modimage = ''<br />
# Setjy knows about this source so we dont need anything more<br />
saveinputs('setjy',prefix+'.setjy.saved')<br />
setjy()<br />
</source><br />
<br />
A summary of the operation is sent to the logger window. Here's a listing of the output.<br />
<br />
<pre><br />
2011-04-25 19:50:53 INFO setjy ##########################################<br />
2011-04-25 19:50:53 INFO setjy ##### Begin Task: setjy #####<br />
2011-04-25 19:50:53 INFO setjy::::casa<br />
2011-04-25 19:50:53 INFO setjy Adding MODEL_DATA and CORRECTED_DATA columns<br />
2011-04-25 19:50:53 INFO setjy Initializing MODEL_DATA (to unity) and CORRECTED_DATA (to DATA)<br />
2011-04-25 19:50:54 INFO setjy Initialized 22653 rows.<br />
2011-04-25 19:50:54 INFO setjy 1331+30500002_0 spwid= 0 [I=14.79, Q=0, U=0, V=0] Jy, (Perley-Butler 2010)<br />
2011-04-25 19:50:54 INFO setjy Selecting data<br />
2011-04-25 19:50:54 INFO setjy Selected 4509 out of 22653 visibilities.<br />
2011-04-25 19:50:54 INFO setjy Selecting 63 channels, starting at 0, stepped by 1, for spw 0<br />
2011-04-25 19:50:54 INFO setjy Fourier transforming: replacing MODEL_DATA column<br />
2011-04-25 19:50:54 INFO setjy Processing after subtracting componentlist .setjy_1331+30500002_0_1.41342GHz49820d.cl<br />
2011-04-25 19:50:54 INFO setjy Performing interferometric gridding...<br />
2011-04-25 19:50:55 INFO setjy::::casa<br />
2011-04-25 19:50:55 INFO setjy ##### End Task: setjy #####<br />
2011-04-25 19:50:55 INFO setjy ##########################################<br />
</pre><br />
<br />
=== Bandpass Calibration ===<br />
<br />
The flux calibrator 1331+305 = 3C 286 now has a point-source model assigned to it. Since the point-source model doesn't change over this narrow range of frequencies, we can use the model to determine amplitude and phase (gain) corrections for each channel independently. The result is the bandpass calibration.<br />
<br />
As for any antenna-based calibration scheme, we have to pick an antenna to act as the reference point for the calibration. Any antenna will do, but it's better to pick one near the center of the array. For the remainder of the calibration, we will use '''refant = '15''''.<br />
<br />
<source lang="python"><br />
default('bandpass')<br />
# We can first do the bandpass on the single 5min scan on 1331+305<br />
# At 1.4GHz phase stablility should be sufficient to do this without<br />
# a first (rough) gain calibration. This will give us the relative<br />
# antenna gain as a function of frequency.<br />
vis = msfile<br />
# set the name for the output bandpass caltable<br />
btable = prefix + '.bcal'<br />
caltable = btable<br />
# No gain tables yet<br />
gaintable = ''<br />
gainfield = ''<br />
interp = ''<br />
# Use flux calibrator 1331+305 = 3C286 (FIELD_ID 0) as bandpass calibrator<br />
field = '0'<br />
# all channels<br />
spw = ''<br />
# No other selection<br />
selectdata = False<br />
# In this band we do not need a-priori corrections for<br />
# antenna gain-elevation curve or atmospheric opacity<br />
# (at 8GHz and above you would want these)<br />
opacity = 0.0<br />
# Choose bandpass solution type<br />
# Pick standard time-binned B (rather than BPOLY)<br />
bandtype = 'B'<br />
# set solution interval arbitrarily long (get single bpass)<br />
solint = 'inf'<br />
combine = 'scan'<br />
# reference antenna Name 15 (15=VLA:N2) (Id 14)<br />
refant = '15'<br />
saveinputs('bandpass',prefix+'.bandpass.saved')<br />
bandpass()<br />
</source><br />
<br />
=== Inspect the Bandpass Response Curve ===<br />
<br />
[[File:ngc5921-bandpass.png | thumb | Bandpass response curves generated by {{plotcal}}. The solutions for different antennas are indicated by differently colored plotting symbols. Plots for individual antennas can be generated by setting '''iteration = 'antenna'''' for {{plotcal}}.]]<br />
<br />
In the [[#Gain Calibration | gain calibration]] to follow, we will effectively convert the spectral line data into a continuum data set. Before proceeding, we need to inspect the bandpass calibration to make sure that it contains no bad values and also to inspect which channels to average to produce the continuum data. {{plotcal | Plotcal}} is the standard tool for plotting calibration solutions. The following script produces the figure at right. <br />
<br />
By inspection, the amplitude response curve is flat over channels 6~56; that channel range will be used to generate the continuum data for gain calibration.<br />
<br />
Notice that {{plotcal}} is run twice: once to display gain amplitudes as a function of channel (frequency), and again to plot gain phases as a function of channel.<br />
<br />
If the '''scriptmode''' is set to '''False''', the plot is saved to ngc5921.demo.plotcal.png.<br />
<br />
<source lang="python"><br />
default('plotcal')<br />
caltable = btable<br />
field = '0'<br />
# Set up 2x1 panels - upper panel amp vs. channel<br />
subplot = 211<br />
yaxis = 'amp'<br />
saveinputs('plotcal',prefix+'.plotcal.b.amp.saved')<br />
showgui = True<br />
plotcal()<br />
#<br />
# Set up 2x1 panels - lower panel phase vs. channel<br />
subplot = 212<br />
yaxis = 'phase'<br />
saveinputs('plotcal',prefix+'.plotcal.b.phase.saved')<br />
#<br />
# Note the rolloff in the start and end channels. Looks like<br />
# channels 6-56 (out of 0-62) are the best<br />
# If you want to do this interactively and iterate over antenna, set<br />
# iteration = 'antenna'<br />
plotcal()<br />
</source><br />
<br />
=== Gain Calibration ===<br />
<br />
From [[#Inspect the Bandpass Response Curve | inspection of the bandpass response curve]], we can average channels 6~56 to produce continuum data for the calibrators. For VLA data, this averaging is specified through the '''spw''' (spectral window) parameter, which takes the form IF:Channel-range, as follows.<br />
<br />
<source lang="python"><br />
spw = '0:6~56'<br />
</source><br />
<br />
That is, there is only one spectral window (IF), '''spw = 0''', and we want to average channels '''6~56''' within that spectral window.<br />
<br />
Gain calibrations are otherwise determined as for [[Calibrating a VLA 5 GHz continuum survey#Calibration | continuum data]]. <br />
* {{gaincal}}() is run only on the calibrators, 1331+305 (flux calibrator) and 1445+099 (phase calibrator).<br />
* The default model for gain calibrations is a 1 Jy point-source. The flux scale is overridden by {{setjy}}, which has been performed for the flux calibrator. We need to transfer that flux scale to the phase calibrator using {{fluxscale}}(). <br />
* Note that fluxscale() determines the flux density of the phase calibrator and accordingly adjusts its model and calibration solutions. A report of the results are sent to the logger window.<br />
<br />
<source lang="python"><br />
default('gaincal')<br />
# Armed with the bandpass, we now solve for the<br />
# time-dependent antenna gains<br />
vis = msfile<br />
# set the name for the output gain caltable<br />
gtable = prefix + '.gcal'<br />
caltable = gtable<br />
# Use our previously determined bandpass<br />
# Note this will automatically be applied to all sources<br />
# not just the one used to determine the bandpass<br />
gaintable = btable<br />
gainfield = ''<br />
# Use nearest (there is only one bandpass entry)<br />
interp = 'nearest'<br />
# Gain calibrators are 1331+305 and 1445+099 (FIELD_ID 0 and 1)<br />
field = '0,1'<br />
# We have only a single spectral window (SPW 0)<br />
# Choose 51 channels 6-56 out of the 63<br />
# to avoid end effects.<br />
# Channel selection is done inside spw<br />
spw = '0:6~56'<br />
# No other selection<br />
selectdata = False<br />
# In this band we do not need a-priori corrections for<br />
# antenna gain-elevation curve or atmospheric opacity<br />
# (at 8GHz and above you would want these)<br />
opacity = 0.0<br />
# scan-based G solutions for both amplitude and phase<br />
gaintype = 'G'<br />
solint = 'inf'<br />
combine = ''<br />
calmode = 'ap'<br />
# minimum SNR allowed<br />
minsnr = 1.0<br />
# reference antenna 15 (15=VLA:N2)<br />
refant = '15'<br />
saveinputs('gaincal',prefix+'.gaincal.saved')<br />
gaincal()<br />
#=====================================================================<br />
#<br />
# Bootstrap flux scale<br />
#<br />
print '--Fluxscale--'<br />
default('fluxscale')<br />
vis = msfile<br />
# set the name for the output rescaled caltable<br />
ftable = prefix + '.fluxscale'<br />
fluxtable = ftable<br />
# point to our first gain cal table<br />
caltable = gtable<br />
# we will be using 1331+305 (the source we did setjy on) as<br />
# our flux standard reference - note its extended name as in<br />
# the FIELD table summary above (it has a VLA seq number appended)<br />
reference = '1331*'<br />
# we want to transfer the flux to our other gain cal source 1445+099<br />
transfer = '1445*'<br />
saveinputs('fluxscale',prefix+'.fluxscale.saved')<br />
fluxscale()<br />
</source><br />
<br />
The output from {{fluxscale}} follows. A relatively large uncertainty for the phase calibrator is a sign that something went wrong, perhaps bad solutions in {{gaincal}}. Here, the phase calibrator scaled to 2.486 &plusmn; 0.001 Jy, which looks reasonable.<br />
<br />
<pre><br />
2011-04-25 19:54:25 INFO fluxscale ##########################################<br />
2011-04-25 19:54:25 INFO fluxscale ##### Begin Task: fluxscale #####<br />
2011-04-25 19:54:25 INFO fluxscale::::casa<br />
2011-04-25 19:54:25 INFO fluxscale Opening MS: ngc5921.demo.ms for calibration.<br />
2011-04-25 19:54:25 INFO fluxscale Initializing nominal selection to the whole MS.<br />
2011-04-25 19:54:26 INFO fluxscale Beginning fluxscale--(MSSelection version)-------<br />
2011-04-25 19:54:26 INFO fluxscale Found reference field(s): 1331+30500002_0<br />
2011-04-25 19:54:26 INFO fluxscale Found transfer field(s): 1445+09900002_0<br />
2011-04-25 19:54:26 INFO fluxscale Flux density for 1445+09900002_0 in SpW=0 (freq=1.41342e+09 Hz) is: 2.53882 +/- 0.00218946 (SNR = 1159.57, N = 54)<br />
2011-04-25 19:54:26 INFO fluxscale Storing result in ngc5921.demo.fluxscale<br />
2011-04-25 19:54:26 INFO fluxscale Writing solutions to table: ngc5921.demo.fluxscale<br />
2011-04-25 19:54:26 INFO fluxscale::::casa<br />
2011-04-25 19:54:26 INFO fluxscale ##### End Task: fluxscale #####<br />
2011-04-25 19:54:26 INFO fluxscale ##########################################<br />
</pre><br />
<br />
=== Inspect the Calibration Solutions ===<br />
<br />
[[File:ngc5921-gaincal.png|thumb|Gain calibration solutions from {{gaincal}} and {{fluxscale}}.]]<br />
<br />
Now inspect the results of {{gaincal}}. The setup is identical to that used to plot the [[#Inspect the Bandpass Response Curve | bandpass response curve]]. The only change is that we are plotting the gaintable ngc5921.demo.gcal, and we're looking at solutions for both of the calibrator sources. The results are shown at right.<br />
<br />
<source lang="python"><br />
default('plotcal')<br />
caltable = ftable<br />
field = '0,1'<br />
# Set up 2x1 panels - upper panel amp vs. time<br />
subplot = 211<br />
yaxis = 'amp'<br />
# No output file yet (wait to plot next panel)<br />
saveinputs('plotcal',prefix+'.plotcal.gscaled.amp.saved')<br />
showgui = True<br />
plotcal()<br />
#<br />
# Set up 2x1 panels - lower panel phase vs. time<br />
subplot = 212<br />
yaxis = 'phase'<br />
saveinputs('plotcal',prefix+'.plotcal.gscaled.phase.saved')<br />
#<br />
# The amp and phase coherence looks good<br />
# Pause script if you are running in scriptmode<br />
# If you want to do this interactively and iterate over antenna, set<br />
#iteration = 'antenna'<br />
plotcal()<br />
</source><br />
<br />
=== Apply the Solutions ===<br />
<br />
Next, apply the calibration solutions to the calibrators themselves, and finally transfer the calibration solutions by interpolation (or nearest-neighbor sampling) to the source. The relevant task is {{applycal}}, which fills out a new column (CORRECTED_DATA) of calibrated data in the measurement set without wiping out the raw data column. The application is identical to that used for [[Calibrating a VLA 5 GHz continuum survey|continuum data]], except that the bandpass table is also included in the calibration. To apply multiple calibrations at once, provide the '''gaintable''' parameter with a list of calibration tables, as follows.<br />
<br />
<source lang="python"><br />
gaintable = ['ngc5921.demo.gcal', 'ngc5921.demo.bcal']<br />
</source><br />
<br />
In the script snippet below, the python global variables ''ftable'' and ''btable'' replace the full table names.<br />
<br />
<source lang="python"><br />
default('applycal')<br />
vis = msfile<br />
# We want to correct the calibrators using themselves<br />
# and transfer from 1445+099 to itself and the target N5921<br />
# Start with the fluxscale/gain and bandpass tables<br />
gaintable = [ftable,btable]<br />
# pick the 1445+099 out of the gain table for transfer<br />
# use all of the bandpass table<br />
gainfield = ['1','*']<br />
# interpolation using linear for gain, nearest for bandpass<br />
interp = ['linear','nearest']<br />
# only one spw, do not need mapping<br />
spwmap = []<br />
# all channels<br />
spw = ''<br />
selectdata = False<br />
# as before<br />
opacity = 0.0<br />
# select the fields for 1445+099 and N5921<br />
field = '1,2'<br />
applycal()<br />
# Now for completeness apply 1331+305 to itself<br />
field = '0'<br />
gainfield = ['0','*']<br />
# The CORRECTED_DATA column now contains the calibrated visibilities<br />
saveinputs('applycal',prefix+'.applycal.saved')<br />
applycal()<br />
</source><br />
<br />
== Plot the Spectrum ==<br />
<br />
[[File:spectrum-plotting.png|thumb|{{plotms|Plotms}} settings to produce the integrated spectrum from the calibrated visibilities data.]]<br />
<br />
Before we attempt to image the 21 cm cube of the source, we need to subtract off the underlying continuum, which means we need to plot the integrated spectrum of the source to determine the continuum channels.<br />
<br />
We can do this in {{plotms}}.<br />
<br />
<source lang="bash"><br />
plotms(vis=msfile, selectdata=True, field='N5921*', spw='0:6~56', \<br />
averagedata=True, avgtime='3600', avgscan=True, avgbaseline=True, \<br />
xaxis='channel', yaxis='amp', ydatacolumn='corrected')<br />
</source><br />
<br />
Note that we have entered all the relevent parameters via the task interface, as an alternative to entering each option into the GUI. If the symbols appear too small, the size may be increased via the Display tab: change the Unflagged Points Symbol to 'Custom' and increase the number of pixels for the plotting symbol. The resulting plot is illustrated in the figure at right. Briefly, we want to average both in time and over baselines to get the signal-to-noise necessary to reveal the 21 cm profile (see [[Averaging data in plotms]] for more details on averaging options). If you wish to enter the values directly into the GUI, you can follow the (Tab)'''Command''' convention of the [[Data flagging with casaplotms|flagging tutorial]] with the following settings :<br />
* (Data)'''field''' = N5921*<br />
* (Data)'''spw''' = 0:6~56<br />
* (Data)'''Averaging &rarr; Time''' = 3600 (average over some long time window)<br />
* (Data)'''Averaging &rarr; Scan''' = True (checkmark; average in time across scan boundaries)<br />
* (Data)'''Averaging &rarr; All Baselines''' = True (checkmark)<br />
* (Axes)'''X Axis''' = Channel<br />
* (Axes)'''Y Axis''' = Amp<br />
<br />
From inspection of this plot, it looks like channels 4~6 and 50~59 contain line-free channels, suitable to use for continuum subtraction.<br />
<br />
== Continuum Subtraction ==<br />
<br />
The next step is to split off the NGC 5921 data from the multisource measurement set and subtract the continuum. Splitting uses the {{split}} command, as follows.<br />
<br />
<source lang="python"><br />
default('split')<br />
vis = msfile<br />
splitms = prefix + '.src.split.ms'<br />
outputvis = splitms<br />
field = 'N5921*'<br />
spw = ''<br />
datacolumn = 'corrected'<br />
saveinputs('split',prefix+'.split.n5921.saved')<br />
split()<br />
print "Created "+splitms<br />
</source><br />
<br />
This action generated a new measurement set called ''ngc5921.demo.src.split.ms'' and copied the ''calibrated'' source data (datacolumn = 'corrected') into it.<br />
<br />
{{uvcontsub|Uvcontsub}} subtracts the continuum from the data in the visibility (''u'', ''v'') plane. <br />
<br />
<source lang="python"><br />
default('uvcontsub')<br />
vis = splitms<br />
field = 'N5921*'<br />
# Use channels 4-6 and 50-59 for continuum<br />
fitspw='0:4~6;50~59'<br />
# Output all of spw 0<br />
spw = '0'<br />
# Averaging time (none)<br />
solint = 0.0<br />
# Fit only a mean level<br />
fitorder = 0<br />
# Do the uv-plane subtraction<br />
fitmode = 'subtract'<br />
# Let it split out the data automatically for us<br />
splitdata = True<br />
saveinputs('uvcontsub',prefix+'.uvcontsub.saved')<br />
uvcontsub()<br />
</source><br />
<br />
Notice that {{uvcontsub}} splits two new measurement sets, 'ngc5921.demo.ms.cont', which contains an average of the continuum channels, and 'ngc5921.demo.ms.contsub', which contains the continuum-subtracted spectral line data.<br />
<br />
== Imaging ==<br />
[[File:spectrum-uvplot.png|thumb|Plot of amplitude vs. projected baseline length (in units of the observing wavelength) produced by casaplotms. The maximum baseline is just below 5 kilo-lambda.]]<br />
<br />
<br />
Now we can generate the primary science product, a {{clean}} data cube (ra, dec, velocity) from the continuum-subtracted (''u'', ''v'', channel) measurement set, ''ngc5921.demo.ms.contsub''. Things to consider in using {{clean}}:<br />
* To ensure channels aren't averaged prior to imaging, choose '''mode='channel''''.<br />
* Specify the channels to image using '''start = 5''', '''width = 1''' (no averaging over channels), '''nchan = 46'''; only channels 5~51 will be imaged.<br />
* The maximum baseline is just under 5 kilolambda (see the figure at right), and so the expected synthetic beam is roughly 1.22 &times; 206265 / 5000 = 50 arcseconds (subject to the details of ''u'', ''v'' weighting). Pixels should sample the beam better than 3 times, so 15 arcseconds is a good choice of pixel size ('''cell = ['15.0arcsec','15.0arcsec']''').<br />
* We only want to {{clean}} down to the noise, which is easily determined by trial-and-error imaging of a single channel (choosing '''nchan=1''' and '''start''' appropriately). Here, {{clean}} stops when the maximum residual on the channel is below '''threshold='8.0mJy''''. <br />
<br />
<source lang="python"><br />
# identify the continuum subtracted measurement set<br />
srcsplitms = splitms + '.contsub'<br />
#=====================================================================<br />
#<br />
# Now clean an image cube of N5921<br />
#<br />
default('clean')<br />
# Pick up our split source continuum-subtracted data<br />
vis = srcsplitms<br />
# Make an image root file name<br />
imname = prefix + '.cleanimg'<br />
imagename = imname<br />
# Set up the output image cube<br />
mode = 'channel'<br />
nchan = 46<br />
start = 5<br />
width = 1<br />
# This is a single-source MS with one spw<br />
field = '0'<br />
spw = ''<br />
# Standard gain factor 0.1<br />
gain = 0.1<br />
# Set the output image size and cell size (arcsec)<br />
imsize = [256,256]<br />
# Do a simple Clark clean<br />
psfmode = 'clark'<br />
# No Cotton-Schwab iterations<br />
csclean = False<br />
# Pixel size 15 arcsec for this data (1/3 of 45" beam)<br />
# VLA D-config L-band<br />
cell = ['15.0arcsec','15.0arcsec']<br />
# Fix maximum number of iterations<br />
niter = 6000<br />
# Also set flux residual threshold (in mJy)<br />
threshold='8.0mJy'<br />
# Set up the weighting<br />
# Use Briggs weighting (a moderate value, on the uniform side)<br />
weighting = 'briggs'<br />
robust = 0.5<br />
# Set a cleanbox +/-20 pixels around the center 128,128<br />
mask = [108,108,148,148]<br />
# If you want interactive clean set to True<br />
#interactive=True<br />
interactive=False<br />
saveinputs('clean',prefix+'.clean.saved')<br />
clean()<br />
</source><br />
<br />
Use {{imhead}} to look at the cube header:<br />
<br />
<source lang="python"><br />
clnimage = imname+'.image' # store the clean image in a python global for future use<br />
default('imhead')<br />
imagename = clnimage<br />
mode = 'summary'<br />
imhead()<br />
</source><br />
<br />
The output, as follows, appears in the logger window.<br />
<br />
<pre><br />
2011-04-25 20:10:29 INFO imhead ##########################################<br />
2011-04-25 20:10:29 INFO imhead ##### Begin Task: imhead #####<br />
2011-04-25 20:10:29 INFO imhead::::casa<br />
2011-04-25 20:10:29 INFO ImageAnalysis::summary<br />
2011-04-25 20:10:29 INFO ImageAnalysis Image name : ngc5921.demo.cleanimg.image<br />
2011-04-25 20:10:29 INFO ImageAnalysis Object name : N5921_2<br />
2011-04-25 20:10:29 INFO ImageAnalysis Image type : PagedImage<br />
2011-04-25 20:10:29 INFO ImageAnalysis Image quantity : Intensity<br />
2011-04-25 20:10:29 INFO ImageAnalysis Pixel mask(s) : None<br />
2011-04-25 20:10:29 INFO ImageAnalysis Region(s) : None<br />
2011-04-25 20:10:29 INFO ImageAnalysis Image units : Jy/beam<br />
2011-04-25 20:10:29 INFO ImageAnalysis Restoring Beam : 51.7693 arcsec, 47.2192 arcsec, -170.658 deg<br />
2011-04-25 20:10:29 INFO ImageAnalysis::summary<br />
2011-04-25 20:10:29 INFO ImageAnalysis Direction reference : J2000<br />
2011-04-25 20:10:29 INFO ImageAnalysis Spectral reference : LSRK<br />
2011-04-25 20:10:29 INFO ImageAnalysis Velocity type : RADIO<br />
2011-04-25 20:10:29 INFO ImageAnalysis Rest frequency : 1.42041e+09 Hz<br />
2011-04-25 20:10:29 INFO ImageAnalysis Pointing center : 15:22:00.000000 +05.04.00.000000<br />
2011-04-25 20:10:29 INFO ImageAnalysis Telescope : VLA<br />
2011-04-25 20:10:29 INFO ImageAnalysis Observer : TEST<br />
2011-04-25 20:10:29 INFO ImageAnalysis Date observation : 1995/04/13/09:33:00<br />
2011-04-25 20:10:29 INFO ImageAnalysis Telescope position: [-1.60119e+06m, -5.04198e+06m, 3.55488e+06m] (ITRF)<br />
2011-04-25 20:10:29 INFO ImageAnalysis::summary+<br />
2011-04-25 20:10:29 INFO ImageAnalysis Axis Coord Type Name Proj Shape Tile Coord value at pixel Coord incr Units<br />
2011-04-25 20:10:29 INFO ImageAnalysis ------------------------------------------------------------------------------------------------ <br />
2011-04-25 20:10:29 INFO ImageAnalysis 0 0 Direction Right Ascension SIN 256 64 15:22:00.000 128.00 -1.500000e+01 arcsec<br />
2011-04-25 20:10:29 INFO ImageAnalysis 1 0 Direction Declination SIN 256 64 +05.04.00.000 128.00 1.500000e+01 arcsec<br />
2011-04-25 20:10:29 INFO ImageAnalysis 2 1 Stokes Stokes 1 1 I<br />
2011-04-25 20:10:29 INFO ImageAnalysis 3 2 Spectral Frequency 46 8 1.41279e+09 0.00 2.4414062e+04 Hz<br />
2011-04-25 20:10:29 INFO ImageAnalysis Velocity 1607.99 0.00 -5.152860e+00 km/s<br />
2011-04-25 20:10:29 INFO imhead::::casa<br />
2011-04-25 20:10:29 INFO imhead ##### End Task: imhead #####<br />
2011-04-25 20:10:29 INFO imhead ##########################################<br />
</pre><br />
<br />
== Additional Science Products ==<br />
<br />
If things went well, you should now have a spectral line cube (''ngc5921.demo.cleanimg.img'') as a primary science product. The demo script illustrates further how to generate cube statistics (using {{imstat}}), an integrated spectrum, and moment maps. <br />
<br />
=== Cube Statistics ===<br />
<br />
{{imstat|Imstat}} is the tool for displaying statistics of images and cubes. The following example displays the statistics for the whole cube.<br />
<br />
<source lang="python"><br />
default('imstat')<br />
imagename = clnimage # or imagename = "ngc5921.demo.cleanimg.img"<br />
# Do whole image<br />
box = ''<br />
# or you could stick to the cleanbox<br />
#box = '108,108,148,148'<br />
cubestats = imstat()<br />
</source><br />
<br />
The output goes to the logger window.<br />
<br />
<pre><br />
2011-04-25 20:12:13 INFO imstat ##########################################<br />
2011-04-25 20:12:13 INFO imstat ##### Begin Task: imstat #####<br />
2011-04-25 20:12:13 INFO imstat::::casa<br />
2011-04-25 20:12:13 INFO imstat No region specified. Using full positional plane.<br />
2011-04-25 20:12:13 INFO imstat Using all spectral channels.<br />
2011-04-25 20:12:13 INFO imstat Using polarizations I<br />
2011-04-25 20:12:13 INFO imstat Determining stats for image ngc5921.demo.cleanimg.image<br />
2011-04-25 20:12:13 INFO imstat Selected bounding box : <br />
2011-04-25 20:12:13 INFO imstat [0, 0, 0, 0] to [255, 255, 0, 45] (15:24:08.404, +04.31.59.181, I, 1.41279e+09Hz to 15:19:52.390, +05.35.44.246, I, 1.41389e+09Hz)<br />
2011-04-25 20:12:13 INFO imstat Regions --- <br />
2011-04-25 20:12:13 INFO imstat -- bottom-left corner (pixel) [blc]: [0, 0, 0, 0]<br />
2011-04-25 20:12:13 INFO imstat -- top-right corner (pixel) [trc]: [255, 255, 0, 45]<br />
2011-04-25 20:12:13 INFO imstat -- bottom-left corner (world) [blcf]: 15:24:08.404, +04.31.59.181, I, 1.41279e+09Hz<br />
2011-04-25 20:12:13 INFO imstat -- top-right corner (world) [trcf]: 15:19:52.390, +05.35.44.246, I, 1.41389e+09Hz<br />
2011-04-25 20:12:13 INFO imstat Values --- <br />
2011-04-25 20:12:13 INFO imstat -- flux density [flux]: 4.14845 Jy<br />
2011-04-25 20:12:13 INFO imstat -- number of points [npts]: 3.01466e+06<br />
2011-04-25 20:12:13 INFO imstat -- maximum value [max]: 0.0555024 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- minimum value [min]: -0.0107881 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- position of max value (pixel) [maxpos]: [134, 134, 0, 38]<br />
2011-04-25 20:12:13 INFO imstat -- position of min value (pixel) [minpos]: [230, 0, 0, 15]<br />
2011-04-25 20:12:13 INFO imstat -- position of max value (world) [maxposf]: 15:21:53.976, +05.05.29.998, I, 1.41371e+09Hz<br />
2011-04-25 20:12:13 INFO imstat -- position of min value (world) [maxposf]: 15:20:17.679, +04.31.59.470, I, 1.41315e+09Hz<br />
2011-04-25 20:12:13 INFO imstat -- Sum of pixel values [sum]: 51.0692 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- Sum of squared pixel values [sumsq]: 12.2781 Jy/beam.Jy/beam<br />
2011-04-25 20:12:13 INFO imstat::::<br />
2011-04-25 20:12:13 INFO imstat Statistics --- <br />
2011-04-25 20:12:13 INFO imstat -- Mean of the pixel values [mean]: 1.69403e-05 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- Variance of the pixel values : 4.07252e-06 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- Standard deviation of the Mean [sigma]: 0.00201805 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- Root mean square [rms]: 0.00201812 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- Median of the pixel values [median]: -1.18009e-05 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- Median of the deviations [medabsdevmed]: 0.00126199 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat -- Quartile [quartile]: 0.00252393 Jy/beam<br />
2011-04-25 20:12:13 INFO imstat::::casa<br />
2011-04-25 20:12:13 INFO imstat ##### End Task: imstat #####<br />
2011-04-25 20:12:13 INFO imstat ##########################################<br />
</pre><br />
<br />
=== The Integrated Spectrum ===<br />
<br />
[[File:spectrum-integrated.png|thumb|Example of the {{viewer}} rectangle selection tool on one channel of the NGC&nbsp;5921 21&nbsp;cm data cube. The spectral profile window is shown at right.]]<br />
<br />
<br />
We saw earlier [[#Plot the Spectrum|how to generate an integrated spectrum]] from the (''u'', ''v'') measurement set. Here's how to produce the integrated spectrum from the spectral line cube. First, load the cube into {{viewer}}.<br />
<br />
<source lang="python"><br />
# Store the name of the clean image into a python global<br />
clnimage = imname+'.image'<br />
viewer(clnimage)<br />
</source><br />
<br />
To generate the integrated spectrum, perform the following tasks.<br />
* Use the player controls [[File:vcrNext.png]] to inspect the cube one channel at a time.<br />
* From the {{viewer}} Tools menu, select '''Spectral Profile'''. A new graphics window should appear.<br />
* By default, the rectangle selection tool [[File:drawingSelector.png]] is assigned to the right mouse button, and you can just right-click and drag a box over the region where you want to (spatially) integrate the spectrum. See the figure at upper right. <br />
* Alternatively, you can assign one of the other selection tools by right-clicking on the appropriate button.<br />
* The spectrum now appears in the graphics window; see the figure at right.<br />
<br />
You can save the integrated spectrum to a text file by clicking the [[File:save-as-text-file.png]] button on the graphics window. There are also buttons to print the figure or save the figure to disk.<br />
<br />
=== Cube Moments ===<br />
<br />
[[File:spectrum-momzero.png|thumb|The moment 0 (integrated intensity) 21&nbsp;cm image of NGC&nbsp;5921, produced using {{immoments}}]]<br />
<br />
Cube moments are maps of weighted sums along the velocity axis. In CASA, they are generated by the task {{immoments}}. The zeroth moment (moments = 0) is a sum of intensities along the velocity axis (the integrated intensity map); the first moment (moment = 1) is the sum of velocities weighted by intensity (the ''velocity field''); the second moment (moment = 2) is a map of the velocity dispersion; see the {{immoments|helpfile}} for additional options.<br />
<br />
The following example produces maps of the zeroth and first moments, or the integrated intensity and velocity field. The respective measurement sets are ''ngc5921.demo.moments.integrated'' and ''ngc5921.demo.moments.weighted_coord'', stored in the python globals ''momzeroimage'' and ''momoneimage''.<br />
<br />
<br />
<source lang="python"><br />
default('immoments')<br />
imagename = clnimage<br />
# Do zeroth and first moments<br />
moments = [0,1]<br />
# Need to mask out noisy pixels, currently done<br />
# using hard global limits<br />
excludepix = [-100,0.009]<br />
# Collapse along the spectral (channel) axis<br />
axis = 'spectral'<br />
# Include all planes<br />
chans = ''<br />
# Output root name<br />
momfile = prefix + '.moments'<br />
outfile = momfile<br />
saveinputs('immoments',prefix+'.immoments.saved')<br />
immoments()<br />
momzeroimage = momfile + '.integrated'<br />
momoneimage = momfile + '.weighted_coord'<br />
</source><br />
<br />
To examine the moment images, use {{viewer}}; the resulting moment zero image is displayed at right.<br />
<br />
<source lang="python"><br />
viewer(momzeroimage)<br />
</source><br />
<br />
== Export the Data ==<br />
<br />
To export the (''u'', ''v'') data and image cube as FITS files, use {{exportuvfits}} and {{exportfits}}, respectively.<br />
<br />
Here's how to export the continuum-subtracted (''u'', ''v'') data. Note that this snippet has {{exportuvfits}} spawned to the background (async = True).<br />
<br />
<source lang="python"><br />
default('exportuvfits')<br />
srcuvfits = prefix + '.contsub.uvfits' # recall: prefix = 'ngc5921.demo'<br />
vis = splitms + '.contsub'<br />
fitsfile = srcuvfits<br />
datacolumn = 'corrected'<br />
multisource = True<br />
async = True<br />
myhandle = exportuvfits()<br />
</source><br />
<br />
And now, the FITS cube.<br />
<source lang="python"><br />
default('exportfits')<br />
clnfits = prefix + '.cleanimg.fits'<br />
imagename = clnimage<br />
fitsimage = clnfits<br />
async = True<br />
saveinputs('exportfits',prefix+'.exportfits.saved')<br />
myhandle2 = exportfits()<br />
</source><br />
<br />
The moment maps (or any CASA images) can be similarly exported using {{exportfits}}.<br />
<br />
== Appendix: Python Notes ==<br />
<br />
=== os.system ===<br />
<br />
'''os.system''' allows you to run shell commands from within python / CASA. For example:<br />
<br />
<pre><br />
import os<br />
os.system('ls -sF')<br />
</pre><br />
<br />
will give an OS-level listing of the current directory's contents.<br />
<br />
=== os.environ.get ===<br />
It's worth having a look at the output of the '''os.environ.get''' command to understand the python syntax (alternative: '''os.getenv'''). You can think of '''os.environ''' as a list of operating system environment variables, and '''get''' is a method that extracts information about the requested environment variable, here, CASAPATH. '''Get''' returns a string of whitespace separated information, and '''.split()''' turns that string into a list. The array index '''[0]''' extracts the first element of that list, which contains the path. <br />
<br />
To illustrate, here is some example python I/O in CASA.<br />
<br />
<pre><br />
CASA <12>: print os.environ.get('CASAPATH')<br />
/usr/lib64/casapy/30.0.9709test-001 linux local el5bld64b<br />
<br />
CASA <13>: print os.environ.get('CASAPATH').split()<br />
['/usr/lib64/casapy/30.0.9709test-001', 'linux', 'local', 'el5bld64b']<br />
<br />
CASA <14>: print os.environ.get('CASAPATH').split()[0]<br />
/usr/lib64/casapy/30.0.9709test-001<br />
</pre><br />
<br />
[[Pre-upgrade VLA Tutorials | &#8629; '''Pre-upgrade VLA Tutorials''']]<br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_5_GHz_continuum_survey_of_Seyfert_galaxies_5.7.2&diff=21905VLA 5 GHz continuum survey of Seyfert galaxies 5.7.22017-06-23T21:01:56Z<p>Emomjian: </p>
<hr />
<div>[[Category:VLA]] [[Category:Calibration]] [[Category:Imaging]] [[Category:Self-Calibration]] [[Category:Continuum]]<br />
<br />
Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here. <br />
<br />
<br />
This particular tutorial demonstrates how to calibrate a VLA continuum survey, consisting of a flux calibrator, target sources, and their respective phase calibrators. What distinguishes this example is that it includes multiple sources and multiple phase calibrators.<br />
<br />
<br />
== Importing VLA Archive Data ==<br />
<br />
Download your data from the [http://archive.nrao.edu: VLA Archive]; in this example we'll use the publicly available survey [https://archive.nrao.edu/archive/ArchiveQuery?QUERY_ID=9999&QUERY_MODE=Prepare+Download&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&PROJECT_CODE=AG733&OBSERVER=&TELESCOPE=ALL&TIMERANGE1=&TIMERANGE2=&PASSWD=&QUERYTYPE=ARCHIVE&SUBMIT=Submit+Query: AG733], which consists of VLA C-array, C-band (5 GHz) continuum observations of a handful of Seyfert galaxies. <br />
<br />
With some sensible fiddling of the web interface, you should be able to download two archive formatted files, one from 06-Dec-09 and one from 06-Dec-13, with names like '''AG733_A061209.xp1''' and '''AG733_B061213.xp1''' or '''2006-12/vla2006-12-09.dat''' and '''2006-12/vla2006-12-13.dat'''.<br />
<br />
[[Image:Screenshot-Log_Messages_(ashlesha--users-jgallimo-AG733-casapy.log).png|thumb|Example snippet of listobs output for program AG733.]]<br />
<br />
It's helpful to make a subdirectory to store the VLA archive data and the CASA measurement sets before starting CASA. Linux-level commands are given assuming you are using bash as your default shell.<br />
<br />
<source lang="bash"><br />
# In bash<br />
mkdir ~/AG733<br />
mv AG733_* ~/AG733<br />
cd ~/AG733<br />
casa<br />
</source><br />
<br />
To load the data in CASA, we employ the '''{{importvla}}''' command.[[Image:casa_vla_antenna_plot.png|thumb|Graphics window produced by {{plotants}}. Shown are the antennas of the VLA for the program AG733.]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Before importing, get rid of any original version of the measurement set<br />
if os.path.exists('ag733.ms'): rmtables('ag733.ms')<br />
# let glob find the archive files, store in a python global "myFiles"<br />
from glob import glob as filesearch <br />
myFiles = filesearch("AG733*.xp?")<br />
importvla(archivefiles=myFiles,vis='ag733.ms')<br />
listobs('ag733.ms')<br />
vishead('ag733.ms')<br />
</source><br />
<br />
The output measurement set, i.e., the data to be worked on, is ''ag733.ms.'' Note that the (interesting) output of {{listobs}} and {{vishead}} go to the CASA message log window.<br />
<br />
{{listobs}} is similar to LISTR in AIPS and provides a list of scans, sources and their respective indices, and antennas. You can also get a more useful plot of antennas, akin to the output of the AIPS task PRTAN, using {{plotants}}. The antennas plot aids the selection of a reference antenna; the antenna ''VA05'' looks like a reasonable choice.<br />
<br />
<source lang="python"><br />
# In CASA<br />
vis = 'ag733.ms'<br />
plotants()<br />
</source><br />
<br />
It is also currently important to '''close''' {{plotants}} before moving on in the script to avoid locked tables. --><br />
<br />
''' <pre style="color: red"> Note: Some older VLA data may give errors in more recent CASA versions when attempting to generate a plot of antennas. The instructions that follow are a workaround.<br />
</pre>'''<br />
<br />
<br />
'''Step 1:''' Download the [ftp://ftp.cv.nrao.edu/pub/casaguides/analysis_scripts.tar Analysis Utilities] and extract the tar ball.<br />
<br />
<source lang="bash"><br />
$ tar xvf analysis_scripts.tar<br />
</source><br />
<br />
'''Step 2:''' Run the following commands.<br />
<br />
<source lang="Python"><br />
import sys<br />
sys.path.append("/PATH_TO_ANALYSIS_SCRIPTS/")<br />
import analysisUtils as au<br />
</source><br />
<br />
where <tt>/PATH_TO_ANALYSIS_SCRIPTS/</tt> is the complete path to the directory you just extracted from the tar ball.<br />
<br />
<br />
'''Step 3:''' Generate the antenna plots with Analysis Utilities.<br />
<br />
<source lang="Python"><br />
au.buildConfigurationFile('ag733.ms')<br />
au.plotconfig(config='ag733.ms.cfg', figfile='ag733.plotants.png')<br />
</source><br />
<br />
== Editing the Measurement Set ==<br />
<br />
This technique is detailed in the tutorial [[Data flagging with plotms]]. Here's a shorter version. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms() <br />
</source><br />
<br />
[[plotms|Plotms]] brings up the following graphical display.<br />
<br />
[[Image:Screenshot-PlotMS.png|300px|The plotms GUI.]]<br />
<br />
In this example, the measurement set was selected by navigating to and highlighting the measurement set ag733.ms (which will look like a directory to your file browser, because measurement sets are actually directory structures). A specific subset of data (the flux calibrator 3C48 = 0137+331 on the first observing day) was selected by specifying in the appropriate "Data -> Selection" boxes,<br />
<br />
* field = 0137+331, and<br />
* timerange = 2006/12/09/0:0:0~23:0:0.<br />
<br />
For this specific field, we see no obvious bad data. Note that the low points are actually cross-polarization data; if you want to only look at the right and left polarizations, enter<br />
<br />
* corr = RR, LL.<br />
<br />
For the sake of this example, we'll clip out the early part of the observations: <br />
<br />
Firstly, box the region to be flagged by using the "Mark Regions" button [[Image:MarkRegionsButton.png]]. Next, go the the "Flag" tab at top, and select "Extend flags" and "Correlation", so that we will delete the RL and LR data as well (despite the fact that we're not plotting it). Then, press the "Go" button, or "Flag" button [[Image:FlagThoseData.png]] to zap those data. <br />
<br />
The {{viewer}} tool provides utilities to flag data in a manner similar to the AIPS task TVFLG; see the tutorial [[Data flagging with viewer]]. <br />
<br />
If we leave the field and timerange filter boxes blank and plot again, we see a number of visibilities with extremely large amplitudes. By highlighting and locating the bad points (described in detail in the [[data flagging with plotms]] tutorial), we find most of the bad data can be attributed to one antenna, 'VA15', at specific times, and one time range where all antennas contribute. In the text below we flag these points using {{flagdata}}; you can also flag data interactively in the {{plotms}} display (though this is generally not encouraged since it does not save backup versions of the flags, and it can be easy to flag data by mistake or miss data that's not plotted). <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='ag733.ms', antenna='VA15', timerange='2006/12/09/04:45:10.0~04:45:20.0')<br />
flagdata(vis='ag733.ms', antenna='VA15', timerange='2006/12/09/06:12:30.0~06:14:10.0')<br />
flagdata(vis='ag733.ms', antenna='VA15', timerange='2006/12/13/04:32:40.0~04:35:30.0')<br />
flagdata(vis='ag733.ms', antenna='VA15', timerange='2006/12/09/06:12:30.0~06:12:40.0')<br />
flagdata(vis='ag733.ms', antenna='VA15', timerange='2006/12/09/06:13:00.0~06:13:40.0')<br />
flagdata(vis='ag733.ms', antenna='VA15', timerange='2006/12/09/06:13:50.0~06:14:00.0')<br />
flagdata(vis='ag733.ms', antenna='VA15', timerange='2006/12/09/06:14:00.0~06:14:10.0')<br />
flagdata(vis='ag733.ms', timerange='2006/12/09/06:24:50.0~06:25:00.0')<br />
</source><br />
<br />
Plot the data again to make sure the amplitudes look reasonable (if {{plotms}} is already open, just hit Shift+Plot to reload with the current flags): <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms() <br />
</source><br />
<br />
=== Careful with Cross-Pols ===<br />
<br />
By default, {{plotms}} plots all of the data, including cross-polarization. All those low-amplitude points in your data are probably not junk. They are real cross-polarization data. Flagging those data will cause grief later on. Take a look at the tutorial [[Data flagging with plotms#Cross-Pol Data of Bright Sources | data flagging with plotms]].<br />
<br />
Tread lightly! And, whenever possible, flag with {{flagdata}} and keep careful track of what you flag in a separate file.<br />
<br />
==A Python Interlude==<br />
<br />
CASA uses python as an interactive wrapper and scripting language. Python offers the ability to define global variables on the fly, and it's worth using such variables to take care of bookkeeping. First we'll define some ''list'' variables that store calibrator and source information. <br />
<br />
<source lang="python"><br />
ampcallist = ['0137+331']<br />
phasecallist = ['2250+143', '0119+321', '0237+288', '0239-025',<br />
'0323+055', '0339-017', '0423-013'] <br />
sourcelist = ['NGC7469', 'MRK0993', 'MRK1040', 'NGC1056', 'NGC1068',<br />
'NGC1194', 'NGC1241', 'NGC1320', 'F04385-0828',<br />
'NGC1667'] <br />
allcals = ampcallist + phasecallist<br />
print allcals<br />
</source><br />
<br />
It will also be helpful later to have a way of referencing which phase calibrator should be assigned to which source. We can define the matching explicitly by using a ''dictionary'' variable.<br />
<br />
<source lang="python"><br />
calDict = {'NGC7469':'2250+143',<br />
'MRK0993':'0119+321', <br />
'MRK1040':'0237+288', <br />
'NGC1056':'0237+288',<br />
'NGC1068':'0239-025',<br />
'NGC1194':'0323+055',<br />
'NGC1241':'0323+055',<br />
'NGC1320':'0339-017',<br />
'F04385-0828':'0423-013',<br />
'NGC1667':'0423-013'}<br />
</source><br />
<br />
Those variables being defined, we can proceed with the calibration (and not have to type source and calibrator names over and over).<br />
<br />
== Calibration ==<br />
<br />
=== Set the Flux Scale ===<br />
As in AIPS, CASA sets the flux scale using an observation of a flux calibrator (here, 0137+331) and the task {{setjy}}. For this example, we'll determine the calibration in reference to a model image of the calibrator source. Look at the inputs of {{setjy}} and get it ready to go!<br />
<br />
<source lang="python"> <br />
# in CASA<br />
# perhaps restore setjy back to its defaults<br />
default setjy<br />
inp<br />
vis = 'ag733.ms'<br />
field = '0137+331'<br />
# For the model image, need to specify the correct directory, something like the following<br />
#<br />
# load the CASA path from environment variable<br />
import os<br />
casapath = os.environ.get('CASAPATH').split()[0]<br />
# generate full path to the image<br />
modimage = casapath + '/data/nrao/VLA/CalModels/3C48_C.im'<br />
go<br />
</source><br />
<br />
That example was set up to demonstrate some AIPS heritage. You could equally well run {{setjy}} the following way.<br />
<source lang="python"> <br />
# in CASA<br />
setjy('ag733.ms', field = '0137+331', modimage = casapath + '/data/nrao/VLA/CalModels/3C48_C.im')<br />
</source><br />
<br />
=== Determine calibration solutions ===<br />
<br />
At this stage the data have an overall flux scaling determined, but full gain solutions aren't there yet. The relevant task is {{gaincal}} (analogous to the AIPS task CALIB).<br />
<br />
Rather than generate solution tables (SN tables in AIPS) that are attached to the measurement set, {{gaincal}} will produce a separate table, which we'll call ''cal.G.''<br />
<br />
<source lang="python"><br />
rmtables('cal.G') # get rid of any old, unmeritorious versions of the calibration<br />
rmtables('gc.cal')<br />
<br />
# First we'll generate an antenna zenith-angle dependent gain curve calibration table<br />
gencal(vis='ag733.ms', caltype='gc', caltable='gc.cal') <br />
<br />
#<br />
# Some caution on the following step. The solution interval is set to 'inf,' meaning to average over entire<br />
# scans of each calibrator in turn. If a given scan is however longer than the coherence time (esp. on long baselines), <br />
# some decorrelation will occur and the flux calibration will be thrown off. Suggest, e.g., solint='1min' or solint='2min' <br />
# as reasonable alternatives to solint='inf'. Trial and error may be your friend here.<br />
#<br />
gaincal(vis='ag733.ms',caltable='cal.G', field=','.join(allcals), solint='inf',<br />
refant='VA05', gaintable=['gc.cal'], append=False)<br />
plotcal('cal.G',yaxis='amp',timerange='2006/12/09/04:0:0~06:30:0')<br />
</source><br />
<br />
There's some python trickery buried in this example. If you are unfamiliar with the string method ''join,'' try the following example. Recall that we had defined the variable allcals above in [[#A_Python_Interlude|A Python Interlude]].<br />
<source lang="python"><br />
print allcals <br />
print ','.join(allcals)<br />
</source><br />
<br />
It's worth playing with the timerange, etc. in {{plotcal}} to explore the solutions. You may notice that the antenna 'EA26' looks a little junky on the second day. Here's how to flag it manually and then repeat the calibration (be sure to close the {{plotcal}} window displaying the previous solutions first).<br />
<br />
<source lang="python"><br />
flagdata(vis='ag733.ms',antenna='EA26', timerange='2006/12/13/0:0:0~24:0:0')<br />
rmtables('cal.G') <br />
gaincal(vis='ag733.ms',caltable='cal.G', field=','.join(allcals), solint='inf',<br />
refant='VA05', gaintable=['gc.cal'], append=False )<br />
</source><br />
<br />
Go back and re-plot the calibration; this example shows even more AIPS heritage.<br />
<br />
<source lang="python"><br />
tget plotcal<br />
plotcal()<br />
</source><br />
<br />
Finally, bootstrap the flux scale of the flux calibrator onto the phase calibrators. AIPS called it GETJY, but CASA calls it {{fluxscale}}.<br />
<br />
<source lang="python"><br />
rmtables('cal.Gflx') # remove any old versions of the calibration table, if necessary<br />
myFluxes = fluxscale(vis='ag733.ms', caltable='cal.G', reference=','.join(ampcallist), <br />
transfer=','.join(phasecallist), fluxtable='cal.Gflx', append=False)<br />
</source><br />
<br />
Here, the calibration table ''cal.G'' is modified and stored as 'cal.Gflx.' The Python dictionary "myFluxes" will also contain information about the flux scaling and other useful tidbits (type "myFluxes" to see its contents). So far, solutions have been generated only for the calibrators, and they have not yet been transfered to the sources.<br />
<br />
We could equally well have used reference='0137+337', but use of the ''join'' method allows for the more general possibility of multiple flux calibrators.<br />
<br />
=== Apply the Calibrations ===<br />
<br />
In AIPS, the task CLCAL applies calibration solutions to a calibration table. In CASA the equivalent task is {{applycal}}. We loop over sources and calibrators to properly match them:<br />
<!-- for now, just use applycal, since accum appears to be broken in 3.4 (see CAS-4240)<br />
<br />
However, we need to apply the calibration ''incrementally,'' so that calibration solutions for sources are properly matched with their respective phase calibrator. The task {{accum}} performs incremental calibration. <br />
<br />
In this example, the solutions in ''cal.Gflx'' will be incrementally compiled into a new calibration table called ''multisource.gcal.''<br />
<br />
<source lang="python"><br />
# first, generate a "blank" table.<br />
accum(vis='ag733.ms',tablein='',accumtime=10.0,incrtable='cal.Gflx',<br />
interp='linear', caltable='multisource.gcal',<br />
field=','.join(allcals), calfield=','.join(allcals)) <br />
<br />
# now accumulate solutions onto that table<br />
# loop over the source and calibrator pairs in the calDict dictionary variable<br />
for source, calibrator in calDict.iteritems():<br />
accum(vis='ag733.ms',tablein='multisource.gcal',accumtime=10.0,<br />
incrtable='cal.Gflx',<br />
interp='linear', field=source,<br />
calfield=calibrator)<br />
<br />
# Finally, apply the calibrations to the measurement set<br />
default(applycal)<br />
applycal(vis='ag733.ms', gaintable='multisource.gcal')<br />
</source><br />
<br />
--><br />
<br />
<source lang="python"><br />
for source, calibrator in calDict.iteritems():<br />
applycal(vis='ag733.ms', field=source,<br />
gaintable='cal.Gflx', gainfield=calibrator)<br />
</source><br />
<br />
=== Baseline-based corrections ===<br />
<br />
[[File:NGC1068_without_blcal.png|thumb|Initial clean image of NGC1068 ''without'' blcal.]] <br />
[[File:NGC1068_with_blcal.png|thumb|Initial clean image of NGC1068 ''with'' blcal applied.]]<br />
<br />
Although it makes only modest improvements for the present example, observations including a mix of EVLA and VLA telescopes may benefit from performing baseline-based corrections on a strong calibrator. Now that we have initial antenna-based solutions, we can use {{blcal}}.<br />
<br />
<source lang="python"><br />
default('blcal')<br />
vis = 'ag733.ms'<br />
# output baseline-based calibration solutions<br />
caltable = 'cal.BL' <br />
# use the strong flux calibrator to determine baseline-based solutions<br />
field = '0137+331' <br />
# generate a solution for each calibrator scan<br />
solint = 'inf' <br />
gaintable = 'cal.Gflx'<br />
# calibration table with the best antenna-based solutions, so far<br />
gainfield = '0137+331' <br />
# calibrator for the BL calibrator; in this example, gain cal and bl cal are the same<br />
interp = 'nearest'<br />
# do the first day's data<br />
timerange = '2006/12/09/0:0:0~24:0:0'<br />
blcal()<br />
# now the second day<br />
timerange = '2006/12/13/0:0:0~24:0:0'<br />
blcal()<br />
</source><br />
<br />
This operation generates the calibration table ''cal.BL''. Solutions can be inspected with {{plotcal}}. Now we have to use {{applycal}} again to apply both the antenna-based and baseline-based calibration solutions to the data.<br />
<br />
<source lang="python"><br />
for source, calibrator in calDict.iteritems():<br />
applycal(vis='ag733.ms', field=source,<br />
gaintable=['cal.Gflx','cal.BL'], gainfield=calibrator)<br />
</source><br />
<br />
Did the {{blcal}} operation make a difference here? Getting ahead of ourselves a little, the plots at right show the initial {{clean}} images of NGC1068, one with {{blcal}} applied, and one without, and both images are displayed with the same stretch. There does appear a slight reduction of artifacts with {{blcal}} applied, but of course your mileage will vary.<br />
<br />
=== Splitting the calibrated source data from the multisource measurement set ===<br />
<br />
To split the calibrated source data out of the measurement set, use the CASA task {{split}}. <br />
<br />
<source lang="python"><br />
splitfile = 'NGC1667.split.ms'<br />
rmtables(splitfile) # get rid of any old versions before splitting<br />
split(vis='ag733.ms',outputvis=splitfile, datacolumn='corrected', field='NGC1667')<br />
</source><br />
<br />
Alternatively, you can take advantage of scripting to loop over the sources.<br />
<br />
<source lang="python"><br />
for source in sourcelist:<br />
splitfile = source + '.split.ms'<br />
rmtables(splitfile)<br />
split(vis='ag733.ms',outputvis=splitfile, <br />
datacolumn='corrected', field=source)<br />
</source><br />
<br />
== Imaging ==<br />
<br />
[[Image:NGC1667_example.png|thumb|An example desktop display of ''{{viewer}},'' showing the image display panel, the data display options window, and the data loader.]]<br />
<br />
Use {{clean}} for [http://www.aoc.nrao.edu/events/synthesis/2008/lectures/wilner_synthesis08.pdf imaging] and [http://adsabs.harvard.edu/abs/1974A%26AS...15..417H clean deconvolution]. Like IMAGR in AIPS, it's worth some time mulling over all of the options. A complete list of keywords is given in the example, below. Most of the options should be relatively familiar to the AIPS user. <!--, but one keyword bears mentioning in this example: '''calready = True'''. By selecting this option, the CLEAN model is transformed back to (''u'',''v'') space and stored as as a new column in the visibility data. The transformed CLEAN model will be necessary for [[#Self-calibration|self calibration]], described below. --><br />
<br />
For this snapshot survey, the example chooses a fairly conservative, slow clean to accommodate the less-than-ideal beam. You could probably get away with more efficient cleaning, but it's better to play it safe in scripted self-calibration.<br />
<br />
<source lang="python"><br />
default(clean)<br />
mode = 'mfs' <br />
niter = 2000 <br />
gain = 0.1 <br />
threshold = '4.5e-5Jy' <br />
psfmode = 'clark' <br />
imagermode = 'csclean' <br />
cyclefactor = 3 <br />
cyclespeedup = -1 <br />
multiscale = [] <br />
interactive = False <br />
mask = [] <br />
imsize = [1024, 1024] <br />
cell = ['0.75arcsec', '0.75arcsec']<br />
stokes = 'I' <br />
weighting = 'natural' <br />
uvtaper = False <br />
pbcor = False <br />
minpb = 0.1 <br />
usescratch = False <br />
async = False <br />
for source in sourcelist:<br />
splitfile = source + '.split.ms'<br />
vis = splitfile<br />
imagename = source + '_img'<br />
rmtables(imagename + '*') # get rid of earlier versions of the image<br />
clean()<br />
</source><br />
<br />
Take a look at the image using {{viewer}}.<br />
<br />
<source lang="python"><br />
viewer()<br />
</source><br />
<br />
From the GUI, select the .image file you want to view, select ''Raster Image'', and then dismiss and close your way through to the display.<br />
<br />
==Self-calibration==<br />
<br />
At this point we have a set of calibrated visibilities (the "splits") and a set of images. There's not much more that can be done for the fainter sources, but the brighter sources will almost certainly show some residual calibration artifacts that can be cleaned up with self-calibration. The strategy is to repeat using the primary calibration application {{gaincal}}, only this time we use the source itself as a calibrator.<br />
<br />
NGC 1068 is a good example of a difficult source. Firstly, it's very bright, so residual phase calibration errors should be obvious. Secondly, like the larger fraction of the sky, it's located painfully close to the celestial equator, meaning that the (''u'',''v'') coverage is particularly poor. <br />
<br />
To view the calibration artifacts, use {{viewer}} to display the image of NGC 1068, but adjust the display levels so that the displayed image saturates at 10 mJy. To do this, select the leftmost button bearing a wrench icon. The "Data Display Options" GUI opens, and you can enter the desired display stretch in the '''Data Range''' text box. To cutoff the stretch at 30 mJy,<br />
<br />
* Data Range: [-0.005,0.03]<br />
<br />
You can see some residual systematics in the background and sidelobe artifacts that suggest the phase calibration isn't as good as it might be.<br />
<br />
The process of self-calibration will involve iterating between (''i'') improved {{clean}} models of the source <!--(which are updated in the "split" visibility file so long as calready = True)--> and (''i''+1) improved calibrations generated by {{gaincal}}. For simplicity, in this example we will repeatedly generate new calibration tables so that we can avoid using ''accum.''<br />
<br />
=== Step ''i'' ===<br />
<br />
In principle, we could start with the initial {{clean}} model that we generated above. For this example, we'll take a more cautious approach and re-run {{clean}} interactively.<br />
<br />
<source lang="python"><br />
source = 'NGC1068' # set up a global variable to store the source we're working on<br />
tget clean<br />
vis = source + '.split.ms'<br />
imagename = source + '_img0'<br />
interactive = True<br />
npercycle = 10<br />
go<br />
</source><br />
<br />
The trick to interactive cleaning is to draw ''clean boxes'' around the sources as they appear in the residual image. By default, clean boxes are set by ''right-clicking'' on the image and dragging. The box can be edited until you ''double right-click'' inside the box, which then sets that box for cleaning. Press the green icon to continue cleaning the image; on the next major cycle, a new residual map will be displayed, affording the option to add clean boxes as new sources appear where they were once swamped by sidelobes.<br />
<br />
Of course, for self-calibration, it pays to stop the cleaning early when artifacts start to appear in the residuals!<br />
<br />
=== Step ''i'' + 1 ===<br />
<br />
To start, we should already have a clean model for NGC 1068 stored in the visibility database. Now we can set up {{gaincal}}.<br />
<br />
<source lang="python"><br />
default('gaincal') # reset inputs for gaincal (akin to RESTORE in AIPS)<br />
vis = source + '.split.ms'<br />
caltable = source + '.gcal0'<br />
gaintype = 'G'<br />
calmode = 'p' # phase-only self-calibration at this stage<br />
solint = '10 min' # set up a fairly coarse correction interval in this early stage<br />
minsnr = 3.0<br />
gaintable = ''<br />
field = source<br />
gainfield = source # are these necessary?<br />
gaincal() # or go!<br />
</source><br />
<br />
=== Pause for Reflection and Troubleshooting ===<br />
<br />
Self-calibration is a method to fine-tune your original calibration, and, as such, should produce relatively small phase corrections assuming the primary calibration was applied correctly. Now is a good time to run {{plotcal}} on your new calibration table (''NGC1068.gcal0'') to make sure the phase corrections range from a few to perhaps 10 or 20 degrees. If you see phase corrections greatly exceeding 20 degrees, then there's a chance of either<br />
<br />
* the primary calibration wasn't applied to the split database correctly, and should be checked, or<br />
<!--* the model wasn't updated in the last iteration of {{clean}}, meaning you should verify that ''calready = True.''--><br />
<br />
<source lang="python"><br />
plotcal(caltable, iteration='antenna', xaxis='time', yaxis='phase', plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
=== Step ''i'' + 2 ===<br />
<br />
Assuming {{plotcal}} gave reasonable results, next apply the calibration.<br />
<br />
<source lang="python"><br />
default('applycal')<br />
vis = source + '.split.ms'<br />
gaintable = source + '.gcal0'<br />
gainfield = source<br />
go<br />
</source><br />
<br />
And now we can generate a new {{clean}} model.<br />
<br />
<source lang="python"><br />
tget clean<br />
imagename=source + '_img1' # choose an image root name that matches the future cal table to facilitate bookkeeping<br />
go<br />
</source><br />
<br />
=== Further Steps ===<br />
<br />
Assuming things are going well, we can now continue to repeat the process until we are satisfied with the image. <br />
<br />
[[Image:N1068final.png|thumb|Final, self-calibrated image of NGC1068.]]<br />
<br />
<source lang="python"><br />
tget gaincal<br />
caltable = source +'.gcal1'<br />
solint = "5 min" # halve the solution interval. <br />
go<br />
<br />
# good idea to plotcal() again here!<br />
<br />
tget applycal<br />
gaintable = source + '.gcal1'<br />
go<br />
<br />
tget clean<br />
imagename = source + '_img2'<br />
go<br />
</source><br />
<br />
And so on, decreasing the solution interval as much as the S/N and sampling allow.<br />
<br />
Note that {{gaincal}} always uses the 'data' column (not the recently self-calibrated 'corrected' column) to find gain solutions. Therefore, these iterations simply serve to improve the data model, and gain solutions at each iteration are independent (e.g., the solution from the second self calibration iteration does ''not'' accumulate on top of the solution from the first interval. Use the CASA task {{accum}} if you wish to accumulate gain solutions). <br />
<br />
=== One Last Iteration: Amplitude & Phase Self Calibration ===<br />
<br />
After you have improved the phase calibration as much as possible, one last iteration where both phase ''and'' amplitude are calibrated can be beneficial. A&P self calibration should be performed with long solution intervals---typically the length of a scan. <br />
<br />
<source lang="python"><br />
default('gaincal') # reset inputs for gaincal (akin to RESTORE in AIPS)<br />
vis = source + '.split.ms'<br />
caltable = source + '.gcalap'<br />
gaintype = 'G'<br />
calmode = 'ap' # it's finally time to calibrate amplitudes, too<br />
solint = 'inf' # one solution interval per scan (in conjunction with combine='') <br />
combine = ''<br />
minsnr = 3.0<br />
gaintable = source + '.gcal1' # apply your best phase calibration on the fly. <br />
gaincal() # or go!<br />
</source><br />
<br />
Here, we are applying the best phase calibration solution on the fly by setting the ''gaintable'' parameter, so that this A&P self calibration iteration will only make small tweaks on top of the phase solutions we just worked so hard to find. Check your solutions in {{plotcal}}---amplitudes should not vary much and should be close to unity. Phase tweaks should be minimal and close to zero. If the solutions look reasonable, go ahead and apply the calibration:<br />
<br />
<source lang="python"><br />
default('applycal')<br />
vis = source + '.split.ms'<br />
gaintable = [source + '.gcal1', source + '.gcalap']<br />
gainfield = source<br />
go<br />
</source><br />
<br />
The critical difference here from phase-only self calibration iterations is that we are applying ''two'' gain tables---the best phase solutions (source + '.gcal1') and the A&P solutions (source + '.gcalap'). Now you should be ready for one final imaging run with {{clean}}! For consistency, we will run it with the same command we used the first time:<br />
<br />
<source lang="python"><br />
default(clean)<br />
mode = 'mfs' <br />
niter = 2000 <br />
gain = 0.1 <br />
threshold = '4.5e-5Jy' <br />
psfmode = 'clark' <br />
imagermode = 'csclean' <br />
cyclefactor = 3 <br />
cyclespeedup = -1 <br />
multiscale = [] <br />
interactive = False <br />
mask = [] <br />
imsize = [1024, 1024] <br />
cell = ['0.75arcsec', '0.75arcsec']<br />
stokes = 'I' <br />
weighting = 'natural' <br />
uvtaper = False <br />
pbcor = False <br />
minpb = 0.1 <br />
async = False <br />
usescratch = False <br />
vis = 'NGC1068.split.ms'<br />
imagename = 'NGC1068_img4'<br />
rmtables(imagename + '*') # get rid of earlier versions of the image<br />
clean()<br />
</source><br />
<br />
<br />
{{Checked 4.7.2}}<br />
<br />
[[Pre-upgrade VLA Tutorials | &#8629; '''Pre-upgrade VLA Tutorials''']]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Data_flagging_with_plotms&diff=21904Data flagging with plotms2017-06-23T21:01:01Z<p>Emomjian: </p>
<hr />
<div>[[Category: Interactive Tools]][[Category: VLA]][[Category: Flagging]]<br />
Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
[[plotms|Plotms]] is a tool to inspect and edit measurement sets. This tutorial demonstrates how to use {{plotms}} to edit a multisource continuum data set: VLA program [https://archive.nrao.edu/archive/ArchiveQuery?PASSWD=&QUERYTYPE=ARCHIVE&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&MAX_ROWS=NO+LIMIT&SORT_PARM2=Starttime&SORT_ORDER2=Asc&QUERY_ID=9999&QUERY_MODE=Prepare+Download&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&WRITELOG=0&PROJECT_CODE=AU0079&SEGMENT=&OBSERVER=&ARCHIVE_VOLUME=&TIMERANGE1=&TIMERANGE2=&SOURCE_ID=&SRC_SEARCH_TYPE=SIMBAD+or+NED+Resolver&CALIB_TYPE=ALL+Srcs&CENTER_RA=&LONG_RANGE=&FRAME=Equatorial&CENTER_DEC=&LAT_RANGE=&EQUINOX=J2000&SRAD=1.0%27&MIN_EXPOSURE=&OBS_BANDS=L&TELESCOPE=VLA&OBS_MODE=ALL&CORR_MODE=ALL&TELESCOPE_CONFIG=A&OBS_POLAR=ALL&OBSFREQ1=&DATATYPE=ALL&OBSBW1=&ARCHFORMAT=ALL&SUBMIT=Submit+Query AU079], which consists of L-band (20 cm) continuum observations of galaxies and calibrator sources. It is the same data set used in the [[Data flagging with viewer]] tutorial.<br />
<br />
== Loading the Measurement Set into [[plotms|Plotms]] ==<br />
<br />
The data for this tutorial may be loaded into CASA using the {{importvla}} command. The following commands import the data into the measurement set ''au079.ms'' and exit CASA to the command line.<br />
<br />
<source lang="python"><br />
# import the glob command for filename searching with wildcards<br />
from glob import glob<br />
<br />
# Define the list of files for reading. Use glob to perform wildcard matching with VLA archive filenames.<br />
fileList = glob('AU079_*.xp?')<br />
<br />
importvla(archivefiles=fileList,vis='au079.ms')<br />
exit()<br />
</source><br />
<br />
Now start up {{plotms}} from within CASA.<br />
<br />
<source lang="bash"><br />
# in CASA<br />
plotms()<br />
</source><br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
This command brings up the PlotMS window, shown with annotations at right. The window comprises three panels: the '''control panel''' (outlined in blue), the '''graphics panel''' (green), and the '''tools panel''' (red). The control panel controls the selection of data for display and the graphing parameters (axis selection, axis limits, and so on). The graphics panel is the display panel for two-dimensional (''x'', ''y'') projections of the data. The tools panel provides commands to interact with the graphics panel.<br />
<br />
The control panel further breaks down into a series of tabs, annotated as '''Top Tabs''' and '''Side Tabs''', which contain related plotting and editing control parameters. This tutorial employs only the '''Plots''' tab among the Top Tabs and the following Side Tabs.<br />
<br />
* '''Data''', which controls the selection of the measurement set and the selection of data within the measurement set.<br />
* '''Axes''', which controls the selection of data and plotting parameters for the (''x'', ''y'') graph.<br />
* '''Plot''', which affects the style of plotting symbols, whether or not flagged data points are shown, and axis labels.<br />
* '''Flagging''', which controls how flagging commands are extended (as of 1 Dec 2009, these flagging extensions are very limited but will likely improve as plotms continues development).<br />
<br />
In this tutorial, interactive commands in the PlotMS window will be summarized as (Tab)'''Command,''', where (Tab) represents the Side Tab where the command is found, and '''Command''' is the appropriate GUI interaction (button press, text field, checkbox, etc.). <br />
<br />
Use the (Data)'''Browse''' button, or enter the full pathname, to navigate to and select the measurement set<br />
(here, '''au079.ms'''). <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-1a.png| 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
<br />
|}<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Beta Alert:''' At the time of this writing, {{plotms}} was undergoing rapid development. The specific locations of tabs, whether side tabs or top tabs, for example, will likely change prior to release. Watch this space!<br />
</div><br />
<br />
== Identifying Bad Data by Discrepant Amplitudes ==<br />
<br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Have a first look at the data by hitting the (Data)'''Plot''' button. By default, the axes will be visibility amplitude vs. time. The ''y''-axis amplitudes aren't yet calibrated, but for the sake of the discussion we'll refer to them as flux densities in Jy. <br />
<br />
The ''x''-axis labeling is a little garbled in this development version of the software, but straightaway there appear some wildly discrepant data. For a typical decimeter-wave continuum data set, sources and calibrators are expected to show visibility amplitudes of a few Jy or less; visibilities with amplitudes in the 100s of Jy range are <s>likely</s> bogus. Here's how to flag them.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-01.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;"<br />
|<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
There's a simple pattern to flagging in plotms.<br />
* Highlight the data to be flagged using the '''Mark Regions''' [[Image:MarkRegionsButton.png]] tool; see the tools panel at the bottom of the Plot MS window.<br />
* Flag the data in the highlighted region using the '''Flag''' [[Image:FlagThoseData.png]] tool.<br />
<br />
The figure at right shows a highlighted region selected using the '''Mark Regions''' tool. After flagging, those data will be removed from the display unless (Plot)'''Flagged Points Symbol''' is set. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-3.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
|<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
The figure at right shows a close-up of the data that remain. The ''y''-axis scale was reduced to the range (0, 100 Jy) by using the (Axes)'''Range''' controls. <br />
<br />
Notice that you can set more than one region with the '''Mark Regions''' tool before flagging.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-4.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
'''Tip:''' The automatic scaling of the data axes are cached and so are unaffected by flagging. To rescale (semi-) automatically, change the <br />
(Axes)'''X Axis''' to some other arbitrary projection (say, Scan), (Axes)'''Plot''', and then reset (Axes)'''X Axis''' to its original state<br />
(say, Time).<br />
<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;"<br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
After zapping those obviously high visibilities, things become a little more challenging. The figure at right shows a close up of the remaining visibilities between 0 and 1 Jy flux density. <br />
<br />
There probably remain bad data there, but it's hard to tell on the crowded plot. At this point it's better to examine individual sources within this multisource measurement set.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-5.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
|<br />
<br />
|}<br />
<br />
== Examining Individual Sources within a Measurement Set ==<br />
<br />
In this part of the tutorial we'll look at the first two sources of the multisource measurement set (field = 0 and field = 1). The techniques are easily generalized to the remaining sources by incrementing the field identifier (field = 2, 3, 4, ...). <br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Use the following settings to look specifically at the first source of the measurement set.<br />
<br />
* (Data)'''Field''' = 0<br />
* (Axes)'''Y Axis Range''' = Automatic<br />
* (Plot)'''Unflagged Plots Symbol''' to Custom, 4 px (make the data points larger for clarity)<br />
* (Axes)'''Plot'''<br />
<br />
Notice that the '''Plot''' button is available from more than one tab. <br />
<br />
The figure at right shows the result. The bad data have been highlighted using the '''Mark Regions''' tool. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-02.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
<br />
The figure at right shows the unflagged data for the first source (field = 0) after flagging the obviously discrepant points. Things look deceptively OK, but in fact there remain bad data from one antenna. The antenna contributed poor data for the entire observation of this source, and, since the problem is not isolated in time, it is difficult to see it in this projection.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-03.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|-<br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
<br />
Here are the same data reprojected onto baseline separations, (Axes)'''X Axis''' = '''UVDist_L''' (projected baseline separations in units of the observing wavelength). The misbehaving antenna shows up as spikes in these snapshot observations, because each baseline with that antenna spans only a narrow range of baseline separations. (A longer observation would produce broader spikes, because the projected baseline separations would span a greater range as they rotate with the earth under the source.)<br />
<br />
The idea now would be to highlight a subset of the discrepant data as shown in the figure and extend the flags to the common antenna of these baselines. At the time of this writing, this option is not available in the development build of {{plotms}}, but keep an eye on (Flagging)'''Extend flags''' = '''Antenna'''. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-04.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
<br />
|}<br />
<br />
=== Antenna-Based Flagging ===<br />
<br />
Clearly we have a bad antenna on this field, and the question remains how to deal with it within {{plotms}}. One option we have is to plot baselines to one antenna at a time, using (Data)'''antenna''', until the bad data show up. This approach is not very practical, however, considering that a given measurement set may have dozens of telescopes to search through and perhaps as many sources.<br />
<br />
Instead, use the '''Locate''' [[File:casaplotms-locate-tool.png]] tool to list the properties of the data within the highlighted region. Here's a subset of the listing that results and is reported to the terminal running {{plotms}}.<br />
<br />
<pre><br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:25:55.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131224 Y=0.0534296 (27877/21/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:05.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=90195.5 Y=0.05591 (28759/22/159)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:05.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131224 Y=0.0503287 (29179/22/579)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:15.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131223 Y=0.0568316 (30477/23/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:15.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131223 Y=0.0611954 (30479/23/579)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=90205.1 Y=0.052479 (32659/25/159)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=10-11 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=147929 Y=0.0619749 (33151/25/651)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=8-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=115989 Y=0.0547681 (33219/25/719)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:45.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=7 X=90208.3 Y=0.0577008 (33958/26/158)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:45.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131221 Y=0.0504603 (34377/26/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:55.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131220 Y=0.0518869 (35679/27/579)<br />
</pre><br />
<br />
The antenna pairs are listed as '''BL=''' (baseline = ). The common, or culprit, antenna for the highlighted data is #10. Plot just the data to that antenna by selecting (Data)'''antenna'''&nbsp;=&nbsp;10.<br />
<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;"<br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Considering that the healthier data fall below 0.2 Jy, and all of the antenna #10 baselines commonly produce amplitudes in excess of 0.2 Jy, it's safe to conclude that antenna #10 produced corrupted data for this particular source. Here's how to zap those data.<br />
<br />
* As described above, use the (Data) tab to select the field and antenna (here, field = 0 and antenna = 10).<br />
* (Data)'''Plot'''<br />
* Use the '''Mark Regions''' [[Image:MarkRegionsButton.png]] tool to highlight '''all''' of the data on the screen; the goal here is to zap the data from that antenna.<br />
* Apply the '''Flag''' [[Image:FlagThoseData.png]] tool. <br />
<br />
All of the data in the graphics panel should have disappeared!<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-05.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Clear the (Data)'''antenna''' entry to plot again data from all baselines. The result is shown at right. The data look clean! Or, for the less sanguine, there remain no obviously bad data.<br />
<br />
Time to proceed to the next field, (Data)field = 1.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-06.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
=== Cross-Pol Data of Bright Sources ===<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
(Data)'''field''' = 1 is shown at right. This is a bright calibrator source. '''The data values near 0 Jy are cross-pol data rather than discrepant data'''; these cross-pol data won't be as obvious in the plots for fainter sources.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' To plot only a subset of the correlations, set the '''corr''' parameter in the Data tab. Typing '''RR,LL''' in this window should exclude the cross-pol data. <br />
</div><br />
<br />
<br />
<br />
We see some of the familiar spiking that we saw in field = 0. Use again the '''locate''' tool on the highlighted selection. Here's a trimmed snippet of the output.<br />
<br />
<pre><br />
Time=1999/08/30/05:16:35.0 BL=5-10 <br />
Time=1999/08/30/05:16:35.0 BL=10-13 <br />
Time=1999/08/30/05:16:35.0 BL=2-10 <br />
Time=1999/08/30/05:16:45.0 BL=10-18 <br />
Time=1999/08/30/05:16:45.0 BL=6-10 <br />
Time=1999/08/30/05:16:45.0 BL=3-10 <br />
Time=1999/08/30/05:16:45.0 BL=10-16 <br />
Time=1999/08/30/05:16:45.0 BL=10-24 <br />
</pre><br />
<br />
Notice that antenna #10 reappears as a culprit.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-07.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
=== Manual Flagging ===<br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-08.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
On the strength of inspection of two sources, antenna #10 appears to be producing consistently wonky data. At this point, it's worth getting out of {{plotms}} and returning to CASA to perform manual editing using {{flagdata}}. We'll throw caution to the wind and assume that antenna #10 was acting up during the entire observation.<br />
<br />
<source lang="python"><br />
# in casapy<br />
default("flagdata")<br />
vis = "au079.ms"<br />
antenna = "10"<br />
flagdata()<br />
</source><br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
Now, load the data back into {{plotms}}, and set things up to plot (Data)'''field''' = 1 again; see figure at right. The data look much cleaner, with a few wonky points that can be easily cleaned up with interactive flagging.<br />
<br />
== Displaying a Subset of Spectral Windows and Channels ==<br />
<br />
By default, {{plotms}} will plot all spectral windows and all channels. To select only a subset of spectral windows and/or channels, use the (Data)'''spw''' field. The normal CASA syntax ([[Selecting_Spectral_Windows_and_Channels |described here]]) should work in most cases. One notable exception is if you would like to display only one channel; in this case, in {{plotms}}, you still need to write it as a range. For example, to plot channel 30 of spectral window 0, type <tt>0:30~30</tt> in the (Data)'''spw''' field.<br />
<br />
== Displaying and Flagging Calibrated Data ==<br />
<br />
Once you have applied a calibration to your data with {{applycal}}, take note of the drop down menus called (Axes)'''Data Column'''. If you choose 'data', {{plotms}} will plot the raw uncalibrated data. To plot the calibrated version of the data, select 'corrrected'. If {{plotms}} is open and running in the background, and you apply an updated calibration to your data, just refresh the plot of the 'corrected' data, and you should see the newest calibration. <br />
<br />
[[Averaging data in plotms | &#8629; '''Averaging Data in Plotms''']]<br><br />
[[Pre-upgrade VLA Tutorials | &#8629; '''Pre-upgrade VLA Tutorials''']]<br><br />
[[Main Page | &#8629; '''CASA Guides''']]<br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Data_flagging_with_plotms&diff=21903Data flagging with plotms2017-06-23T20:59:12Z<p>Emomjian: </p>
<hr />
<div>[[Category: Interactive Tools]][[Category: VLA]][[Category: Flagging]]<br />
Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
[[plotms|Plotms]] is a tool to inspect and edit measurement sets. This tutorial demonstrates how to use {{plotms}} to edit a multisource continuum data set: VLA program [https://archive.nrao.edu/archive/ArchiveQuery?PASSWD=&QUERYTYPE=ARCHIVE&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&MAX_ROWS=NO+LIMIT&SORT_PARM2=Starttime&SORT_ORDER2=Asc&QUERY_ID=9999&QUERY_MODE=Prepare+Download&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&WRITELOG=0&PROJECT_CODE=AU0079&SEGMENT=&OBSERVER=&ARCHIVE_VOLUME=&TIMERANGE1=&TIMERANGE2=&SOURCE_ID=&SRC_SEARCH_TYPE=SIMBAD+or+NED+Resolver&CALIB_TYPE=ALL+Srcs&CENTER_RA=&LONG_RANGE=&FRAME=Equatorial&CENTER_DEC=&LAT_RANGE=&EQUINOX=J2000&SRAD=1.0%27&MIN_EXPOSURE=&OBS_BANDS=L&TELESCOPE=VLA&OBS_MODE=ALL&CORR_MODE=ALL&TELESCOPE_CONFIG=A&OBS_POLAR=ALL&OBSFREQ1=&DATATYPE=ALL&OBSBW1=&ARCHFORMAT=ALL&SUBMIT=Submit+Query AU079], which consists of L-band (20 cm) continuum observations of galaxies and calibrator sources. It is the same data set used in the [[Data flagging with viewer]] tutorial.<br />
<br />
== Loading the Measurement Set into [[plotms|Plotms]] ==<br />
<br />
The data for this tutorial may be loaded into CASA using the {{importvla}} command. The following commands import the data into the measurement set ''au079.ms'' and exit CASA to the command line.<br />
<br />
<source lang="python"><br />
# import the glob command for filename searching with wildcards<br />
from glob import glob<br />
<br />
# Define the list of files for reading. Use glob to perform wildcard matching with VLA archive filenames.<br />
fileList = glob('AU079_*.xp?')<br />
<br />
importvla(archivefiles=fileList,vis='au079.ms')<br />
exit()<br />
</source><br />
<br />
Now start up {{plotms}} from within CASA.<br />
<br />
<source lang="bash"><br />
# in CASA<br />
plotms()<br />
</source><br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
This command brings up the PlotMS window, shown with annotations at right. The window comprises three panels: the '''control panel''' (outlined in blue), the '''graphics panel''' (green), and the '''tools panel''' (red). The control panel controls the selection of data for display and the graphing parameters (axis selection, axis limits, and so on). The graphics panel is the display panel for two-dimensional (''x'', ''y'') projections of the data. The tools panel provides commands to interact with the graphics panel.<br />
<br />
The control panel further breaks down into a series of tabs, annotated as '''Top Tabs''' and '''Side Tabs''', which contain related plotting and editing control parameters. This tutorial employs only the '''Plots''' tab among the Top Tabs and the following Side Tabs.<br />
<br />
* '''Data''', which controls the selection of the measurement set and the selection of data within the measurement set.<br />
* '''Axes''', which controls the selection of data and plotting parameters for the (''x'', ''y'') graph.<br />
* '''Plot''', which affects the style of plotting symbols, whether or not flagged data points are shown, and axis labels.<br />
* '''Flagging''', which controls how flagging commands are extended (as of 1 Dec 2009, these flagging extensions are very limited but will likely improve as plotms continues development).<br />
<br />
In this tutorial, interactive commands in the PlotMS window will be summarized as (Tab)'''Command,''', where (Tab) represents the Side Tab where the command is found, and '''Command''' is the appropriate GUI interaction (button press, text field, checkbox, etc.). <br />
<br />
Use the (Data)'''Browse''' button, or enter the full pathname, to navigate to and select the measurement set<br />
(here, '''au079.ms'''). <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-1a.png| 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
<br />
|}<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Beta Alert:''' At the time of this writing, {{plotms}} was undergoing rapid development. The specific locations of tabs, whether side tabs or top tabs, for example, will likely change prior to release. Watch this space!<br />
</div><br />
<br />
== Identifying Bad Data by Discrepant Amplitudes ==<br />
<br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Have a first look at the data by hitting the (Data)'''Plot''' button. By default, the axes will be visibility amplitude vs. time. The ''y''-axis amplitudes aren't yet calibrated, but for the sake of the discussion we'll refer to them as flux densities in Jy. <br />
<br />
The ''x''-axis labeling is a little garbled in this development version of the software, but straightaway there appear some wildly discrepant data. For a typical decimeter-wave continuum data set, sources and calibrators are expected to show visibility amplitudes of a few Jy or less; visibilities with amplitudes in the 100s of Jy range are <s>likely</s> bogus. Here's how to flag them.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-01.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;"<br />
|<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
There's a simple pattern to flagging in plotms.<br />
* Highlight the data to be flagged using the '''Mark Regions''' [[Image:MarkRegionsButton.png]] tool; see the tools panel at the bottom of the Plot MS window.<br />
* Flag the data in the highlighted region using the '''Flag''' [[Image:FlagThoseData.png]] tool.<br />
<br />
The figure at right shows a highlighted region selected using the '''Mark Regions''' tool. After flagging, those data will be removed from the display unless (Plot)'''Flagged Points Symbol''' is set. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-3.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
|<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
The figure at right shows a close-up of the data that remain. The ''y''-axis scale was reduced to the range (0, 100 Jy) by using the (Axes)'''Range''' controls. <br />
<br />
Notice that you can set more than one region with the '''Mark Regions''' tool before flagging.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-4.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
'''Tip:''' The automatic scaling of the data axes are cached and so are unaffected by flagging. To rescale (semi-) automatically, change the <br />
(Axes)'''X Axis''' to some other arbitrary projection (say, Scan), (Axes)'''Plot''', and then reset (Axes)'''X Axis''' to its original state<br />
(say, Time).<br />
<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;"<br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
After zapping those obviously high visibilities, things become a little more challenging. The figure at right shows a close up of the remaining visibilities between 0 and 1 Jy flux density. <br />
<br />
There probably remain bad data there, but it's hard to tell on the crowded plot. At this point it's better to examine individual sources within this multisource measurement set.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-5.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
|<br />
<br />
|}<br />
<br />
== Examining Individual Sources within a Measurement Set ==<br />
<br />
In this part of the tutorial we'll look at the first two sources of the multisource measurement set (field = 0 and field = 1). The techniques are easily generalized to the remaining sources by incrementing the field identifier (field = 2, 3, 4, ...). <br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Use the following settings to look specifically at the first source of the measurement set.<br />
<br />
* (Data)'''Field''' = 0<br />
* (Axes)'''Y Axis Range''' = Automatic<br />
* (Plot)'''Unflagged Plots Symbol''' to Custom, 4 px (make the data points larger for clarity)<br />
* (Axes)'''Plot'''<br />
<br />
Notice that the '''Plot''' button is available from more than one tab. <br />
<br />
The figure at right shows the result. The bad data have been highlighted using the '''Mark Regions''' tool. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-02.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
<br />
The figure at right shows the unflagged data for the first source (field = 0) after flagging the obviously discrepant points. Things look deceptively OK, but in fact there remain bad data from one antenna. The antenna contributed poor data for the entire observation of this source, and, since the problem is not isolated in time, it is difficult to see it in this projection.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-03.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|-<br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
<br />
Here are the same data reprojected onto baseline separations, (Axes)'''X Axis''' = '''UVDist_L''' (projected baseline separations in units of the observing wavelength). The misbehaving antenna shows up as spikes in these snapshot observations, because each baseline with that antenna spans only a narrow range of baseline separations. (A longer observation would produce broader spikes, because the projected baseline separations would span a greater range as they rotate with the earth under the source.)<br />
<br />
The idea now would be to highlight a subset of the discrepant data as shown in the figure and extend the flags to the common antenna of these baselines. At the time of this writing, this option is not available in the development build of {{plotms}}, but keep an eye on (Flagging)'''Extend flags''' = '''Antenna'''. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-04.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
<br />
|}<br />
<br />
=== Antenna-Based Flagging ===<br />
<br />
Clearly we have a bad antenna on this field, and the question remains how to deal with it within {{plotms}}. One option we have is to plot baselines to one antenna at a time, using (Data)'''antenna''', until the bad data show up. This approach is not very practical, however, considering that a given measurement set may have dozens of telescopes to search through and perhaps as many sources.<br />
<br />
Instead, use the '''Locate''' [[File:casaplotms-locate-tool.png]] tool to list the properties of the data within the highlighted region. Here's a subset of the listing that results and is reported to the terminal running {{plotms}}.<br />
<br />
<pre><br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:25:55.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131224 Y=0.0534296 (27877/21/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:05.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=90195.5 Y=0.05591 (28759/22/159)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:05.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131224 Y=0.0503287 (29179/22/579)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:15.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131223 Y=0.0568316 (30477/23/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:15.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131223 Y=0.0611954 (30479/23/579)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=90205.1 Y=0.052479 (32659/25/159)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=10-11 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=147929 Y=0.0619749 (33151/25/651)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=8-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=115989 Y=0.0547681 (33219/25/719)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:45.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=7 X=90208.3 Y=0.0577008 (33958/26/158)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:45.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131221 Y=0.0504603 (34377/26/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:55.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131220 Y=0.0518869 (35679/27/579)<br />
</pre><br />
<br />
The antenna pairs are listed as '''BL=''' (baseline = ). The common, or culprit, antenna for the highlighted data is #10. Plot just the data to that antenna by selecting (Data)'''antenna'''&nbsp;=&nbsp;10.<br />
<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;"<br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Considering that the healthier data fall below 0.2 Jy, and all of the antenna #10 baselines commonly produce amplitudes in excess of 0.2 Jy, it's safe to conclude that antenna #10 produced corrupted data for this particular source. Here's how to zap those data.<br />
<br />
* As described above, use the (Data) tab to select the field and antenna (here, field = 0 and antenna = 10).<br />
* (Data)'''Plot'''<br />
* Use the '''Mark Regions''' [[Image:MarkRegionsButton.png]] tool to highlight '''all''' of the data on the screen; the goal here is to zap the data from that antenna.<br />
* Apply the '''Flag''' [[Image:FlagThoseData.png]] tool. <br />
<br />
All of the data in the graphics panel should have disappeared!<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-05.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Clear the (Data)'''antenna''' entry to plot again data from all baselines. The result is shown at right. The data look clean! Or, for the less sanguine, there remain no obviously bad data.<br />
<br />
Time to proceed to the next field, (Data)field = 1.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-06.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
=== Cross-Pol Data of Bright Sources ===<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
(Data)'''field''' = 1 is shown at right. This is a bright calibrator source. '''The data values near 0 Jy are cross-pol data rather than discrepant data'''; these cross-pol data won't be as obvious in the plots for fainter sources.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' To plot only a subset of the correlations, set the '''corr''' parameter in the Data tab. Typing '''RR,LL''' in this window should exclude the cross-pol data. <br />
</div><br />
<br />
<br />
<br />
We see some of the familiar spiking that we saw in field = 0. Use again the '''locate''' tool on the highlighted selection. Here's a trimmed snippet of the output.<br />
<br />
<pre><br />
Time=1999/08/30/05:16:35.0 BL=5-10 <br />
Time=1999/08/30/05:16:35.0 BL=10-13 <br />
Time=1999/08/30/05:16:35.0 BL=2-10 <br />
Time=1999/08/30/05:16:45.0 BL=10-18 <br />
Time=1999/08/30/05:16:45.0 BL=6-10 <br />
Time=1999/08/30/05:16:45.0 BL=3-10 <br />
Time=1999/08/30/05:16:45.0 BL=10-16 <br />
Time=1999/08/30/05:16:45.0 BL=10-24 <br />
</pre><br />
<br />
Notice that antenna #10 reappears as a culprit.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-07.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
=== Manual Flagging ===<br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-08.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
On the strength of inspection of two sources, antenna #10 appears to be producing consistently wonky data. At this point, it's worth getting out of {{plotms}} and returning to CASA to perform manual editing using {{flagdata}}. We'll throw caution to the wind and assume that antenna #10 was acting up during the entire observation.<br />
<br />
<source lang="python"><br />
# in casapy<br />
default("flagdata")<br />
vis = "au079.ms"<br />
antenna = "10"<br />
flagdata()<br />
</source><br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
Now, load the data back into {{plotms}}, and set things up to plot (Data)'''field''' = 1 again; see figure at right. The data look much cleaner, with a few wonky points that can be easily cleaned up with interactive flagging.<br />
<br />
== Displaying a Subset of Spectral Windows and Channels ==<br />
<br />
By default, {{plotms}} will plot all spectral windows and all channels. To select only a subset of spectral windows and/or channels, use the (Data)'''spw''' field. The normal CASA syntax ([[Selecting_Spectral_Windows_and_Channels |described here]]) should work in most cases. One notable exception is if you would like to display only one channel; in this case, in {{plotms}}, you still need to write it as a range. For example, to plot channel 30 of spectral window 0, type <tt>0:30~30</tt> in the (Data)'''spw''' field.<br />
<br />
== Displaying and Flagging Calibrated Data ==<br />
<br />
Once you have applied a calibration to your data with {{applycal}}, take note of the drop down menus called (Axes)'''Data Column'''. If you choose 'data', {{plotms}} will plot the raw uncalibrated data. To plot the calibrated version of the data, select 'corrrected'. If {{plotms}} is open and running in the background, and you apply an updated calibration to your data, just refresh the plot of the 'corrected' data, and you should see the newest calibration. <br />
<br />
[[Averaging data in plotms | &#8629; '''Averaging Data in Plotms''']]<br><br />
[[Pre-Upgrade VLA Tutorials | &#8629; '''Pre-Upgrade VLA Tutorials''']]<br><br />
[[Main Page | &#8629; '''CASA Guides''']]<br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Data_flagging_with_plotms&diff=21902Data flagging with plotms2017-06-23T20:58:41Z<p>Emomjian: </p>
<hr />
<div>[[Category: Interactive Tools]][[Category: VLA]][[Category: Flagging]]<br />
Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
{{VLA Intro}}<br />
<br />
{{Interactive Tools Intro}}<br />
<br />
[[plotms|Plotms]] is a tool to inspect and edit measurement sets. This tutorial demonstrates how to use {{plotms}} to edit a multisource continuum data set: VLA program [https://archive.nrao.edu/archive/ArchiveQuery?PASSWD=&QUERYTYPE=ARCHIVE&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&MAX_ROWS=NO+LIMIT&SORT_PARM2=Starttime&SORT_ORDER2=Asc&QUERY_ID=9999&QUERY_MODE=Prepare+Download&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&WRITELOG=0&PROJECT_CODE=AU0079&SEGMENT=&OBSERVER=&ARCHIVE_VOLUME=&TIMERANGE1=&TIMERANGE2=&SOURCE_ID=&SRC_SEARCH_TYPE=SIMBAD+or+NED+Resolver&CALIB_TYPE=ALL+Srcs&CENTER_RA=&LONG_RANGE=&FRAME=Equatorial&CENTER_DEC=&LAT_RANGE=&EQUINOX=J2000&SRAD=1.0%27&MIN_EXPOSURE=&OBS_BANDS=L&TELESCOPE=VLA&OBS_MODE=ALL&CORR_MODE=ALL&TELESCOPE_CONFIG=A&OBS_POLAR=ALL&OBSFREQ1=&DATATYPE=ALL&OBSBW1=&ARCHFORMAT=ALL&SUBMIT=Submit+Query AU079], which consists of L-band (20 cm) continuum observations of galaxies and calibrator sources. It is the same data set used in the [[Data flagging with viewer]] tutorial.<br />
<br />
== Loading the Measurement Set into [[plotms|Plotms]] ==<br />
<br />
The data for this tutorial may be loaded into CASA using the {{importvla}} command. The following commands import the data into the measurement set ''au079.ms'' and exit CASA to the command line.<br />
<br />
<source lang="python"><br />
# import the glob command for filename searching with wildcards<br />
from glob import glob<br />
<br />
# Define the list of files for reading. Use glob to perform wildcard matching with VLA archive filenames.<br />
fileList = glob('AU079_*.xp?')<br />
<br />
importvla(archivefiles=fileList,vis='au079.ms')<br />
exit()<br />
</source><br />
<br />
Now start up {{plotms}} from within CASA.<br />
<br />
<source lang="bash"><br />
# in CASA<br />
plotms()<br />
</source><br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
This command brings up the PlotMS window, shown with annotations at right. The window comprises three panels: the '''control panel''' (outlined in blue), the '''graphics panel''' (green), and the '''tools panel''' (red). The control panel controls the selection of data for display and the graphing parameters (axis selection, axis limits, and so on). The graphics panel is the display panel for two-dimensional (''x'', ''y'') projections of the data. The tools panel provides commands to interact with the graphics panel.<br />
<br />
The control panel further breaks down into a series of tabs, annotated as '''Top Tabs''' and '''Side Tabs''', which contain related plotting and editing control parameters. This tutorial employs only the '''Plots''' tab among the Top Tabs and the following Side Tabs.<br />
<br />
* '''Data''', which controls the selection of the measurement set and the selection of data within the measurement set.<br />
* '''Axes''', which controls the selection of data and plotting parameters for the (''x'', ''y'') graph.<br />
* '''Plot''', which affects the style of plotting symbols, whether or not flagged data points are shown, and axis labels.<br />
* '''Flagging''', which controls how flagging commands are extended (as of 1 Dec 2009, these flagging extensions are very limited but will likely improve as plotms continues development).<br />
<br />
In this tutorial, interactive commands in the PlotMS window will be summarized as (Tab)'''Command,''', where (Tab) represents the Side Tab where the command is found, and '''Command''' is the appropriate GUI interaction (button press, text field, checkbox, etc.). <br />
<br />
Use the (Data)'''Browse''' button, or enter the full pathname, to navigate to and select the measurement set<br />
(here, '''au079.ms'''). <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-1a.png| 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
<br />
|}<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Beta Alert:''' At the time of this writing, {{plotms}} was undergoing rapid development. The specific locations of tabs, whether side tabs or top tabs, for example, will likely change prior to release. Watch this space!<br />
</div><br />
<br />
== Identifying Bad Data by Discrepant Amplitudes ==<br />
<br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Have a first look at the data by hitting the (Data)'''Plot''' button. By default, the axes will be visibility amplitude vs. time. The ''y''-axis amplitudes aren't yet calibrated, but for the sake of the discussion we'll refer to them as flux densities in Jy. <br />
<br />
The ''x''-axis labeling is a little garbled in this development version of the software, but straightaway there appear some wildly discrepant data. For a typical decimeter-wave continuum data set, sources and calibrators are expected to show visibility amplitudes of a few Jy or less; visibilities with amplitudes in the 100s of Jy range are <s>likely</s> bogus. Here's how to flag them.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-01.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;"<br />
|<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
There's a simple pattern to flagging in plotms.<br />
* Highlight the data to be flagged using the '''Mark Regions''' [[Image:MarkRegionsButton.png]] tool; see the tools panel at the bottom of the Plot MS window.<br />
* Flag the data in the highlighted region using the '''Flag''' [[Image:FlagThoseData.png]] tool.<br />
<br />
The figure at right shows a highlighted region selected using the '''Mark Regions''' tool. After flagging, those data will be removed from the display unless (Plot)'''Flagged Points Symbol''' is set. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-3.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
|<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
The figure at right shows a close-up of the data that remain. The ''y''-axis scale was reduced to the range (0, 100 Jy) by using the (Axes)'''Range''' controls. <br />
<br />
Notice that you can set more than one region with the '''Mark Regions''' tool before flagging.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-4.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
'''Tip:''' The automatic scaling of the data axes are cached and so are unaffected by flagging. To rescale (semi-) automatically, change the <br />
(Axes)'''X Axis''' to some other arbitrary projection (say, Scan), (Axes)'''Plot''', and then reset (Axes)'''X Axis''' to its original state<br />
(say, Time).<br />
<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;"<br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
After zapping those obviously high visibilities, things become a little more challenging. The figure at right shows a close up of the remaining visibilities between 0 and 1 Jy flux density. <br />
<br />
There probably remain bad data there, but it's hard to tell on the crowded plot. At this point it's better to examine individual sources within this multisource measurement set.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-5.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
|<br />
<br />
|}<br />
<br />
== Examining Individual Sources within a Measurement Set ==<br />
<br />
In this part of the tutorial we'll look at the first two sources of the multisource measurement set (field = 0 and field = 1). The techniques are easily generalized to the remaining sources by incrementing the field identifier (field = 2, 3, 4, ...). <br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Use the following settings to look specifically at the first source of the measurement set.<br />
<br />
* (Data)'''Field''' = 0<br />
* (Axes)'''Y Axis Range''' = Automatic<br />
* (Plot)'''Unflagged Plots Symbol''' to Custom, 4 px (make the data points larger for clarity)<br />
* (Axes)'''Plot'''<br />
<br />
Notice that the '''Plot''' button is available from more than one tab. <br />
<br />
The figure at right shows the result. The bad data have been highlighted using the '''Mark Regions''' tool. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-02.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
<br />
The figure at right shows the unflagged data for the first source (field = 0) after flagging the obviously discrepant points. Things look deceptively OK, but in fact there remain bad data from one antenna. The antenna contributed poor data for the entire observation of this source, and, since the problem is not isolated in time, it is difficult to see it in this projection.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-03.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|-<br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
<br />
Here are the same data reprojected onto baseline separations, (Axes)'''X Axis''' = '''UVDist_L''' (projected baseline separations in units of the observing wavelength). The misbehaving antenna shows up as spikes in these snapshot observations, because each baseline with that antenna spans only a narrow range of baseline separations. (A longer observation would produce broader spikes, because the projected baseline separations would span a greater range as they rotate with the earth under the source.)<br />
<br />
The idea now would be to highlight a subset of the discrepant data as shown in the figure and extend the flags to the common antenna of these baselines. At the time of this writing, this option is not available in the development build of {{plotms}}, but keep an eye on (Flagging)'''Extend flags''' = '''Antenna'''. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-04.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
<br />
|}<br />
<br />
=== Antenna-Based Flagging ===<br />
<br />
Clearly we have a bad antenna on this field, and the question remains how to deal with it within {{plotms}}. One option we have is to plot baselines to one antenna at a time, using (Data)'''antenna''', until the bad data show up. This approach is not very practical, however, considering that a given measurement set may have dozens of telescopes to search through and perhaps as many sources.<br />
<br />
Instead, use the '''Locate''' [[File:casaplotms-locate-tool.png]] tool to list the properties of the data within the highlighted region. Here's a subset of the listing that results and is reported to the terminal running {{plotms}}.<br />
<br />
<pre><br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:25:55.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131224 Y=0.0534296 (27877/21/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:05.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=90195.5 Y=0.05591 (28759/22/159)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:05.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131224 Y=0.0503287 (29179/22/579)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:15.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131223 Y=0.0568316 (30477/23/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:15.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131223 Y=0.0611954 (30479/23/579)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=90205.1 Y=0.052479 (32659/25/159)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=10-11 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=147929 Y=0.0619749 (33151/25/651)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:35.0 BL=8-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=115989 Y=0.0547681 (33219/25/719)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:45.0 BL=10-19 Spw=0 Chan=0 Freq=1.4649 Corr=7 X=90208.3 Y=0.0577008 (33958/26/158)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:45.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=6 X=131221 Y=0.0504603 (34377/26/577)<br />
PlotMS::locate+ Scan=1 Field=0 Time=1999/08/30/05:26:55.0 BL=0-10 Spw=0 Chan=0 Freq=1.4649 Corr=8 X=131220 Y=0.0518869 (35679/27/579)<br />
</pre><br />
<br />
The antenna pairs are listed as '''BL=''' (baseline = ). The common, or culprit, antenna for the highlighted data is #10. Plot just the data to that antenna by selecting (Data)'''antenna'''&nbsp;=&nbsp;10.<br />
<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;"<br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Considering that the healthier data fall below 0.2 Jy, and all of the antenna #10 baselines commonly produce amplitudes in excess of 0.2 Jy, it's safe to conclude that antenna #10 produced corrupted data for this particular source. Here's how to zap those data.<br />
<br />
* As described above, use the (Data) tab to select the field and antenna (here, field = 0 and antenna = 10).<br />
* (Data)'''Plot'''<br />
* Use the '''Mark Regions''' [[Image:MarkRegionsButton.png]] tool to highlight '''all''' of the data on the screen; the goal here is to zap the data from that antenna.<br />
* Apply the '''Flag''' [[Image:FlagThoseData.png]] tool. <br />
<br />
All of the data in the graphics panel should have disappeared!<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-05.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Clear the (Data)'''antenna''' entry to plot again data from all baselines. The result is shown at right. The data look clean! Or, for the less sanguine, there remain no obviously bad data.<br />
<br />
Time to proceed to the next field, (Data)field = 1.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-06.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
=== Cross-Pol Data of Bright Sources ===<br />
{|vspace="100"<br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
(Data)'''field''' = 1 is shown at right. This is a bright calibrator source. '''The data values near 0 Jy are cross-pol data rather than discrepant data'''; these cross-pol data won't be as obvious in the plots for fainter sources.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' To plot only a subset of the correlations, set the '''corr''' parameter in the Data tab. Typing '''RR,LL''' in this window should exclude the cross-pol data. <br />
</div><br />
<br />
<br />
<br />
We see some of the familiar spiking that we saw in field = 0. Use again the '''locate''' tool on the highlighted selection. Here's a trimmed snippet of the output.<br />
<br />
<pre><br />
Time=1999/08/30/05:16:35.0 BL=5-10 <br />
Time=1999/08/30/05:16:35.0 BL=10-13 <br />
Time=1999/08/30/05:16:35.0 BL=2-10 <br />
Time=1999/08/30/05:16:45.0 BL=10-18 <br />
Time=1999/08/30/05:16:45.0 BL=6-10 <br />
Time=1999/08/30/05:16:45.0 BL=3-10 <br />
Time=1999/08/30/05:16:45.0 BL=10-16 <br />
Time=1999/08/30/05:16:45.0 BL=10-24 <br />
</pre><br />
<br />
Notice that antenna #10 reappears as a culprit.<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-07.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
=== Manual Flagging ===<br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:casaplotms-add-08.png | 500px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
On the strength of inspection of two sources, antenna #10 appears to be producing consistently wonky data. At this point, it's worth getting out of {{plotms}} and returning to CASA to perform manual editing using {{flagdata}}. We'll throw caution to the wind and assume that antenna #10 was acting up during the entire observation.<br />
<br />
<source lang="python"><br />
# in casapy<br />
default("flagdata")<br />
vis = "au079.ms"<br />
antenna = "10"<br />
flagdata()<br />
</source><br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
Now, load the data back into {{plotms}}, and set things up to plot (Data)'''field''' = 1 again; see figure at right. The data look much cleaner, with a few wonky points that can be easily cleaned up with interactive flagging.<br />
<br />
== Displaying a Subset of Spectral Windows and Channels ==<br />
<br />
By default, {{plotms}} will plot all spectral windows and all channels. To select only a subset of spectral windows and/or channels, use the (Data)'''spw''' field. The normal CASA syntax ([[Selecting_Spectral_Windows_and_Channels |described here]]) should work in most cases. One notable exception is if you would like to display only one channel; in this case, in {{plotms}}, you still need to write it as a range. For example, to plot channel 30 of spectral window 0, type <tt>0:30~30</tt> in the (Data)'''spw''' field.<br />
<br />
== Displaying and Flagging Calibrated Data ==<br />
<br />
Once you have applied a calibration to your data with {{applycal}}, take note of the drop down menus called (Axes)'''Data Column'''. If you choose 'data', {{plotms}} will plot the raw uncalibrated data. To plot the calibrated version of the data, select 'corrrected'. If {{plotms}} is open and running in the background, and you apply an updated calibration to your data, just refresh the plot of the 'corrected' data, and you should see the newest calibration. <br />
<br />
[[Averaging data in plotms | &#8629; '''Averaging Data in Plotms''']]<br><br />
[[Pre-Upgrade VLA Tutorials | &#8629; '''Pre-Upgrade VLA Tutorials''']]<br><br />
[[Main Page | &#8629; '''CASA Guides''']]<br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Data_flagging_with_viewer&diff=21901Data flagging with viewer2017-06-23T20:56:03Z<p>Emomjian: </p>
<hr />
<div>[[Category: Interactive Tools]][[Category: VLA]][[Category: Flagging]]<br />
Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
{{VLA Intro}}<br />
<br />
{{Interactive Tools Intro}}<br />
<br />
== Overview ==<br />
<br />
Your data will inevitably have some bad data. Perhaps a receiver on a given antenna is acting up, or perhaps the pointing on a given antenna is poor, and these problematic data will (hopefully) appear obviously in your measurement set. It's a good idea to inspect your data carefully before calibration and imaging, and CASA offers several tools to flag bad data interactively. For loyal users of TVFLG and SPFLG in AIPS, you'll be happy to know that {{viewer}} reproduces and even enhances these functionalities.<br />
<br />
This tutorial illustrates how to use {{viewer}} to flag poor quality data. It will use the continuum dataset [https://archive.nrao.edu/archive/ArchiveQuery?PASSWD=&QUERYTYPE=ARCHIVE&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&MAX_ROWS=NO+LIMIT&SORT_PARM2=Starttime&SORT_ORDER2=Asc&QUERY_ID=9999&QUERY_MODE=Prepare+Download&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&WRITELOG=0&PROJECT_CODE=AU0079&SEGMENT=&OBSERVER=&ARCHIVE_VOLUME=&TIMERANGE1=&TIMERANGE2=&SOURCE_ID=&SRC_SEARCH_TYPE=SIMBAD+or+NED+Resolver&CALIB_TYPE=ALL+Srcs&CENTER_RA=&LONG_RANGE=&FRAME=Equatorial&CENTER_DEC=&LAT_RANGE=&EQUINOX=J2000&SRAD=1.0%27&MIN_EXPOSURE=&OBS_BANDS=L&TELESCOPE=VLA&OBS_MODE=ALL&CORR_MODE=ALL&TELESCOPE_CONFIG=A&OBS_POLAR=ALL&OBSFREQ1=&DATATYPE=ALL&OBSBW1=&ARCHFORMAT=ALL&SUBMIT=Submit+Query AU079] from the VLA archive; this dataset is also used in the [[Data flagging with casaplotms]] tutorial.<br />
<br />
== Loading the Measurement Set into Viewer ==<br />
<br />
The data may be loaded into CASA using the {{importvla}} command. The following commands import the data into the measurement set ''au079.ms'' and send them to the {{viewer}}.<br />
<br />
<source lang="python"><br />
# from loaddata.py<br />
from glob import glob<br />
<br />
# Define the list of files for reading. Use glob to perform wildcard matching with VLA archive filenames.<br />
fileList = glob('AU079_*.xp?')<br />
<br />
importvla(archivefiles=fileList,vis='au079.ms')<br />
viewer('au079.ms') # load the data into the viewer for editing<br />
</source><br />
<br />
The {{viewer}} command can also be invoked without argument, in which case a file selector GUI will appear.<br />
<br />
== Setting Up for Antenna-based Flagging ==<br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
The figure at right shows the default {{viewer}} display for a measurement set. The axes are (''y'') time and (''x'') baseline, or antenna pair, enumerated by the common antenna number of the pair (like TVFLG in AIPS). Flagged data are highlighted in pale blue. <br />
<br />
Another window, titled ''Data Display Options'', will also appear, allowing you to alter the display and flagging options. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:au079_ms_in_viewer_with_flags.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
The blue highlights can be initially distracting, and they can be optionally suppressed by altering the '''Flagging Options''' in the '''Data Display Options'''. The figure at right shows the options.<br />
<br />
* Set '''Show Flagged Regions...''' to '''Masked to Background'''. This operation will effectively set the flagged regions to black. <br />
<br />
In anticipation of the antenna-based editing to come, it's also worth setting the antenna flag.<br />
<br />
* Set '''Flag / Unflag Entire Antenna''' to '''Yes'''. <br />
<br />
Now, when multiple baselines are selected, all baselines to the common antenna will be flagged.<br />
<br />
There are also options to flag all '''correlations''' or '''channels''', the latter being particularly helpful for spectral-line experiments. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_fig2.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
<br />
There is a choice of data to display. Select the '''MS and Visibility Selection''' tab on the '''Data Display Options''' window; see the figure at right. The '''Visibility Component''' menu allows you to choose to display visibility data from the following list.<br />
* Amplitude<br />
* Phase<br />
* Real (the real part of the visibilities)<br />
* Imaginary (the imaginary part of the visibilities)<br />
* Amp Diff (amplitudes after subtracting a local, rolling time-average)<br />
* Amp RMS (based on a rolling time window)<br />
* Phase Diff (phases after subtracting a local, rolling time-average)<br />
* Phase RMS (based on a rolling time window)<br />
<br />
The rolling time windows needed to calculate the '''Diff''' and '''RMS''' options are tuned using the '''Moving Average Size''' slider. <br />
<br />
All of these options are useful to investigate aberrant data, but '''Amp Diff''' can be efficient to start with. Reasonable options are shown in the figure at right. Be sure to turn down the display maximum ('''Data Maximum''' under '''Basic Settings''') to bring out the fainter variations.<br />
<br />
You can also select a single source, or subset of sources, to display using the '''Field IDs''' blank. After you change the '''Field IDs''' box, you'll probably need to click on the '''Apply''' button. If you don't see the button, scroll all the way down in the '''Data Display Options''' window (Note that you don't see the '''Apply''' button in the figure to the immediate right because several submenus are open, but you can see it in the figure above).<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_ampdiffselect.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
<br />
The result of these operations is illustrated in the figure at right. Data from the RR correlation are shown, but you can easily select the other correlations (RL, LR, LL) by using the blue, DVD-style control buttons [[File:vcrNext.png]] below the main display.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' For spectral line datasets, you'll probably also want to use the DVD-style control buttons to step through spectral channels. You can change which data dimension is affected by these control buttons with the '''Animation Axis''' menu, located under '''Display Axes''' in the '''Data Display Options''' window. <br />
</div><br />
<br />
<br />
The RR data look overall pretty good. The brighter horizontal stripes mark calibrator observations; brighter sources show greater noise. Antenna 13 looks a little junky, however, especially between time marks 1000 and 2500. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:ampDiffExample.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' For spectral line datasets, you might also like to display (''y'') time against (''x'') spectral channel (as in AIPS SPFLG). You can do this by going to '''Display Axes''' in the '''Data Display Options''' and selecting '''X Axis''' as '''Channel'''. <br />
</div><br />
<br />
== Antenna-Based Flagging ==<br />
<br />
The following steps outline flagging with {{viewer}}.<br />
<br />
* Zoom in on junky-looking data using the zoom tool [[File:viewer_zoomselect.png]].<br />
* Use one of the drawing selectors [[File:drawingSelector.png]] to identify the bad data. <br />
* Zoom out---either incrementally with [[File:zoomOut.png]] or to show the entire image with [[File:zoomTool.png]]---and repeat.<br />
* '''Important:''' save your edits! Under '''Flagging Options''', use the '''Save Edits''' button.<br />
<br />
This section of the tutorial provides a more detailed walkthrough.<br />
<br />
{|<br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Time to zap those junky antenna 13 data. First, use the zoom tool [[File:viewer_zoomselect.png]], which by default is activated by the '''left mouse button''', to click and drag a box around the region of interest. '''Left double-click''' within the box to zoom. The figure at right shows the zoom selector box and the resulting zoom. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_zoom.png | 400px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Now, use the drawing rectangle selector tool [[File:drawingSelector.png]], which by default is activated by the '''right mouse button'''. Right-click and drag to draw a box around the data to be flagged. Since we're flagging all to a common antenna, it's better to make the box narrow in the baseline (''x'') direction to avoid accidentally flagging the neighboring antenna. Then, '''right double-click''' within the selector box to flag.<br />
<br />
The figure at right illustrates the data selection tool and the result of data flagging. Although only a subset of the Antenna 13 baselines were selected, all of the Antenna 13 baselines were flagged because of our choice of '''Flag / Unflag Entire Antenna''' in the '''Data Display Options - Flagging Options''' menu.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
<br />
[[File:viewerFlag_flags.png | 400px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
== Finishing up ==<br />
<div style="float: right; width: 50%; text-align: center;"><br />
<br />
[[File:viewerSaveFlags.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
To flag additional data, zoom out [[File: zoomOut.png]], select correlations [[File:vcrNext.png]] as necessary, and repeat the steps outlined above. <br />
<br />
It is absolutely vital to save your edits before leaving {{viewer}}. From the '''Data Display Options''' window, select '''Flagging Options''' and '''Save Edits''' (see figure at right). You are now ready for calibration and imaging!<br />
<br />
== Displaying Calibrated Data ==<br />
<br />
You might like to double-check the data after calibration, to ensure that the calibration looks good and no new bad data has popped up. You can do this easily by selecting 'corrected' as the '''Visibility Type''', which can be found in '''MS and Visibility Selection''' section of the '''Data Display Options'''.<br />
<br />
[[Pre-upgrade VLA Tutorials | &#8629; '''Pre-upgrade VLA Tutorials''']]<br><br />
[[Main Page | &#8629; '''CASA Guides''']]<br />
<br />
<br />
<br />
<br />
<br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Data_flagging_with_viewer&diff=21898Data flagging with viewer2017-06-23T20:54:44Z<p>Emomjian: </p>
<hr />
<div>[[Category: Interactive Tools]][[Category: VLA]][[Category: Flagging]]<br />
Disclaimer: Due to continuous CASA software updates, GUI images may look different on more recent versions than those shown here.<br />
<br />
{{VLA Intro}}<br />
<br />
{{Interactive Tools Intro}}<br />
<br />
== Overview ==<br />
<br />
Your data will inevitably have some bad data. Perhaps a receiver on a given antenna is acting up, or perhaps the pointing on a given antenna is poor, and these problematic data will (hopefully) appear obviously in your measurement set. It's a good idea to inspect your data carefully before calibration and imaging, and CASA offers several tools to flag bad data interactively. For loyal users of TVFLG and SPFLG in AIPS, you'll be happy to know that {{viewer}} reproduces and even enhances these functionalities.<br />
<br />
This tutorial illustrates how to use {{viewer}} to flag poor quality data. It will use the continuum dataset [https://archive.nrao.edu/archive/ArchiveQuery?PASSWD=&QUERYTYPE=ARCHIVE&PROTOCOL=HTML&SORT_PARM=Starttime&SORT_ORDER=Asc&MAX_ROWS=NO+LIMIT&SORT_PARM2=Starttime&SORT_ORDER2=Asc&QUERY_ID=9999&QUERY_MODE=Prepare+Download&LOCKMODE=PROJECT&SITE_CODE=AOC&DBHOST=CHEWBACCA&WRITELOG=0&PROJECT_CODE=AU0079&SEGMENT=&OBSERVER=&ARCHIVE_VOLUME=&TIMERANGE1=&TIMERANGE2=&SOURCE_ID=&SRC_SEARCH_TYPE=SIMBAD+or+NED+Resolver&CALIB_TYPE=ALL+Srcs&CENTER_RA=&LONG_RANGE=&FRAME=Equatorial&CENTER_DEC=&LAT_RANGE=&EQUINOX=J2000&SRAD=1.0%27&MIN_EXPOSURE=&OBS_BANDS=L&TELESCOPE=VLA&OBS_MODE=ALL&CORR_MODE=ALL&TELESCOPE_CONFIG=A&OBS_POLAR=ALL&OBSFREQ1=&DATATYPE=ALL&OBSBW1=&ARCHFORMAT=ALL&SUBMIT=Submit+Query AU079] from the VLA archive; this dataset is also used in the [[Imaging Flanking Fields]] and [[Data flagging with casaplotms]] tutorials.<br />
<br />
== Loading the Measurement Set into Viewer ==<br />
<br />
As described in the [[Imaging Flanking Fields]] tutorial, the data may be loaded into CASA using the {{importvla}} command. The following commands import the data into the measurement set ''au079.ms'' and send them to the {{viewer}}.<br />
<br />
<source lang="python"><br />
# from loaddata.py<br />
from glob import glob<br />
<br />
# Define the list of files for reading. Use glob to perform wildcard matching with VLA archive filenames.<br />
fileList = glob('AU079_*.xp?')<br />
<br />
importvla(archivefiles=fileList,vis='au079.ms')<br />
viewer('au079.ms') # load the data into the viewer for editing<br />
</source><br />
<br />
The {{viewer}} command can also be invoked without argument, in which case a file selector GUI will appear.<br />
<br />
== Setting Up for Antenna-based Flagging ==<br />
<br />
{|vspace="100"<br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
The figure at right shows the default {{viewer}} display for a measurement set. The axes are (''y'') time and (''x'') baseline, or antenna pair, enumerated by the common antenna number of the pair (like TVFLG in AIPS). Flagged data are highlighted in pale blue. <br />
<br />
Another window, titled ''Data Display Options'', will also appear, allowing you to alter the display and flagging options. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:au079_ms_in_viewer_with_flags.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
The blue highlights can be initially distracting, and they can be optionally suppressed by altering the '''Flagging Options''' in the '''Data Display Options'''. The figure at right shows the options.<br />
<br />
* Set '''Show Flagged Regions...''' to '''Masked to Background'''. This operation will effectively set the flagged regions to black. <br />
<br />
In anticipation of the antenna-based editing to come, it's also worth setting the antenna flag.<br />
<br />
* Set '''Flag / Unflag Entire Antenna''' to '''Yes'''. <br />
<br />
Now, when multiple baselines are selected, all baselines to the common antenna will be flagged.<br />
<br />
There are also options to flag all '''correlations''' or '''channels''', the latter being particularly helpful for spectral-line experiments. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_fig2.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
<br />
There is a choice of data to display. Select the '''MS and Visibility Selection''' tab on the '''Data Display Options''' window; see the figure at right. The '''Visibility Component''' menu allows you to choose to display visibility data from the following list.<br />
* Amplitude<br />
* Phase<br />
* Real (the real part of the visibilities)<br />
* Imaginary (the imaginary part of the visibilities)<br />
* Amp Diff (amplitudes after subtracting a local, rolling time-average)<br />
* Amp RMS (based on a rolling time window)<br />
* Phase Diff (phases after subtracting a local, rolling time-average)<br />
* Phase RMS (based on a rolling time window)<br />
<br />
The rolling time windows needed to calculate the '''Diff''' and '''RMS''' options are tuned using the '''Moving Average Size''' slider. <br />
<br />
All of these options are useful to investigate aberrant data, but '''Amp Diff''' can be efficient to start with. Reasonable options are shown in the figure at right. Be sure to turn down the display maximum ('''Data Maximum''' under '''Basic Settings''') to bring out the fainter variations.<br />
<br />
You can also select a single source, or subset of sources, to display using the '''Field IDs''' blank. After you change the '''Field IDs''' box, you'll probably need to click on the '''Apply''' button. If you don't see the button, scroll all the way down in the '''Data Display Options''' window (Note that you don't see the '''Apply''' button in the figure to the immediate right because several submenus are open, but you can see it in the figure above).<br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_ampdiffselect.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<br />
<div style="float: left; width: 50%; align: left; vertical-align=middle;"><br />
<br />
The result of these operations is illustrated in the figure at right. Data from the RR correlation are shown, but you can easily select the other correlations (RL, LR, LL) by using the blue, DVD-style control buttons [[File:vcrNext.png]] below the main display.<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' For spectral line datasets, you'll probably also want to use the DVD-style control buttons to step through spectral channels. You can change which data dimension is affected by these control buttons with the '''Animation Axis''' menu, located under '''Display Axes''' in the '''Data Display Options''' window. <br />
</div><br />
<br />
<br />
The RR data look overall pretty good. The brighter horizontal stripes mark calibrator observations; brighter sources show greater noise. Antenna 13 looks a little junky, however, especially between time marks 1000 and 2500. <br />
<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:ampDiffExample.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
<div style="background-color: #dddddd;"><br />
'''Tip:''' For spectral line datasets, you might also like to display (''y'') time against (''x'') spectral channel (as in AIPS SPFLG). You can do this by going to '''Display Axes''' in the '''Data Display Options''' and selecting '''X Axis''' as '''Channel'''. <br />
</div><br />
<br />
== Antenna-Based Flagging ==<br />
<br />
The following steps outline flagging with {{viewer}}.<br />
<br />
* Zoom in on junky-looking data using the zoom tool [[File:viewer_zoomselect.png]].<br />
* Use one of the drawing selectors [[File:drawingSelector.png]] to identify the bad data. <br />
* Zoom out---either incrementally with [[File:zoomOut.png]] or to show the entire image with [[File:zoomTool.png]]---and repeat.<br />
* '''Important:''' save your edits! Under '''Flagging Options''', use the '''Save Edits''' button.<br />
<br />
This section of the tutorial provides a more detailed walkthrough.<br />
<br />
{|<br />
<br />
|- style="background-color:#eeeeee;" <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Time to zap those junky antenna 13 data. First, use the zoom tool [[File:viewer_zoomselect.png]], which by default is activated by the '''left mouse button''', to click and drag a box around the region of interest. '''Left double-click''' within the box to zoom. The figure at right shows the zoom selector box and the resulting zoom. <br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
[[File:viewerFlag_zoom.png | 400px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|- <br />
| <br />
<br />
<div style="float: left; width: 50%; align: left;"><br />
Now, use the drawing rectangle selector tool [[File:drawingSelector.png]], which by default is activated by the '''right mouse button'''. Right-click and drag to draw a box around the data to be flagged. Since we're flagging all to a common antenna, it's better to make the box narrow in the baseline (''x'') direction to avoid accidentally flagging the neighboring antenna. Then, '''right double-click''' within the selector box to flag.<br />
<br />
The figure at right illustrates the data selection tool and the result of data flagging. Although only a subset of the Antenna 13 baselines were selected, all of the Antenna 13 baselines were flagged because of our choice of '''Flag / Unflag Entire Antenna''' in the '''Data Display Options - Flagging Options''' menu.<br />
</div><br />
<br />
<div style="float: right; width: 50%; text-align: center;"><br />
<br />
[[File:viewerFlag_flags.png | 400px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
|}<br />
<br />
== Finishing up ==<br />
<div style="float: right; width: 50%; text-align: center;"><br />
<br />
[[File:viewerSaveFlags.png | 200px]] <br />
<br />
''Click to enlarge''<br />
</div><br />
<br />
To flag additional data, zoom out [[File: zoomOut.png]], select correlations [[File:vcrNext.png]] as necessary, and repeat the steps outlined above. <br />
<br />
It is absolutely vital to save your edits before leaving {{viewer}}. From the '''Data Display Options''' window, select '''Flagging Options''' and '''Save Edits''' (see figure at right). You are now ready for calibration and imaging!<br />
<br />
== Displaying Calibrated Data ==<br />
<br />
You might like to double-check the data after calibration, to ensure that the calibration looks good and no new bad data has popped up. You can do this easily by selecting 'corrected' as the '''Visibility Type''', which can be found in '''MS and Visibility Selection''' section of the '''Data Display Options'''.<br />
<br />
[[VLA Tutorials | &#8629; '''VLA Tutorials''']]<br><br />
[[Main Page | &#8629; '''CASA Guides''']]<br />
<br />
<br />
<br />
<br />
<br />
<br />
{{Checked 4.7.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=21508Karl G. Jansky VLA Tutorials2017-04-18T04:24:34Z<p>Emomjian: /* VLA CASA Pipeline Guide */</p>
<hr />
<div>== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 4.7) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 4.7) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 4.7.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 4.7)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 4.7)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 4.7)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.5.3)<br />
* This tutorial does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA CASA pipeline] run. The guide explains the pipeline requirements, pipeline execution options, and pipeline output products. The bulk of the tutorial consists of a detailed walk through the weblog and provides interpretations of the plots and tables. The guide also provides instructions on how to run spectral line and polarization data sets and explains how to edit the scan intents if required.<br />
</blockquote><br />
<br />
== CASA 4.7 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_basic_data_reduction-CASA4.7 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.0 VLA CASA Pipeline Guide (CASA 4.7.0)]<br />
--><br />
<br />
== CASA 4.6 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
== CASA 4.5 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
== CASA 4.4 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
== CASA 4.3 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
== CASA 4.2 versions ==<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
== CASA 4.1 versions ==<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
== CASA 3.4 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
== CASA 3.3 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=20813Karl G. Jansky VLA Tutorials2017-01-09T14:00:28Z<p>Emomjian: </p>
<hr />
<div>== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
If you are new to CASA, you may start with <h3>[https://casaguides.nrao.edu/index.php?title=Getting_Started_in_CASA Getting Started in CASA]</h3><br />
<br />
<!--<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
--><br />
<br />
== VLA Data Reduction Tutorials ==<br />
<br />
<br />
<br />
=== Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] (CASA 4.7) <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
<br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking ===<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] (CASA 4.7) <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
<br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
=== 3C129 P-band continuum imaging data reduction tutorial ===<br />
<blockquote><br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial 3C129 P-band Tutorial] (CASA 4.7.0)<br />
<br />
* This tutorial requires to download a dataset <font color=red>26GB</font> in size <br />
<br />
This low frequency VLA tutorial for the observation of radio galaxy 3C 129 focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime and demonstrates multi-frequency synthesis imaging with w-projection. This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
* Widefield imaging <br />
</blockquote><br />
<br />
== VLA Topical Guides ==<br />
<br />
<br />
=== Flagging VLA Data in CASA === <br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Flagging Topical Guide: Flagging VLA Data] (CASA 4.7)<br />
<br />
* This tutorial requires to download a dataset <font color=green>4.9GB</font> in size<br />
<br />
This tutorial describes different flagging techniques, including:<br />
<br />
* Identifying Problematic Antennas from the Operator Logs<br />
* Online Flags (shadowing, zeroes, quacking, flagmanager, Hanning smoothing)<br />
* Automatic RFI excision (TFCrop, RFlag)<br />
* Interactive Flagging<br />
* Flagging Summary & Report<br />
</blockquote><br />
<br />
<br />
=== Imaging VLA Data in CASA ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Imaging Topical Guide: Imaging VLA Data] (CASA 4.7)<br />
<br />
* This tutorial requires to download a dataset <font color=green>1.2GB</font> in size<br />
<br />
The Imaging topical guide shows annotated examples of different imaging techniques that are applicable to different science applications. This tutorial includes:<br />
<br />
* An Introduction to the CLEAN algorithm, data and imaging weights, primary and synthesized beams<br />
* Clean Output Images<br />
* Dirty Image, Regular CLEAN & RMS Noise, CLEAN with different Weights<br />
* Multi-Scale CLEAN<br />
* Multi-Scale, Wide-Field CLEAN (w-projection)<br />
* Multi-Scale, Multi-Term Frequency Synthesis<br />
* Multi-Scale, Multi-Term Frequency, Widefield CLEAN<br />
* Imaging Outlier Fields<br />
* Primary Beam Correction<br />
* Imaging Spectral Cubes, Beam per Plane<br />
* Image Header, Image Conversion<br />
</blockquote><br />
<br />
<br />
=== Correcting for a Spectral Index in Bandpass Calibration ===<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope Topical Guide: Correcting for a Spectral Index in Bandpass Calibration] (CASA 4.7)<br />
* This tutorial requires to download a dataset <font color=green>3.4GB</font> in size<br />
<br />
This tutorial explains how to obtain a bandpass solution when the bandpass calibrator is a source with an unknown spectral index. This is typically encountered at high frequency observations, where the standard VLA flux density calibrators are too weak for high signal-to-noise bandpass solutions.<br />
</blockquote><br />
<br />
<br />
<br />
<!-- <br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
<br />
<!--<br />
== Specific Topics ==<br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
<br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
--><br />
<br />
== VLA CASA Pipeline Guide ==<br />
<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide] (CASA 4.5.3)<br />
* This tutorial requires does not require any download. <br />
<br />
The VLA CASA Pipeline Guide provides an example of a [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA CASA pipeline] run. The guide explains the pipeline requirements, pipeline execution options, and pipeline output products. The bulk of the tutorial consists of a detailed walk through the weblog and provides interpretations of the plots and tables. The guide also provides instructions on how to run spectral line and polarization data sets and explains how to edit the scan intents if required.<br />
</blockquote><br />
<br />
<br />
== CASA 4.7 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.7 IRC+10216 Tutorial (CASA4.7)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.7 3C391 Continuum Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_Radio_galaxy_3C_129:_P-band_continuum_tutorial-CASA4.7.0 3C129 P-band Tutorial (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.7.0 Topical Guide: Flagging VLA Data (CASA 4.7)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.7.0 Topical Guide: Imaging VLA Data (CASA 4.7)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.7.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.7)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/P-band_basic_data_reduction-CASA4.7 P-band Tutorial (CASA 4.7)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.7.0 VLA CASA Pipeline Guide (CASA 4.7.0)]<br />
--><br />
<br />
== CASA 4.6 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.6 0313-192 P-band Tutorial (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging-CASA4.6.0 Topical Guide: Flagging VLA Data (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Imaging-CASA4.6.0 Topical Guide: Imaging VLA Data (CASA 4.6)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Bandpass_Slope-CASA4.6.0 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.6)]<br />
<br />
== CASA 4.5 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.5 0313-192 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Topical Guide: Flagging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Topical Guide: Imaging VLA Data (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Topical Guide: Correcting for a Spectral Index in Bandpass Calibration (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline-CASA4.5.3 VLA CASA Pipeline Guide (CASA 4.5.3)]<br />
<br />
== CASA 4.4 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/0313-192_P-band_basic_data_reduction-CASA4.4 0313-192 P-band basic data reduction (CASA 4.4)]<br />
<br />
== CASA 4.3 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
== CASA 4.2 versions ==<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
== CASA 4.1 versions ==<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
== CASA 3.4 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
== CASA 3.3 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Bandpass_Slope-CASA4.6.0&diff=20023VLA CASA Bandpass Slope-CASA4.6.02016-06-14T14:44:34Z<p>Emomjian: </p>
<hr />
<div><br />
<b>This CASA Guide is for CASA version 4.6</b><br />
<br />
<br />
== Overview ==<br />
<br />
For the standard VLA flux density calibrators 3C138, 3C147, 3C286 and 3C48, CASA includes spatial and spectral models that are applied during calibration. The models account for the source characteristics, resulting in calibration solutions that represent the instrumental and atmospheric corrections. These VLA standard calibrators, however, exhibit a negative spectral index and are relatively weak at high frequencies.<br />
<br />
Although the standard VLA flux density calibrators are usually still bright enough for absolute flux density calibration, a good bandpass determination&#151;which is very important for spectral line observations&#151;requires large signal-to-noise ratios derived from either a long integration time or a very strong source (see the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/line#section-8 Spectral Line Guide for Observing]). Observations of non-standard, but strong, bandpass calibrators are therefore common at high frequencies. Unfortunately, such sources are likely variable and no ''a priori'' flux density model is available. In particular, these sources exhibit an unknown and maybe variable spectral slope, which, if not accounted for, will create an error in the bandpass calibration. This tutorial describes how to model a spectral slope and how to correct the bandpass solution for this effect. <br />
<br />
Data used in this guide are taken in wide 3-bit mode for the protostar G192.16-3.84 in Ka-band with basebands centered at 29 and 36.5 GHz. Each baseband has over 4 GHz of bandwidth comprising thirty-two 128-MHz spectral windows.<br />
<br />
If you are new to CASA, or with VLA data reduction in CASA, it is '''strongly''' recommended that you start with the [[Getting Started in CASA]] guide, the [https://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 spectral line tutorial], or the [[VLA Continuum Tutorial 3C391]] before proceeding with this tutorial.<br />
<br />
== Obtaining the Data ==<br />
<br />
As this tutorial concerns bandpass calibration, all sources other than the flux density and bandpass calibrator scans were removed from the measurement set (MS). All pre-calibration steps including flagging, antenna position offsets, requantizer gains, opacity corrections, and gain-elevation curves were applied. The original data (<tt>TVER0004.sb14459364.eb14492359.56295.26287841435</tt>) can be obtained through the [https://archive.nrao.edu NRAO archive] and has a raw size of 57.04 GB.<br />
<br />
The trimmed measurement set can be downloaded directly from [http://casa.nrao.edu/Data/EVLA/G192/G192-BP.ms.tar.gz http://casa.nrao.edu/Data/EVLA/G192/G192-BP.ms.tar.gz] (dataset size: 3.4 GB)<br />
<br />
Your first step will be to unzip and untar the file in a terminal (before you start CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf G192-BP.ms.tar.gz <br />
</source><br />
<br />
== Starting CASA ==<br />
<br />
To start CASA, type:<br />
<br />
<source lang="bash"><br />
casa<br />
</source><br />
<br />
This will run a script initializing CASA and setting paths appropriately. The script will also create two files called ''ipython-<unique-stamp>.log'' (which contains a record of all the text you enter at the CASA prompt) as well as ''casapy-<unique-stamp>.log'' (which will contain all the messages that are printed to the CASA logger window). It is recommended that you keep your log files intact&#151;you may need them to remind you of the last step you completed in your data reduction. (It is also a good idea to include your log files when submitting a help desk ticket).<br />
<br />
Once CASA has started, a logger window will appear. Note that you can rescale this window or change the font size (under the '''View''' menu option) as desired.<br />
<br />
== Examining the Measurement Set (MS) ==<br />
<br />
We use {{listobs}} to summarize our MS:<br />
<source lang="python"><br />
# In CASA: listobs on the initial data set<br />
listobs('G192-BP.ms', listfile='G192_listobs.txt')<br />
</source><br />
<br />
This will write the output to a file called <tt>G192_listobs.txt</tt>, which we can print to the terminal using various Unix/Linux commands such as <tt>cat</tt>, <tt>less</tt>, or <tt>more</tt>:<br />
<br />
<source lang="python"><br />
# In CASA<br />
cat G192_listobs.txt<br />
</source><br />
<br />
<pre><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/jott/casa/topicalguide/bandpass/new/G192-BP.ms MS Version 2<br />
================================================================================<br />
Observer: Dr. Debra Shepherd Project: uid://evla/pdb/7303457 <br />
Observation: EVLA<br />
Data records: 1769355 Total elapsed time = 4563 seconds<br />
Observed from 03-Jan-2013/06:31:48.0 to 03-Jan-2013/07:47:51.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
03-Jan-2013/06:31:48.0 - 06:36:42.0 6 0 3C147 704865 [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63] [6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5.94, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6] [CALIBRATE_FLUX#UNSPECIFIED,OBSERVE_TARGET#UNSPECIFIED]<br />
07:40:27.0 - 07:47:51.0 64 1 3c84-J0319+413 1064490 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63] [6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6] [CALIBRATE_BANDPASS#UNSPECIFIED,OBSERVE_TARGET#UNSPECIFIED]<br />
(nRows = Total number of rows per scan) <br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 E 3C147 05:42:36.137916 +49.51.07.23356 J2000 0 704865<br />
1 F 3c84-J0319+413 03:19:48.160102 +41.30.42.10305 J2000 1 1064490<br />
Spectral Windows: (64 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 EVLA_KA#A1C1#2 128 TOPO 34476.000 1000.000 128000.0 34539.5000 10 RR LL<br />
1 EVLA_KA#A1C1#3 128 TOPO 34604.000 1000.000 128000.0 34667.5000 10 RR LL<br />
2 EVLA_KA#A1C1#4 128 TOPO 34732.000 1000.000 128000.0 34795.5000 10 RR LL<br />
3 EVLA_KA#A1C1#5 128 TOPO 34860.000 1000.000 128000.0 34923.5000 10 RR LL<br />
<br />
<snip><br />
<br />
13 EVLA_KA#A1C1#15 128 TOPO 36140.000 1000.000 128000.0 36203.5000 10 RR LL<br />
14 EVLA_KA#A1C1#16 128 TOPO 36268.000 1000.000 128000.0 36331.5000 10 RR LL<br />
15 EVLA_KA#A1C1#17 128 TOPO 36396.000 1000.000 128000.0 36459.5000 10 RR LL<br />
16 EVLA_KA#A2C2#18 128 TOPO 36476.000 1000.000 128000.0 36539.5000 11 RR LL<br />
17 EVLA_KA#A2C2#19 128 TOPO 36604.000 1000.000 128000.0 36667.5000 11 RR LL<br />
18 EVLA_KA#A2C2#20 128 TOPO 36732.000 1000.000 128000.0 36795.5000 11 RR LL<br />
<br />
<snip><br />
<br />
29 EVLA_KA#A2C2#31 128 TOPO 38140.000 1000.000 128000.0 38203.5000 11 RR LL<br />
30 EVLA_KA#A2C2#32 128 TOPO 38268.000 1000.000 128000.0 38331.5000 11 RR LL<br />
31 EVLA_KA#A2C2#33 128 TOPO 38396.000 1000.000 128000.0 38459.5000 11 RR LL<br />
32 EVLA_KA#B1D1#34 128 TOPO 26976.000 1000.000 128000.0 27039.5000 13 RR LL<br />
33 EVLA_KA#B1D1#35 128 TOPO 27104.000 1000.000 128000.0 27167.5000 13 RR LL<br />
34 EVLA_KA#B1D1#36 128 TOPO 27232.000 1000.000 128000.0 27295.5000 13 RR LL<br />
<br />
<snip><br />
<br />
45 EVLA_KA#B1D1#47 128 TOPO 28640.000 1000.000 128000.0 28703.5000 13 RR LL<br />
46 EVLA_KA#B1D1#48 128 TOPO 28768.000 1000.000 128000.0 28831.5000 13 RR LL<br />
47 EVLA_KA#B1D1#49 128 TOPO 28896.000 1000.000 128000.0 28959.5000 13 RR LL<br />
48 EVLA_KA#B2D2#50 128 TOPO 28976.000 1000.000 128000.0 29039.5000 14 RR LL<br />
49 EVLA_KA#B2D2#51 128 TOPO 29104.000 1000.000 128000.0 29167.5000 14 RR LL<br />
50 EVLA_KA#B2D2#52 128 TOPO 29232.000 1000.000 128000.0 29295.5000 14 RR LL<br />
<br />
<snip><br />
<br />
61 EVLA_KA#B2D2#63 128 TOPO 30640.000 1000.000 128000.0 30703.5000 14 RR LL<br />
62 EVLA_KA#B2D2#64 128 TOPO 30768.000 1000.000 128000.0 30831.5000 14 RR LL<br />
63 EVLA_KA#B2D2#65 128 TOPO 30896.000 1000.000 128000.0 30959.5000 14 RR LL<br />
Sources: 128<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 3C147 0 - - <br />
0 3C147 1 - - <br />
0 3C147 2 - - <br />
<br />
<snip><br />
<br />
0 3C147 61 - - <br />
0 3C147 62 - - <br />
0 3C147 63 - - <br />
1 3c84-J0319+413 0 - - <br />
1 3c84-J0319+413 1 - - <br />
1 3c84-J0319+413 2 - - <br />
<br />
<snip><br />
<br />
1 3c84-J0319+413 61 - - <br />
1 3c84-J0319+413 62 - - <br />
1 3c84-J0319+413 63 - - <br />
<br />
<snip><br />
</pre><br />
This small MS contains only scans on the flux density calibrator 3C147 (field 0) and the bandpass calibrator 3C84 (field 1). Both fields have 64 spectral windows (spws); each spw is comprised of 128 channels, each channel being 1 MHz wide, for a total bandwidth of 128 MHz / spw.<br />
<br />
== Setting the Model of the Flux Density Calibrator ==<br />
<br />
<br />
To start, we insert the spectral (using the 'Perley-Butler 2013' standard) and spatial (3C147_A.im for Ka-band) models for the flux density calibrator 3C147 (field 0) with the {{setjy}} task:<br />
<br />
<source lang="python"><br />
# In CASA: model for the flux density calibrator<br />
setjy(vis='G192-BP.ms', field='0', scalebychan=True, \<br />
standard='Perley-Butler 2013', model='3C147_A.im')<br />
</source><br />
<br />
[[Image:ScreenshotPlotG192-setjy-4.5.png|200px|thumb|right|Figure 1: plotms of model amp vs freq for 3C147]]<br />
* <tt>scalebychan=True</tt>: If ''scalebychan=False'' {{setjy}} would use a single value per spectral window.<br />
<br />
Inspecting the logger report shows that 3C147 has amplitudes ranging from ~1.0-1.47 Jy across all spws. <br />
<br />
We can plot the model data using {{plotms}} (Figure 1):<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='G192-BP.ms', field='0', antenna='ea03', \<br />
xaxis='freq', yaxis='amp', ydatacolumn='model',coloraxis='ant2')<br />
</source><br />
<br />
This plot shows baselines to antenna ea03. Since we provided both a spectral and a spatial model for this well resolved calibrator, each baseline has a somewhat different behavior.<br />
<br />
== Calibrating delays and initial bandpass solutions ==<br />
<br />
As a first step, we need to specify a reference antenna for all phase calibrations. It is desirable to use an antenna that is near the center of the array and has the least amount of calibrator data flagged. The array can be mapped with {{plotants}}:<br />
<br />
<source lang="python"><br />
# In CASA: plotting antenna locations<br />
plotants(vis='G192-BP.ms')<br />
</source><br />
Although the plot is a bit crowded (Figure 2), a zooming in (with the magnifying glass icon) shows that ea05 is located close to the center. It also has a comparably small number of flags and we will use this antenna as our reference.<br />
<br />
[[Image:plotG192_plotants.png|200px|thumb|right|Figure 2: plotants plotter]]<br />
<br />
We start with a phase-only, time-dependent calibration solution for the bandpass calibrator. Solutions for each integration will remove most of the decorrelation of the signal. For best results, we will derive the phase variations from a narrow range of channels (60~68) near the centers of each spws:<br />
<br />
<source lang="python"><br />
# In CASA: phase only calibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G0', \<br />
field='1', spw='*:60~68', \<br />
gaintype='G', refant='ea05', calmode='p', \<br />
solint='int', minsnr=3)<br />
</source><br />
<br />
* <tt>refant='ea05'</tt> : Use ea05 as the reference antenna<br />
* <tt>solint='int'</tt> : Do a per-integration solve (every 6 seconds, since we've time-averaged the data).<br />
* <tt>minsnr=3</tt> : Apply a minimum signal-to-noise cutoff. Solutions with less than this value will be flagged.<br />
* <tt>gaintable</tt> is not set here as we have already applied pre-calibrations.<br />
<br />
Plot the phase solutions (using full phase range -180 to 180 instead of autorange):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
Click on the Next button to navigate through the antennas. Click on the Quit button to exit the viewer.<br />
<br />
We will now produce multipanel plots of the phase solutions, writing the plots to output files as well as on the screen (Figures 3a & 3b). The output files generated are PNG files and can be viewed within CASA by executing an external viewer program, e.g., <tt>!xv plotG192_plotcal_G0p1.png</tt>; or by running an image viewing application such as xv, Preview, Gimp, Photoshop, etc., external to CASA at the OS level. <!-- (Note that the hardcopy only shows the first page): --><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
antenna='0~10,12~15', subplot=531, iteration='antenna', \<br />
plotrange=[-1,-1,-180,180], fontsize=8.0, \<br />
markersize=3.0, figfile='plotG192_plotcal_G0p1.png')<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
antenna='16~26', subplot=531, iteration='antenna', \<br />
plotrange=[-1,-1,-180,180], fontsize=8.0, \<br />
markersize=3.0, figfile='plotG192_plotcal_G0p2.png')<br />
</source><br />
<br />
{|<br />
| [[Image:plotG192_plotcal_G0p1_4.0.png|200px|thumb|left|Figure 3a: plotcal G0 phase ant 0~15]]<br />
| [[Image:plotG192_plotcal_G0p2_4.0.png|200px|thumb|center|Figure 3b: plotcal G0 phase ant 16~26]]<br />
|}<br />
<br />
<!--<br />
NOTE: When you are done plotting and want to use the calibration table in another task (e.g., for subsequent calibration or viewing with plotms), use the Quit button on the GUI to dismiss the plotter and free-up the lock on the calibration table. You should see a message in your terminal window saying "Resetting plotcal" which means you are good to go!<br />
--><br />
<br />
We can now solve for the residual delays using the parameter ''gaintype='K' ''option in {{gaincal}}. Note that this currently does not do a global fringe-fitting solution for delays, but instead does a baseline-based delay solution per spw for all baselines to the reference antenna, treating these as antenna-based delays. In most cases, with high enough S/N to get baseline-based delay solutions, this will suffice. We avoid the edge channels of each spectral window by selecting channels 5~122:<br />
<br />
<source lang="python"><br />
# In CASA: residual delays<br />
gaincal(vis='G192-BP.ms', caltable='calG192.K0', \<br />
field='1', spw='*:5~122', gaintype='K', \<br />
gaintable=['calG192.G0'],<br />
refant='ea05', solint='inf', minsnr=3)<br />
</source><br />
<br />
Note that we have also pre-applied our initial phase table <tt>calG192.G0</tt>. <br />
<br />
Alternatively, you can also derive a delay across all spws of a baseband. If this is desired, use parameter ''combine='spw''' in {{gaincal}} and run the task for each baseband separately. The solutions from the second and following runs can be appended to the same calibration table via parameter ''append=T''.<br />
<br />
[[Image:plotG192_plotcal_delays.png|200px|thumb|right|Figure 4: plotcal K0 delay vs. antenna]]<br />
<br />
Now plot the delays, in nanoseconds, as a function of antenna index (you will get one for each spw and polarization):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.K0', xaxis='antenna', yaxis='delay')<br />
</source><br />
<br />
The delays range from around -5 to 4 nanoseconds (Figure 4).<br />
<br />
Now solve for the antenna bandpasses using the previously generated tables <tt>calG192.G0</tt> and <tt>calG192.K0</tt>:<br />
<br />
<source lang="python"><br />
# In CASA: antenna bandpasses<br />
bandpass(vis='G192-BP.ms', caltable='calG192.B0', \<br />
gaintable=['calG192.G0', 'calG192.K0'], \<br />
field='1', refant='ea05', solnorm=False, \<br />
bandtype='B', solint='inf')<br />
</source><br />
<br />
'''WARNING''': You must set <tt>solnorm=False</tt> here or later on you will find some offsets<br />
among spws due to the way the amplitude scaling adjusts weights internally during solving.<br />
<br />
<br />
You will see in the terminal window some reports of solutions failing due to "Insufficient unflagged antennas"&#151;note that these are for bad channels that have been pre-flagged.<br />
<br />
Plot the resulting bandpasses in amplitude and phase. Note that the first panel with ea01 is empty as it is completely flagged. Proceed to ea06 to see the plots as shown in Figures 5a, 5b, 6a, and 6b:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='amp', \<br />
spw='0~31', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='amp', \<br />
spw='32~63', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='0~31', \<br />
plotrange=[-1,-1,-180,180])<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='32~63', \<br />
plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
{|<br />
| [[Image:plotG192_plotcal_B0a1_4.0.png|200px|thumb|left|Figure 5a: plotcal B0 bandpass amp ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0a2_4.0.png|200px|thumb|center|Figure 5b: plotcal B0 bandpass amp ant ea06 spw 32-63]]<br />
| [[Image:plotG192_plotcal_B0p1_4.0.png|200px|thumb|center|Figure 6a: plotcal B0 bandpass phase ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0p2_4.0.png|200px|thumb|right|Figure 6b: plotcal B0 bandpass phase ant ea06 spw 32-63]]<br />
|}<br />
<br />
== Bootstrapping the bandpass calibrator spectrum ==<br />
<br />
<br />
Since there is no ''a priori'' spectral information for our chosen bandpass calibrator of 3C84, we need to bootstrap to find its spectral index, then recalibrate with this information in order to avoid folding the intrinsic spectral shape of 3C84 into our calibration.<br />
<br />
First, we again do a phase-only calibration solution, this time for both the bandpass and the flux density calibrator. This will correct for decorrelation of the signals. We again use the channel range 60~68 and apply the bandpass and delay calibration tables: <br />
<br />
<source lang="python"><br />
# In CASA: flux and bandpass calibrators gain<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G1p', field='0,1', \<br />
gaintable=['calG192.K0', 'calG192.B0'], \<br />
gaintype='G', refant='ea05', calmode='p', solint='int', minsnr=3)<br />
</source><br />
<br />
Now we are ready to solve for both phase and gain for each scan: <br />
<br />
<source lang="python"><br />
# In CASA: flux and bandpass calibrators gain<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G1', field='0,1', \<br />
gaintable=['calG192.K0', 'calG192.B0','calG192.G1p'], \<br />
gaintype='G', refant='ea05', calmode='ap', solint='inf', minsnr=3)<br />
</source><br />
<br />
<!--<br />
Let's have a look at the phase and amplitude solutions, iterating over antenna. We will look at the flux density calibrator (3C147) and bandpass calibrator (3C84) individually since they're widely separated in time:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='amp', \<br />
field='0', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='amp', \<br />
field='1', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180], \<br />
field='0')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180], \<br />
field='1')<br />
</source><br />
<br />
The solutions all look reasonable and relatively constant with time. <br />
--><br />
<br />
With gain solutions for the flux density and bandpass calibrators, we can now use {{fluxscale}} to scale the gain amplitudes of the bandpass calibrator using those of the flux density calibrator:<br />
<br />
<br />
<source lang="python"><br />
# In CASA: bandpass calibrator gain amplitudes scaling<br />
flux1 = fluxscale(vis='G192-BP.ms', caltable='calG192.G1', \<br />
fluxtable='calG192.F1', reference='0', \<br />
transfer='1', listfile='3C84.fluxinfo', fitorder=1)<br />
</source><br />
* <tt>flux1 = fluxscale(...)</tt>: we allow {{fluxscale}} to use the variable <tt>flux1</tt> for the Python dictionary that is returned, which has information about the flux scaling. You can inspect the dictionary flux1 by typing "print flux1" at the CASA command line.<br />
* <tt>fluxtable='calG192.F1'</tt>: this is the output scaled gain table. Since we are only using this to find the spectral index of 3C84, we won't be using this table.<br />
* <tt>listfile='3C84.fluxinfo'</tt>: an output file that contains the derived flux values and fit information.<br />
* <tt>fitorder=1</tt>: only find a spectral index, ignoring curvature in the spectrum.<br />
* <tt>reference='0'</tt>: the reference field ''from'' which the flux scaling is transferred (here: the flux density calibrator 3C147, field 0)<br />
* <tt>transfer='1'</tt>: the target field ''to'' which the flux scaling is transferred (here: the bandpass calibrator 3C84, field 1)<br />
<br />
<br />
The last line in the file (and displayed in the logger) shows:<br />
<pre style="background-color: #fffacd;"><br />
Fitted spectrum for 3c84-J0319+413 with fitorder=1: Flux density = 29.0282 +/- 0.0308648 (freq=32.5128 GHz) spidx=-0.538758 +/- 0.00882913<br />
</pre><br />
<br />
[[Image:PlotG192-3C84-fluxspec-4.5.png|200px|thumb|right|Figure 7: 3C84 flux values returned by fluxscale]]<br />
<br />
Using the information in the returned <tt>flux1</tt> dictionary, we can plot the derived spectrum (Figure 7):<br />
<source lang="python"><br />
# In CASA<br />
freq = flux1['freq'] / 1e9<br />
spw_list = range(0,64)<br />
spw_str = []<br />
for i in spw_list:<br />
thisspw = str(i)<br />
spw_str.append(thisspw)<br />
</source><br />
<br />
(Note that in order to close indented python loops, conditions etc. you will have to press ''Enter'' again to execute the indented commands and to return to the CASA prompt.) <br />
<br />
<source lang="python"><br />
# In CASA<br />
bootstrapped_fluxes = []<br />
for j in spw_str:<br />
thisflux = flux1['1'][j]['fluxd'][0]<br />
if thisflux ==None:<br />
continue<br />
else:<br />
bootstrapped_fluxes.append(thisflux)<br />
</source><br />
<br />
<br />
<source lang="python"><br />
# In CASA - this section creates the plot seen in Figure 7<br />
pl.clf()<br />
pl.plot(freq, bootstrapped_fluxes, 'bo')<br />
pl.xlabel('Frequency (GHz)')<br />
pl.ylabel('Flux Density (Jy)')<br />
pl.title('3C84')<br />
pl.show()<br />
</source><br />
<br />
We can use the model from {{fluxscale}} to fill the MODEL column with 3C84's spectral information using {{setjy}}. With ''standard='fluxscale''', we can directly use the <tt>flux1</tt> Python dictionary as input via ''fluxdict'':<br />
<br />
[[Image:ScreenshotPlotG192-setjy-bp-4.5.png|200px|thumb|right|Figure 8: plotms of model amp vs freq for 3C84]]<br />
<br />
<source lang="python"><br />
# In CASA: spectral information<br />
setjy(vis='G192-BP.ms', field='1', scalebychan=True, \<br />
standard = 'fluxscale', fluxdict=flux1)<br />
</source><br />
<br />
Check with plotms that the data have been appropriately filled (Figure 8):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='G192-BP.ms', field='1', antenna='ea05&ea02', \<br />
xaxis='freq', yaxis='amp', ydatacolumn='model')<br />
</source><br />
<br />
Next, we redo the previous calibration using this new model information. Although the commands are the same as issued earlier, keep in mind that the model values for the bandpass calibrator have changed and, therefore, the results of these calibration calculations will differ:<br />
<br />
<source lang="python"><br />
# In CASA: phase only recalibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G0.b', \<br />
field='1', spw='*:60~68', \<br />
gaintype='G', refant='ea05', calmode='p', \<br />
solint='int', minsnr=3) <br />
# In CASA: residual delays recalibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.K0.b', \<br />
gaintable=['calG192.G0.b'], \<br />
field='1', spw='*:5~122', gaintype='K', \<br />
refant='ea05', solint='inf', minsnr=3)<br />
# In CASA: antenna bandpasses recalibration<br />
bandpass(vis='G192-BP.ms', caltable='calG192.B0.b', \<br />
gaintable=['calG192.G0.b', 'calG192.K0.b'], \<br />
field='1', refant='ea05', solnorm=False, \<br />
bandtype='B', solint='inf')<br />
</source><br />
<br />
Finally, we inspect these solutions (Figures 9a, 9b, 10a, and 10b):<br />
<br />
<source lang="python"><br />
# In CASA - Figure 9a<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='amp', \<br />
spw='0~31', iteration='antenna')<br />
# Figure 9b<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='amp', \<br />
spw='32~63', iteration='antenna')<br />
# Figure 10a<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='0~31', \<br />
plotrange=[-1,-1,-180,180])<br />
# Figure 10b<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='32~63', \<br />
plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
{|<br />
| [[Image:plotG192_plotcal_B0a1.b_4.1.png|200px|thumb|left|Figure 9a: plotcal B0 bootstrapped bandpass amp ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0a2.b_4.1.png|200px|thumb|center|Figure 9b: plotcal B0 bootstrapped bandpass amp ant ea06 spw 32-63]]<br />
| [[Image:plotG192_plotcal_B0p1.b_4.1.png|200px|thumb|center|Figure 10a: plotcal B0 bootstrapped bandpass phase ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0p2.b_4.1.png|200px|thumb|right|Figure 10b: plotcal B0 bootstrapped bandpass phase ant ea06 spw 32-63]] <br />
|}<br />
<br />
They look virtually unchanged from the previous solutions with the exception that the amplitude scaling is corrected for the spectrum of 3C84. We have the final version of our delay and bandpass calibration tables, '''calG192.K0.b''' and '''calG192.B0.b''', which can be used for all subsequent calibration steps. <br />
<br />
{{Checked 4.6.0}}<br />
<!--<br />
-- Original: Miriam Hartman (full G192 guide) <br /><br />
-- Modifications: Jose Salcido (4.5.2. 2016/04/14) <br /><br />
-- Topical Guide: Juergen Ott (4.5.2, 2016/04/14) <br /><br />
-- Edits to Guide: Tony Perreault (4.5.2, 2016/05/18) <br /><br />
--></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Bandpass_Slope-CASA4.5.2&diff=19333VLA CASA Bandpass Slope-CASA4.5.22016-05-09T21:57:55Z<p>Emomjian: </p>
<hr />
<div><br />
<b>This CASA Guide is for CASA version 4.5.2</b><br />
<br />
<br />
== Overview ==<br />
<br />
For the standard VLA flux density calibrators 3C138, 3C147, 3C286, and 3C48, CASA includes spatial and spectral models that are applied during calibration. The models remove the source characteristics, resulting in calibration solutions that represent the instrumental and atmospheric corrections. These VLA standard calibrators, however, exhibit a negative spectral index and are relatively weak at high frequencies.<br />
<br />
Although the standard VLA flux density calibrators are usually still bright enough for absolute flux density calibration, a good bandpass determination&#151;which is very important for spectral line observations&#151;requires large signal-to-noise ratios derived from either a long integration time or a very strong source (see the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/line#section-8 Spectral Line Guide for Observing]). Observations of non-standard, but strong, bandpass calibrators are therefore common at high frequencies. Unfortunately, such sources are likely variable and no ''a priori'' flux density model is available. In particular, these sources exhibit an unknown and maybe variable spectral slope, which, if not accounted for, will create an error in the bandpass calibration. This tutorial describes how to model a spectral slope and how to correct the bandpass solution for this effect. <br />
<br />
Data used in this guide are taken in wide 3-bit mode for the protostar G192.16-3.84 in Ka-band with basebands centered at 29 and 36.5 GHz. Each baseband has over 4 GHz of bandwidth comprising thirty-two 128-MHz spectral windows.<br />
<br />
If you are new to CASA, or with VLA data reduction in CASA, it is '''strongly''' recommended that you start with either the [[VLA Continuum Tutorial 3C391]] or [[Getting Started in CASA]] before proceeding with this tutorial.<br />
<br />
== Obtaining the Data ==<br />
<br />
As this tutorial concerns bandpass calibration, all sources other than the flux density and bandpass calibrator scans were removed from the MS. All pre-calibration steps including flagging, antenna position offsets, requantizer gains, opacity corrections, and gain-elevation curves were applied. The original data (<tt>TVER0004.sb14459364.eb14492359.56295.26287841435</tt>) can be obtained through the [https://archive.nrao.edu NRAO archive] and has a raw size of 57.04 GB.<br />
<br />
The trimmed measurement set can be downloaded directly from [http://casa.nrao.edu/Data/EVLA/G192/G192-BP.ms.tar.gz http://casa.nrao.edu/Data/EVLA/G192/G192-BP.ms.tar.gz] (dataset size: 3.4 GB)<br />
<br />
Your first step will be to unzip and untar the file in a terminal (before you start CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf G192-BP.ms.tar.gz <br />
</source><br />
<br />
== Starting CASA ==<br />
<br />
To start CASA, type:<br />
<br />
<source lang="bash"><br />
casa<br />
</source><br />
<br />
This will run a script initializing CASA and setting paths appropriately. The script will also create two files called ''ipython-<unique-stamp>.log'' (which contains a record of all the text you enter at the CASA prompt) as well as ''casapy-<unique-stamp>.log'' (which will contain all the messages that are printed to the CASA logger window). It is recommended that you keep your log files intact&#151;you may need them to remind you of the last step you completed in your data reduction. (It is also a good idea to include your log files when submitting a help desk ticket).<br />
<br />
Once CASA has started, a logger window will appear. Note that you can rescale this window or change the font size (under the '''View''' menu option) as desired.<br />
<br />
== Examining the Measurement Set (MS) ==<br />
<br />
We use {{listobs}} to summarize our MS:<br />
<source lang="python"><br />
# In CASA: listobs on the initial data set<br />
listobs('G192-BP.ms', listfile='G192_listobs.txt')<br />
</source><br />
<br />
This will write the output to a file called <tt>G192_listobs.txt</tt>, which we can print to the terminal using various Unix/Linux commands such as <tt>cat</tt>, <tt>less</tt>, or <tt>more</tt>:<br />
<br />
<source lang="python"><br />
# In CASA<br />
cat G192_listobs.txt<br />
</source><br />
<br />
<pre><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/jott/casa/topicalguide/bandpass/new/G192-BP.ms MS Version 2<br />
================================================================================<br />
Observer: Dr. Debra Shepherd Project: uid://evla/pdb/7303457 <br />
Observation: EVLA<br />
Data records: 1769355 Total elapsed time = 4563 seconds<br />
Observed from 03-Jan-2013/06:31:48.0 to 03-Jan-2013/07:47:51.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
03-Jan-2013/06:31:48.0 - 06:36:42.0 6 0 3C147 704865 [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63] [6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5.94, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6] [CALIBRATE_FLUX#UNSPECIFIED,OBSERVE_TARGET#UNSPECIFIED]<br />
07:40:27.0 - 07:47:51.0 64 1 3c84-J0319+413 1064490 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63] [6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6] [CALIBRATE_BANDPASS#UNSPECIFIED,OBSERVE_TARGET#UNSPECIFIED]<br />
(nRows = Total number of rows per scan) <br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 E 3C147 05:42:36.137916 +49.51.07.23356 J2000 0 704865<br />
1 F 3c84-J0319+413 03:19:48.160102 +41.30.42.10305 J2000 1 1064490<br />
Spectral Windows: (64 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 EVLA_KA#A1C1#2 128 TOPO 34476.000 1000.000 128000.0 34539.5000 10 RR LL<br />
1 EVLA_KA#A1C1#3 128 TOPO 34604.000 1000.000 128000.0 34667.5000 10 RR LL<br />
2 EVLA_KA#A1C1#4 128 TOPO 34732.000 1000.000 128000.0 34795.5000 10 RR LL<br />
3 EVLA_KA#A1C1#5 128 TOPO 34860.000 1000.000 128000.0 34923.5000 10 RR LL<br />
<br />
<snip><br />
<br />
13 EVLA_KA#A1C1#15 128 TOPO 36140.000 1000.000 128000.0 36203.5000 10 RR LL<br />
14 EVLA_KA#A1C1#16 128 TOPO 36268.000 1000.000 128000.0 36331.5000 10 RR LL<br />
15 EVLA_KA#A1C1#17 128 TOPO 36396.000 1000.000 128000.0 36459.5000 10 RR LL<br />
16 EVLA_KA#A2C2#18 128 TOPO 36476.000 1000.000 128000.0 36539.5000 11 RR LL<br />
17 EVLA_KA#A2C2#19 128 TOPO 36604.000 1000.000 128000.0 36667.5000 11 RR LL<br />
18 EVLA_KA#A2C2#20 128 TOPO 36732.000 1000.000 128000.0 36795.5000 11 RR LL<br />
<br />
<snip><br />
<br />
29 EVLA_KA#A2C2#31 128 TOPO 38140.000 1000.000 128000.0 38203.5000 11 RR LL<br />
30 EVLA_KA#A2C2#32 128 TOPO 38268.000 1000.000 128000.0 38331.5000 11 RR LL<br />
31 EVLA_KA#A2C2#33 128 TOPO 38396.000 1000.000 128000.0 38459.5000 11 RR LL<br />
32 EVLA_KA#B1D1#34 128 TOPO 26976.000 1000.000 128000.0 27039.5000 13 RR LL<br />
33 EVLA_KA#B1D1#35 128 TOPO 27104.000 1000.000 128000.0 27167.5000 13 RR LL<br />
34 EVLA_KA#B1D1#36 128 TOPO 27232.000 1000.000 128000.0 27295.5000 13 RR LL<br />
<br />
<snip><br />
<br />
45 EVLA_KA#B1D1#47 128 TOPO 28640.000 1000.000 128000.0 28703.5000 13 RR LL<br />
46 EVLA_KA#B1D1#48 128 TOPO 28768.000 1000.000 128000.0 28831.5000 13 RR LL<br />
47 EVLA_KA#B1D1#49 128 TOPO 28896.000 1000.000 128000.0 28959.5000 13 RR LL<br />
48 EVLA_KA#B2D2#50 128 TOPO 28976.000 1000.000 128000.0 29039.5000 14 RR LL<br />
49 EVLA_KA#B2D2#51 128 TOPO 29104.000 1000.000 128000.0 29167.5000 14 RR LL<br />
50 EVLA_KA#B2D2#52 128 TOPO 29232.000 1000.000 128000.0 29295.5000 14 RR LL<br />
<br />
<snip><br />
<br />
61 EVLA_KA#B2D2#63 128 TOPO 30640.000 1000.000 128000.0 30703.5000 14 RR LL<br />
62 EVLA_KA#B2D2#64 128 TOPO 30768.000 1000.000 128000.0 30831.5000 14 RR LL<br />
63 EVLA_KA#B2D2#65 128 TOPO 30896.000 1000.000 128000.0 30959.5000 14 RR LL<br />
Sources: 128<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 3C147 0 - - <br />
0 3C147 1 - - <br />
0 3C147 2 - - <br />
<br />
<snip><br />
<br />
0 3C147 61 - - <br />
0 3C147 62 - - <br />
0 3C147 63 - - <br />
1 3c84-J0319+413 0 - - <br />
1 3c84-J0319+413 1 - - <br />
1 3c84-J0319+413 2 - - <br />
<br />
<snip><br />
<br />
1 3c84-J0319+413 61 - - <br />
1 3c84-J0319+413 62 - - <br />
1 3c84-J0319+413 63 - - <br />
<br />
<snip><br />
</pre><br />
<br />
We have trimmed the MS to contain only scans on the flux calibrator 3C147 (field 0) and the bandpass calibrator 3C84 (field 1) but retained all 64 spectral windows, each 128 MHz wide and containing one hundred twenty-eight 1 MHz channels.<br />
<br />
== Setting the Model of the Flux Density Calibrator ==<br />
<br />
<br />
To start, we insert the spectral (using the 'Perley-Butler 2013' standard) and spatial (3C147_A.im for Ka-band) models for the flux density calibrator 3C147 (field 0) with the {{setjy}} task:<br />
<br />
<source lang="python"><br />
# In CASA: model for the flux density calibrator<br />
setjy(vis='G192-BP.ms', field='0', scalebychan=True, \<br />
standard='Perley-Butler 2013', model='3C147_A.im')<br />
</source><br />
<br />
[[Image:ScreenshotPlotG192-setjy-4.5.png|200px|thumb|right|Figure 1: plotms of model amp vs freq for 3C147]]<br />
* <tt>scalebychan=True</tt>: If ''scalebychan=False'' {{setjy}} would use a single value per spectral window.<br />
<br />
Inspecting the logger report shows that 3C147 has a flux density ranging from ~1.0-1.47 Jy across all spws. <br />
<br />
We can plot the model data using {{plotms}} (Figure 1):<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='G192-BP.ms', field='0', antenna='ea03', \<br />
xaxis='freq', yaxis='amp', ydatacolumn='model',coloraxis='ant2')<br />
</source><br />
<br />
This plot shows baselines to antenna ea03. Since we provided both a spectral and a spatial model for this well resolved calibrator, each baseline has a somewhat different behavior.<br />
<br />
== Calibrating delays and initial bandpass solutions ==<br />
<br />
As a first step, we need to specify a reference antenna for all phase calibrations. It is desirable to use an antenna that is near the center of the array and that has a minimum of flags. The array can be mapped with {{plotants}}:<br />
<br />
<source lang="python"><br />
# In CASA: plotting antenna locations<br />
plotants(vis='G192-BP.ms')<br />
</source><br />
Although the plot is a bit crowded (Figure 2), a zooming in (the icon with the magnifying glass) shows that ea05 sits close to the center and appears to be a good choice. <br />
[[Image:plotG192_plotants.png|200px|thumb|right|Figure 2: plotants plotter]]<br />
<br />
[[Image:plotG192_plotcal_G0p1_4.0.png|200px|thumb|right|Figure 3a: plotcal G0 phase ant 0~15]]<br />
[[Image:plotG192_plotcal_G0p2_4.0.png|200px|thumb|right|Figure 3b: plotcal G0 phase ant 16~26]]<br />
<br />
[[Image:plotG192_plotcal_delays.png|200px|thumb|right|Figure 4: plotcal K0 delay vs. antenna]]<br />
<br />
<br />
We start with a phase-only calibration solution on a narrow range of channels near the center of each spectral window on the bandpass calibrator 3C84. We select a narrow range of channels near the center to flatten with respect to time before solving for the bandpass. This action will remove most of the decorrelation of the signal. The channel range of 60~68 should work, and we want derive a solution for each individual integration:<br />
<br />
<source lang="python"><br />
# In CASA: phase only calibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G0', \<br />
field='1', spw='*:60~68', \<br />
gaintype='G', refant='ea05', calmode='p', \<br />
solint='int', minsnr=3)<br />
</source><br />
<br />
* <tt>refant='ea05'</tt> : Use ea05 as the reference antenna<br />
* <tt>solint='int'</tt> : Do a per-integration solve (every 6 seconds, since we've time-averaged the data).<br />
* <tt>minsnr=3</tt> : Apply a minimum signal-to-noise cutoff. Solutions with less than this value will be flagged.<br />
* <tt>gaintable</tt> is not set here as we have already applied pre-calibrations.<br />
<br />
Plot the phase solutions (using full phase range -180 to 180 instead of autorange):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
Click on the Next button to navigate through the antennas. Click on the Quit button to exit the viewer.<br />
<br />
We will now produce multipanel plots of the phase solutions, writing the plots to output files as well as on the screen (Figures 3a & 3b). The output files generated are PNG files and can be viewed within CASA by executing an external viewer program, e.g., <tt>!xv plotG192_plotcal_G0p1.png</tt>; or by running any number of image viewing applications such as xv, Preview, Gimp, Photoshop, etc., external to CASA at the OS level. <!-- (Note that the hardcopy only shows the first page): --><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
antenna='0~10,12~15', subplot=531, iteration='antenna', \<br />
plotrange=[-1,-1,-180,180], fontsize=8.0, \<br />
markersize=3.0, figfile='plotG192_plotcal_G0p1.png')<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
antenna='16~26', subplot=531, iteration='antenna', \<br />
plotrange=[-1,-1,-180,180], fontsize=8.0, \<br />
markersize=3.0, figfile='plotG192_plotcal_G0p2.png')<br />
</source><br />
<br />
<!--<br />
NOTE: When you are done plotting and want to use the calibration table in another task (e.g., for subsequent calibration or viewing with plotms), use the Quit button on the GUI to dismiss the plotter and free-up the lock on the calibration table. You should see a message in your terminal window saying "Resetting plotcal" which means you are good to go!<br />
--><br />
<br />
We can now solve for the residual delays using the <tt>gaintype='K'</tt> option in {{gaincal}}. Note that this currently does not do a global fringe-fitting solution for delays, but instead does a baseline-based delay solution per spw for all baselines to the reference antenna, treating these as antenna-based delays. In most cases, with high enough S/N to get baseline-based delay solutions, this will suffice. We avoid the edge channels of each spectral window by selecting channels 5~122:<br />
<br />
<source lang="python"><br />
# In CASA: residual delays<br />
gaincal(vis='G192-BP.ms', caltable='calG192.K0', \<br />
field='1', spw='*:5~122', gaintype='K', \<br />
gaintable=['calG192.G0'],<br />
refant='ea05', solint='inf', minsnr=3)<br />
</source><br />
<br />
Note that we have also pre-applied our initial phase table <tt>calG192.G0</tt>. <br />
<br />
Alternatively, you can also derive a delay across all spws of a baseband. If this is desired, use ''combine='spw''' in {{gaincal}} and run the task for each baseband separately. The solutions from the second and following runs can be appended to the same calibration table via ''append=T''.<br />
<br />
Now plot the delays, in nanoseconds, as a function of antenna index (you will get one for each spw and polarization):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.K0', xaxis='antenna', yaxis='delay')<br />
</source><br />
<br />
The delays range from around -5 to 4 nanoseconds (Figure 4).<br />
<br />
Now solve for the antenna bandpasses using the previously generated tables <tt>calG192.G0</tt> and <tt>calG192.K0</tt>:<br />
<br />
<source lang="python"><br />
# In CASA: antenna bandpasses<br />
bandpass(vis='G192-BP.ms', caltable='calG192.B0', \<br />
gaintable=['calG192.G0', 'calG192.K0'], \<br />
field='1', refant='ea05', solnorm=False, \<br />
bandtype='B', solint='inf')<br />
</source><br />
<br />
'''WARNING''': You must set <tt>solnorm=False</tt> here or later on you will find some offsets<br />
among spws due to the way the amplitude scaling adjusts weights internally during solving.<br />
<br />
<br />
You will see in the terminal window some reports of solutions failing due to "Insufficient unflagged antennas"&#151;note that these are for bad channels that have been pre-flagged.<br />
<br />
Plot the resulting bandpasses in amplitude and phase. Note that the first panel with ea01 is empty as it is completely flagged. Proceed to ea06 to see the plots as shown in Figures 5a, 5b, 6a, and 6b:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='amp', \<br />
spw='0~31', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='amp', \<br />
spw='32~63', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='0~31', \<br />
plotrange=[-1,-1,-180,180])<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='32~63', \<br />
plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
{|<br />
| [[Image:plotG192_plotcal_B0a1_4.0.png|200px|thumb|left|Figure 5a: plotcal B0 bandpass amp ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0a2_4.0.png|200px|thumb|center|Figure 5b: plotcal B0 bandpass amp ant ea06 spw 32-63]]<br />
| [[Image:plotG192_plotcal_B0p1_4.0.png|200px|thumb|center|Figure 6a: plotcal B0 bandpass phase ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0p2_4.0.png|200px|thumb|right|Figure 6b: plotcal B0 bandpass phase ant ea06 spw 32-63]]<br />
|}<br />
<br />
== Bootstrapping the bandpass calibrator spectrum ==<br />
<br />
<br />
Since there is no ''a priori'' spectral information for our chosen bandpass calibrator&#151;3C84&#151;we need to bootstrap to find its spectral index, then recalibrate with this information in order to avoid folding the intrinsic spectral shape of 3C84 into our calibration.<br />
<br />
First, we again do a phase-only calibration solution, this time for both the bandpass and the flux density calibrator. This will correct for decorrelation of the signals. Again we use the channel range 60~68 and apply the bandpass and delay calibration tables: <br />
<br />
<source lang="python"><br />
# In CASA: flux and bandpass calibrators gain<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G1p', field='0,1', \<br />
gaintable=['calG192.K0', 'calG192.B0'], \<br />
gaintype='G', refant='ea05', calmode='p', solint='int', minsnr=3)<br />
</source><br />
<br />
Now we are ready to solve for both, phase and gain for each scan: <br />
<br />
<source lang="python"><br />
# In CASA: flux and bandpass calibrators gain<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G1', field='0,1', \<br />
gaintable=['calG192.K0', 'calG192.B0','calG192.G1p'], \<br />
gaintype='G', refant='ea05', calmode='ap', solint='inf', minsnr=3)<br />
</source><br />
<br />
<!--<br />
Let's have a look at the phase and amplitude solutions, iterating over antenna. We will look at the flux density calibrator (3C147) and bandpass calibrator (3C84) individually since they're widely separated in time:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='amp', \<br />
field='0', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='amp', \<br />
field='1', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180], \<br />
field='0')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180], \<br />
field='1')<br />
</source><br />
<br />
The solutions all look reasonable and relatively constant with time. <br />
--><br />
<br />
With gain solutions for the flux density and bandpass calibrators, we can now use {{fluxscale}} to scale the gain amplitudes of the bandpass calibrator using those of the flux density calibrator:<br />
<br />
<br />
<source lang="python"><br />
# In CASA: bandpass calibrator gain amplitudes scaling<br />
flux1 = fluxscale(vis='G192-BP.ms', caltable='calG192.G1', \<br />
fluxtable='calG192.F1', reference='0', \<br />
transfer='1', listfile='3C84.fluxinfo', fitorder=1)<br />
</source><br />
* <tt>flux1 = fluxscale(...)</tt>: we allow {{fluxscale}} to use the variable <tt>flux1</tt> for the output Python dictionary that is returned, which has information about the flux scaling. You can inspect the output dictionary flux1 by typing "print flux1" at the CASA command line.<br />
* <tt>fluxtable='calG192.F1'</tt>: this is the output scaled gain table. Since we are only using this to find the spectral index of 3C84, we won't be using this table.<br />
* <tt>listfile='3C84.fluxinfo'</tt>: an output file that contains the derived flux values and fit information.<br />
* <tt>fitorder=1</tt>: only find a spectral index, ignoring curvature in the spectrum.<br />
* <tt>reference='0'</tt>: the reference field ''from'' which the flux scaling is transferred (here: the flux calibrator 3C147, field 0)<br />
* <tt>transfer='1'</tt>: the target field ''to'' which the flux scaling is transferred (here: the bandpass calibrator 3C84, field 1)<br />
<br />
<br />
The last line in the file (and displayed in the logger) shows:<br />
<pre style="background-color: #fffacd;"><br />
Fitted spectrum for 3c84-J0319+413 with fitorder=1: Flux density = 29.0282 +/- 0.0308648 (freq=32.5128 GHz) spidx=-0.538758 +/- 0.00882913<br />
</pre><br />
<br />
[[Image:PlotG192-3C84-fluxspec-4.5.png|200px|thumb|right|Figure 7: 3C84 flux values returned by fluxscale]]<br />
[[Image:ScreenshotPlotG192-setjy-bp-4.5.png|200px|thumb|right|Figure 8: plotms of model amp vs freq for 3C84]]<br />
<br />
Using the information in the returned <tt>flux1</tt> dictionary, we can plot the derived spectrum (Figure 7):<br />
<source lang="python"><br />
# In CASA<br />
freq = flux1['freq'] / 1e9<br />
spw_list = range(0,64)<br />
spw_str = []<br />
for i in spw_list:<br />
thisspw = str(i)<br />
spw_str.append(thisspw)<br />
</source><br />
Note that in order to close indented python loops, conditions etc. you will have to press ''Enter'' again to execute the indented commands and to return to the CASA prompt. <br />
<br />
<source lang="python"><br />
# In CASA<br />
bootstrapped_fluxes = []<br />
for j in spw_str:<br />
thisflux = flux1['1'][j]['fluxd'][0]<br />
if thisflux ==None:<br />
continue<br />
else:<br />
bootstrapped_fluxes.append(thisflux)<br />
</source><br />
<br />
<br />
<source lang="python"><br />
# In CASA - this section creates the plot seen in Figure 7<br />
pl.clf()<br />
pl.plot(freq, bootstrapped_fluxes, 'bo')<br />
pl.xlabel('Frequency (GHz)')<br />
pl.ylabel('Flux Density (Jy)')<br />
pl.title('3C84')<br />
pl.show()<br />
</source><br />
<br />
We can use the model from {{fluxscale}} to fill the MODEL column with 3C84's spectral information using {{setjy}}. With ''standard='fluxscale''', we can directly use the <tt>flux1</tt> Python dictionary as input via ''fluxdict'':<br />
<br />
<source lang="python"><br />
# In CASA: spectral information<br />
setjy(vis='G192-BP.ms', field='1', scalebychan=True, \<br />
standard = 'fluxscale', fluxdict=flux1)<br />
</source><br />
<br />
Check with plotms that the data have been appropriately filled (Figure 8):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='G192-BP.ms', field='1', antenna='ea05&ea02', \<br />
xaxis='freq', yaxis='amp', ydatacolumn='model')<br />
</source><br />
<br />
We next redo the previous calibration using this new model information. Although the commands are the same as what we issued earlier, keep in mind that the model values for the bandpass calibrator have changed, and therefore the results of these calibration calculations will differ:<br />
<br />
<source lang="python"><br />
# In CASA: phase only recalibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G0.b', \<br />
field='1', spw='*:60~68', \<br />
gaintype='G', refant='ea05', calmode='p', \<br />
solint='int', minsnr=3) <br />
# In CASA: residual delays recalibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.K0.b', \<br />
gaintable=['calG192.G0.b'], \<br />
field='1', spw='*:5~122', gaintype='K', \<br />
refant='ea05', solint='inf', minsnr=3)<br />
# In CASA: antenna bandpasses recalibration<br />
bandpass(vis='G192-BP.ms', caltable='calG192.B0.b', \<br />
gaintable=['calG192.G0.b', 'calG192.K0.b'], \<br />
field='1', refant='ea05', solnorm=False, \<br />
bandtype='B', solint='inf')<br />
</source><br />
<br />
Finally, we inspect these solutions (Figures 9a, 9b, 10a, and 10b):<br />
<br />
<source lang="python"><br />
# In CASA - Figure 9a<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='amp', \<br />
spw='0~31', iteration='antenna')<br />
# Figure 9b<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='amp', \<br />
spw='32~63', iteration='antenna')<br />
# Figure 10a<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='0~31', \<br />
plotrange=[-1,-1,-180,180])<br />
# Figure 10b<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='32~63', \<br />
plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
{|<br />
| [[Image:plotG192_plotcal_B0a1.b_4.1.png|200px|thumb|left|Figure 9a: plotcal B0 bootstrapped bandpass amp ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0a2.b_4.1.png|200px|thumb|center|Figure 9b: plotcal B0 bootstrapped bandpass amp ant ea06 spw 32-63]]<br />
| [[Image:plotG192_plotcal_B0p1.b_4.1.png|200px|thumb|center|Figure 10a: plotcal B0 bootstrapped bandpass phase ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0p2.b_4.1.png|200px|thumb|right|Figure 10b: plotcal B0 bootstrapped bandpass phase ant ea06 spw 32-63]] <br />
|}<br />
<br />
They look virtually unchanged from the previous solutions, with the exception that the amplitude scaling is corrected for the spectrum of 3C84. We have the final version of our delay and bandpass calibration tables, calG192.K0.b and calG192.B0.b, which can be used for all subsequent calibration steps. <br />
<br />
{{Checked 4.5.2}}<br />
<!--<br />
-- Original: Miriam Hartman (full G192 guide) <br /><br />
-- Modifications: Jose Salcido (4.5.2. 2016/04/14) <br /><br />
-- Topical Guide: Juergen Ott (4.5.2, 2016/04/14) <br /><br />
-- Edits to Guide: Tony Perreault (4.5.2, 2016/04/26) <br /><br />
--></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Bandpass_Slope-CASA4.5.2&diff=19332VLA CASA Bandpass Slope-CASA4.5.22016-05-09T21:31:22Z<p>Emomjian: </p>
<hr />
<div><br />
<b>This CASA Guide is for CASA version 4.5.2</b><br />
<br />
<br />
== Overview ==<br />
<br />
For the standard VLA flux density calibrators 3C138, 3C147, 3C286, and 3C48, CASA includes a spatial and spectral model that is applied for bandpass calibration. The model removes the source characteristics, resulting in a calibration solution that is a representation of the instrumental and atmospheric corrections. These VLA standard calibrators, however, exhibit a negative spectral index and are relatively weak at high frequencies.<br />
<br />
Although the standard VLA flux density calibrators are usually still bright enough for absolute flux density calibration, a good bandpass determination&#151;which is very important for spectral line observations&#151;requires large signal-to-noise ratios derived from either a long integration time or a very strong source (see the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/line#section-8 Spectral Line Guide for Observing]). Observations of non-standard, but strong, bandpass calibrators are therefore common at high frequencies. Unfortunately, such sources are likely variable and no ''a priori'' flux density model is available. In particular, these sources exhibit an unknown and maybe variable spectral slope, which, if not accounted for, will create an error in the bandpass calibration. This tutorial describes how to model a spectral slope and how to correct the bandpass solution for this effect. <br />
<br />
Data used in this guide are taken in wide 3-bit mode for the protostar G192.16-3.84 in Ka-band with basebands centered at 29 and 36.5 GHz. Each baseband has over 4 GHz of bandwidth comprising thirty-two 128-MHz spectral windows.<br />
<br />
If you are new to CASA, or with VLA data reduction in CASA, it is '''strongly''' recommended that you start with either the [[VLA Continuum Tutorial 3C391]] or [[Getting Started in CASA]] before proceeding with this tutorial.<br />
<br />
== Obtaining the Data ==<br />
<br />
As this tutorial concerns bandpass calibration, all sources other than the flux density and bandpass calibrator scans were removed from the MS. All pre-calibration steps including flagging, antenna position offsets, requantizer gains, opacity corrections, and gain-elevation curves were applied. The original data (<tt>TVER0004.sb14459364.eb14492359.56295.26287841435</tt>) can be obtained through the [https://archive.nrao.edu NRAO archive] and has a raw size of 57.04 GB.<br />
<br />
The trimmed measurement set can be downloaded directly from [http://casa.nrao.edu/Data/EVLA/G192/G192-BP.ms.tar.gz http://casa.nrao.edu/Data/EVLA/G192/G192-BP.ms.tar.gz] (dataset size: 3.4 GB)<br />
<br />
Your first step will be to unzip and untar the file in a terminal (before you start CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf G192-BP.ms.tar.gz <br />
</source><br />
<br />
== Starting CASA ==<br />
<br />
To start CASA, type:<br />
<br />
<source lang="bash"><br />
casa<br />
</source><br />
<br />
This will run a script initializing CASA and setting paths appropriately. The script will also create two files called ''ipython-<unique-stamp>.log'' (which contains a record of all the text you enter at the CASA prompt) as well as ''casapy-<unique-stamp>.log'' (which will contain all the messages that are printed to the CASA logger window). It is recommended that you keep your log files intact&#151;you may need them to remind you of the last step you completed in your data reduction. (It is also a good idea to include your log files when submitting a help desk ticket).<br />
<br />
Once CASA has started, a logger window will appear. Note that you can rescale this window or change the font size (under the '''View''' menu option) as desired.<br />
<br />
== Examining the Measurement Set (MS) ==<br />
<br />
We use {{listobs}} to summarize our MS:<br />
<source lang="python"><br />
# In CASA: listobs on the initial data set<br />
listobs('G192-BP.ms', listfile='G192_listobs.txt')<br />
</source><br />
<br />
This will write the output to a file called <tt>G192_listobs.txt</tt>, which we can print to the terminal using various Unix/Linux commands such as <tt>cat</tt>, <tt>less</tt>, or <tt>more</tt>:<br />
<br />
<source lang="python"><br />
# In CASA<br />
cat G192_listobs.txt<br />
</source><br />
<br />
<pre><br />
================================================================================<br />
MeasurementSet Name: /lustre/aoc/sciops/jott/casa/topicalguide/bandpass/new/G192-BP.ms MS Version 2<br />
================================================================================<br />
Observer: Dr. Debra Shepherd Project: uid://evla/pdb/7303457 <br />
Observation: EVLA<br />
Data records: 1769355 Total elapsed time = 4563 seconds<br />
Observed from 03-Jan-2013/06:31:48.0 to 03-Jan-2013/07:47:51.0 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
03-Jan-2013/06:31:48.0 - 06:36:42.0 6 0 3C147 704865 [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63] [6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 5.94, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6] [CALIBRATE_FLUX#UNSPECIFIED,OBSERVE_TARGET#UNSPECIFIED]<br />
07:40:27.0 - 07:47:51.0 64 1 3c84-J0319+413 1064490 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63] [6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6] [CALIBRATE_BANDPASS#UNSPECIFIED,OBSERVE_TARGET#UNSPECIFIED]<br />
(nRows = Total number of rows per scan) <br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 E 3C147 05:42:36.137916 +49.51.07.23356 J2000 0 704865<br />
1 F 3c84-J0319+413 03:19:48.160102 +41.30.42.10305 J2000 1 1064490<br />
Spectral Windows: (64 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 EVLA_KA#A1C1#2 128 TOPO 34476.000 1000.000 128000.0 34539.5000 10 RR LL<br />
1 EVLA_KA#A1C1#3 128 TOPO 34604.000 1000.000 128000.0 34667.5000 10 RR LL<br />
2 EVLA_KA#A1C1#4 128 TOPO 34732.000 1000.000 128000.0 34795.5000 10 RR LL<br />
3 EVLA_KA#A1C1#5 128 TOPO 34860.000 1000.000 128000.0 34923.5000 10 RR LL<br />
<br />
<snip><br />
<br />
13 EVLA_KA#A1C1#15 128 TOPO 36140.000 1000.000 128000.0 36203.5000 10 RR LL<br />
14 EVLA_KA#A1C1#16 128 TOPO 36268.000 1000.000 128000.0 36331.5000 10 RR LL<br />
15 EVLA_KA#A1C1#17 128 TOPO 36396.000 1000.000 128000.0 36459.5000 10 RR LL<br />
16 EVLA_KA#A2C2#18 128 TOPO 36476.000 1000.000 128000.0 36539.5000 11 RR LL<br />
17 EVLA_KA#A2C2#19 128 TOPO 36604.000 1000.000 128000.0 36667.5000 11 RR LL<br />
18 EVLA_KA#A2C2#20 128 TOPO 36732.000 1000.000 128000.0 36795.5000 11 RR LL<br />
<br />
<snip><br />
<br />
29 EVLA_KA#A2C2#31 128 TOPO 38140.000 1000.000 128000.0 38203.5000 11 RR LL<br />
30 EVLA_KA#A2C2#32 128 TOPO 38268.000 1000.000 128000.0 38331.5000 11 RR LL<br />
31 EVLA_KA#A2C2#33 128 TOPO 38396.000 1000.000 128000.0 38459.5000 11 RR LL<br />
32 EVLA_KA#B1D1#34 128 TOPO 26976.000 1000.000 128000.0 27039.5000 13 RR LL<br />
33 EVLA_KA#B1D1#35 128 TOPO 27104.000 1000.000 128000.0 27167.5000 13 RR LL<br />
34 EVLA_KA#B1D1#36 128 TOPO 27232.000 1000.000 128000.0 27295.5000 13 RR LL<br />
<br />
<snip><br />
<br />
45 EVLA_KA#B1D1#47 128 TOPO 28640.000 1000.000 128000.0 28703.5000 13 RR LL<br />
46 EVLA_KA#B1D1#48 128 TOPO 28768.000 1000.000 128000.0 28831.5000 13 RR LL<br />
47 EVLA_KA#B1D1#49 128 TOPO 28896.000 1000.000 128000.0 28959.5000 13 RR LL<br />
48 EVLA_KA#B2D2#50 128 TOPO 28976.000 1000.000 128000.0 29039.5000 14 RR LL<br />
49 EVLA_KA#B2D2#51 128 TOPO 29104.000 1000.000 128000.0 29167.5000 14 RR LL<br />
50 EVLA_KA#B2D2#52 128 TOPO 29232.000 1000.000 128000.0 29295.5000 14 RR LL<br />
<br />
<snip><br />
<br />
61 EVLA_KA#B2D2#63 128 TOPO 30640.000 1000.000 128000.0 30703.5000 14 RR LL<br />
62 EVLA_KA#B2D2#64 128 TOPO 30768.000 1000.000 128000.0 30831.5000 14 RR LL<br />
63 EVLA_KA#B2D2#65 128 TOPO 30896.000 1000.000 128000.0 30959.5000 14 RR LL<br />
Sources: 128<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 3C147 0 - - <br />
0 3C147 1 - - <br />
0 3C147 2 - - <br />
<br />
<snip><br />
<br />
0 3C147 61 - - <br />
0 3C147 62 - - <br />
0 3C147 63 - - <br />
1 3c84-J0319+413 0 - - <br />
1 3c84-J0319+413 1 - - <br />
1 3c84-J0319+413 2 - - <br />
<br />
<snip><br />
<br />
1 3c84-J0319+413 61 - - <br />
1 3c84-J0319+413 62 - - <br />
1 3c84-J0319+413 63 - - <br />
<br />
<snip><br />
</pre><br />
<br />
We have trimmed the MS to contain only scans on the flux calibrator 3C147 (field 0) and the bandpass calibrator 3C84 (field 1) but retained all 64 spectral windows, each 128 MHz wide and containing one hundred twenty-eight 1 MHz channels.<br />
<br />
== Setting the Model of the Flux Density Calibrator ==<br />
<br />
<br />
To start, we insert the spectral (using the 'Perley-Butler 2013' standard) and spatial (3C147_A.im for Ka-band) models for the flux density calibrator 3C147 (field 0) with the {{setjy}} task:<br />
<br />
<source lang="python"><br />
# In CASA: model for the flux density calibrator<br />
setjy(vis='G192-BP.ms', field='0', scalebychan=True, \<br />
standard='Perley-Butler 2013', model='3C147_A.im')<br />
</source><br />
<br />
[[Image:ScreenshotPlotG192-setjy-4.5.png|200px|thumb|right|Figure 1: plotms of model amp vs freq for 3C147]]<br />
* <tt>scalebychan=True</tt>: If ''scalebychan=False'' {{setjy}} would use a single value per spectral window.<br />
<br />
Inspecting the logger report shows that 3C147 has a flux density ranging from ~1.0-1.47 Jy across all spws. <br />
<br />
We can plot the model data using {{plotms}} (Figure 1):<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='G192-BP.ms', field='0', antenna='ea03', \<br />
xaxis='freq', yaxis='amp', ydatacolumn='model',coloraxis='ant2')<br />
</source><br />
<br />
This plot shows baselines to antenna ea03. Since we provided both a spectral and a spatial model for this well resolved calibrator, each baseline has a somewhat different behavior.<br />
<br />
== Calibrating delays and initial bandpass solutions ==<br />
<br />
As a first step, we need to specify a reference antenna for all phase calibrations. It is desirable to use an antenna that is near the center of the array and that has a minimum of flags. The array can be mapped with {{plotants}}:<br />
<br />
<source lang="python"><br />
# In CASA: plotting antenna locations<br />
plotants(vis='G192-BP.ms')<br />
</source><br />
Although the plot is a bit crowded (Figure 2), a zooming in (the icon with the magnifying glass) shows that ea05 sits close to the center and appears to be a good choice. <br />
[[Image:plotG192_plotants.png|200px|thumb|right|Figure 2: plotants plotter]]<br />
<br />
[[Image:plotG192_plotcal_G0p1_4.0.png|200px|thumb|right|Figure 3a: plotcal G0 phase ant 0~15]]<br />
[[Image:plotG192_plotcal_G0p2_4.0.png|200px|thumb|right|Figure 3b: plotcal G0 phase ant 16~26]]<br />
<br />
[[Image:plotG192_plotcal_delays.png|200px|thumb|right|Figure 4: plotcal K0 delay vs. antenna]]<br />
<br />
<br />
We start with a phase-only calibration solution on a narrow range of channels near the center of each spectral window on the bandpass calibrator 3C84. We select a narrow range of channels near the center to flatten with respect to time before solving for the bandpass. This action will remove most of the decorrelation of the signal. The channel range of 60~68 should work, and we want derive a solution for each individual integration:<br />
<br />
<source lang="python"><br />
# In CASA: phase only calibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G0', \<br />
field='1', spw='*:60~68', \<br />
gaintype='G', refant='ea05', calmode='p', \<br />
solint='int', minsnr=3)<br />
</source><br />
<br />
* <tt>refant='ea05'</tt> : Use ea05 as the reference antenna<br />
* <tt>solint='int'</tt> : Do a per-integration solve (every 6 seconds, since we've time-averaged the data).<br />
* <tt>minsnr=3</tt> : Apply a minimum signal-to-noise cutoff. Solutions with less than this value will be flagged.<br />
* <tt>gaintable</tt> is not set here as we have already applied pre-calibrations.<br />
<br />
Plot the phase solutions (using full phase range -180 to 180 instead of autorange):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
Click on the Next button to navigate through the antennas. Click on the Quit button to exit the viewer.<br />
<br />
We will now produce multipanel plots of the phase solutions, writing the plots to output files as well as on the screen (Figures 3a & 3b). The output files generated are PNG files and can be viewed within CASA by executing an external viewer program, e.g., <tt>!xv plotG192_plotcal_G0p1.png</tt>; or by running any number of image viewing applications such as xv, Preview, Gimp, Photoshop, etc., external to CASA at the OS level. <!-- (Note that the hardcopy only shows the first page): --><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
antenna='0~10,12~15', subplot=531, iteration='antenna', \<br />
plotrange=[-1,-1,-180,180], fontsize=8.0, \<br />
markersize=3.0, figfile='plotG192_plotcal_G0p1.png')<br />
plotcal(caltable='calG192.G0', xaxis='time', yaxis='phase', \<br />
antenna='16~26', subplot=531, iteration='antenna', \<br />
plotrange=[-1,-1,-180,180], fontsize=8.0, \<br />
markersize=3.0, figfile='plotG192_plotcal_G0p2.png')<br />
</source><br />
<br />
<!--<br />
NOTE: When you are done plotting and want to use the calibration table in another task (e.g., for subsequent calibration or viewing with plotms), use the Quit button on the GUI to dismiss the plotter and free-up the lock on the calibration table. You should see a message in your terminal window saying "Resetting plotcal" which means you are good to go!<br />
--><br />
<br />
We can now solve for the residual delays using the <tt>gaintype='K'</tt> option in {{gaincal}}. Note that this currently does not do a global fringe-fitting solution for delays, but instead does a baseline-based delay solution per spw for all baselines to the reference antenna, treating these as antenna-based delays. In most cases, with high enough S/N to get baseline-based delay solutions, this will suffice. We avoid the edge channels of each spectral window by selecting channels 5~122:<br />
<br />
<source lang="python"><br />
# In CASA: residual delays<br />
gaincal(vis='G192-BP.ms', caltable='calG192.K0', \<br />
field='1', spw='*:5~122', gaintype='K', \<br />
gaintable=['calG192.G0'],<br />
refant='ea05', solint='inf', minsnr=3)<br />
</source><br />
<br />
Note that we have also pre-applied our initial phase table <tt>calG192.G0</tt>. <br />
<br />
Alternatively, you can also derive a delay across all spws of a baseband. If this is desired, use ''combine='spw''' in {{gaincal}} and run the task for each baseband separately. The solutions from the second and following runs can be appended to the same calibration table via ''append=T''.<br />
<br />
Now plot the delays, in nanoseconds, as a function of antenna index (you will get one for each spw and polarization):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.K0', xaxis='antenna', yaxis='delay')<br />
</source><br />
<br />
The delays range from around -5 to 4 nanoseconds (Figure 4).<br />
<br />
Now solve for the antenna bandpasses using the previously generated tables <tt>calG192.G0</tt> and <tt>calG192.K0</tt>:<br />
<br />
<source lang="python"><br />
# In CASA: antenna bandpasses<br />
bandpass(vis='G192-BP.ms', caltable='calG192.B0', \<br />
gaintable=['calG192.G0', 'calG192.K0'], \<br />
field='1', refant='ea05', solnorm=False, \<br />
bandtype='B', solint='inf')<br />
</source><br />
<br />
'''WARNING''': You must set <tt>solnorm=False</tt> here or later on you will find some offsets<br />
among spws due to the way the amplitude scaling adjusts weights internally during solving.<br />
<br />
<br />
You will see in the terminal window some reports of solutions failing due to "Insufficient unflagged antennas"&#151;note that these are for bad channels that have been pre-flagged.<br />
<br />
Plot the resulting bandpasses in amplitude and phase. Note that the first panel with ea01 is empty as it is completely flagged. Proceed to ea06 to see the plots as shown in Figures 5a, 5b, 6a, and 6b:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='amp', \<br />
spw='0~31', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='amp', \<br />
spw='32~63', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='0~31', \<br />
plotrange=[-1,-1,-180,180])<br />
#<br />
plotcal(caltable='calG192.B0', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='32~63', \<br />
plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
{|<br />
| [[Image:plotG192_plotcal_B0a1_4.0.png|200px|thumb|left|Figure 5a: plotcal B0 bandpass amp ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0a2_4.0.png|200px|thumb|center|Figure 5b: plotcal B0 bandpass amp ant ea06 spw 32-63]]<br />
| [[Image:plotG192_plotcal_B0p1_4.0.png|200px|thumb|center|Figure 6a: plotcal B0 bandpass phase ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0p2_4.0.png|200px|thumb|right|Figure 6b: plotcal B0 bandpass phase ant ea06 spw 32-63]]<br />
|}<br />
<br />
== Bootstrapping the bandpass calibrator spectrum ==<br />
<br />
<br />
Since there is no ''a priori'' spectral information for our chosen bandpass calibrator&#151;3C84&#151;we need to bootstrap to find its spectral index, then recalibrate with this information in order to avoid folding the intrinsic spectral shape of 3C84 into our calibration.<br />
<br />
First, we again do a phase-only calibration solution, this time for both the bandpass and the flux density calibrator. This will correct for decorrelation of the signals. Again we use the channel range 60~68 and apply the bandpass and delay calibration tables: <br />
<br />
<source lang="python"><br />
# In CASA: flux and bandpass calibrators gain<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G1p', field='0,1', \<br />
gaintable=['calG192.K0', 'calG192.B0'], \<br />
gaintype='G', refant='ea05', calmode='p', solint='int', minsnr=3)<br />
</source><br />
<br />
Now we are ready to solve for both, phase and gain for each scan: <br />
<br />
<source lang="python"><br />
# In CASA: flux and bandpass calibrators gain<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G1', field='0,1', \<br />
gaintable=['calG192.K0', 'calG192.B0','calG192.G1p'], \<br />
gaintype='G', refant='ea05', calmode='ap', solint='inf', minsnr=3)<br />
</source><br />
<br />
<!--<br />
Let's have a look at the phase and amplitude solutions, iterating over antenna. We will look at the flux density calibrator (3C147) and bandpass calibrator (3C84) individually since they're widely separated in time:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='amp', \<br />
field='0', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='amp', \<br />
field='1', iteration='antenna')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180], \<br />
field='0')<br />
#<br />
plotcal(caltable='calG192.G1', xaxis='time', yaxis='phase', \<br />
iteration='antenna', plotrange=[-1,-1,-180,180], \<br />
field='1')<br />
</source><br />
<br />
The solutions all look reasonable and relatively constant with time. <br />
--><br />
<br />
With gain solutions for the flux density and bandpass calibrators, we can now use {{fluxscale}} to scale the gain amplitudes of the bandpass calibrator using those of the flux density calibrator:<br />
<br />
<br />
<source lang="python"><br />
# In CASA: bandpass calibrator gain amplitudes scaling<br />
flux1 = fluxscale(vis='G192-BP.ms', caltable='calG192.G1', \<br />
fluxtable='calG192.F1', reference='0', \<br />
transfer='1', listfile='3C84.fluxinfo', fitorder=1)<br />
</source><br />
* <tt>flux1 = fluxscale(...)</tt>: we allow {{fluxscale}} to use the variable <tt>flux1</tt> for the output Python dictionary that is returned, which has information about the flux scaling. You can inspect the output dictionary flux1 by typing "print flux1" at the CASA command line.<br />
* <tt>fluxtable='calG192.F1'</tt>: this is the output scaled gain table. Since we are only using this to find the spectral index of 3C84, we won't be using this table.<br />
* <tt>listfile='3C84.fluxinfo'</tt>: an output file that contains the derived flux values and fit information.<br />
* <tt>fitorder=1</tt>: only find a spectral index, ignoring curvature in the spectrum.<br />
* <tt>reference='0'</tt>: the reference field ''from'' which the flux scaling is transferred (here: the flux calibrator 3C147, field 0)<br />
* <tt>transfer='1'</tt>: the target field ''to'' which the flux scaling is transferred (here: the bandpass calibrator 3C84, field 1)<br />
<br />
<br />
The last line in the file (and displayed in the logger) shows:<br />
<pre style="background-color: #fffacd;"><br />
Fitted spectrum for 3c84-J0319+413 with fitorder=1: Flux density = 29.0282 +/- 0.0308648 (freq=32.5128 GHz) spidx=-0.538758 +/- 0.00882913<br />
</pre><br />
<br />
[[Image:PlotG192-3C84-fluxspec-4.5.png|200px|thumb|right|Figure 7: 3C84 flux values returned by fluxscale]]<br />
[[Image:ScreenshotPlotG192-setjy-bp-4.5.png|200px|thumb|right|Figure 8: plotms of model amp vs freq for 3C84]]<br />
<br />
Using the information in the returned <tt>flux1</tt> dictionary, we can plot the derived spectrum (Figure 7):<br />
<source lang="python"><br />
# In CASA<br />
freq = flux1['freq'] / 1e9<br />
spw_list = range(0,64)<br />
spw_str = []<br />
for i in spw_list:<br />
thisspw = str(i)<br />
spw_str.append(thisspw)<br />
</source><br />
Note that in order to close indented python loops, conditions etc. you will have to press ''Enter'' again to execute the indented commands and to return to the CASA prompt. <br />
<br />
<source lang="python"><br />
# In CASA<br />
bootstrapped_fluxes = []<br />
for j in spw_str:<br />
thisflux = flux1['1'][j]['fluxd'][0]<br />
if thisflux ==None:<br />
continue<br />
else:<br />
bootstrapped_fluxes.append(thisflux)<br />
</source><br />
<br />
<br />
<source lang="python"><br />
# In CASA - this section creates the plot seen in Figure 7<br />
pl.clf()<br />
pl.plot(freq, bootstrapped_fluxes, 'bo')<br />
pl.xlabel('Frequency (GHz)')<br />
pl.ylabel('Flux Density (Jy)')<br />
pl.title('3C84')<br />
pl.show()<br />
</source><br />
<br />
We can use the model from {{fluxscale}} to fill the MODEL column with 3C84's spectral information using {{setjy}}. With ''standard='fluxscale''', we can directly use the <tt>flux1</tt> Python dictionary as input via ''fluxdict'':<br />
<br />
<source lang="python"><br />
# In CASA: spectral information<br />
setjy(vis='G192-BP.ms', field='1', scalebychan=True, \<br />
standard = 'fluxscale', fluxdict=flux1)<br />
</source><br />
<br />
Check with plotms that the data have been appropriately filled (Figure 8):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='G192-BP.ms', field='1', antenna='ea05&ea02', \<br />
xaxis='freq', yaxis='amp', ydatacolumn='model')<br />
</source><br />
<br />
We next redo the previous calibration using this new model information. Although the commands are the same as what we issued earlier, keep in mind that the model values for the bandpass calibrator have changed, and therefore the results of these calibration calculations will differ:<br />
<br />
<source lang="python"><br />
# In CASA: phase only recalibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.G0.b', \<br />
field='1', spw='*:60~68', \<br />
gaintype='G', refant='ea05', calmode='p', \<br />
solint='int', minsnr=3) <br />
# In CASA: residual delays recalibration<br />
gaincal(vis='G192-BP.ms', caltable='calG192.K0.b', \<br />
gaintable=['calG192.G0.b'], \<br />
field='1', spw='*:5~122', gaintype='K', \<br />
refant='ea05', solint='inf', minsnr=3)<br />
# In CASA: antenna bandpasses recalibration<br />
bandpass(vis='G192-BP.ms', caltable='calG192.B0.b', \<br />
gaintable=['calG192.G0.b', 'calG192.K0.b'], \<br />
field='1', refant='ea05', solnorm=False, \<br />
bandtype='B', solint='inf')<br />
</source><br />
<br />
Finally, we inspect these solutions (Figures 9a, 9b, 10a, and 10b):<br />
<br />
<source lang="python"><br />
# In CASA - Figure 9a<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='amp', \<br />
spw='0~31', iteration='antenna')<br />
# Figure 9b<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='amp', \<br />
spw='32~63', iteration='antenna')<br />
# Figure 10a<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='0~31', \<br />
plotrange=[-1,-1,-180,180])<br />
# Figure 10b<br />
plotcal(caltable='calG192.B0.b', xaxis='freq', yaxis='phase', \<br />
iteration='antenna', spw='32~63', \<br />
plotrange=[-1,-1,-180,180])<br />
</source><br />
<br />
{|<br />
| [[Image:plotG192_plotcal_B0a1.b_4.1.png|200px|thumb|left|Figure 9a: plotcal B0 bootstrapped bandpass amp ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0a2.b_4.1.png|200px|thumb|center|Figure 9b: plotcal B0 bootstrapped bandpass amp ant ea06 spw 32-63]]<br />
| [[Image:plotG192_plotcal_B0p1.b_4.1.png|200px|thumb|center|Figure 10a: plotcal B0 bootstrapped bandpass phase ant ea06 spw 0-31]]<br />
| [[Image:plotG192_plotcal_B0p2.b_4.1.png|200px|thumb|right|Figure 10b: plotcal B0 bootstrapped bandpass phase ant ea06 spw 32-63]] <br />
|}<br />
<br />
They look virtually unchanged from the previous solutions, with the exception that the amplitude scaling is corrected for the spectrum of 3C84. We have the final version of our delay and bandpass calibration tables, calG192.K0.b and calG192.B0.b, which can be used for all subsequent calibration steps. <br />
<br />
{{Checked 4.5.2}}<br />
<!--<br />
-- Original: Miriam Hartman (full G192 guide) <br /><br />
-- Modifications: Jose Salcido (4.5.2. 2016/04/14) <br /><br />
-- Topical Guide: Juergen Ott (4.5.2, 2016/04/14) <br /><br />
-- Edits to Guide: Tony Perreault (4.5.2, 2016/04/26) <br /><br />
--></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA:_Getting_Started&diff=19331VLA: Getting Started2016-05-09T21:23:32Z<p>Emomjian: </p>
<hr />
<div>== Introduction ==<br />
<br />
Before delving into data calibration, editing, etc., it is very important to understand the structure and contents of a measurement set (the format in which the data are stored). These pages will guide the new user through loading and initial inspection of their data, as well as on the structure and organization of the measurement set (MS). For a Guide on getting started with the CASA software, please see [[Getting Started in CASA]].<br />
<br />
If you are using AIPS to reduce your data please refer to [http://www.aips.nrao.edu/CookHTML/CookBookap5.html#x178-414000E Appendix E of the AIPS cookbook].<br />
<br />
== Getting Started CASA Guides ==<br />
<br />
* '''[[Loading Data]]'''<br />
<br />
<blockquote><br />
* Obtaining data from the archive<br />
** Selecting a set of scans<br />
** Choosing to average data<br />
** Retrieval process<br />
* Starting CASA and initial inspection: listobs<br />
* Initial data flagging: online flags, zero flags, and shadow flags<br />
</blockquote><br />
<br />
* '''[[Inspecting Data]]'''<br />
<br />
<blockquote><br />
* Checking the online flags<br />
* Checking the weather information<br />
* Initial inspection: listobs<br />
* Graphical display: plotms<br />
* Graphical display: viewer<br />
</blockquote><br />
<br />
* '''[[Measurement Set Contents]]'''<br />
<br />
<blockquote><br />
* The measurement set directory structure and contents<br />
* Inspecting MS contents in CASA<br />
* Advanced: using information about MS structure<br />
</blockquote><br />
<br />
* '''More advanced: EVLA Guides'''<br />
<br />
<blockquote><br />
* [[EVLA Tutorials | Tutorials]]<br />
* [[CASA Hints, Tips, and Tricks | Hints, Tips, & Tricks]]<br />
* [[CASA User Scripts and Tasks]]<br />
</blockquote></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA:_Getting_Started&diff=19330VLA: Getting Started2016-05-09T21:23:05Z<p>Emomjian: </p>
<hr />
<div>== Introduction ==<br />
<br />
Before delving into data calibration, editing, etc., it is very important to understand the structure and contents of a measurement set (the format in which the data are stored). These pages will guide the new user through loading and initial inspection of their data, as well as on the structure and organization of the measurement set (MS). For a Guide on getting started with the CASA software, please see [[Getting Started in CASA]].<br />
<br />
If you are using AIPS to reduce your data please refer to [http://www.aips.nrao.edu/CookHTML/CookBookap5.html#x178-414000E Appendix E of the AIPS cookbook].<br />
<br />
== Getting Started CASA Guides ==<br />
<br />
* '''[[Loading Data]]'''<br />
<br />
<blockquote><br />
* Obtaining data from the archive<br />
** Selecting a set of scans<br />
** Choosing to average data<br />
** Retrieval process<br />
* Starting CASA and initial inspection: listobs<br />
* Initial data flagging: online flags, zero flags, and shadow flags<br />
</blockquote><br />
<br />
* '''[[Inspecting Data]]'''<br />
<br />
<blockquote><br />
* Checking the online flags<br />
* Checking the weather information<br />
* Initial inspection: listobs<br />
* Graphical display: plotms<br />
* Graphical display: viewer<br />
</blockquote><br />
<br />
* '''[[Measurement Set Contents]]'''<br />
<br />
<blockquote><br />
* The measurement set directory structure and contents<br />
* Inspecting MS contents in CASA<br />
* Advanced: using information about MS structure<br />
</blockquote><br />
<br />
* '''More advanced: EVLA Guides'''<br />
<br />
<blockquote><br />
* [[VLA Tutorials | Tutorials]]<br />
* [[CASA Hints, Tips, and Tricks | Hints, Tips, & Tricks]]<br />
* [[CASA User Scripts and Tasks]]<br />
</blockquote></div>Emomjianhttps://casaguides.nrao.edu/index.php?title=Karl_G._Jansky_VLA_Tutorials&diff=19329Karl G. Jansky VLA Tutorials2016-05-09T21:19:12Z<p>Emomjian: </p>
<hr />
<div>== Introduction ==<br />
<br />
The Karl G. Jansky Very Large Array (VLA) Tutorials are meant to guide the observer through some common types of data analysis, using example datasets, and including explanations of the individual steps. These analyses will be broadly applicable to many VLA data sets. <br />
<br />
<br />
If you are new to VLA Data, you may start with <h3>[[VLA: Getting Started | Getting Started with VLA data]]</h3><br />
<br />
== VLA Tutorials ==<br />
<br />
<br />
<br />
* '''Carbon Star IRC+10216: high frequency (36GHz), spectral line data reduction''' (CASA 4.6)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 Tutorial] <br />
<br />
* This tutorial requires to download a dataset <font color=green>1.1GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and make image cubes of the line emission from this asymptotic giant branch star. This is a high-frequency VLA dataset. Includes:<br />
* Inspecting data; basic flagging & calibration<br />
* Subtracting continuum emission <br />
* Imaging the spectral lines<br />
* Imaging the continuum<br />
* Image analysis<br />
* Self-calibration <br />
</blockquote><br />
<br />
<br />
* '''Supernova Remnant 3C391: 6cm Polarimetry and Continuum Imaging, Mosaicking''' (CASA 4.6)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391 3C391 Tutorial] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>3.1GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate VLA full polarization data, image a mosaic of the region in full Stokes and create a spectral index map. Includes:<br />
* Inspecting data; basic flagging <br />
* Calibration<br />
* Image Analysis and Manipulation <br />
* Polarization Imaging<br />
* Spectral Index imaging<br />
* Self-calibration <br />
</blockquote><br />
<br />
== VLA Tutorials for more Advanced Users ==<br />
<br />
* '''Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ Supernova SN2010FZ Tutorial ] <br />
<br />
* This tutorial requires to download a dataset <font color=orange>2.9GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a galaxy field for this mid-frequency VLA observation. This includes:<br />
* Excising RFI from the data<br />
* Basic flagging and calibration steps<br />
* Imaging from one spectral window to full dataset<br />
* Using multi-scale, multi-frequency synthesis<br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192 Protostar G192.16-3.84 Tutorial]<br />
<br />
* This tutorial requires the download of an <font color=red>18GB</font> dataset<br />
</blockquote><br />
<br />
<blockquote><br />
Calibrate and image a protostar for this high-frequency VLA observation taken using the 3-bit samplers. Note that this tutorial can be used as a guide for 8-bit data as well. This includes:<br />
* Deleting bad data and RFI<br />
* Basic calibration, including tropospheric opacity and requantizer gain corrections<br />
* Bootstrapping to determine the spectrum of the bandpass calibration source<br />
* Imaging from a single spectral window to the full dataset<br />
* Using multi-frequency synthesis imaging <br />
* Image analysis<br />
</blockquote><br />
<br />
* '''Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)''' (CASA 4.4)<br />
<blockquote><br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4 G55.7+3.4 Tutorial]<br />
<br />
* This tutorial requires to download a dataset <font color=red>14GB</font> in size<br />
</blockquote><br />
<br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a supernova remnant focuses on RFI excision as well as imaging wide-field and wide-fractional-bandwidth data. This includes:<br />
* Basic flagging and evaluation<br />
* Different methods for automatic RFI excision<br />
* Calibrating the data<br />
* Exploring different imaging options: widefield, multi-scale, and multi-frequency synthesis<br />
</blockquote><br />
<br />
* '''P-band continuum imaging data reduction tutorial''' (CASA 4.6)<br />
<blockquote><br />
* [[P-band basic data reduction]]<br />
<br />
* This tutorial requires to download a dataset <font color=orange>21GB</font> in size <br />
<br />
</blockquote><br />
<blockquote><br />
This low-frequency VLA tutorial for the observation of a giant radio remnant focuses on low frequency wide-fractional-bandwidth data in the sub GHz regime . This includes:<br />
* Basic flagging and evaluation<br />
* Automatic RFI Excision<br />
* Ionospheric Calibration<br />
* Self - Calibration<br />
</blockquote><br />
<br />
== Specific Topics ==<br />
<br />
<!--<br />
* [http://casaguides.nrao.edu/index.php?title=WorkshopSelfcal_(Caltech) Self-calibration]<br />
--><br />
<br />
* CASA: [http://casaguides.nrao.edu/index.php?title=Writing_a_CASA_Task Writing a CASA task]<br />
<br />
<blockquote><br />
Presents the basics of writing a CASA task, and provides examples of tasks that will be helpful and instructive. <br />
</blockquote><br />
<br />
* [[Baseband Stitching-CASA4.1]]<br />
<br />
<blockquote><br />
Demonstrates the use of the contributed "stitch" task to edit data which has been obtained in an overlapping subband mode.<br />
</blockquote><br />
<br />
* [[Imaging Flanking Fields]]<br />
<br />
* [[Transient reduction pipeline]]<br />
<br />
* [[Obtaining EVLA Data: 3C 391 Example]]<br />
<br />
<br />
== Other ==<br />
* Data Calibration, Imaging, and Analysis - simplified for Community Day Events: [[EVLA high frequency Spectral Line Tutorial - CDE]]<br />
<br />
<br />
== CASA 4.6 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.6 IRC+10216 Tutorial (CASA4.6)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6 3C391 Continuum Tutorial (CASA 4.6)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/P-band_basic_data_reduction-CASA4.6 P-band continuum imaging data reduction tutorial (CASA 4.6)]<br />
<br />
== CASA 4.5 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.5 IRC+10216 Tutorial (CASA4.5)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 Continuum Tutorial (CASA 4.5)] <br />
<br />
* [https://casaguides.nrao.edu/index.php/P-band_basic_data_reduction-CASA4.5 P-band continuum imaging data reduction tutorial (CASA 4.5)]<br />
<br />
<!--<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Flagging-CASA4.5.2 Guide for Flagging in CASA]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Bandpass-Slope-CASA4.5.2 Guide on Correcting a Bandpass Slope (CASA 4.5.2)]<br />
<br />
* [https://casaguides.nrao.edu/index.php/VLA-CASA-Imaging-CASA4.5.2 Imaging Guide (CASA 4.5.2)]<br />
--><br />
<br />
== CASA 4.4 versions ==<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.4 IRC+10216 Tutorial (CASA4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.4 3C391 Continuum Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.4 Supernova SN2010FZ Tutorial (CASA 4.4)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.4 Protostar G192.16-3.84 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.4 G55.7+3.4 Tutorial (CASA 4.4)]<br />
<br />
* [http://casaguides.nrao.edu/index.php/P-band_basic_data_reduction-CASA4.4 P-band basic data reduction (CASA 4.4)]<br />
<br />
<br />
<br />
== CASA 4.3 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA4.3 IRC+10216 Tutorial (CASA4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA4.3 3C391 Tutorial Part 1: calibration, imaging (CASA 4.3)] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA4.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration (CASA 4.3)] <br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA4.3 Supernova SN2010FZ Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_3-bit_Tutorial_G192-CASA4.3 Protostar G192.16-3.84 Tutorial (CASA 4.3)]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA4.3 G55.7+3.4 Tutorial (CASA 4.3)]<br />
<br />
== CASA 4.2 versions ==<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.2]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.2 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.2 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.2 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.2 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.2 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
<br />
== CASA 4.1 versions ==<br />
* [[EVLA high frequency Spectral Line tutorial - IRC+10216 - CASA 4.1]]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391_-_CASA_4.1 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=Advanced_Topics_3C391_-_CASA4.1 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ_-_CASA_4.1 Supernova SN2010FZ: Wide-band, narrow-field imaging; C-band (5-7 GHz)]<br />
<br />
* [[EVLA 3-bit Tutorial G192 - CASA4.1 | Protostar G192.16-3.84: Wide-band, narrow-field imaging using 3-bit sampled data; Ka-band (27-38 GHz)]]<br />
<br />
* [[EVLA Wide-Band Wide-Field Imaging: G55.7 - CASA4.1 | Supernova G55.7_3.4: Wide-band, wide-field imaging; L-band (1-2 GHz)]]<br />
<br />
<br />
== CASA 3.4 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216-CASA3.4 IRC+10216 Tutorial] <br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.4 SN2010FZ Tutorial] <br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.4 G55.7+3.4 Tutorial]<br />
<br />
<br />
<br />
== CASA 3.3 versions ==<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part1-CASA3.3 IRC+10216 Part 1: editing, calibration] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216_part2-CASA3.3 IRC+10216 Part 2: continuum subtraction, imaging, selfcal] <br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.3 3C391 Tutorial Part 1: calibration, imaging] <br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.3 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration] <br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_6cmWideband_Tutorial_SN2010FZ-CASA3.3 SN2010FZ Tutorial] <br />
<br />
<br />
* [http://casaguides.nrao.edu/index.php?title=EVLA_Wide-Band_Wide-Field_Imaging:_G55.7_3.4-CASA3.3 G55.7+3.4 Tutorial]</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_Continuum_Tutorial_3C391-CASA4.6&diff=19064VLA Continuum Tutorial 3C391-CASA4.62016-04-21T17:10:17Z<p>Emomjian: /* Examining and Editing the Data */</p>
<hr />
<div>[[Category:EVLA]][[Category:Calibration]]<br />
<br />
<b>This CASA Guide is for Version 4.6 of CASA.</b> If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this casaguide, as we try to limit script breaking changes in CASA development. <br />
<br />
== Overview ==<br />
This CASA guide describes the calibration and imaging of a multiple-pointing Jansky VLA continuum dataset on the supernova remnant <br />
[http://simbad.u-strasbg.fr/simbad/sim-id?Ident=3C+391&NbIdent=1&Radius=2&Radius.unit=arcmin&submit=submit+id 3C 391]. The data were taken in JVLA early science OSRO1 mode, with 128 MHz of bandwidth in each of two widely spaced spectral windows, centered at 4.6 and 7.5 GHz, and were set up for full polarization calibration.<br />
<br />
<!-- Original content <br />
There are a number of possible ways to run CASA, described in more detail in [[Getting Started in CASA]]. In brief, there are at least three different ways to run CASA:<br />
* Interactively examining task inputs. In this mode, one types <tt>default taskname</tt> to load the task, <tt>inp</tt> to examine the inputs, and <tt>go</tt> once those inputs have been set to your satisfaction. Allowed inputs are shown in blue, and bad inputs are colored red. The inputs themselves are changed one by one, e.g., <tt>selectdata=T</tt>. Screenshots of the inputs to various tasks used in the data reduction below are provided, to illustrate which parameters need to be set.<br />
More detailed help can be obtained on any task by typing <tt>help taskname</tt>. Once a task is run, the set of inputs are stored and can be retrieved via <tt>tget taskname</tt>; subsequent runs will overwrite the previous <tt>tget</tt> file.<br />
* Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set. <br />
* Non-interactively via a script. A series of task function calls can be combined together into a script, and run from within CASA via <tt>execfile('scriptname.py')</tt>. This (and other) CASA Guides have been designed to be extracted into a script using the [[Extracting_scripts_from_these_tutorials | script extractor]]. Should one use the script generated by the [[Extracting_scripts_from_these_tutorials | script extractor]] for this CASA Guide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is in fact unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered). <!--, and in one case (that of {{plotms}}), you '''must''' leave the graphics window open, as the GUI cannot be reopened without first exiting from CASA. (note 5/6/13: tested and this no longer seems to be an issue [MK]) --><br />
<!-- If you are a relative novice (and <em>particularly</em> for this tutorial), it is <em>strongly</em> recommended that you start with the interactive mode, graduating to the pseudo- or non-interactive mode as you gain experience. Work at your own pace, look at the inputs to the tasks to see what other options exist, and read the help files.<br />
End original content --><br />
<br />
== Obtaining the Data ==<br />
<br />
For the purposes of this tutorial, we have created a starting data set, upon which several initial processing steps have already been conducted. You may obtain the data set from here:<br />
[http://casa.nrao.edu/Data/EVLA/3C391/3c391_ctm_mosaic_10s_spw0.ms.tgz http://casa.nrao.edu/Data/EVLA/3C391/3c391_ctm_mosaic_10s_spw0.ms.tgz] (dataset size: 3.1GB).<br />
<br />
If you wish to start from the very beginning, you may download the dataset from the [https://archive.nrao.edu/archive/archiveproject.jsp NRAO Archive]: TDEM0001_sb1218006_1.55310.33439732639<br />
<br />
We are providing this starting data set, rather than the true initial data set for at least two reasons. First, many of these initial processing steps can be rather time consuming (> 1 hr). Second, while necessary, many of these steps are not fundamental to the calibration and imaging process, which is the focus of this tutorial. For completeness, however, here are the steps that were taken from the initial data set to produce the starting data set. <br />
<br />
* The initial Science Data Model (SDM) file was converted into a measurement set.<br />
* Basic data flagging was applied, to account for shadowing of the antennas. These data are from the D configuration, in which antennas are particularly susceptible to being blocked (shadowed) by other antennas in the array, depending upon the elevation of the source.<br />
* The data were averaged from the initial 1-second correlator sample time to 10-second samples. In the D configuration, the fringe rate is relatively slow and time-average smearing is less of a concern.<br />
* The data were acquired with two subbands (spectral windows) around 4.6 and 7.5 GHz. Because of disk space concerns on some machines, the focus will be on only one of the two spectral windows.<br />
<br />
All of these steps can be done directly from the NRAO Archive's Download page, by selecting '''CASA MS''' as the download format (it's a good idea to also check the '''Create MS or SDM tar file''' box), checking the '''Apply flags generated during observing''' box, and setting '''Time Averaging''' to 10s.<br />
<br />
<!-- We emphasize that, were this a real science observation, all of these steps would need to be run. Detailed instructions on obtaining the data from the archive and creating this "starting" data set may be found in the [[Obtaining EVLA Data: 3C 391 Example]] tutorial. 5/6/13: removed this b/c the page is outdated. All steps can now be done directly from the Archive download page (added note). MK --><br />
Once the download is complete, unzip and unpack the file (within a working directory, which you will then run CASA):<br />
<br />
<source lang="bash"><br />
# In a Terminal:<br />
tar xzvf 3c391_ctm_mosaic_10s_spw0.ms.tgz<br />
</source><br />
<br />
== How to Use This CASA Guide ==<br />
<br />
Here are a number of possible ways to run CASA, described in more detail in [[Getting Started in CASA]]. In brief, there are at least three different ways to run CASA:<br />
<br />
* Interactively examining task inputs. In this mode, one types '''taskname''' to load the task, '''inp''' to examine the inputs, and '''go''' once those inputs have been set to your satisfaction. Allowed inputs are shown in blue and bad inputs are colored red. The input parameters themselves are changed one by one, e.g., ''selectdata=T''. Screenshots of the inputs to various tasks used in the data reduction below are provided, to illustrate which parameters need to be set. More detailed help can be obtained on any task by typing '''help ''taskname'''''. Once a task is run, the set of inputs are stored and can be retrieved via '''tget ''taskname'''''; subsequent runs will overwrite the previous '''tget''' file.<br />
<br />
* Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set.<br />
<br />
* Non-interactively via a script. A series of task function calls can be combined together into a script, and run from within CASA via '''execfile('scriptname.py')'''. This and other CASA Tutorial Guides have been designed to be extracted into a script via the script extractor by using the method described at the [[Extracting_scripts_from_these_tutorials]] page. Should you use the script generated by the script extractor for this CASA Guide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is in fact unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered).<br />
<br />
If you are a relative novice or just new to CASA, it is strongly recommended to work through this tutorial by cutting and pasting the task function calls provided below after you have read all the associated explanations. Work at your own pace, look at the inputs to the tasks to see what other options exist, and read the help files. Later, when you are more comfortable, you might try to extract the script, modify it for your purposes, and begin to reduce other data.<br />
<br />
== The Observation ==<br />
<br />
Before starting the calibration process, we want to get some basic information about the data set. To examine the observing conditions during the observing run, and to find out any known problems with the data, download the [http://www.vla.nrao.edu/cgi-bin/oplogs.cgi observer log]. Simply fill in the known observing date (in our case 2010-Apr-24) as both the Start and Stop date, and click on the '''Show Logs''' button. The relevant log is labeled with the project code, TDEM0001, and can be downloaded as a PDF file. From this, we find the following:<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
Information from observing log:<br />
There is no C-band receivers on ea13<br />
Antenna ea06 is out of the array<br />
Antenna ea15 has some corrupted data<br />
Antennas ea10, ea12, ea22 do not have good baseline positions<br />
Gusty winds, mixed clouds, API rms up to 11.5.<br />
</pre><br />
<br />
Before beginning our data reduction, we must start CASA. If you have not used CASA before, some helpful tips are available on the [[Getting Started in CASA]] page.<br />
<br />
Once you have CASA up and running in the directory containing the data, then start your data reduction by getting some basic information about the data. The task {{listobs}} can be used to get a listing of the individual scans comprising the observation, the frequency setup, source list, and antenna locations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c391_ctm_mosaic_10s_spw0.ms')<br />
</source><br />
<br />
One will note that there are nine sources observed. Here the various sources are introduced briefly, with more detail contained in the sections below in which they are used.<br />
* J1331+3030 = 3C 286, which will later serve as a calibrator for the visibility amplitudes, i.e., it is assumed to have a precisely known flux density; the spectral bandpass; and the polarization position angle;<br />
* J1822-0938, which will serve as a calibrator for the visibility phases;<br />
* J0319+4130 = 3C 84, which will serve as a polarization calibrator; and<br />
* 3C391 C1&#150;C7, which are 7 fields centered on and surrounding the supernova remnant.<br />
This observation was set up as a 7-pointing mosaic because the supernova remnant is so large that it essentially fills the primary beam.<br />
<br />
<br />
<pre style="background-color: #fffacd;"><br />
##########################################<br />
##### Begin Task: listobs #####<br />
listobs(vis="3c391_ctm_mosaic_10s_spw0.ms",selectdata=True,spw="",field="",antenna="",<br />
uvrange="",timerange="",correlation="",scan="",intent="",<br />
feed="",array="",observation="",verbose=True,listfile="",<br />
listunfl=False,cachesize=50,overwrite=False)<br />
================================================================================<br />
MeasurementSet Name: 3c391_ctm_mosaic_10s_spw0.ms MS Version 2<br />
================================================================================<br />
Observer: Dr. James Miller-Jones Project: T.B.D. <br />
Observation: EVLA<br />
Data records: 845379 Total integration time = 28681.5 seconds<br />
Observed from 24-Apr-2010/08:02:10.0 to 24-Apr-2010/16:00:11.5 (UTC)<br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
24-Apr-2010/08:02:10.0 - 08:02:30.0 1 0 J1331+3030 650 [0] [10] <br />
08:02:20.0 - 08:09:30.0 2 0 J1331+3030 13975 [0] [10] <br />
08:09:20.0 - 08:16:28.0 3 0 J1331+3030 13975 [0] [10] <br />
08:19:38.0 - 08:24:26.5 4 1 J1822-0938 7035 [0] [10] <br />
08:24:48.0 - 08:29:48.0 5 2 3C391 C1 7590 [0] [10] <br />
08:29:38.0 - 08:34:48.0 6 3 3C391 C2 7821 [0] [10] <br />
08:34:38.0 - 08:39:48.0 7 4 3C391 C3 7821 [0] [10] <br />
08:39:38.0 - 08:44:48.0 8 5 3C391 C4 7821 [0] [10] <br />
08:44:38.0 - 08:49:48.0 9 6 3C391 C5 7843 [0] [10] <br />
08:49:38.0 - 08:54:48.0 10 7 3C391 C6 7843 [0] [10] <br />
08:54:38.0 - 08:59:43.5 11 8 3C391 C7 7843 [0] [10] <br />
09:00:03.0 - 09:01:31.0 12 1 J1822-0938 2925 [0] [10] <br />
09:01:52.0 - 09:06:52.0 13 2 3C391 C1 7941 [0] [10] <br />
09:06:42.0 - 09:11:52.0 14 3 3C391 C2 9801 [0] [10] <br />
09:11:42.0 - 09:16:52.0 15 4 3C391 C3 10075 [0] [10] <br />
09:16:42.0 - 09:21:52.0 16 5 3C391 C4 10050 [0] [10] <br />
09:21:42.0 - 09:26:52.0 17 6 3C391 C5 10075 [0] [10] <br />
09:26:42.0 - 09:31:52.0 18 7 3C391 C6 10075 [0] [10] <br />
09:31:42.0 - 09:36:47.5 19 8 3C391 C7 10075 [0] [10] <br />
09:37:07.0 - 09:38:35.0 20 1 J1822-0938 2900 [0] [10] <br />
09:38:57.0 - 09:43:57.0 21 2 3C391 C1 9700 [0] [10] <br />
09:43:47.0 - 09:48:57.0 22 3 3C391 C2 10050 [0] [10] <br />
09:48:47.0 - 09:53:57.0 23 4 3C391 C3 10075 [0] [10] <br />
09:53:47.0 - 09:58:57.0 24 5 3C391 C4 10075 [0] [10] <br />
09:58:47.0 - 10:03:57.0 25 6 3C391 C5 10075 [0] [10] <br />
10:03:47.0 - 10:08:57.0 26 7 3C391 C6 10075 [0] [10] <br />
10:08:47.0 - 10:13:47.0 27 8 3C391 C7 9750 [0] [10] <br />
10:14:12.0 - 10:15:39.5 28 1 J1822-0938 2925 [0] [10] <br />
10:16:01.0 - 10:21:01.0 29 2 3C391 C1 9000 [0] [10] <br />
10:20:51.0 - 10:26:01.0 30 3 3C391 C2 10050 [0] [10] <br />
10:25:51.0 - 10:31:01.0 31 4 3C391 C3 10075 [0] [10] <br />
10:30:51.0 - 10:36:01.0 32 5 3C391 C4 10075 [0] [10] <br />
10:35:51.0 - 10:41:01.0 33 6 3C391 C5 10075 [0] [10] <br />
10:40:51.0 - 10:46:01.0 34 7 3C391 C6 10075 [0] [10] <br />
10:45:51.0 - 10:50:51.0 35 8 3C391 C7 9750 [0] [10] <br />
10:51:15.0 - 10:52:42.5 36 1 J1822-0938 2925 [0] [10] <br />
10:55:14.0 - 10:57:42.0 37 0 J1331+3030 3364 [0] [10] <br />
11:00:13.0 - 11:02:41.0 38 1 J1822-0938 3883 [0] [10] <br />
11:03:03.0 - 11:08:03.0 39 2 3C391 C1 9750 [0] [10] <br />
11:07:53.0 - 11:12:53.0 40 3 3C391 C2 9725 [0] [10] <br />
11:12:43.0 - 11:17:43.0 41 4 3C391 C3 9750 [0] [10] <br />
11:17:33.0 - 11:22:33.0 42 5 3C391 C4 9750 [0] [10] <br />
11:22:23.0 - 11:27:23.0 43 6 3C391 C5 9750 [0] [10] <br />
11:27:13.0 - 11:32:13.0 44 7 3C391 C6 9750 [0] [10] <br />
11:32:03.0 - 11:36:53.0 45 8 3C391 C7 9425 [0] [10] <br />
11:37:21.0 - 11:38:47.0 46 1 J1822-0938 2700 [0] [10] <br />
11:39:11.0 - 11:44:11.0 47 2 3C391 C1 9750 [0] [10] <br />
11:44:01.0 - 11:49:01.0 48 3 3C391 C2 9700 [0] [10] <br />
11:48:51.0 - 11:53:41.0 49 4 3C391 C3 8355 [0] [10] <br />
11:53:41.0 - 11:58:31.0 50 5 3C391 C4 9425 [0] [10] <br />
11:58:21.0 - 12:03:21.0 51 6 3C391 C5 9725 [0] [10] <br />
12:03:11.0 - 12:08:11.0 52 7 3C391 C6 9701 [0] [10] <br />
12:08:01.0 - 12:12:59.0 53 8 3C391 C7 9725 [0] [10] <br />
12:13:29.0 - 12:14:48.0 54 1 J1822-0938 2600 [0] [10] <br />
12:15:18.0 - 12:20:08.0 55 2 3C391 C1 9425 [0] [10] <br />
12:19:58.0 - 12:24:58.0 56 3 3C391 C2 9750 [0] [10] <br />
12:24:48.0 - 12:29:48.0 57 4 3C391 C3 9750 [0] [10] <br />
12:29:38.0 - 12:34:38.0 58 5 3C391 C4 9725 [0] [10] <br />
12:34:28.0 - 12:39:28.0 59 6 3C391 C5 9725 [0] [10] <br />
12:39:18.0 - 12:44:18.0 60 7 3C391 C6 9750 [0] [10] <br />
12:44:08.0 - 12:49:04.5 61 8 3C391 C7 9750 [0] [10] <br />
12:49:35.0 - 12:50:53.0 62 1 J1822-0938 2600 [0] [10] <br />
12:51:24.0 - 12:56:14.0 63 2 3C391 C1 9425 [0] [10] <br />
12:56:04.0 - 13:01:04.0 64 3 3C391 C2 9000 [0] [10] <br />
13:00:54.0 - 13:05:54.0 65 4 3C391 C3 9750 [0] [10] <br />
13:05:44.0 - 13:10:44.0 66 5 3C391 C4 9750 [0] [10] <br />
13:10:34.0 - 13:15:34.0 67 6 3C391 C5 9725 [0] [10] <br />
13:15:24.0 - 13:20:24.0 68 7 3C391 C6 9750 [0] [10] <br />
13:20:14.0 - 13:25:10.0 69 8 3C391 C7 9000 [0] [10] <br />
13:25:40.0 - 13:26:57.5 70 1 J1822-0938 2600 [0] [10] <br />
13:27:28.0 - 13:32:18.0 71 2 3C391 C1 9425 [0] [10] <br />
13:32:08.0 - 13:37:08.0 72 3 3C391 C2 9750 [0] [10] <br />
13:36:58.0 - 13:41:58.0 73 4 3C391 C3 9750 [0] [10] <br />
13:41:48.0 - 13:46:48.0 74 5 3C391 C4 9750 [0] [10] <br />
13:46:38.0 - 13:51:38.0 75 6 3C391 C5 9725 [0] [10] <br />
13:51:28.0 - 13:56:28.0 76 7 3C391 C6 9750 [0] [10] <br />
13:56:18.0 - 14:01:14.0 77 8 3C391 C7 9750 [0] [10] <br />
14:01:44.0 - 14:03:01.5 78 1 J1822-0938 2024 [0] [10] <br />
14:03:33.0 - 14:08:23.0 79 2 3C391 C1 8900 [0] [10] <br />
14:08:13.0 - 14:13:13.0 80 3 3C391 C2 9750 [0] [10] <br />
14:13:03.0 - 14:18:03.0 81 4 3C391 C3 9750 [0] [10] <br />
14:17:53.0 - 14:22:53.0 82 5 3C391 C4 9350 [0] [10] <br />
14:22:43.0 - 14:27:43.0 83 6 3C391 C5 9000 [0] [10] <br />
14:27:33.0 - 14:32:33.0 84 7 3C391 C6 8595 [0] [10] <br />
14:32:23.0 - 14:37:18.5 85 8 3C391 C7 7590 [0] [10] <br />
14:37:48.0 - 14:39:05.5 86 1 J1822-0938 1848 [0] [10] <br />
14:39:36.0 - 14:44:26.0 87 2 3C391 C1 7337 [0] [10] <br />
14:44:16.0 - 14:49:16.0 88 3 3C391 C2 7568 [0] [10] <br />
14:49:06.0 - 14:54:06.0 89 4 3C391 C3 7590 [0] [10] <br />
14:53:56.0 - 14:58:56.0 90 5 3C391 C4 7527 [0] [10] <br />
14:58:46.0 - 15:03:46.0 91 6 3C391 C5 7568 [0] [10] <br />
15:03:36.0 - 15:08:36.0 92 7 3C391 C6 7590 [0] [10] <br />
15:08:26.0 - 15:13:22.0 93 8 3C391 C7 7590 [0] [10] <br />
15:13:51.0 - 15:15:09.0 94 1 J1822-0938 1680 [0] [10] <br />
15:15:40.0 - 15:20:30.0 95 2 3C391 C1 7337 [0] [10] <br />
15:20:20.0 - 15:25:20.0 96 3 3C391 C2 7568 [0] [10] <br />
15:25:10.0 - 15:30:10.0 97 4 3C391 C3 7590 [0] [10] <br />
15:30:00.0 - 15:35:00.0 98 5 3C391 C4 7564 [0] [10] <br />
15:34:50.0 - 15:39:50.0 99 6 3C391 C5 7260 [0] [10] <br />
15:39:40.0 - 15:44:40.0 100 7 3C391 C6 6930 [0] [10] <br />
15:44:30.0 - 15:49:26.0 101 8 3C391 C7 6930 [0] [10] <br />
15:49:55.0 - 15:51:13.5 102 1 J1822-0938 1088 [0] [10] <br />
15:54:52.0 - 16:00:11.5 103 9 J0319+4130 8768 [0] [10] <br />
(nRows = Total number of rows per scan) <br />
Fields: 10<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 N J1331+3030 13:31:08.287984 +30.30.32.95886 J2000 0 31964<br />
1 J J1822-0938 18:22:28.704200 -09.38.56.83501 J2000 1 39733<br />
2 NONE 3C391 C1 18:49:24.244000 -00.55.40.58001 J2000 2 105580<br />
3 NONE 3C391 C2 18:49:29.149001 -00.57.48.00001 J2000 3 110533<br />
4 NONE 3C391 C3 18:49:19.339000 -00.57.48.00001 J2000 4 110331<br />
5 NONE 3C391 C4 18:49:14.434001 -00.55.40.58001 J2000 5 110862<br />
6 NONE 3C391 C5 18:49:19.339000 -00.53.33.16000 J2000 6 110546<br />
7 NONE 3C391 C6 18:49:29.149001 -00.53.33.16000 J2000 7 109884<br />
8 NONE 3C391 C7 18:49:34.054000 -00.55.40.58001 J2000 8 107178<br />
9 Z J0319+4130 03:19:48.160102 +41.30.42.10305 J2000 9 8768<br />
Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) Corrs <br />
0 Subband:0 64 TOPO 4536.000 2000.000 128000.0 4599.0000 RR RL LR LL<br />
Sources: 10<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1331+3030 0 - - <br />
1 J1822-0938 0 - - <br />
2 3C391 C1 0 - - <br />
3 3C391 C2 0 - - <br />
4 3C391 C3 0 - - <br />
5 3C391 C4 0 - - <br />
6 3C391 C5 0 - - <br />
7 3C391 C6 0 - - <br />
8 3C391 C7 0 - - <br />
9 J0319+4130 0 - - <br />
Antennas: 26:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 ea01 W09 25.0 m -107.37.25.2 +33.53.51.0 -521.9407 -332.7782 -1.1977 -1601710.017000 -5042006.928200 3554602.355600<br />
1 ea02 E02 25.0 m -107.37.04.4 +33.54.01.1 9.8247 -20.4292 -2.7808 -1601150.059500 -5042000.619800 3554860.729400<br />
2 ea03 E09 25.0 m -107.36.45.1 +33.53.53.6 506.0591 -251.8666 -3.5832 -1600715.948000 -5042273.187000 3554668.184500<br />
3 ea04 W01 25.0 m -107.37.05.9 +33.54.00.5 -27.3562 -41.3030 -2.7418 -1601189.030140 -5042000.493300 3554843.425700<br />
4 ea05 W08 25.0 m -107.37.21.6 +33.53.53.0 -432.1158 -272.1493 -1.5032 -1601614.091000 -5042001.655700 3554652.509300<br />
5 ea07 N06 25.0 m -107.37.06.9 +33.54.10.3 -54.0667 263.8720 -4.2292 -1601162.593200 -5041829.000000 3555095.890500<br />
6 ea08 N01 25.0 m -107.37.06.0 +33.54.01.8 -30.8810 -1.4664 -2.8597 -1601185.634945 -5041978.156586 3554876.424700<br />
7 ea09 E06 25.0 m -107.36.55.6 +33.53.57.7 236.9058 -126.3369 -2.4443 -1600951.588000 -5042125.911000 3554773.012300<br />
8 ea11 E04 25.0 m -107.37.00.8 +33.53.59.7 102.8046 -63.7684 -2.6412 -1601068.791200 -5042051.910200 3554824.835300<br />
9 ea12 E08 25.0 m -107.36.48.9 +33.53.55.1 407.8394 -206.0057 -3.2252 -1600801.916000 -5042219.371000 3554706.449900<br />
10 ea13 N07 25.0 m -107.37.07.2 +33.54.12.9 -61.1040 344.2335 -4.6144 -1601155.635800 -5041783.843000 3555162.374100<br />
11 ea14 E05 25.0 m -107.36.58.4 +33.53.58.8 164.9788 -92.8032 -2.5268 -1601014.462000 -5042086.252000 3554800.799800<br />
12 ea15 W06 25.0 m -107.37.15.6 +33.53.56.4 -275.8288 -166.7451 -2.0590 -1601447.198000 -5041992.502500 3554739.687600<br />
13 ea16 W02 25.0 m -107.37.07.5 +33.54.00.9 -67.9687 -26.5614 -2.7175 -1601225.255200 -5041980.383590 3554855.675000<br />
14 ea17 W07 25.0 m -107.37.18.4 +33.53.54.8 -349.9866 -216.7507 -1.7978 -1601526.386100 -5041996.840100 3554698.327400<br />
15 ea18 N09 25.0 m -107.37.07.8 +33.54.19.0 -77.4352 530.6274 -5.5867 -1601139.485500 -5041679.036000 3555316.532800<br />
16 ea19 W04 25.0 m -107.37.10.8 +33.53.59.1 -152.8599 -83.8054 -2.4614 -1601315.893000 -5041985.320170 3554808.304600<br />
17 ea20 N05 25.0 m -107.37.06.7 +33.54.08.0 -47.8454 192.6015 -3.8723 -1601168.786100 -5041869.054000 3555036.936000<br />
18 ea21 E01 25.0 m -107.37.05.7 +33.53.59.2 -23.8638 -81.1510 -2.5851 -1601192.467800 -5042022.856800 3554810.438800<br />
19 ea22 N04 25.0 m -107.37.06.5 +33.54.06.1 -42.5986 132.8623 -3.5431 -1601173.953700 -5041902.660400 3554987.536500<br />
20 ea23 E07 25.0 m -107.36.52.4 +33.53.56.5 318.0523 -164.1848 -2.6960 -1600880.570000 -5042170.388000 3554741.457400<br />
21 ea24 W05 25.0 m -107.37.13.0 +33.53.57.8 -210.0944 -122.3885 -2.2581 -1601377.008000 -5041988.665500 3554776.393400<br />
22 ea25 N02 25.0 m -107.37.06.2 +33.54.03.5 -35.6245 53.1806 -3.1345 -1601180.861480 -5041947.453400 3554921.628700<br />
23 ea26 W03 25.0 m -107.37.08.9 +33.54.00.1 -105.3429 -51.7191 -2.6054 -1601265.151700 -5041982.533050 3554834.856300<br />
24 ea27 E03 25.0 m -107.37.02.8 +33.54.00.5 50.6647 -39.4832 -2.7249 -1601114.365500 -5042023.153700 3554844.945600<br />
25 ea28 N08 25.0 m -107.37.07.5 +33.54.15.8 -68.9057 433.1889 -5.0602 -1601147.940400 -5041733.837000 3555235.956000<br />
##### End Task: listobs #####<br />
##########################################<br />
<br />
</pre><br />
<br />
Note that the antenna IDs (which are numbered sequentially up to the total number of antennas in the array; 0 through 25 in this instance) do not correspond to the actual antenna names (ea01 through ea28; these numbers correspond to those painted on the side of the dishes). The antennas can be referenced using either convention; ''antenna='22' '' would correspond to ea25, whereas ''antenna='ea22' '' would correspond to ea22. Note that the antenna numbers in the observer log correspond to the actual antenna names, i.e., the 'ea??' numbers given in listobs.<br />
<br />
Both to get a sense of the array, as well as identify an antenna for later use in calibration, use the task {{plotants}} (see Figure 1). In general, for calibration purposes, one would like to select an antenna that is close to the center of the array (and that is not listed in the operator's log as having had problems!). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis='3c391_ctm_mosaic_10s_spw0.ms',figfile='plotants_3c391_antenna_layout.png')<br />
clearstat() # This removes the table lock generated by plotants in script mode<br />
</source><br />
<br />
<br />
[[Image:3C391_mosaic-plotants.png|200px|thumb|center|Figure 1: plotants figure]]<br />
<br />
== Examining and Editing the Data ==<br />
<br />
It is always a good idea, particularly with a new system like the JVLA, to examine the data. Moreover, from the observer's log, we already know that one antenna will need to be flagged because it does not have a C-band receiver. Start by flagging data known to be bad, then examine the data.<br />
<br />
In the scheduling block configuration, it is common to insert a setup scan as the first scan. From the '''{{listobs}}''' output above, one may have noticed that the first scan is less than 1 minute long. This first scan can safely be flagged.<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', flagbackup=T, <br />
mode='manual', scan='1')<br />
</source><br />
<br />
* ''flagbackup=T'' : A comment is warranted on the setting of flagbackup (here set to T for True). If set to True, '''{{flagdata}}''' will save a copy of the existing set of flags ''before'' entering any new flags. The setting of flagbackup is therefore a matter of some taste. You could choose not to save any flags or only save major flags, or you could save every flag. <!-- (One of the authors of this document was glad that flagbackup was set to True as he recently ran '''{{flagdata}}''' with a typo in one of the entries.) --><br />
* ''mode='manualflag' '': Specific data are going to be selected to be edited. <br />
<!--* <tt>selectdata=T</tt> : In order to select the specific data to be flagged, selectdata has to be set to True. Once selectdata is set to True, then the various data selection options become visible (use ''help flagdata'' to see the possible options). In this case, --><br />
* ''scan='1' '': is chosen to select only the first scan. Note that scan expects an entry in the form of a string (scan=1 would generate an error).<br />
<br />
If satisfied with the inputs, run this task. The initial display in the logger will include <br />
<br />
<pre style="background-color: #fffacd;"><br />
##########################################<br />
##### Begin Task: flagdata #####<br />
.<br />
.<br />
.<br />
.<br />
Backup original flags before applying new flags<br />
Table type is Measurement Set<br />
Creating new backup flag file called flagdata_1<br />
Table type is Measurement Set<br />
Manual mode is active<br />
Initializing the agents<br />
Manual autocorr is 0<br />
There are 1 valid agents in list<br />
Running the agentflagger tool<br />
------------------------------------------------------------------------------------ <br />
Chunk = 1 [progress: 100%], Observation = 0, Array = 0, Scan = 1, Field = 0 (J1331+3030), Spw = 0, Channels = 64, Corrs = [ RR RL LR LL ], Total Rows = 650<br />
Manual => Data flagged so far 100%<br />
==================================================================================== <br />
=> Percentage of data flagged in table selection: 100%<br />
=> Writing flags to the MS<br />
##### End Task: flagdata #####<br />
##########################################<br />
</pre><br />
<br />
which indicates that, among other things, the flags that existed in the data set prior to this run will be saved to another file called ''flagdata_1''. Should you ever desire to revert to the data prior to this run, the task '''{{flagmanager}}''' could be used. Also note that the values of all the task parameters (explicit or hidden) are given at the start of the task listing.<br />
<br />
From the observer's log, we know that antenna ea13 does not have a C-band receiver and antenna ea15 had some corrupted data, so they should be flagged as well. The parameters are similar as before.<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', flagbackup=T, mode='manual', <br />
antenna='ea13,ea15')<br />
</source><br />
* ''antenna='ea13,ea15' '': Once again, this parameter requires a string input. Remember that ''antenna='ea13' ''and'' 'antenna='13' ''are '''not''' the same antenna. (See the discussion after our call to '''{{listobs}}''' above.)<br />
<br />
Finally, it is common for the array to require a small amount of time to settle down at the start of a scan. Consequently, it has become standard practice to flag the initial samples from the start of each scan.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', mode='quack', <br />
quackinterval=10.0, quackmode='beg')<br />
</source><br />
* ''mode='quack' '': Quack is another mode in which the same edit will be applied to all scans for all baselines.<br />
* ''quackmode='beg' '': In this case, data from the start of each scan will be flagged. Other options include flagging data at the end of the scan.<br />
* ''quackinterval=10 '': In this data set, the sampling time is 10 seconds, so this choice flags the first sample from all scans on all baselines.<br />
<br />
Having now done some basic editing of the data, based in part on ''a priori'' information, it is time to look at the data to determine if there are any other obvious problems. One task to examine the data themselves is '''{{plotms}}'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
clearstat() # This removes any existing table locks generated by flagdata<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms', selectdata=True, field='0')<br />
</source><br />
<br />
* ''selectdata=T '': One can choose to plot only subsets of the data.<br />
* ''field='0' '': The entire dataset is rather large, and different sources have very different amplitudes, so it is advisable to start by loading a subset of the data. You can later loop through the different fields (i.e. sources) and spectral windows using the GUI interface.<br />
<br />
In this case, many other values have been left to defaults, but it is possible to select them from within the '''{{plotms}}''' GUI. <br />
<br />
Task '''{{plotms}}''' should produce a GUI, with the default view being to show the visibility amplitude as a function of time. Figure 2 shows the result of running '''{{plotms}}''' with the field selection discussed above. You can save your own version of any plotms plot by using the menu bar in the '''{{plotms}}''' GUI to select the ''Export...'' option under the Export menu.<br />
[[Image:Colorbyfield_4.6.jpeg|200px|right|thumb|Figure 2: Overview of the observation: amplitude vs time, color-coded by field.]]<br />
<br />
Task '''{{plotms}}''' allows one to select and view the data in many ways. Across the top of the left panel are a set of tabs labeled Plot, Flag, Tools, Annotate, and Options. The option under the Flag tab is to Extend flags; thus, even though '''{{plotms}}''' was started with ''extendflag=F'', if you decide that it does make sense to extend the flags, you can do so here.<br />
<br />
In the default view, the Plot tab is visible, and there are a number of tabs running down the side of the left hand panel, including Data, Calibration, Axes, Page, Transform, Display, and Canvas. These allow you to make changes without having to re-launch '''{{plotms}}'''. Even if '''{{plotms}}''' was started with ''xaxis=' ' '', you can select a different X-axis by selecting the Axes tab, then using the dropdown menu to switch to (for example) ''xaxis='Frequency' ''.<br />
<br />
You should spend several minutes displaying the data in various formats. For instance, you could select the Data tab and specify ''field 0'' (source J1331+3030, a.k.a. 3C 286) to display data associated with the amplitude calibrator, then select the Axes tab and change the X-axis to be ''UVdist'' (baseline length in meters), and plot the data. The result should be similar to Figure 3A. The amplitude distribution is relatively constant as a function of u&#150;v distance or baseline length (i.e., <math>\sqrt{u^2+v^2}</math> ). You should recognize that a relatively constant visibility amplitude, as a function of baseline length, means that the source is very nearly a point source. (The Fourier transform of a constant is a delta function, a.k.a., a point source.) <br />
<br />
By contrast, if you opt for0 ''field 3'' (one of the 3C 391 fields) in the Data tab and plot these data, one sees a visibility function that falls rapidly with increasing baseline length (see Figure 3B). Such a visibility function indicates a highly resolved source. By noting the baseline length at which the visibility function falls to some fiducial value (e.g., 1/2 of its peak value), a rough estimate of the angular scale of the source can be obtained. (Angular scale [in radians] ~ 1/baseline [in wavelengths]. To plot baseline length in wavelengths rather than meters, one needs to select ''UVwave'' as the X-axis in the '''{{plotms}}''' GUI.)<br />
<br />
[[Image:plotms-3C286-UVDist_vs_Amp.png|200px|left|thumb|Figure 3A: plotms view vs. uvdist of 3C 286]]<br />
[[Image:plotms-3C391-UVDist_vs_Amp.png|200px|center|thumb|Figure 3B: plotms view vs. uvdist of 3C 391]]<br />
<br />
<br />
At this stage in the data reduction process, the general data editing and examination strategy is to focus is on the calibrators. The reduction strategy is to determine various corrections from the calibrators, then apply these correction factors to the science data. The 3C 286 data look relatively clean in that there are no wildly egregious data (e.g., amplitudes that are 100,000x larger than the rest of the data). One may notice that there are antenna-to-antenna variations (under the Display tab select Colorize by Antenna1). These antenna-to-antenna variations are acceptable, that's what calibration will help determine.<br />
<br />
[[Image:plotms_3c391-datastream.png|200px|right|thumb|Figure 4: datastream view of MS]]<br />
One final useful plot we will make is a datastream plot of the antenna2 in a baseline for the data versus ea01. This shows, assuming that ea01 is in the entire observation, when various antennas drop out.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='',correlation='RR,LL',<br />
timerange='',antenna='ea01',spw='0:31',<br />
xaxis='time',yaxis='antenna2',<br />
plotrange=[-1,-1,0,26],coloraxis='field')<br />
</source><br />
<br />
From this display (see Figure 4), you can see that antenna 3 (ea04) is missing the last scan on 3C84 (which will have some implications later on); antennas 1, 6, and 13 (ea02, ea08, and ea16) have some missing blocks of time in the middle of the run; and 23 (ea26) is missing a scan or two near the end.<br />
<br />
== Calibrating the Data ==<br />
<br />
It is now time to begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For more discussion of the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see [http://casa.nrao.edu/docs/cookbook/casa_cookbook005.html Synthesis Calibration] in the CASA Reference Manual.<br />
<br />
Recall that the observed visibility <math>V^{\prime}</math> between two antennas <math>(i,j)</math> is related to the true visibility <math>V</math> by: <br />
<br />
<math><br />
V^{\prime}_{i,j}(u,v,f) = b_{ij}(t)\,[B_i(f,t) B^{*}_j(f,t)]\,g_i(t) g_j(t)\,V_{i,j}(u,v,f)\,e^{i [\theta_i(t) - \theta_j(t)]} <br />
</math><br />
<br />
Here, for generality, we show the visibility as a function of frequency <math>f</math> and spatial wave numbers <math>u</math> and <math>v</math>. The other terms are: <br />
* <math>g_i</math> and <math>\theta_i</math> are the amplitude and phase portions of what is commonly termed the complex gain. They are shown separately here because they are usually determined separately. For completeness, these are shown as a function of time <math>t</math> to indicate that they can change with temperature, atmospheric conditions, etc.<br />
* <math>B_i</math> is the complex bandpass, the instrumental response as a function of frequency <math>f</math>. As shown here, the bandpass may also vary as a function of time.<br />
* <math>b(t)</math> is the often-neglected baseline term. It can be important to include for the highest dynamic range images or shortly after a configuration change at the JVLA, when antenna positions may not be known well. <br />
Strictly, the equation above is a simplification of a more general measurement equation formalism, but it is a useful simplification in many cases.<br />
<br />
=== ''A priori'' Antenna Position Corrections ===<br />
<br />
As mentioned in the observing log above, antennas ea10, ea12, and ea22 do not have good baseline positions. Antenna ea10 was not in the array, but, for the other two antennas, any improved baseline positions need to be incorporated. The importance of this step is that the visibility function is a function of <math>u</math> and <math>v</math>. If the baseline positions are incorrect, then <math>u</math> and <math>v</math> will be calculated incorrectly and there will be errors in the image. These corrections could also be determined later by a baseline-based calibration incorporating the <math>b_{ij}</math> term from the equation above, but since they are known ''a priori'' it makes sense to incorporate them now.<br />
<br />
The calculations are inserted via '''{{gencal}}''' which, since CASA 3.4, allows automated lookup of the corrections. To see how to calculate corrections manually, go to the [http://www.vla.nrao.edu/astro/archive/baselines/ EVLA/VLA Baseline Corrections] site.<br />
<br />
<source lang="python"><br />
# In CASA<br />
gencal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.antpos',caltype='antpos')<br />
</source><br />
<br />
In the logger you can see the corrections reported:<br />
<pre style="background-color: #fffacd;"><br />
##########################################<br />
##### Begin Task: gencal #####<br />
gencal(vis="3c391_ctm_mosaic_10s_spw0.ms",caltable="3c391_ctm_mosaic_10s_spw0.antpos",caltype="antpos",infile="",spw="",<br />
antenna="",pol="",parameter=[])<br />
Opening MS: 3c391_ctm_mosaic_10s_spw0.ms for calibration.<br />
Initializing nominal selection to the whole MS.<br />
Determine antenna position offests from the baseline correction database<br />
offsets for antenna ea01 : 0.00000 0.00300 0.00000<br />
offsets for antenna ea02 : -0.00080 0.00000 0.00000<br />
offsets for antenna ea03 : -0.00280 0.00000 0.00000<br />
offsets for antenna ea05 : 0.00000 0.00280 0.00000<br />
offsets for antenna ea11 : 0.00090 0.00000 0.00000<br />
offsets for antenna ea12 : -0.01000 0.00450 -0.00170<br />
offsets for antenna ea13 : 0.00000 -0.00080 0.00000<br />
offsets for antenna ea17 : -0.00120 0.00000 0.00000<br />
offsets for antenna ea18 : 0.00040 -0.00080 0.00040<br />
offsets for antenna ea22 : -0.02570 0.00270 -0.01900<br />
offsets for antenna ea23 : -0.00140 0.00000 0.00000<br />
offsets for antenna ea24 : -0.00150 0.00000 0.00000<br />
offsets for antenna ea26 : -0.00190 0.00000 0.00210<br />
offsets for antenna ea27 : 0.00000 0.00190 -0.00160<br />
Beginning specifycal-----------------------<br />
Creating KAntPos Jones table from specified parameters.<br />
Writing solutions to table: 3c391_ctm_mosaic_10s_spw0.antpos<br />
##### End Task: gencal #####<br />
##########################################<br />
</pre><br />
<br />
This corresponds to what is found in the database for 2010:<br />
<pre style="background-color: #E0FFFF;"><br />
; 2010 BASELINE CORRECTIONS IN METERS<br />
;ANT<br />
;MOVED OBSDATE Put_In_ MC(IAT) ANT PAD Bx By Bz<br />
;<br />
JAN27 FEB12 FEB21 01:57 11 E04 0.0000 0.0000 0.0000<br />
JAN27 FEB12 FEB21 01:57 26 W03 -0.0170 0.0204 0.0041<br />
MAR24 MAR25 MAR26 18:28 17 W07 -0.0061 -0.0069 -0.0055<br />
APR21 MAY02 MAY04 23:25 12 E08 -0.0072 0.0045 -0.0017<br />
MAR09 MAY02 MAY04 23:25 22 N04 -0.0220 0.0040 -0.0190<br />
JUN08 JUN20 JUN22 03:00 10 N03 0.0046 -0.0196 0.0090<br />
JUL17 JUL18 21:44 1 W09 0.0000 0.0030 0.0000<br />
JUL17 JUL18 21:44 2 E02 -0.0008 0.0000 0.0000<br />
JUL17 JUL18 21:44 3 E09 -0.0028 0.0000 0.0000<br />
JUL17 JUL18 21:44 5 W08 0.0000 0.0028 0.0000<br />
JUL01 JUL17 JUL18 21:44 6 N06 0.0022 0.0010 0.0059<br />
JUL17 JUL18 21:44 10 N03 0.0008 0.0030 -0.0014<br />
JUL17 JUL18 21:44 11 E04 0.0009 0.0000 0.0000<br />
JUL17 JUL18 21:44 12 E08 -0.0028 0.0000 0.0000<br />
JUL17 JUL18 21:44 13 N07 0.0000 -0.0008 0.0000<br />
JUL17 JUL18 21:44 17 W07 -0.0012 0.0000 0.0000 <br />
JUL17 JUL18 21:44 18 N09 0.0004 -0.0008 0.0004<br />
JUL17 JUL18 21:44 22 N04 -0.0037 -0.0013 0.0000<br />
JUL17 JUL18 21:44 23 E07 -0.0014 0.0000 0.0000<br />
JUL17 JUL18 21:44 24 W05 -0.0015 0.0000 0.0000<br />
JUL17 JUL18 21:44 26 W03 -0.0019 0.0000 0.0021<br />
JUL17 JUL18 21:44 27 E03 0.0000 0.0019 -0.0016<br />
AUG05 SEP15 SEP16 00:11 07 E05 0.0087 0.0137 0.0000<br />
SEP13 SEP15 SEP16 00:11 08 N10 0.0098 -0.0063 0.0059<br />
SEP13 SEP15 SEP16 00:11 10 N14 0.0124 -0.0050 0.0110<br />
SEP14 SEP15 SEP16 00:11 13 N12 0.0020 -0.0060 -0.0077<br />
SEP13 SEP15 SEP16 00:11 18 N16 -0.0035 0.0000 -0.0027<br />
SEP13 SEP15 SEP16 00:11 20 N18 0.0044 -0.0020 0.0000 <br />
OCT04 OCT06 OCT11 05:03 1 W18 -0.0087 0.0077 -0.0041<br />
OCT05 OCT06 OCT11 05:03 3 E18 -0.0131 -0.0143 0.0099<br />
OCT04 OCT06 OCT11 05:03 4 W10 0.0041 0.0032 0.0005<br />
OCT04 OCT06 OCT11 05:03 7 E14 0.0027 0.0154 0.0049<br />
OCT05 OCT06 OCT11 05:03 17 W14 -0.0141 0.0095 -0.0046<br />
OCT04 OCT06 OCT11 05:03 23 E16 0.0044 0.0045 -0.0076<br />
OCT05 OCT06 OCT11 05:03 24 W16 -0.0017 0.0004 0.0038<br />
OCT05 OCT06 OCT11 05:03 27 E10 0.0039 0.0188 -0.0060<br />
OCT06 OCT07 OCT12 20:12 21 E12 -0.0125 -0.0140 -0.0100<br />
OCT20 OCT22 OCT23 19:54 14 W12 0.0191 -0.0126 0.0032<br />
OCT22 OCT23 19:54 21 E12 0.0196 -0.0062 0.0051 <br />
DEC01 DEC04 DEC08 20:28 26 W02 -0.0130 0.0265 0.0187 <br />
</pre><br />
The observations were taken 24 April 2010, so antennas that were moved BEFORE that date, but whose corrections were not applied before that date, need to be corrected. Note in reading this table a blank in the first column means that the antenna was not moved at that time but has updated locations that need to be applied. One complication specific to this and other observations with the early JVLA (early 2010) is that previous location determinations are not necessarily in the JVLA database, but instead are in the old VLA database, as they were determined before the changeover in Jan 2010. They will have been applied at the time of observation.<br />
<br />
=== Flux Density Scale ===<br />
<br />
The next step is to provide a flux density value for the amplitude calibrator J1331+3030 (a.k.a. 3C 286). For the VLA, the ultimate flux density scale at most frequencies was set by 3C 295, which was then transferred to a small number of primary flux density calibrators, including 3C 286. For the JVLA, the flux density scale at most frequencies is determined from WMAP observations of the planet Mars, which, in turn, was then transferred to a small number of primary flux density calibrators. The procedure is to assume that the flux density of a primary calibrator source is known and, by comparison with the observed data for that calibrator, determine the <math>g_i</math> values.<br />
<br />
To start, let's find the available calibrator models with '''{{setjy}}''' and setting the parameter ''listmodels=True'':<br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis='3c391_ctm_mosaic_10s_spw0.ms', listmodels=T)<br />
</source><br />
<br />
This command will show all available calibrator models:<br />
<pre style="background-color: #fffacd;"><br />
No candidate modimages matching '*.im* *.mod*' found in .<br />
<br />
Candidate modimages (*) in /home/casa/packages/RHEL6/release/casa-release-4.4.0/data/nrao/VLA/CalModels:<br />
3C138_A.im 3C138_S.im 3C147_K.im 3C147_X.im 3C286_Q.im 3C48_C.im 3C48_U.im<br />
3C138_C.im 3C138_U.im 3C147_L.im 3C286_A.im 3C286_S.im 3C48_K.im 3C48_X.im<br />
3C138_K.im 3C138_X.im 3C147_Q.im 3C286_C.im 3C286_U.im 3C48_L.im README<br />
3C138_L.im 3C147_A.im 3C147_S.im 3C286_K.im 3C286_X.im 3C48_Q.im<br />
3C138_Q.im 3C147_C.im 3C147_U.im 3C286_L.im 3C48_A.im 3C48_S.im<br />
</pre><br />
Since any image could be a potential calibrator model, '''{{setjy}}''' will list all ''*.im'' and ''*.mod'' images in the working directory. In addition, it will list all models that are provided by NRAO with the CASA package, and they will be picked by their names. We will be using the C-band JVLA standard model for 3C286 which is aptly named '3C286_C.im':<br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis='3c391_ctm_mosaic_10s_spw0.ms',field='J1331+3030',standard='Perley-Butler 2013',<br />
model='3C286_C.im',usescratch=False,scalebychan=True,spw='')<br />
</source><br />
<br />
<!--[[Image:3C391_setjy.png|200px|thumb|right|setjy inputs]]--><br />
* ''field='J1331+3030' '': if the flux density calibrator is not specified then ''all'' sources will be assumed to have the same flux density.<br />
* ''standard='Perley-Butler 2013' '': The flux density scale at the JVLA is periodically revised, updated, or expanded. The specified value represents the most recent determination of the flux density scale by R. Perley and B. Butler in 2013, ApJS, 204, 19 (now the default); older scales can also be specified, and might be important if, for example, one was attempting to conduct a careful comparison with a previously published result. <br />
* ''model='3C286_C.im' '': From plotms above, it was estimated that 3C 286 is roughly a point source. Depending upon the frequency and configuration, the source may be slightly resolved. Fiducial model images have been determined from a painstaking set of observations, and, if one is available, it should be used to compensate for slight resolution effects. In this case, spectral window 0 (at 4.536 GHz) is in the C-band, so we use the C-band model image. <br />
* ''usescratch=False '': To save disk space, we will NOT force the writing of the model visibilities to the MODEL_DATA scratch column.<br />
* ''scalebychan=True '': In order to take out the intrinsic spectral index of our flux density calibrator 3C286 when we use it as our bandpass calibrator, we let it calculate a flux density value per channel.<br />
* ''spw=' ' '': The original data contained two spectral windows. Having split off spectral window 0, it is not necessary to specify spw. Had the spectral window 0 not been split off, we might wish to specify the spectral window because, in this observation, the spectral windows were sufficiently separated that two different model images for 3C 286 would be appropriate; 3C286_C.im at 4.6 GHz and 3C286_X.im at 7.5 GHz. This would require two separate runs of '''{{setjy}}''', one for each spectral window. If the spectral windows were much closer together, it might be possible to calibrate both using the same model.<br />
<br />
In this case, a model image of a primary flux density calibrator exists. However, for some kinds of polarization calibration or in extreme situations (e.g., there are problems with the scan on the flux density calibrator), it can be useful (or required) to set the flux density of the source explicitly.<br />
<br />
The most important output from '''{{setjy}}''' should look similar to the following:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Selected 31964 out of 845379 rows.<br />
J1331+3030 (fld ind 0) spw 0 [I=7.6677, Q=0, U=0, V=0] Jy @ 4.536e+09Hz, (Perley-Butler 2013)<br />
Scaling spw(s) [0]'s model image by channel to I = 7.66874, 7.59996, 7.53261 Jy @(4.535e+09, 4.599e+09, 4.663e+09)Hz ...<br />
</pre><br />
<br />
As set, the flux density scale is being calculated only for spectral window 0 (''spw='0' ''), as it is the only one in the dataset. The flux density in each Stokes (IQUV) for the reference channel 0 is reported, followed by the I flux density in each channel of the spectral window that will be used to scale the data. This value is determined from an analytical formula for the spectrum of the source as a function of frequency; this value must be determined so that the flux density in the image can be scaled to it, as it is unlikely that the observation was taken at exactly the same frequency as the model image. Also, setjy will clear any previous calibration model that fits the selection. In this case, no such previous model data was found.<br />
<br />
Note that setjy also returns a python dictionary (CASA record) containing the reference flux density used. In our case, you will find the return value in the CASA command line window:<br />
<pre><br />
{'0': {'0': {'fluxd': array([ 7.66765308, 0. , 0. , 0. ])},<br />
'fieldName': 'J1331+3030'},<br />
'format': "{field Id: {spw Id: {fluxd: [I,Q,U,V] in Jy}, 'fieldName':field name }}"}<br />
</pre><br />
If desired, this can be captured by calling the task by setting it to a variable, e.g. '''myset = setjy(...'''.<br />
<br />
=== Initial Phase Calibration ===<br />
<br />
Before solving for the bandpass, we will do an initial phase calibration. Depending upon frequency and configuration, there could be gain variations between the different scans of the bandpass calibrator, particularly if the scans happen at much different elevations. One can solve for an initial set of antenna-based gains, which will later be discarded, in order to moderate the effects of variations from scan to scan on the bandpass calibrator. While amplitude variations will have little effect on the bandpass solutions, it is important to solve for any phase variations with time to prevent decorrelation when vector averaging the data in computing the bandpass solutions.<br />
<br />
We use the CASA task '''{{gaincal}}''' to solve for phase versus time for the central channels on our three calibrators:<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.G0all', <br />
field='0,1,9', refant='ea21', spw='0:27~36',<br />
gaintype='G',calmode='p', solint='int', <br />
minsnr=5, gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'])<br />
</source><br />
<br />
* ''caltable='3c391_ctm_mosaic_10s_spw0.G0all' '': The gain solutions will be stored in an external table.<br />
* ''field='0,1,9' '': Specify the calibrators. We will use this opportunity to inspect solutions for all calibrators to weed out bad data.<br />
* ''refant='ea21' '': Earlier, by looking at the output from '''{{plotants}}''', a reference antenna near the center of the array was noted. Here is the first time that that choice will be used. Strictly, all of the gain corrections derived will be relative to this reference antenna.<br />
* ''spw='0:27~36' '': Choose a subset of the channels from which to determine the gain corrections. These should be near the center of the band, and there should be enough channels chosen so that a reasonable signal-to-noise ratio can be obtained, the central 10% of the channels is a good guideline. Particularly at lower frequencies where RFI can manifest itself, one should choose RFI-free frequency channels; the [http://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/rfi VLA Observing Guide RFI page] lists the known RFI frequencies for each band. Also note that, even though these data have only a single spectral window, the syntax requires specifying the spectral window in order to specify the spectral channels.<br />
* ''gaintype='G' '': Compute the complex gain solutions, one per antenna per spw per polarization per solution interval<br />
* ''calmode='p' '': Solve for only the phase portion of the gain.<br />
* ''solint='int' '': To track the phases, a short solution interval is chosen. (''int'' refers to a single integration time or 10 seconds for this case)<br />
* ''minsnr=5 '': Restrict the solutions to be at relatively high signal-to-noise ratios, although this parameter may need to be varied depending upon the source and frequency.<br />
* ''gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'] '': Having produced antenna position corrections, they should now be applied.<br />
<br />
[[Image:plotcal-3C286-G0all-phase-R-ea05.png|200px|thumb|right|Figure 5: Initial gain phases, R polarization, stepped through to show ea05]]<br />
To really see what is going on, we use '''{{plotcal}}''' to inspect the solutions from '''{{gaincal}}''' for a single antenna at a time, iterating through each antenna in sequence, starting with the R polarization:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',xaxis='time',yaxis='phase',<br />
poln='R',iteration='antenna',plotrange=[-1,-1,-180,180])<br />
</source><br />
Use the Next button on the GUI to advance the displayed antenna. In Figure 5 we have stepped through to antenna ''ea05''.<br />
<br />
'''NOTE:''' Sometimes plotcal will lock a table and/or keep it in the table cache beyond the end of plotting. This can hang up further use of that table, or cause errors if you delete the table outside of CASA and want to re-create it. To deal with that issue, simply close the plotcal GUI using the Quit button when you are done looking at the plot.<br />
<br />
Antennas that have been flagged will show a blank plot, as there are no solutions for these antennas. For most antennas, we see a fairly smooth variation with time, so we expect to be able to calibrate the data nicely. However, when you get to ea05, note that there are phase jumps where it appears to be bouncing between two states. We can plot this antenna separately to recreate Figure 5:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',xaxis='time',yaxis='phase',antenna='ea05',<br />
poln='R',iteration='antenna',plotrange=[-1,-1,-180,180],figfile='plotcal_3c391-G0all-phase-R-ea05.png')<br />
</source><br />
We see the same thing for the L polarization solutions:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',<br />
xaxis='time', yaxis='phase', poln='L', <br />
iteration='antenna', plotrange=[-1,-1,-180,180])<br />
</source><br />
Antennas other than ''ea05'' look OK.<br />
<br />
We will not be able to transfer calibration for antenna ea05 so we will flag it from the data:<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms',<br />
flagbackup=T, mode='manual', antenna='ea05')<br />
</source><br />
<br />
For the following bandpass solution we need only solve for our bandpass calibrator, and we will do so now after flagging. The following call to '''{{gaincal}}''' is similar to the one above, but selects only the bandpass calibrator (using the ''field'' parameter). This is the calibration table we will use when solving for the bandpass solution, below.<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.G0', <br />
field='J1331+3030', refant='ea21', spw='0:27~36', calmode='p', solint='int', <br />
minsnr=5, gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'])<br />
</source><br />
You can inspect this with '''{{plotcal}}''' as we did above. For example, plot R for the first block of 3C286 data only:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0',<br />
xaxis='time',yaxis='phase',poln='R',field='J1331+3030',iteration='antenna',<br />
plotrange=[-1,-1,-180,180],timerange='08:02:00~08:17:00')<br />
</source><br />
You can do the same for L if you like.<br />
<br />
=== Delay Calibration ===<br />
<br />
The first stage of bandpass calibration involves solving for the antenna-based delays which put a phase ramp versus frequency channel in each spectral window. The K gain type in '''{{gaincal}}''' solves for the relative delays of each antenna relative to the reference antenna (parameter ''refant''), so be sure you pick one that is there for this entire scan and good. This is not a full global delay, but gives one value per spw per polarization.<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.K0', <br />
field='J1331+3030',refant='ea21',spw='0:5~58',gaintype='K', <br />
solint='inf',combine='scan',minsnr=5,<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.G0'])<br />
</source><br />
<br />
* ''field='J1331+3030' '': For the bandpass calibrator<br />
* ''refant='ea21' '': Delays will be relative to this antenna, make sure it is there!<br />
* ''spw='0:5~58' '': Widest possible frequency range in the spw, avoiding edge channels because they have lower sensitivity<br />
* ''gaintype='K' '': Compute K (i.e., delay) solutions, one per antenna per spw per polarization per solution interval<br />
* ''solint='inf ',combine='scan' '': Only need one solution averaged over all times and scans. ''solint='inf ' '' sets the solution interval to 'infinite' but respects scan boundaries; ''combine='scan' '' combines data across scan boundaries<br />
* ''minsnr=5 '': Restrict the solutions to be at relatively high signal-to-noise ratios, although this parameter may need to be varied depending upon the source and frequency.<br />
* ''gaintable=['3c391_ctm_mosaic_10s_spw0.antpos','3c391_ctm_mosaic_10s_spw0.G0'] '': Use the antpos and G0 tables that were created earlier<br />
<br />
[[Image:plotcal_3c391-K0-delay.png|200px|thumb|right|Figure 6: delay solutions]]<br />
We can plot these solutions (in nanoseconds) as a function of antenna:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.K0',xaxis='antenna',yaxis='delay',<br />
figfile='plotcal_3c391-K0-delay.png')<br />
</source><br />
These are within about 4 nanoseconds, as expected for the early JVLA science observations.<br />
<br />
=== Bandpass Calibration ===<br />
<br />
This step solves for the complex bandpass, <math>B_i</math>. <br />
[[Image:plotms-3C286-RRbandpass2.png|200px|thumb|right|Figure 7: bandpass illustration]]<br />
For the VLA, in its old continuum modes, this step could be skipped. With the JVLA, all data are spectral line, even if the science that one is conducting is continuum. Solving for the bandpass won't hurt for continuum data, and, for moderate or high dynamic range image, it is essential. To motivate the need for solving for the bandpass, consider Figure 7. It shows the right circularly polarized data (RR polarization) for the source J1331+3030, which will serve as the bandpass calibrator. The data are color coded by spectral window, and they are averaged over all baselines, as earlier plots from '''{{plotms}}''' indicated that the visibility data are nearly constant with baseline length. Ideally, the visibility data would be constant as a function of frequency as well. The variations with frequency are a reflection of the (slightly) different antenna bandpasses. (''Exercise for the reader, reproduce Figure 7 using '''{{plotms}}'''.'') <font color="CDCDCD"> (x-axis is Channel, y-axis is Amp (data column), field=0, antenna=ea01, correlator=RR, amp range is 0-0.25, time averaging is 1e10 seconds)</font><br />
<br />
Now form the bandpass, using the phase solutions just derived.<br />
<source lang="python"><br />
# In CASA<br />
bandpass(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.B0',<br />
field='J1331+3030',spw='',refant='ea21',combine='scan', <br />
solint='inf',bandtype='B',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.G0',<br />
'3c391_ctm_mosaic_10s_spw0.K0'])<br />
</source><br />
<br />
* ''caltable='3c391_ctm_mosaic_10s_spw0.B0' '': Specify where to store the bandpass corrections.<br />
* ''solint='inf ', combine='scan' '': This observation contains multiple scans on the bandpass calibrator, J1331+3030. Because these are continuum observations, it is probably acceptable to combine all the scans and compute one bandpass correction per antenna, which is achieved by the combination of '''solint='inf ' ''and ''combine='scan' ''. Had ''combine=' ' ''then there would have been a bandpass correction derived per scan, which might be necessary for the highest dynamic range spectral line observations.<br />
* ''bandtype='B' '': The bandpass solution will be derived on a channel-by-channel basis. There is an alternate, somewhat experimental option of parameter ''bandtype='BPOLY' ''that will attempt to fit an n<sup>th</sup> order polynomial to the bandpass.<br />
* ''gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', '3c391_ctm_mosaic_10s_spw0.G0', '3c391_ctm_mosaic_10s_spw0.K0'] '': Pre-apply antenna positions, phase solutions, and delays.<br />
<br />
Once again, one can use '''{{plotcal}}''' to display the bandpass solutions. Note that in the '''{{plotcal}}''' inputs below, the amplitudes are being displayed as a function of frequency channel and, for compactness, parameter ''subplot=221'' is used to display multiple plots per page. The first two commands show the amplitude solutions (one per each polarization) and the last two show the phase solutions (one per each polarization). Parameter ''iteration='antenna' '' is used to step through separate plots for each antenna.<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='R', <br />
xaxis='chan',yaxis='amp',field= 'J1331+3030',subplot=221, <br />
iteration='antenna',figfile='plotcal_3c391-3C286-B0-R-amp.png')<br />
#<br />
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='L', <br />
xaxis='chan',yaxis='amp',field= 'J1331+3030',subplot=221, <br />
iteration='antenna',figfile='plotcal_3c391-3C286-B0-L-amp.png')<br />
#<br />
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='R', <br />
xaxis='chan',yaxis='phase',field= 'J1331+3030',subplot=221, <br />
iteration='antenna',plotrange=[-1,-1,-180,180],<br />
figfile='plotcal_3c391-3C286-B0-R-phase.png')<br />
#<br />
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='L', <br />
xaxis='chan',yaxis='phase',field= 'J1331+3030',subplot=221, <br />
iteration='antenna',plotrange=[-1,-1,-180,180],<br />
figfile='plotcal_3c391-3C286-B0-L-phase.png')<br />
</source><br />
As expected, the bandpass phases are relatively flat (see Figures 8A&#150;8D), with the slopes removed by the delay calibration. Residual excursions are on the order of a few degrees.<br />
<br />
{|<br />
| [[Image:plotcal_3c391-3C286-B0-R-amp.png|200px|thumb|left|Figure 8A: bandpass amp for 3C 286 R]]<br />
| [[Image:plotcal_3c391-3C286-B0-L-amp.png|200px|thumb|center|Figure 8B: bandpass amp for 3C 286 L]]<br />
| [[Image:plotcal_3c391-3C286-B0-R-phase.png|200px|thumb|center|Figure 8C: bandpass phase for 3C 286 R]]<br />
| [[Image:plotcal_3c391-3C286-B0-L-phase.png|200px|thumb|right|Figure 8D: bandpass phase for 3C 286 L]]<br />
|}<br />
<br />
=== Gain Calibration ===<br />
<br />
The next step is to derive corrections for the complex antenna gains, <math>g_i</math> and <math>\theta_i</math>. As discussed above, the absolute magnitude of the gain amplitudes&#151;<math>g_i</math>&#151;are determined by reference to a standard flux density calibrator. In order to determine the appropriate complex gains for the target source, and to minimize differences through the atmosphere (neutral and/or ionized) between the lines of sight to the phase calibrator and the target source, you want to observe a so-called phase calibrator that is much closer to the target. If we establish the relative gain amplitudes and phases for different antennas using the phase calibrator, we can later determine the absolute flux density scale by comparing the gain amplitudes, <math>g_i</math>, derived for 3C 286 with those derived for the phase calibrator. This will eventually be done using the task '''{{fluxscale}}'''. Since there is no such thing as absolute phase, we determine a zero phase by selecting a reference antenna for which the gain phase is defined to be zero.<br />
<br />
In principle, one could determine the complex antenna gains for all sources with a single invocation of '''{{gaincal}}'''; for clarity here, two separate invocations will be used.<br />
<br />
In the first step, we derive the appropriate complex gains <math>g_i</math> and <math>\theta_i</math> for the flux density calibrator 3C 286.<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.G1',<br />
field='J1331+3030',spw='0:5~58',<br />
solint='inf',refant='ea21',gaintype='G',calmode='ap',solnorm=F,<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0'])<br />
</source><br />
* ''caltable='3c391_ctm_mosaic_10s_spw0.G1' '': Produce a new calibration table containing these gain solutions. In order to make the bookkeeping easier, a '1' is appended to the file name to distinguish it from the earlier set of gain solutions, which are effectively being thrown away.<br />
* ''spw='0:5~58' '': From the inspection of the bandpass, one can determine the range of edge channels that are affected by the bandpass filter rolloff. Because the amplitude is dropping rapidly in these channels, one does not want to include them in the solution.<br />
* ''gaintype='G', calmode='ap', solnorm=F '': Solve for the complex antenna gains for 3C 286. The objective is to relate the measured data values to the (assumed known) flux density of 3C 286, thus the solution is both amplitude and phase ('ap') and the solutions should not be normalized to unity amplitude.<br />
* ''solint='inf ' '': Produce a solution for each scan. Phase coherence for these observations is good.<br />
* ''gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', '3c391_ctm_mosaic_10s_spw0.K0', '3c391_ctm_mosaic_10s_spw0.B0'] '': Use the antenna position corrections, delays, and bandpass solutions determined earlier before solving for the gain amplitudes.<br />
<br />
In the second step, the appropriate complex gains for a direction on the sky close to the target source will be determined from the phase calibrator J1822-0938. We also determine the complex gains for the polarization calibrator source J0319+4130. These will be solved separately, but in practice could be solved together as there are no gaintables that are time dependent at this point (and thus would risk having cross-source interpolation issues), nor are we doing different solution intervals per source.<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.G1',<br />
field='J1822-0938',<br />
spw='0:5~58',solint='inf',refant='ea21',gaintype='G',calmode='ap',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0'],<br />
append=True)<br />
#<br />
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.G1',<br />
field='J0319+4130',<br />
spw='0:5~58',solint='inf',refant='ea21',gaintype='G',calmode='ap',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0'],<br />
append=True)<br />
</source><br />
* ''caltable='3c391_ctm_mosaic_10s_spw0.G1', append=True '': In all previous invocations of '''{{gaincal}}''', append has been set to False. Here, the gain solutions from the phase calibrators are going to be appended to the existing set from 3C 286. In following steps, all of these gain solutions will then be used together to derive a set of complex gains that are applied to the science data for the target source.<br />
<br />
If one checks the gain phase solutions using '''{{plotcal}}''', one should see smooth solutions for each antenna as a function of time (see Figures 9A&#150;9D). <br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='phase',<br />
poln='R',plotrange=[-1,-1,-180,180],figfile='plotcal_3c391-G1-phase-R.png')<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='phase',<br />
poln='L',plotrange=[-1,-1,-180,180],figfile='plotcal_3c391-G1-phase-L.png')<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='amp',<br />
poln='R',figfile='plotcal_3c391-G1-amp-R.png')<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='amp',<br />
poln='L',figfile='plotcal_3c391-G1-amp-L.png')<br />
</source><br />
<br />
{| <br />
| [[Image:plotcal_3c391-G1-phase-R.png|200px|thumb|left|Figure 9A: gain phase solutions, R polarization]]<br />
| [[Image:plotcal_3c391-G1-phase-L.png|200px|thumb|center|Figure 9B: gain phase solutions, L polarization]]<br />
| [[Image:plotcal_3c391-G1-amp-R.png|200px|thumb|left|Figure 9C: gain amp solutions, R polarization]]<br />
| [[Image:plotcal_3c391-G1-amp-L.png|200px|thumb|center|Figure 9D: gain amp solutions, L polarization]]<br />
|}<br />
<br />
This is also a good time to check that our chosen reference antenna (''ea21'') has good phase stability (i.e., the phase difference between the right and left polarizations is stable with time). This is a prerequisite for accurate polarization calibration. To do this, we plot the complex polarization ratio by selecting ''poln=' / ' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1', xaxis='time', yaxis='phase',<br />
poln='/', plotrange=[-1,-1,-180,180], figfile='plotcal_3c391-G1-phase-rat.png')<br />
</source><br />
<br />
As can be seen in Figure 10, there is a bit of drift (a few degrees here and there), but no phase jumps. This means that ea21 is, indeed, a good choice for reference antenna.<br />
<br />
{| <br />
| [[Image:plotcal_3c391-G1-phase-rat.png|200px|thumb|center|Figure 10: complex polarization ratio]]<br />
|}<br />
<br />
=== Polarization Calibration ===<br />
<br />
''[If time is running short, skip this step and proceed to''' Applying the Calibration''' below.]''<br />
<!-- Really? ("Stay on target." Gold Five) Why not, "Danger! Will Robinson. Danger! Danger!" --><br />
<br />
Having set the complex gains, we now need to do the polarization calibration. This should be done prior to running '''{{fluxscale}}''', since it has to run using the un-rescaled gains in the MODEL_DATA column of the measurement set. Polarization calibration is done in two steps. First, we solve for the instrumental polarization (the frequency-dependent leakage terms ('D-terms')), using either an unpolarized source or a source which has sufficiently good parallactic angle coverage. Second, we solve for the polarization position angle using a source with a known polarization position angle (3C 286 is recommended here).<br />
<br />
The initial run of '''{{setjy}}''' above only set the total intensity of our flux calibrator source, 3C 286. This source is known to have a fairly stable fractional polarization (measured to be 11.2% in C-band around the time of the observations), and a polarization position angle of 33 degrees (at most frequencies). For information on polarization calibrators suitable for VLA observations, see the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol VLA Observing Guide on Polarimetry]. The most recently determined values, in a table at the bottom of the Polarimetry guide page, give 11.3% at 4.5GHz, but we use the old value of 11.2% for consistency with the rest of the guide.<br />
<br />
In order to calibrate the position angle, we need to set the appropriate values for Stokes Q and U. Examining our casapy.log file to find the output of '''{{setjy}}''', we find that the total intensity was set to 7.6677 Jy in channel 0 of spw 0 at 4536 MHz according to the '''{{listobs}}''' output. We therefore use the Python interpreter to find the polarized flux (P) and the values of Stokes Q and U at this reference channel:<br />
<source lang="python"><br />
# In CASA<br />
i0=7.6677 # Stokes I value for spw 0 ch 0<br />
p0=0.112*i0 # Fractional polarization=11.2%<br />
q0=p0*cos(66*pi/180) # Stokes Q for spw 0 for pang = 33deg (Q+iU phase = 66deg)<br />
u0=p0*sin(66*pi/180) # Stokes U for spw 0 for pang = 33deg (Q+iU phase = 66deg)<br />
</source><br />
We can introduce a spectral index to the flux densities. This is done by noting that our printout from '''{{setjy}}''' gave 7.53261 Jy in the last channel 127 at 4663 MHz. We again use the Python interpreter to compute a spectral index:<br />
<source lang="python"><br />
# In CASA<br />
alpha = log(7.53261/7.6677)/log(4663.0/4536.0)<br />
</source><br />
which gives ''alpha = -0.64371''. (Type alpha in CASA to see the output.)<br />
<!-- *as of 4.3.1 the following will not work correctly for general cases, so we do not include this here<br />
<source lang="python"><br />
# In CASA<br />
i0=7.81694 # Stokes I value for spw 0 ch 0<br />
</source><br />
Furthermore, in CASA 3.4 onward, we can introduce a spectral index to the flux densities. We do this by noting that our printout from {{setjy}} gave 7.68506 Jy in the last channel 127 (at 4663 MHz). Therefore we use python again to compute a spectral index<br />
<source lang="python"><br />
# In CASA<br />
alpha = log(7.68506/7.81694)/log(4663.0/4536.0)<br />
</source><br />
which gives <tt>alpha = -0.61618</tt>. Finally, in CASA 4.3, we have the ability to set the polarization angle and fraction directly (earlier versions you must set Q and U).<br />
<source lang="python"><br />
# In CASA<br />
fpol = 0.112 # Fractional polarization=11.2%<br />
pang = 33.0*pi/180.0 # 3C286 33deg to radians<br />
</source><br />
--><br />
<br />
We now set the values of Stokes I, Q and U for 3C 286, using '''{{setjy}}''' using the above values.<br />
<br />
Generate the polarization model:<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis='3c391_ctm_mosaic_10s_spw0.ms',field='J1331+3030',standard='manual',<br />
spw='0',fluxdensity=[i0,q0,u0,0],spix=[alpha,0],reffreq='4536.0MHz',<br />
scalebychan=True,usescratch=False)<br />
</source><br />
<!-- *as of 4.3.1 the following will not work correctly for general cases, so we do not include this here<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis='3c391_ctm_mosaic_10s_spw0.ms',field='J1331+3030',standard='manual',<br />
spw='0',fluxdensity=[i0,0,0,0],spix=[alpha,0],reffreq='4536.0MHz',<br />
polindex=[fpol],polangle=[pang],rotmeas=0.0,<br />
scalebychan=True,usescratch=False)<br />
</source><br />
--><br />
* ''standard='manual' '': The user will supply the flux densities rather than giving a model.<br />
* ''fluxdensity=[i0,q0,u0,0] '': Use the IQUV values for channel 0 calculated above.<br />
* ''spix=alpha '': Set the spectral index using the formula above (this will also apply to QU).<br />
* ''reffreq='4536.0MHz' '': This is the frequency of channel 0 from listobs.<br />
<!-- *as of 4.3.1 the following will not work correctly for general cases, so we do not include this here<br />
* <tt>polindex=[fpol] : The fractional polarization of 3C286 at this frequency set earlier.<br />
* <tt>polindex=[pang] : The polarization angle of 3C286 at this frequency set earlier (in radians).<br />
* <tt>rotmeas=0.0 : You can set a rotation measure value for the calibrator, for 3C286 it is zero.<br />
--><br />
* ''scalebychan=True '': This will let it compute a value per channel.<br />
* ''usescratch=False '': DO NOT create/use the MODEL_DATA column explicitly.<br />
<br />
The Stokes V flux value has been set to zero, corresponding to no circular polarization. <br />
<br />
'''Note:''' With CASA 4.2 the '''setjy''' task parameters have changed, in particular the use of ''standard='manual' ''to specify flux densities. CASA 4.3 added the polindex, polangle and rotmeas capability, although as of 4.3.1 it will not work correctly for all values of polangle and thus is still experimental. Beware when following this guide with earlier versions of CASA.<br />
<br />
We can see the results in the model column in '''{{plotms}}''' (Figure 11A):<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='RR',<br />
timerange='08:02:00~08:17:00',antenna='ea01&ea02',<br />
xaxis='channel',yaxis='amp',ydatacolumn='model',<br />
plotfile='plotms_3c391-model-amp-RR.png',overwrite=True)<br />
</source><br />
Showing our model source spectrum. We can see this translates to the spectrum in QU (Figure 11B):<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='RL',<br />
timerange='08:02:00~08:17:00',antenna='ea01&ea02',<br />
xaxis='channel',yaxis='amp',ydatacolumn='model',<br />
plotfile='plotms_3c391-model-amp-RL.png',overwrite=True)<br />
</source><br />
<!-- *as of r24160, this crashes on MacOS X* --><br />
<br />
Finally, our R-L phase difference is constant at 66 degrees (twice the polarization<br />
angle) as desired (Figure 11C):<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='RL',<br />
timerange='08:02:00~08:17:00',antenna='ea01&ea02',<br />
xaxis='channel',yaxis='phase',ydatacolumn='model',<br />
plotrange=[-1,-1,-180,180],plotfile='plotms_3c391-model-phase-RL.png',<br />
overwrite=True)<br />
</source><br />
<!-- *as of r24160, this crashes on MacOS X* --><br />
<br />
{| <br />
| [[Image:plotms_3c391-model-amp-RR.png|200px|thumb|left|Figure 11A: model RR amplitudes]]<br />
| [[Image:plotms_3c391-model-amp-RL.png|200px|thumb|center|Figure 11B: model RL amplitudes]]<br />
| [[Image:plotms_3c391-model-phase-RL.png|200px|thumb|right|Figure 11C: model RL phases]]<br />
|}<br />
<br />
==== Solving for the Cross-Hand delays ====<br />
<br />
Just as we did for the parallel-hand (RR,LL) delays before bandpass calibration, we solve for the cross-hand (RL,LR) delays due to the residual delay difference between the R and L on the reference antenna (''ea21'') used for the original delay calibration. In our case, we simply use 3C286, which has a strong polarized signal in the RL,LR correlations.<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.Kcross',<br />
field='J1331+3030', spw='0:5~58',<br />
gaintype='KCROSS', solint='inf', combine='scan', refant='ea21',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.G1'],<br />
gainfield=['','','','J1331+3030'],<br />
interp=['linear','nearest','nearest','linear'], parang=T)<br />
</source><br />
<br />
[[Image:plotcal_3c391-Kcross-delay.png|200px|thumb|right|Figure 12: cross-hand delay solutions]]<br />
We can plot these (see Figure 12):<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.Kcross',xaxis='antenna',yaxis='delay',<br />
figfile='plotcal_3c391-Kcross-delay.png')<br />
</source><br />
As expected there is a single value for R versus L (with L delay set to zero) across all antennas. The solution is reported in the logger, and is 7.149 nsec. <!-- was 7.157 ns for parang=F --><br />
<br />
Note that if we did not solve for this delay, it would be absorbed into the phases per channel of the following Df and Xf solutions. This would not cause us problems, as we are not solving for the Q+iU polarization of our D-term calibrator (we are using unpolarized 3C84 for that) but if we were (e.g., using our gain calibrator J1822-0938 with parameter ''poltype='Df+QU' '') then this step would be essential.<br />
<br />
==== Solving for the Leakage Terms ====<br />
<br />
The task '''{{polcal}}''' is used for polarization calibration. In this data set, we observed the unpolarized calibrator J0319+4130 (a.k.a. 3C 84) in order to solve for the instrumental polarization. Task '''{{polcal}}''' uses the Stokes I, Q, and U values in the MODEL_DATA column (Q and U being zero for our unpolarized calibrator) to derive the leakage solutions. The function call is:<br />
<br />
<source lang="python"><br />
# In CASA<br />
polcal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.D1',<br />
field='J0319+4130',spw='0:5~58',<br />
refant='ea21',poltype='Df',solint='inf',combine='scan',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.G1',<br />
'3c391_ctm_mosaic_10s_spw0.Kcross'],<br />
gainfield=['','','','J0319+4130',''],<br />
interp=['linear','nearest','nearest','linear','nearest'])<br />
</source><br />
<br />
* ''caltable='3c391_ctm_mosaic_10s_spw0.D1' '': '''{{polcal}}''' will create a new calibration table containing the leakage solutions, which we specify with the ''caltable'' parameter.<br />
* ''field='J0319+4130' '': The unpolarized source J0319+4130 (a.k.a. 3C 84) is used to solve for the leakage terms.<br />
* ''spw='0:5~58' '': In this example, the edge channels are not used in finding the solution. Because the bandpass is one of the calibration tables being applied (in ''gaintable''), this restriction is not necessary. However, if one restricts the spectral window here, it '''must''' also be restricted in the remainder of the calibration steps, particularly '''{{applycal}}''', otherwise the final data set will contain frequency channels for which the leakage terms have not been calibrated.<br />
* ''poltype='Df' '': Solve for the leakages (''D'') on a per-channel basis (''f''), assuming zero source polarization.<br />
* ''solint='inf' '': One solution over the entire run (default)<br />
* ''combine='scan' '': Combine all scans together (default)<br />
* ''gaintable=['3c391_ctm_mosaic_10s_spw0.antpos','3c391_ctm_mosaic_10s_spw0.K0','3c391_ctm_mosaic_10s_spw0.B0','3c391_ctm_mosaic_10s_spw0.G1','3c391_ctm_mosaic_10s_spw0.Kcross'] '': All of the previous corrections&#151;antenna positions, K-delay, bandpass, Kcross-delay, and complex gain&#151;are to be applied <br />
* ''gainfield=['&nbsp;','&nbsp;','&nbsp;','J0319+4130','&nbsp;'] '': The gain caltable that is being applied on the fly, '''3c391_ctm_mosaic_10s_spw0.G1''', contains the solutions for multiple sources. Only the solutions from J0319+4130 should be applied to itself in the process of finding the polarization leakage terms.<br />
* ''interp=['linear','nearest','nearest','linear','nearest'] '': The interpolation to be used for each gaintable.<br />
<br />
After polcal has finished running, you are strongly advised to examine the solutions with '''{{plotcal}}''', to ensure that everything looks good.<br />
[[Image:plotcal_3c391-D1-ea01.png|thumb|Figure 13: Df amp vs. channel for ea01]]<br />
[[Image:plotcal_3c391-D1-phase-ea01.png|thumb|Figure 14: Df phase vs. channel for ea01]]<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D1',xaxis='chan',yaxis='amp', <br />
spw='',field='',iteration='antenna')<br />
#<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D1',xaxis='chan',yaxis='phase', <br />
spw='',field='',iteration='antenna',plotrange=[-1,-1,-180,180])<br />
</source><br />
This will produce plots similar to Figures 13 & `4 (you might get slightly different values).<br />
As ever, you can cycle through the antennas by clicking the Next button. You should see leakages of between 5&#150;15% in most cases. We can also display these in a single plot versus antenna index:<br />
[[Image:plotcal_3c391-D1.png|thumb|Figure 15: Df solutions for J0319+4130 versus antenna index]]<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D1',xaxis='antenna',yaxis='amp', <br />
figfile='plotcal_3c391-D1.png')<br />
</source><br />
Note that there are no solutions for antenna ea04&#151 (see Figure 15);a bit of sleuthing will turn up that ea04 was missing for the scans on 3C84.<br />
<br />
If we want to rescue ea04, which otherwise seems OK, then we turn to our gain calibrator. If we plot data for field 1 versus ParAngle in '''{{plotms}}''' then we see that it has sufficient range (>60 deg) in parallactic angle so should be useable as a calibrator with unknown polarization. We can make a new set of solutions:<br />
<source lang="python"><br />
# In CASA<br />
polcal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.D2',<br />
field='J1822-0938',spw='0:5~58',<br />
refant='ea21',poltype='Df+QU',solint='inf',combine='scan',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.G1',<br />
'3c391_ctm_mosaic_10s_spw0.Kcross'],<br />
gainfield=['','','','J1822-0938',''],<br />
interp=['linear','nearest','nearest','linear','nearest'])<br />
</source><br />
<br />
* ''field='J1822-0938' '': Our gain calibrator observed throughout the scheduling block.<br />
* ''poltype='Df+QU' '': Solve for the leakages using a calibrator with unknown polarization but with good parallactic angle coverage, and simultaneously for the source polarization (averaged over frequency).<br />
* ''gainfield=['&nbsp;','&nbsp;','&nbsp;','J1822-0938','&nbsp;'] '': For '''3c391_ctm_mosaic_10s_spw0.G1''' use only the solutions from J1822-0938 itself.<br />
<br />
[[Image:plotcal_3c391-D2.png|thumb|Figure 16: Df solutions for J1822-0938 versus antenna index]]<br />
We now plot this as we did before:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D2',xaxis='antenna',yaxis='amp', <br />
figfile='plotcal_3c391-D2.png')<br />
</source><br />
Comparison of this plot (see Figure 16) with that for the D1 caltable shows that we get nearly identical results, but now ea04 (index 3) is present! That should give us some confidence in our leakage calibration as well.<br />
<br />
==== Solving for the R-L polarization angle ====<br />
<br />
Having calibrated the instrumental polarization, the total polarization is now correct, but the <br />
R-L phase still needs to be calibrated in order to obtain an accurate polarization position angle. We use the same task, '''{{polcal}}''', but this time set parameter ''poltype='Xf' '', which specifies a frequency-dependent (''f'') position angle (''X'') calibration, using the source J1331+3030 (a.k.a. 3C 286), whose position angle is known, having set this earlier using '''{{setjy}}'''. Note that we must correct for the leakages before determining the R-L phase, which we do by adding the calibration table made in the previous step ('''3c391_ctm_mosaic_10s_spw0.D2''') to the gain tables that are applied on-the-fly. Note that we are using the second D table we made as it included ea04:<br />
<br />
<source lang="python"><br />
# In CASA<br />
polcal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.X1',<br />
field='J1331+3030',combine='scan',<br />
poltype='Xf',solint='inf',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.G1',<br />
'3c391_ctm_mosaic_10s_spw0.Kcross',<br />
'3c391_ctm_mosaic_10s_spw0.D2'],<br />
gainfield=['','','','J1331+3030','',''],<br />
interp=['linear','nearest','nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
[[Image:plotcal_3c391-X1.png|thumb|Figure 17: Xf solutions versus channel.]]<br />
Again, it is strongly suggested that you check the calibration worked properly, by plotting up the newly-generated calibration table using '''{{plotcal}}''' (see Figure 17):<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.X1',xaxis='chan',yaxis='phase',<br />
figfile='plotcal_3c391-X1.png')<br />
</source><br />
Because the Xf term captures the residual R-L phase on the reference antenna over the array, there is one value for all antennas. Also, as we took out the RL delays using the Kcross solution, these Xf variations only span about 6 degress across the spectral window.<br />
<br />
At this point, you have all the necessary polarization calibration tables.<br />
<br />
=== Scaling the Amplitude Gains ===<br />
<br />
While we know the flux density of our primary calibrator (J1331+3030<math>\equiv</math>3C 286), the model assumed for the secondary calibrator (J1822-0938) was a point source of 1 Jy located at the phase center. While the secondary calibrator was chosen to be a point source (at least, over some limited range of ''uv''-distance; see [https://science.nrao.edu/facilities/vla/docs/manuals/observing/callist the VLA calibrator manual] for any ''u''-''v'' restrictions on your calibrator of choice at the observing frequency), its absolute flux density is unknown. Being point-like, secondary calibrators typically vary on timescales of months to years, in some cases by up to 50&#150;100%. <br />
<br />
We use the primary (flux) calibrator to determine the system response to a source of known flux density and assume that the mean gain amplitudes for the primary calibrator are the same as those for the secondary calibrator. This allows us to find the true flux density of the secondary calibrator. To do this, we use the task '''{{fluxscale}}''', which produces a new calibration table containing properly-scaled amplitude gains for the secondary calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
myscale = fluxscale(vis='3c391_ctm_mosaic_10s_spw0.ms',<br />
caltable='3c391_ctm_mosaic_10s_spw0.G1', <br />
fluxtable='3c391_ctm_mosaic_10s_spw0.fluxscale1', <br />
reference=['J1331+3030'],<br />
transfer=['J1822-0938,J0319+4130'],<br />
incremental=False)<br />
</source><br />
* ''myscale = fluxscale(...) '': '''{{fluxscale}}''' returns a dictionary of results, which we capture in the variable '''myscale'''<br />
* ''caltable='3c391_ctm_mosaic_10s_spw0.G1' '': We provide '''{{fluxscale}}''' with the calibration table containing the amplitude gain solutions derived earlier.<br />
* ''fluxtable='3c391_ctm_mosaic_10s_spw0.fluxscale1' '': We specify the name of the new output table to be written, which will contain the properly-scaled amplitude gains.<br />
* ''reference='J1331+3030' '': We specify the source with the known flux density.<br />
* ''transfer=['J1822-0938,J0319+4130'] '': We specify the sources whose amplitude gains are to be rescaled.<br />
* ''incremental=False '': Make a new output fluxtable replacing caltable with rescaled transfer gains. If parameter ''incremental=True'' then the new table would be used in addition to caltable in subsequent applications.<br />
<br />
Task '''{{fluxscale}}''' will print to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' parameter. These are also captured in the return variable from the task. You should examine the output to ensure that it looks sensible. If the data set has more than 1 spectral window, depending upon where they are spaced and the spectrum of the source, it is possible to find quite different flux densities and spectral indexes for the secondary calibrators. Example output would be<br />
<pre style="background-color: #fffacd;"><br />
CASA <99>: myscale['1']<br />
Out[99]: <br />
{'0': {'fluxd': array([ 2.29522371, 0. , 0. , 0. ]),<br />
'fluxdErr': array([ 0.00632494, 0. , 0. , 0. ]),<br />
'numSol': array([ 46., 0., 0., 0.])},<br />
'fieldName': 'J1822-0938',<br />
'fitFluxd': 0.0,<br />
'fitFluxdErr': 0.0,<br />
'fitRefFreq': 0.0,<br />
'spidx': array([ 0., 0., 0.]),<br />
'spidxerr': array([ 0., 0., 0.])}<br />
<br />
<br />
<br />
CASA <100>: myscale['9']<br />
Out[100]: <br />
{'0': {'fluxd': array([ 13.66191225, 0. , 0. , 0. ]),<br />
'fluxdErr': array([ 0.03827984, 0. , 0. , 0. ]),<br />
'numSol': array([ 44., 0., 0., 0.])},<br />
'fieldName': 'J0319+4130',<br />
'fitFluxd': 0.0,<br />
'fitFluxdErr': 0.0,<br />
'fitRefFreq': 0.0,<br />
'spidx': array([ 0., 0., 0.]),<br />
'spidxerr': array([ 0., 0., 0.])}<br />
<br />
<br />
</pre><br />
or in the CASA logger:<br />
<pre style="background-color: #fffacd;"><br />
...<br />
Found reference field(s): J1331+3030<br />
Found transfer field(s): J1822-0938 J0319+4130<br />
Flux density for J1822-0938 in SpW=0 (freq=4.599e+09 Hz) is: 2.29522 +/- 0.00632494 (SNR = 362.885, N = 46)<br />
Flux density for J0319+4130 in SpW=0 (freq=4.599e+09 Hz) is: 13.6619 +/- 0.0382798 (SNR = 356.896, N = 44)<br />
Storing result in 3c391_ctm_mosaic_10s_spw0.fluxscale1<br />
...<br />
</pre><br />
<br />
Again, the VLA Calibrator Manual may be used to check whether the derived flux densities look sensible. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We plot the rescaled amplitudes from this table:<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.fluxscale1',xaxis='time',yaxis='amp',<br />
poln='R',figfile='plotcal_3c391-fluxscale1-amp-R.png')<br />
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.fluxscale1',xaxis='time',yaxis='amp',<br />
poln='L',figfile='plotcal_3c391-fluxscale1-amp-L.png')<br />
</source><br />
You can see in Figures 18A and 18B that the amplitude gain factors are now similar across sources, compared to the raw factors in the G1 table.<br />
<br />
{| <br />
| [[Image:plotcal_3c391-fluxscale1-amp-R.png|200px|thumb|left|Figure 18A: post-fluxscale amp solutions, R pol]]<br />
| [[Image:plotcal_3c391-fluxscale1-amp-L.png|200px|thumb|center|Figure 18B: post-fluxscale amp solutions, L pol]]<br />
|}<br />
<br />
== Applying the Calibration ==<br />
<br />
Now that we have derived all the calibration solutions, we need to apply them to the actual data, using the task '''{{applycal}}'''. The measurement set DATA column contains the original data. To apply the calibration we have derived, we specify the appropriate calibration tables, which are then applied to the DATA column, with the results being written in the CORRECTED_DATA column. If the dataset does not already have a CORRECTED_DATA scratch column, then one will be created in the first '''{{applycal}}''' run.<br />
<br />
First, we apply the calibration to each individual calibrator, using the gain solutions derived on that calibrator alone to compute the CORRECTED_DATA. To do this, we iterate over the different calibrators, in each case specifying the source to be calibrated (using the ''field'' parameter). The relevant function calls are given below, although as explained presently, the calls to '''{{applycal}}''' will differ slightly if you skipped the Polarization Calibration steps above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',<br />
field='J1331+3030',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', <br />
'3c391_ctm_mosaic_10s_spw0.fluxscale1',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.Kcross', <br />
'3c391_ctm_mosaic_10s_spw0.D2',<br />
'3c391_ctm_mosaic_10s_spw0.X1'],<br />
gainfield=['','J1331+3030','','','','',''], <br />
interp=['','nearest','','','','',''],<br />
calwt=[False],<br />
parang=True)<br />
#<br />
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',<br />
field='J0319+4130',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', <br />
'3c391_ctm_mosaic_10s_spw0.fluxscale1',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.Kcross', <br />
'3c391_ctm_mosaic_10s_spw0.D2',<br />
'3c391_ctm_mosaic_10s_spw0.X1'],<br />
gainfield=['','J0319+4130','','','','',''], <br />
interp=['','nearest','','','','',''],<br />
calwt=[False],<br />
parang=True)<br />
#<br />
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',<br />
field='J1822-0938',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', <br />
'3c391_ctm_mosaic_10s_spw0.fluxscale1',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.Kcross', <br />
'3c391_ctm_mosaic_10s_spw0.D2',<br />
'3c391_ctm_mosaic_10s_spw0.X1'],<br />
gainfield=['','J1822-0938','','','','',''], <br />
interp=['','nearest','','','','',''],<br />
calwt=[False],<br />
parang=True)<br />
</source><br />
<br />
* ''gaintable'' : We provide a Python list of the calibration tables to be applied. This list must contain the antenna position corrections (.antpos), the properly-scaled gain calibration for the amplitudes and phases (.fluxscale1) which were just made using '''{{fluxscale}}''', the parallel-hand delays (.K0), the bandpass solutions (.B0), the cross-hand delays (.Kcross), the leakage calibration (.D2 (derived by our second solution)), and the R-L phase corrections (.X1).<br />
* ''gainfield, interp'' : To ensure that we use the correct gain amplitudes and phases for a given calibrator (those derived on that same calibrator), then for each calibrator source, we need to specify the particular subset of gain solutions to be applied. This requires use of the ''gainfield'' and ''interp'' parameters; these are both Python lists, and for the list item corresponding to the calibration table made by '''{{fluxscale}}''', we set ''gainfield'' to the field name corresponding to that calibrator, and the desired interpolation type (''interp'') to ''nearest''.<br />
* ''calwt=[False] '': At the time of writing, we are not yet using system calibration data to compute real (1/Jy<sup>2</sup>) weights, thus trying to calibrate them can produce nonsensical results. In particular, experience has shown that calibrating the weights will lead to problems especially in the self-calibration steps. You can specify ''calwt'' on a per-table basis, here is set all to ''False''.<br />
* ''parang '': If polarization calibration has been performed, set parameter ''parang=True''. If the polarization calibration steps in the section above were skipped, the .Kcross, .D2 and .X1 tables will not exist. In this case, you should leave out these tables from the ''gaintable'' list, and the corresponding sets of elements in the ''gainfield'' list each time you run '''{{applycal}}''' above; and set parameter ''parang=False''.<br />
<br />
Finally, we apply the calibration to the target fields in the mosaic, linearly interpolating the gain solutions from the secondary calibrator, J1822-0938. In this case however, we want to apply the amplitude and phase gains derived from the secondary calibrator, J1822-0938, since that is close to the target source on the sky and we assume that the gains applicable to the target source are very similar to those derived in the direction of the secondary calibrator. Of course, this is not strictly true, since the gains on J1822-0938 were derived at a different time and in a different position on the sky from the target. However, assuming that the calibrator was sufficiently close to the target, and the weather was sufficiently well-behaved, then this is a reasonable approximation and should get us a sufficiently good calibration that we can later use self-calibration to correct for the small inaccuracies thus introduced.<br />
<br />
The procedure for applying the calibration to the target source is very similar to what we just did for the calibrator sources.<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',<br />
field='2~8',<br />
gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', <br />
'3c391_ctm_mosaic_10s_spw0.fluxscale1',<br />
'3c391_ctm_mosaic_10s_spw0.K0',<br />
'3c391_ctm_mosaic_10s_spw0.B0',<br />
'3c391_ctm_mosaic_10s_spw0.Kcross', <br />
'3c391_ctm_mosaic_10s_spw0.D2',<br />
'3c391_ctm_mosaic_10s_spw0.X1'],<br />
gainfield=['','J1822-0938','','','','',''], <br />
interp=['','linear','','','','',''],<br />
calwt=[False],<br />
parang=True)<br />
</source><br />
<br />
* ''field '': We can calibrate all seven target fields at once by setting ''field='2~8' ''. <br />
* ''gainfield '': In this case, we wish to use the gains derived on the secondary calibrator, for the reasons explained in the previous paragraph.<br />
* ''interp '': This time, we linearly interpolate between adjacent calibrator scans, to compute the appropriate gains for the intervening observations of the target.<br />
<br />
<br />
We should now have fully-calibrated visibilities in the CORRECTED_DATA column of the measurement set, and it is worthwhile pausing to inspect them to ensure that the calibration did what we expected it to. We make some standard plots (see Figures 19A&#150;19D):<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='',<br />
timerange='08:02:00~08:17:00',antenna='',avgtime='60s',<br />
xaxis='channel',yaxis='amp',ydatacolumn='corrected',<br />
plotfile='plotms_3c391-fld0-corrected-amp.png')<br />
#<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='',<br />
timerange='08:02:00~08:17:00',antenna='',avgtime='60s',<br />
xaxis='channel',yaxis='phase',ydatacolumn='corrected',<br />
plotrange=[-1,-1,-180,180],coloraxis='corr',<br />
plotfile='plotms_3c391-fld0-corrected-phase.png')<br />
#<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='1',correlation='RR,LL',<br />
timerange='',antenna='',avgtime='60s',<br />
xaxis='channel',yaxis='amp',ydatacolumn='corrected',<br />
plotfile='plotms_3c391-fld1-corrected-amp.png')<br />
#<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='1',correlation='RR,LL',<br />
timerange='',antenna='',avgtime='60s',<br />
xaxis='channel',yaxis='phase',ydatacolumn='corrected',<br />
plotrange=[-1,-1,-180,180],coloraxis='corr',<br />
plotfile='plotms_3c391-fld1-corrected-phase.png')<br />
</source><br />
For 3C286 (leftmost plots) we see the polarized signal in the cross-hands. There is no sign of bad data remaining.<br />
<br />
{|<br />
| [[Image:plotms_3c391-fld0-corrected-amp.png|thumb|Figure 19A: amp vs channel for J1822-0938 RR,RL,LR,LL]]<br />
| [[Image:plotms_3c391-fld0-corrected-phase.png|thumb|Figure 19B: phase vs channel for J1822-0938 RR,RL,LR,LL]]<br />
| [[Image:plotms_3c391-fld1-corrected-amp.png|thumb|Figure 19C: amp vs channel for J1822-0938 RR,LL]]<br />
| [[Image:plotms_3c391-fld1-corrected-phase.png|thumb|Figure 19D: phase vs channel for J1822-0938 RR,LL]]<br />
|}<br />
<br />
[[Image:plotms_3c391-fld1-corrected-ampvsphase.png|thumb|Figure 20: corrected amplitude vs phase for J1822-0938]]<br />
Another nice display is to use '''{{plotms}}''' to plot the amplitude and phase of the CORRECTED_DATA column against one another, for one of the parallel-hand correlations (RR or LL; the signal in the cross-hands, RL and LR, is much smaller and will be noise-like for an unpolarized calibrator). This should then show a nice ball of visibilities, with some scatter, centered at zero phase and the amplitude found for that source in '''{{fluxscale}}'''. An example is shown in Figure 20.<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='1',correlation='RR,LL',<br />
timerange='',antenna='',avgtime='60s',<br />
xaxis='phase',xdatacolumn='corrected',yaxis='amp',ydatacolumn='corrected',<br />
plotrange=[-180,180,0,3],coloraxis='corr',<br />
plotfile='plotms_3c391-fld1-corrected-ampvsphase.png')<br />
</source><br />
<br />
Inspecting the data at this stage may well show up previously-unnoticed bad data. Plotting the '''corrected''' amplitude against UV distance or against time is a good way to find such issues. If you find bad data, you can remove them via interactive flagging in '''{{plotms}}''' or via manual flagging in '''{{flagdata}}''' once you have identified the offending antennas/baselines/channels/times. When you are happy that all data (particularly on your target source) look good, you may proceed.<br />
<br />
Now that the calibration has been applied to the target data we can split off the science targets, creating a new, calibrated measurement set containing all the target fields.<br />
<br />
<source lang="python"><br />
# In CASA<br />
split(vis='3c391_ctm_mosaic_10s_spw0.ms',outputvis='3c391_ctm_mosaic_spw0.ms',<br />
datacolumn='corrected',field='2~8')<br />
</source><br />
<br />
* ''outputvis '': We give the name of the new measurement set to be written, which will contain the calibrated data on the science targets.<br />
* ''datacolumn '': We use the CORRECTED_DATA column, containing the calibrated data which we just wrote using '''{{applycal}}'''.<br />
* ''field '': We wish to put all the mosaic pointings into a single measurement set, for imaging and joint deconvolution.<br />
<br />
== Initial Imaging ==<br />
<br />
Now that we have split off the target data into a separate measurement set with all the calibration applied, it's time to make an image. Recall that the visibility data and the sky brightness distribution (a.k.a. image) are Fourier transform pairs.<br />
<br />
<math><br />
I(l,m) = \int V(u,v) e^{[2\pi i(ul + vm)]} dudv<br />
</math><br />
<br />
The <math>u</math> and <math>v</math> coordinates are the baselines measured in units of the observing wavelength, while the <math>l</math> and <math>m</math> coordinates are the direction cosines on the sky. For generality, the sky coordinates are written in terms of direction cosines, but for most JVLA (and ALMA) observations they can be related simply to the right ascension (<math>l</math>) and declination (<math>m</math>). Also recall that this equation is valid only if the <math>w</math> coordinate of the baselines can be neglected. This assumption is almost always true at high frequencies and smaller JVLA configurations (such as the 4.6 GHz D-configuration observations here). The <math>w</math> coordinate cannot be neglected at lower frequencies and larger configurations (e.g., 0.33 GHz, A-configuration observations). This expression also neglects other factors, such as the shape of the primary beam. For more information on imaging, see [http://casa.nrao.edu/docs/cookbook/casa_cookbook006.html Synthesis Imaging] within the CASA Reference Manual.<br />
<br />
[[Image:plotms_3c391-mosaic0-uvwave.png|thumb|Figure 21: ''plotms'' plot showing Amplitude vs UV Distance in wavelengths for 3C391 at 4600 MHz]]<br />
CASA has a single task, '''{{clean}}''' which both Fourier transforms the data and deconvolves the resulting image. For the purposes of this tutorial (Part 1), we will make a simple mosaic clean image in Stokes I only. In Advance Topics (Part 2), we will proceed to make multiscale mosaic images in IQUV and to analyze them.<br />
<br />
Setting the appropriate pixel size for imaging depends upon basic optics aspects of interferometry. Using '''{{plotms}}''' to look at the newly-calibrated, target-only data set,<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c391_ctm_mosaic_spw0.ms',xaxis='uvwave',yaxis='amp',<br />
ydatacolumn='data', field='0',avgtime='30s',correlation='RR',<br />
plotfile='plotms_3c391-mosaic0-uvwave.png',overwrite=True)<br />
</source><br />
You should obtain a plot similar to Figure 21 with the (calibrated) visibility amplitude as a function of <math>u</math>-<math>v</math> distance.<br />
The maximum baseline is about 16,000 wavelengths, i.e., an angular scale of 12 arcseconds (<math>\lambda/D=1/16000</math>). The most effective cleaning occurs with 3&#150;5 pixels across the synthesized beam. For example, a cell size of 2.5 arcseconds will give just under 5 pixels per beam. <br />
<br />
The supernova remnant itself is known to have a diameter of order 9 arcminutes, corresponding to about 216 pixels for the chosen cell size. The mosaic was set up with 7 fields, 1 centered on the remnant and 6 flanking fields; the spacing of the fields was chosen based on the size of the antenna primary beam. For the choice of ''ftmachine='mosaic' ''(our main mosaicking algorithm), you do not have to fit the mosaic inside the inner quarter of the total image in order to prevent image artifacts arising from aliasing, we just want to have a bit of padding around the outside. Although CASA has the feature that its Fourier transform engine (FFTW) does ''not'' require a strict power of 2 for the number of linear pixels in a given image axis, it is somewhat more efficient if the number of pixels on a side is a composite number divisible by ''any pair'' of 2 and 3 and/or 5. Because '''{{clean}}''' internally applies a padding of 1.2 (=3x2/5) choose 480, which is 2<sup>5</sup> &times; 3 &times; 5 (so 480 &times; 1.2 = 576 = 2<sup>6</sup> &times; 3<sup>2</sup>). We therefore set ''imsize=[480,480]'' and our mosaic fits comfortably inside the image.<br />
<br />
To make this simple clean image interactively:<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='3c391_ctm_mosaic_spw0.ms',<br />
imagename='3c391_ctm_spw0_noms_I',<br />
field='',spw='',<br />
mode='mfs',<br />
niter=5000,<br />
gain=0.1, threshold='1.0mJy',<br />
psfmode='clark',<br />
imagermode='mosaic', ftmachine='mosaic',<br />
multiscale=[0], <br />
interactive=True,<br />
imsize=[480,480], cell=['2.5arcsec','2.5arcsec'],<br />
stokes='I',<br />
weighting='briggs',robust=0.5,<br />
usescratch=False)<br />
</source><br />
<br />
Task {{clean}} is powerful with many inputs and a certain amount of experimentation likely is required.<br />
* ''vis='3c391_ctm_mosaic_spw0.ms' '': this split MS contains our 7-pt mosaic fields, now indexed as 0&#150;6. Field 0 is the central field of the mosaic (you can use '''{{listobs}}''' to verify this).<br />
* ''imagename='3c391_ctm_spw0_noms_I' '': our output images will all start with this, e.g., 3c391_ctm_spw0_noms_I.image<br />
* ''mode='mfs' '': Use multi-frequency synthesis imaging. The fractional bandwidth of these data is non-zero (128 MHz at a central frequency of 4.6 GHz). Recall that the <math>u</math> and <math>v</math> coordinates are defined as the baseline coordinates, measured in wavelengths. Thus, slight changes in the frequency from channel to channel result in slight changes in <math>u</math> and <math>v</math>. There is a concomitant improvement in <math>u</math>-<math>v</math> coverage if the visibility data from the multiple spectral channels are gridded separately onto the <math>u</math>-<math>v</math> plane, as opposed to treating all spectral channels as having the same frequency.<br />
* ''niter=25000,gain=0.1,threshold='1.0mJy' '': Recall that the clean gain is the amount by which a clean component is subtracted during the cleaning process. Parameters ''niter'' and ''threshold'' are (coupled) means of determining when to stop the cleaning process, with ''niter'' specifying to find and subtract that many clean components while ''threshold'' specifies a minimum flux density threshold a clean component can have before clean stops. See also ''interactive'' below. Imaging is an iterative process, and to set the threshold and number of iterations, it is usually wise to clean interactively in the first instance, stopping when spurious emission from sidelobes (arising from gain errors) dominates the residual emission in the field. Here, we have used our experience in interactive mode to set a threshold level based on the rms noise in the resulting image. The number of iterations should then be set high enough to reach this threshold.<br />
* ''interactive=True '': Very often, particularly when one is exploring how a source appears for the first time, it can be valuable to interact with the cleaning process. If True, ''interactive'' causes a '''{{viewer}}''' window to appear. One can then set clean regions, restricting where clean searches for clean components, as well as monitor the cleaning process. A standard procedure is to set a large value for ''niter'', and stop the cleaning when it visually appears to be approaching the noise level. This procedure also allows one to change the cleaning region, in cases when low-level intensity becomes visible as the cleaning process proceeds. <br />
* ''imsize=[480,480],cell=['2.5arcsec'] '': See the discussion above regarding the setting of the image size and cell size. If only one value is specified, the same value is used in both directions.<br />
* ''stokes='I',psfmode='clark' '': A single image will be made for total intensity I.<br />
* ''weighting='briggs',robust=0.5 '': 3C391 has diffuse, extended emission that is (at least partially) resolved out by the interferometer owing to a lack of short spacings. A naturally-weighted image would show large-scale patchiness in the noise. In order to suppress this effect, Briggs weighting is used (intermediate between natural and uniform weighting), with a default robust factor of 0.5 (which corresponds to something between natural and uniform weighting).<br />
* ''imagermode='mosaic', ftmachine='mosaic' '': The data consist of a 7-pointing mosaic, since the supernova remnant fills almost the full primary beam at 4.6 GHz. A mosaic combines the data from all of the fields, with imaging and deconvolution being done jointly on all 7 fields. A mosaic both helps compensate for the shape of the primary beam and reduces the amount of large (angular) scale structure that is resolved out by the interferometer.<br />
* ''multiscale=[0] '': For now, we will clean with a single scale of size 0 (a classic clean delta-function)<br />
* ''usescratch=False '': As in calibration, we will NOT require use of a physical MODEL_DATA scratch column.<br />
<br />
[[Image:screenshot_3c391_interclean_start_v4.0.png|thumb|Figure 22: Interactive clean at the beginning, having selected polygon region and ready to double-click inside to set the mask.]]<br />
[[Image:screenshot_3c391_interclean_finish_v4.0.png|thumb|Figure 23: Interactive clean part-way through cleaning (after 3000 iterations)]]<br />
As mentioned above, we can guide the clean process by allowing it to find clean components only within a user-specified region. The easiest way to do this is via interactive clean. When '''{{clean}}''' runs in interactive mode, a viewer window will pop up as shown in Figure 22. To get a more detailed view of the central regions containing the emission, zoom in by tracing out a rectangle with the left mouse button and double-clicking inside the zoom box you just made. Play with the color scale to bring out the emission better, by holding down the middle mouse button and moving it around. To create a clean box (a region within which components may be found), hold down the right mouse button and trace out a rectangle around the source, then double-click inside that rectangle to set it as a box. Note that the clean box must turn white for it to be registered; if the box is not white, it has not been set! Alternatively, you can trace out a more custom shape to better enclose the irregular outline of the supernova remnant. To do that, right-click on the closed polygonal icon. Then trace out a shape by right-clicking where you want the corners of that shape. Once you have come full circle, the shape will be traced out in green, with small squares at the corners. Double-click inside this region and the green outline will turn white. You have now set the clean region. If you have made a mistake with your clean box, click on the Erase button, trace out a rectangle around your erroneous region, and double-click inside that rectangle. You can also set multiple clean regions. By default, all clean regions will apply only to the plane shown. To change this to select all planes, click the All Channels button at the top (we have only a single mfs (multi-frequency synthesis) channel in the image cube), and likewise for Stokes planes using the All Polarizations (you will need to set this when we image all the Stokes).<br />
<br />
At any stage in the cleaning, you can adjust the number of iterations that clean will do before returning to the GUI. By default this is set to 100 (see the iterations field in mid-upper left of panel). You probably want to set this to a high number for this mosaic due to the complicated structure, values from 1000 to 5000 later on seem to work. Note that this will override the ''niter'' that was set when you started the clean task. Clean will keep going until it reaches threshold or runs out of cycles (the cycles field to the right of the iterations).<br />
<br />
When you are happy with the clean regions, press the green circular arrow button on the far right to continue deconvolution. After completing a cycle, a revised image will come up. As the brightest points are removed from the image (cleaned off), fainter emission may show up. You can adjust the clean boxes each cycle, to enclose all real emission. After many cycles, when only noise is left, you can hit the red and white cross icon to stop cleaning. Figure 23 shows the interactive clean panel later in the process, after cleaning around 3000 iterations. At this point we have used the polygon tool to draw around the emission. There is still signal left in the residual, and it would take forever to clean all that out using our standard clean! You should just quit after doing around 10000&#150;20000 iterations total. We'll improve this in the next part of the tutorial.<br />
<br />
[[Image:screenshot_3c391_viewer_noms_final.png|thumb|Figure 24: '''viewer''' display of the Stokes I mosaic of 3C391]]<br />
<br />
Task '''{{clean}}''' will make several output files, all named with the prefix given as ''imagename''. These include:<br />
* ''.image''&#151;final restored image, with the clean components convolved with a restoring beam and added to the remaining residuals at the end of the imaging process<br />
* ''.flux''&#151;effective response of the telescope (the primary beam)<br />
* ''.flux.pbcoverage''&#151;effective response of the full mosaic image<br />
* ''.mask''&#151;areas where '''{{clean}}''' has been allowed to search for emission<br />
* ''.model''&#151;sum of all the clean components, which also has been stored as the MODEL_DATA column in the measurement set<br />
* ''.psf''&#151;dirty beam, which is being deconvolved from the true sky brightness during the clean process<br />
* ''.residual''&#151;what is left at the end of the deconvolution process; this is useful to diagnose whether or not to clean more deeply<br />
<br />
After the imaging and deconvolution process has finished, you can use the '''{{viewer}}''' to look at your image.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('3c391_ctm_spw0_noms_I.image')<br />
</source><br />
<br />
This will bring up a viewer window containing the image (see Figure 24). You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do).<br />
<br />
Note that the image is cut off in a circular fashion at the edges, corresponding to the default minimum primary beam response (''minpb'') within '''{{clean}}''' of 0.2.<br />
<br />
As we saw during the interactive cleaning, this single-scale delta-function cleaning is having problems cleaning out the large-scale emission. You can see the large clean bowl of negative emission surrounding the object, and the heavy pixelization of the emission at low levels. We'll try to improve on this in the next part of the tutorial.<br />
<br />
= Continuum Observations Data Reduction Tutorial: 3C391 - Advanced Topics =<br />
<!--<br />
Now, we discuss various "advanced topics" for further reduction of the 3C391 continuum data. This tutorial assumes that the reader already has some familiarity with basic continuum data reduction, such as should have been obtained in the [[EVLA Continuum Tutorial 3C391-CASA4.3 | Continuum Data Reduction Tutorial]]. <br />
<!-- on the first day of the NRAO Synthesis Imaging Workshop data reduction tutorials. --><br />
<br />
<br />
== Multi-scale Polarization Clean ==<br />
<br />
[[Image:screenshot_3c391part2_interclean_1.png|thumb|Figure 25: After the first 500 iterations of multi-scale clean for Stokes I plane]]<br />
[[Image:screenshot_3c391_clean_interactive_ms_final.png|thumb|Figure 26: Interactive residuals in I after 15000 iterations of multi-scale clean]]<br />
We now employ a multi-scale clean to image Stokes IQUV:<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='3c391_ctm_mosaic_spw0.ms',imagename='3c391_ctm_spw0_IQUV',<br />
field='',spw='',<br />
mode='mfs',<br />
niter=25000,<br />
gain=0.1, threshold='1.0mJy',<br />
psfmode='clarkstokes',<br />
imagermode='mosaic', ftmachine='mosaic',<br />
multiscale=[0, 6, 18, 54], smallscalebias=0.9,<br />
interactive=True,<br />
imsize=[480,480], cell=['2.5arcsec','2.5arcsec'],<br />
stokes='IQUV',<br />
weighting='briggs',robust=0.5,<br />
pbcor=False,<br />
usescratch=False)<br />
</source><br />
<br />
<br />
For these inputs, we have changed:<br />
* ''stokes='IQUV', psfmode='clarkstokes' '': Separate images will be made in all four polarizations (total intensity I, linear polarizations Q and U, and circular polarization V), and, with parameter ''psfmode='clarkstokes' '', the Clark clean algorithm will deconvolve each Stokes plane separately thereby making the polarization image more independent of the total intensity.<br />
* ''multiscale=[0, 6, 18, 54], smallscalebias=0.9 '': A multi-scale cleaning algorithm is used because the supernova remnant contains both diffuse, extended structure on large spatial scales and finer filamentary structure on smaller scales. The settings for ''multiscale'' are in units of pixels, with 0 pixels equivalent to the traditional delta-function clean. The scales here are chosen to provide delta functions and then three logarithmically scaled sizes to fit to the data. The first scale (6 pixels) is chosen to be comparable to the size of the beam. The ''smallscalebias'' attempts to balance the weight given to larger scales, which often have more flux density, and the smaller scales, which often are brighter. Considerable experimentation is likely to be necessary; one of the authors of this document found that it was useful to clean several rounds with this setting, change to ''multiscale=[]'' and remove much of the smaller scale structure, then return to this setting.<br />
* ''pbcor=False '': by default ''pbcor=False'' and a flat-noise image is produced. We can do the primary beam correction later (see below).<br />
<br />
[[Image:screenshot_3c391_viewer_ms_final_I.png|thumb|Figure 27: Viewer panel for Stokes I of final restored image (using HotMetal1 colormap)]]<br />
[[Image:screenshot_3c391_viewer_ms_final_V.png|thumb|Figure 28: Viewer panel for Stokes V (using scaling power cycles of &minus;1.5)]]<br />
As previously, we guide the clean using polygon regions in the interactive clean GUI. Because we now have the four Stokes planes in the image for IQUV, you should be sure to extend the mask you draw by selecting the All Polarizations radio button '''before''' drawing the mask. As you clean, you can use the tape deck controls to examine the residuals in the QUV planes. In practice, those will reach the 1 mJy threshold fairly quickly and not be cleaned further while the I cleaning proceeds for a long time. In Figures 25 & 26, you can see that the multiscale clean is doing a better job of picking up the larger-scale emission. After about 15000 iterations (Figure 26) the residuals were looking pretty good. As mentioned above restarting clean with different ''multiscale=[...]'' choices can help also.<br />
<br />
After the imaging and deconvolution process has finished, you can use the '''{{viewer}}''' to look at your image.<br />
<source lang="python"><br />
# In CASA<br />
viewer('3c391_ctm_spw0_IQUV.image')<br />
</source><br />
The tape deck control that you see under (or adjacent to) the image can be used to step through the different Stokes parameters (I,Q,U,V). You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do). Also, using the wrench panel to change Display Options will be helpful here. In the I Stokes plane (1) (Figure 27) we chose the Hot Metal 1 colormap. In Figure 28, showing the V Stokes plane (3) we also set the scaling power cycles to &minus;1.5 to enhance low-contrast emission. Our Stokes I image is significantly better than with just a delta-function classic clean. You can also explore the Q and U images, but we will produce more intuitive images from these later on. Finally, note that the residual emission in Stokes V is not real, but likely due to the JVLA beam squint between R and L (they are slightly separated on the sky, and this is not currently compensated for in our production version of clean).<br />
<br />
The clean task naturally operates in a flat noise image, i.e., an image where the effective weighting across the mosaic field of view is set so that the noise is constant. This is so that the clean threshold has a uniform meaning for the stopping criterion and that the image fed into the minor cycles has uniform noise levels. However, this means that the image does not take into account the primary beam fall-off in the edges and interstices of the mosaic. We could have set parameter ''pbcor=True'' in '''{{clean}}''', but it is useful to see the flat-noise image and residuals to evaluate the quality of the clean image. Therefore, we use '''{{impbcor}}''' to divide the ''.image'' by the ''.flux'' image to produce a primary beam corrected restored image:<br />
<source lang="python"><br />
# In CASA<br />
impbcor(imagename='3c391_ctm_spw0_IQUV.image',pbimage='3c391_ctm_spw0_IQUV.flux',<br />
outfile='3c391_ctm_spw0_IQUV.pbcorimage')<br />
</source><br />
<br />
You can open this in the viewer and see that it has indeed raised the noise (and signal) at the edges of the mosaic.<br />
<br />
== Image Analysis and Manipulation ==<br />
<br />
This topic is perhaps not advanced, but it appears to fit more naturally here. It is assumed that an image ''3c391_ctm_spw0_IQUV.image'', resulting from the above clean, exists.<br />
<br />
The three most basic analyses are to determine the peak brightness, the flux density, and the image noise level. These are useful measures of how well the imaging efforts are in approaching the thermal noise limit or in reproducing what is already known about a source. Additional discussion of image analysis and manipulation, including the combination of multiple images, mathematical operations on images, and much more can be found in [http://casa.nrao.edu/docs/cookbook/casa_cookbook007.html Image Analysis] in the CASA Reference Book.<br />
<br />
The most straightforward statistic is the peak brightness, which is determined by '''{{imstat}}'''.<br />
<source lang="python"><br />
mystat = imstat(imagename='3c391_ctm_spw0_IQUV.pbcorimage',stokes='')<br />
</source><br />
* '''mystat=imstat(...) '': '''{{imstat}}''' returns a Python dictionary which we capture in the variable ''mystat''<br />
* ''stokes=' ' '': This example determines the peak brightness in the '''entire''' image, which has all four Stokes planes. To determine the peak brightness in just the Stokes V image, for example, then set parameter ''stokes='V' ''.<br />
<br />
The dictionary contains the values which you can extract for further use. For example, for a particular instance of<br />
the previous clean, we found:<br />
<pre style="background-color: #E0FFFF;"><br />
CASA <4>: mystat<br />
Out[4]: <br />
{'blc': array([0, 0, 0, 0], dtype=int32),<br />
'blcf': '18:50:04.251, -01.05.40.567, I, 4.59901e+09Hz',<br />
'max': array([ 0.15447657]),<br />
'maxpos': array([288, 256, 0, 0], dtype=int32),<br />
'maxposf': '18:49:16.243, -00.55.00.579, I, 4.59901e+09Hz',<br />
'mean': array([ 0.00081497]),<br />
'medabsdevmed': array([ 0.00016437]),<br />
'median': array([ 1.00343077e-05]),<br />
'min': array([-0.00607492]),<br />
'minpos': array([239, 413, 0, 0], dtype=int32),<br />
'minposf': '18:49:24.411, -00.48.28.080, I, 4.59901e+09Hz',<br />
'npts': array([ 481828.]),<br />
'q1': array([-0.00014913]),<br />
'q3': array([ 0.00018026]),<br />
'quartile': array([ 0.0003294]),<br />
'rms': array([ 0.00590687]),<br />
'sigma': array([ 0.00585038]),<br />
'sum': array([ 392.67754914]),<br />
'sumsq': array([ 16.81150507]),<br />
'trc': array([479, 479, 3, 0], dtype=int32),<br />
'trcf': '18:48:44.407, -00.45.43.065, V, 4.59901e+09Hz'}<br />
<br />
CASA <5>: mystat['max'][0]<br />
Out[5]: 0.15447656810283661<br />
</pre><br />
and so the peak flux density is 0.154 Jy/beam (likely in the I Stokes plane).<br />
<br />
[[Image:screenshot_3c391_viewer_polygon.png|thumb|right|Figure 29: viewer polygon region drawing for on-source statistics]]<br />
[[Image:screenshot_3c391_viewer_polygon_forrms.png|thumb|right|Figure 30: viewer polygon region for off-source statistics]]<br />
The other two statistics require slightly more care. The flux density of a source is determined by integrating its brightness or intensity over some solid angle, i.e., <br />
<br />
<center><br />
<math>S = \int d\Omega I</math><br />
</center><br />
<br />
where <math>I</math> is the intensity (measured in units of Jy/beam), <math>\Omega</math> is the solid angle of the source (e.g., number of synthesized beams), and <math>S</math> is the flux density (measured in units of Jy). In general, if the noise is well-behaved in the image, when averaged over a reasonable solid angle, the noise contribution should approach 0 Jy. If that is the case, then the flux density of the source is also reported by '''{{imstat}}'''. However, there are many cases for which a noise contribution of 0 Jy may not be a safe assumption. If the source is in a complicated region (e.g., a star formation region, the Galactic center, near the edge of a galaxy), a better estimate of the source's flux density will be obtained by limiting carefully the solid angle over which the integration is performed.<br />
<br />
Open '''{{viewer}}''' and use it to display the corrected image (Figure 29):<br />
<source lang="python"><br />
# In CASA<br />
viewer('3c391_ctm_spw0_IQUV.pbcorimage')<br />
</source><br />
One can choose the function assigned to each mouse button; after zooming into the desired view, assign polygon region to a desired mouse button (e.g., left button) by selecting the polygon tool [[File:Polygon_btn.png]] to create the polygonal region as shown in Figure 30 with the desired mouse button. <br />
<br />
Using the mouse button just assigned to polygon region, outline the supernova remnant. You start drawing vertices by clicking on points in the image in succession, when you draw the final vertex then you double-click to connect and close the region. When your mouse is inside the region, a bounding box will appear with the vertices shown as draggable solid squares. If you want to adjust the vertices you can do so.<br />
<br />
If you find you don't like your region you can dismiss it with with ESC key or using the remove region "X" button in lower right of the panel. You can also employ the region panel to save a region you have created for later use.<br />
<br />
Double click inside of that region (using the same mouse button used to make the region), and the statistics will be reported. This will include the flux density value within the region selected.<br />
<pre style="background-color: #E0FFFF;"><br />
(3c391_ctm_spw0_IQUV.pbcorimage)<br />
Stokes Velocity Frame Doppler Frequency<br />
I -42.9832km/s LSRK RADIO 4.59901e+09<br />
BrightnessUnit BeamArea Npts Sum FluxDensity<br />
Jy/beam 45.0996 22203 3.857102e+02 8.552405e+00<br />
Mean Rms Std dev Minimum Maximum<br />
1.737199e-02 2.668837e-02 2.026084e-02 -6.180350e-03 1.528023e-01<br />
region count<br />
1<br />
</pre><br />
In our example we find a total Flux density of 8.5 Jy while the max brightness is only 0.0173 Jy/beam. Note that the numbers you get may be slightly different depending on how deeply you cleaned.<br />
<br />
By contrast, for the rms noise level, one can load the original (un-pbcor) image: <br />
<source lang="python"><br />
# In CASA<br />
viewer('3c391_ctm_spw0_IQUV.image')<br />
</source><br />
and to ''exclude'' the source's emission to the extent possible, as the source's emission will bias the estimated noise level high. Likewise, one should avoid the clean bowl around the source emission. One can repeat the procedure above, defining a polygonal region, then double clicking inside it to determine the statistics. You can use the tape deck controls to change which Stokes plane you are viewing and double-click in each to get those statistics. For example, from the region selection shown to the right for off-source statistics:<br />
<pre style="background-color: #E0FFFF;"><br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
(3c391_ctm_spw0_IQUV.pbcorimage)<br />
Stokes Velocity Frame Doppler Frequency<br />
I -42.9832km/s LSRK RADIO 4.59901e+09<br />
BrightnessUnit BeamArea Npts Sum FluxDensity<br />
Jy/beam 45.0996 61753 2.844620e+01 6.307414e-01<br />
Mean Rms Std dev Minimum Maximum<br />
4.606448e-04 1.661159e-03 1.596025e-03 -5.679628e-03 9.719489e-03<br />
region count<br />
1<br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
(3c391_ctm_spw0_IQUV.pbcorimage)<br />
Stokes Velocity Frame Doppler Frequency<br />
Q -42.9832km/s LSRK RADIO 4.59901e+09<br />
BrightnessUnit BeamArea Npts Sum FluxDensity<br />
Jy/beam 45.0996 61753 -5.743594e-01 -1.273535e-02<br />
Mean Rms Std dev Minimum Maximum<br />
-9.300915e-06 1.507897e-04 1.505038e-04 -7.758265e-04 8.262882e-04<br />
region count<br />
1<br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
(3c391_ctm_spw0_IQUV.pbcorimage)<br />
Stokes Velocity Frame Doppler Frequency<br />
U -42.9832km/s LSRK RADIO 4.59901e+09<br />
BrightnessUnit BeamArea Npts Sum FluxDensity<br />
Jy/beam 45.0996 61753 4.498778e-01 9.975201e-03<br />
Mean Rms Std dev Minimum Maximum<br />
7.285117e-06 1.485673e-04 1.483898e-04 -8.174598e-04 8.751833e-04<br />
region count<br />
1<br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
(3c391_ctm_spw0_IQUV.pbcorimage)<br />
Stokes Velocity Frame Doppler Frequency<br />
V -42.9832km/s LSRK RADIO 4.59901e+09<br />
BrightnessUnit BeamArea Npts Sum FluxDensity<br />
Jy/beam 45.0996 61803 2.358344e-01 5.229189e-03<br />
Mean Rms Std dev Minimum Maximum<br />
3.815906e-06 3.209289e-04 3.209089e-04 -1.309853e-03 1.316741e-03<br />
region count<br />
1<br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
</pre><br />
Thus the I rms is 1.6 mJy and Q and U rms are around 0.15 mJy. It will be useful later on to have the flat-noise and pb-corrected images available separately along with the statistics.<br />
<br />
== Constructing Polarization Intensity and Angle Images ==<br />
<br />
At the beginning of this Advanced Tutorial, a full polarization imaging cube of 3C391 was constructed. This cube has 3 dimensions, the standard two angular dimensions (right ascension, declination) and a third dimension containing the polarization information. Considering the image cube as a matrix, <math>Image[l,m,p]</math>, the <math>l</math> and <math>m</math> axis describe the sky brightness or intensity for the given <math>p</math> axis. If one opens the '''{{viewer}}''' and loads the 3C391 continuum image, the default view contains an animator or pane with movie controls. One can step through the polarization axis, displaying the images for the different polarizations.<br />
<br />
As created, the image contains four polarizations, one for each of the four Stokes parameters: I, Q, U, and V. Recall that Stokes Q and U describe linear polarization and V describes circular polarization. Specifically, Q describes the amount of linear polarization aligned with a given axis, and U describes the amount of linear polarization at a 45 deg angle to that axis. The V parameter describes the amount of circular polarization, with the sign (positive or negative) describing the sense of the circular polarization (right- or left-hand circularly polarized).<br />
<br />
Few celestial sources, with the notable exception of masers, are expected to show circular polarization. Terrestrial and satellite sources, however, are often highly circularly polarized. The V image is therefore often worth forming because any V emission could be indicative of unflagged RFI within the data (or problems with the calibration!).<br />
<br />
Because the Q and U images both describe the amount of linear polarization, it is more common to work with a linear polarization intensity image, <math>P = \sqrt{Q^2 +U^2}</math>. (<math>P</math> can also be denoted by <math>L</math>.) Also important can be the polarization position angle <math>tan 2\chi = U/Q</math>.<br />
<br />
The relevant task is '''{{immath}}''', with specific examples for processing of polarization images given in<br />
[http://casa.nrao.edu/docs/cookbook/casa_cookbook007.html#sec373 Polarization Manipulation]. The steps are the following.<br />
<br />
1. Extract the I, Q, U, V planes from the full Stokes image cube, forming separate images for each Stokes parameter. Use the flat-noise images so that non-uniform noise does not bias us.<br />
<source lang="python"><br />
# In CASA<br />
immath(imagename='3c391_ctm_spw0_IQUV.image',outfile='3c391_ctm_spw0.I',expr='IM0',stokes='I')<br />
immath(imagename='3c391_ctm_spw0_IQUV.image',outfile='3c391_ctm_spw0.Q',expr='IM0',stokes='Q')<br />
immath(imagename='3c391_ctm_spw0_IQUV.image',outfile='3c391_ctm_spw0.U',expr='IM0',stokes='U')<br />
immath(imagename='3c391_ctm_spw0_IQUV.image',outfile='3c391_ctm_spw0.V',expr='IM0',stokes='V')<br />
</source><br />
We also extract a pb-corrected I image for later use:<br />
<source lang="python"><br />
# In CASA<br />
immath(outfile='3c391_ctm_spw0.pbcorI',<br />
mode='evalexpr',<br />
imagename='3c391_ctm_spw0_IQUV.pbcorimage',<br />
expr='IM0',stokes='I')<br />
</source><br />
<br />
2. Combine the Q and U images using the parameter ''mode='poli' ''of '''{{immath}}''' to form the linear polarization image.<br />
<source lang="python"><br />
# In CASA<br />
immath(outfile='3c391_ctm_spw0.P',<br />
mode='poli',<br />
imagename=['3c391_ctm_spw0.Q','3c391_ctm_spw0.U'],<br />
sigma='0.0Jy/beam')<br />
</source><br />
Because P is formed from the sum of the squares of the Q and U values, it is biased by the noise. In principle, one can set the '''sigma''' value to debias this (it subtracts the square of this value from the Q<sup>2</sup>+U<sup>2</sup> before taking the square-root). <br />
<source lang="python"><br />
# In CASA<br />
immath(outfile='3c391_ctm_spw0.P_unbias',<br />
mode='poli',<br />
imagename=['3c391_ctm_spw0.Q','3c391_ctm_spw0.U'],<br />
sigma='0.000041Jy/beam')<br />
</source><br />
<!-- Currently in CASA 4.1 and earlier, there is a bug where pixels where the noise subtraction makes it negative are set to NaN. To filter this out use {{immath}}. Also, you must give <tt>sigma</tt> in Jy/beam (not mJy/beam). We do:<br />
<source lang="python"><br />
# In CASA<br />
immath(outfile='3c391_ctm_spw0.P_unbias',<br />
mode='evalexpr',<br />
imagename=['3c391_ctm_spw0.P_unbias'],<br />
expr="IM0[IM0>-10000.0]")<br />
</source> --><br />
You can then pb-correct this (after first extracting one plane of the ''.flux'' image, they are all the same):<br />
<source lang="python"><br />
# In CASA<br />
immath(imagename='3c391_ctm_spw0_IQUV.flux',<br />
outfile='3c391_ctm_spw0.Qflux',<br />
expr='IM0',stokes='Q')<br />
#<br />
immath(outfile='3c391_ctm_spw0.pbcorP',<br />
mode='evalexpr',<br />
imagename=['3c391_ctm_spw0.P_unbias_filtered','3c391_ctm_spw0.Qflux'],<br />
expr="IM0[IM1>0.2]/IM1")<br />
</source><br />
<br />
<br />
3. Combine the Q and U images using the parameter ''mode='pola' ''of '''{{immath}}''' to form the polarization position angle image. Because the polarization position angle is derived from the tangent function, the order in which the Q and U images are specified is important. Also, we use the flat-noise images as only the ratio matters and for thresholding you want a uniform noise:<br />
<source lang="python"><br />
# In CASA<br />
immath(outfile='3c391_ctm_spw0.X',<br />
mode='pola',<br />
imagename=['3c391_ctm_spw0.Q','3c391_ctm_spw0.U'],<br />
polithresh='0.2mJy/beam')<br />
</source><br />
To avoid displaying the position angle of noise, we can set a threshold intensity of the linear polarization for above which we wish to calculate the polarization angle, using the parameter ''polithresh''. An appropriate level here might be the <math>5\sigma</math> level of around 0.2 mJy/beam.<br />
<br />
4. If desired, form the fractional linear polarization image, defined as P/I. We can do this on the flat-noise image.<br />
<source lang="python"><br />
# In CASA<br />
immath(outfile='3c391_ctm_spw0.F',<br />
mode='evalexpr',<br />
imagename=['3c391_ctm_spw0.I','3c391_ctm_spw0.Q','3c391_ctm_spw0.U'],<br />
expr='sqrt((IM1^2+IM2^2)/IM0[IM0>1.2e-3]^2)')<br />
</source><br />
Since the total intensity image can (and hopefully does) approach zero in regions free of source emission, dividing by the total intensity can produce very high pixel values in these regions. We therefore wish to restrict our fractional polarization image to regions containing real emission, which we do by setting a threshold in the total intensity image, which in this case corresponds to three times the I noise level. The computation of the polarized intensity is specified by: <br />
<pre style="background-color: #E0FFFF;"><br />
expr='sqrt((IM1^2+IM2^2)/IM0[IM0>1.2e-3]^2)'<br />
</pre><br />
with the expression in square brackets setting the threshold in IM0 (the total intensity image). Note that IM0, IM1 and IM2 correspond to the three files listed in the ''imagename'' array, '''in that order'''. The order in which the different images are specified are once again critical.<br />
<br />
One can then view these various images using '''{{viewer}}'''. It is instructive to display the I, P and X images (total intensity, total linearly polarized intensity, and polarization position angle) together, to show how the polarized emission relates to the total intensity, and how the magnetic field is structured. We can do this using the viewer.<br />
* Begin by loading the linear polarization image in the viewer:<br />
<source lang="python"><br />
# In CASA<br />
viewer('3c391_ctm_spw0.pbcorP')<br />
</source><br />
* Next, load the pb-corrected total intensity image as a contour image. In the viewer panel, press the Open icon (the leftmost button in the top row of icons in the viewer). This will bring up a Load Data GUI showing all images and MS in the current directory. Select the total intensity image (3c391_ctm_spw0.pbcorI) and click the Contour Map button on the right hand side.<br />
* Finally, load the polarization position angle image (3c391_ctm_spw0.X) as a vector map.<br />
<br />
While we set the parameter ''polithresh'' when we created the position angle (X) image, a digression here is instructive in the use of Lattice Expression Language (LEL) Expressions. Had we not set this parameter, the position angle would have been derived for all pixels within the full IQUV image cube. There is only polarized emission from a limited subset of pixels within this image. Therefore, to avoid plotting vectors corresponding to the position angle of pure noise, we would now wish to select only the regions where the polarized intensity is brighter than some threshold value. To do this, we would use a LEL Expression in the Load Data GUI. For our chosen threshold of 0.4 mJy (the 5 sigma level in the pbcorP image), we would paste the expression: <br />
<pre style="background-color: #E0FFFF;"><br />
'3c391_ctm_spw0.X'['3c391_ctm_spw0.pbcorP'>0.0004]<br />
</pre><br />
into the LEL Expression box in the GUI, and click the Vector Map button. This would load the vectors only for regions where <math>P>0.4</math> mJy.<br />
<br />
While we now have all three images loaded into the viewer (the polarized intensity (3c391_ctm_spw0.pbcorP) in color, the total intensity (3c391_ctm_spw0.pbcorI) as a contour map, and the polarization position angle (3c391_ctm_spw0.X) as a vector map), we still wish to optimize the display for ease of interpretation.<br />
<br />
* Click the wrench icon to open a Data Display Options GUI. This will have 3 tabs, corresponding to the three images loaded (Figures 31A&#150;31C). <br />
* Change the image transfer function. Hold down the middle mouse button and move the mouse until the color scale is optimized for the display of the polarized intensity. You can change the colormap (e.g., to HotMetal1) in the first tab of the Data Display Options GUI.<br />
* Change the contour levels. Select the total intensity tab (3c391_ctm_spw0.pbcorI) (2<sup>nd</sup> tab) in the Data Display Options GUI. Change the relative contour levels from the default levels of [0.2,0.4,0.6,0.8,1.0] to powers of <math>\sqrt{2}</math>, including a couple of negative contours at the beginning to demonstrate the image quality. An appropriate set of levels might be [-1.414,-1,1,1.414,2,2.828,4,5.657,8,11.314,16,22.627,32,45.255,64]. These levels will multiply the Unit Contour Level, which we set at some multiple of the rms noise in the total intensity image. An appropriate value might be 0.004 Jy (<math>5\sigma</math> of pbcorI rms in the outer part of the mosaic). Choose a nice contour color (e.g., magenta).<br />
* Change the vector spacing and color and rotate the vectors. The polarization position angle as calculated is the electric vector position angle (EVPA). If we are interested in the orientation of the magnetic field, then for an optically thin source, the magnetic field orientation is perpendicular to the EVPA, so we must rotate the vectors by <math>90^{\circ}</math>. Select the vector image tab in the Data Display Options GUI (labeled as the LEL expression we entered in the Load Data GUI) and enter ''90'' in the '''Extra rotation''' field. If the vectors appear too densely packed on the image, change the spacing of the vectors by setting ''X-increment'' and ''Y-increment'' to a larger value (4 might be appropriate here). Finally, to be able to distinguish the vectors from the total intensity contours, change the color of the vectors by selecting a different ''Line color'' (green might be a good choice).<br />
<br />
Now that we have altered the display to our satisfaction, it remains only to zoom in to the region containing the emission. Close the animator tab in the viewer, and then drag out a rectangular region around the supernova remnant with your left mouse button. Double-click to zoom in to that region. This will give you a final image looking something like Figure 31D. Your image may very depending on how deep you cleaned.<br />
<br />
{|<br />
| [[Image:screenshot_3c391_viewer_menu_raster.png|200px|thumb|left|Figure 31A: data display options for P raster]]<br />
| [[Image:screenshot_3c391_viewer_menu_contour.png|200px|thumb|center|Figure 31B: data display options for total intensity contours]]<br />
| [[Image:screenshot_3c391_viewer_menu_vector.png|200px|thumb|center|Figure 31C: data display options for position angle vectors]]<br />
| [[Image:screenshot_3c391_viewer_allpol.png|200px|thumb|right|Figure 31D: final full-polarization image of 3C391]]<br />
|}<br />
<br />
== Spectral Index Imaging ==<br />
<br />
The spectral index, defined as the slope of the radio spectrum between two different frequencies, <math>\log(S_{\nu_1}/S_{\nu_2})/\log(\nu_1/\nu_2)</math>, is a useful analytical tool which can convey information about the emission mechanism, the optical depth of the source or the underlying energy distribution of synchrotron-radiating electrons.<br />
<br />
Having used '''{{immath}}''' to manipulate the polarization images, the reader should now have some familiarity with performing mathematical operations within CASA. Task '''{{immath}}''' also has a special mode for calculating the spectral index, ''mode='spix' ''. The two input images at different frequencies should be provided using the parameter ''imagename'' (which is a Python list). With this information, it is left as an exercise for the reader to create a spectral index map.<br />
<br />
The two input images could be the two different spectral windows from the 3C391 continuum data set (see below). If the higher-frequency spectral window (spw1) has not yet been reduced, then two images made with different channel ranges from the lower spectral window, spw0, should suffice. In this latter case, the extreme upper and lower channels are suggested, to provide a sufficient lever arm in frequency to measure a believable spectral index.<br />
<br />
'''Note:''' It would be desirable to produce the spectral index image through the first-order Taylor term expansion in the MFS imaging. This is unfortunately not available with mosaics in the current version of CASA. Stay tuned.<br />
<br />
== Self-Calibration ==<br />
<br />
Even after the initial calibration using the amplitude calibrator and the phase calibrator, there are likely to be residual phase and/or amplitude errors in the data. Self-calibration is the process of using an existing model, often constructed from imaging the data itself, provided that sufficient visibility data have been obtained. This is essentially always the case with the JVLA data, the system of equations is wildly over-constrained for the number of unknowns. <br />
<br />
More specifically, the observed visibility data on the <math>i</math>-<math>j</math> baseline can be modeled as <br />
<br />
<math><br />
V'_{ij} = G_i G^*_j V_{ij}<br />
</math><br />
<br />
where <math>G_i</math> is the complex gain for the <math>i^{\mathrm{th}}</math> antenna and <math>V_{ij}</math> is the true visibility. For an array of <math>N</math> antennas, at any given instant, there are <math>N(N-1)/2</math> visibility data, but only <math>N</math> gain factors. For an array with a reasonable number of antennas, <math>N</math> >~ 8, solutions to this set of coupled equations converge quickly.<br />
<br />
There is a small amount of discussion in the CASA Reference Manual on <br />
[http://casa.nrao.edu/docs/cookbook/casa_cookbook006.html#sec355 self calibration], but we have lectures on [https://science.nrao.edu/facilities/alma/naasc-workshops/nrao-cd-stsci/cde_selfcal.pdf Self-calibration] given at NRAO community days. In self-calibrating data, it is useful to keep in mind the structure of a Measurement Set: there are three columns of interest for an MS: the DATA column, the MODEL column, and the CORRECTED_DATA column. In normal usage, as part of the initial split, the CORRECTED_DATA column is set equal to the DATA column. The self-calibration procedure is then: <br />
<br />
* Produce an image ('''{{clean}}''') using the CORRECTED_DATA column.<br />
* Derive a series of gain corrections ('''{{gaincal}}''') by comparing the DATA columns and the Fourier transform of the image, which is stored in the MODEL column. These corrections are stored in an external table.<br />
* Apply these corrections ('''{{applycal}}''') to the DATA column, to form a new CORRECTED_DATA column, ''overwriting'' the previous contents of CORRECTED_DATA.<br />
<br />
The following example begins with the standard data set, 3c391_ctm_mosaic_spw0.ms (resulting from the steps above). From this we will make an I-only multiscale image (3c391_ctm_spw0_I.image)&#151;and in particular the model (3c391_ctm_spw0_I.model)&#151;to generate a series of gain corrections that will be stored in 3C391_ctm_mosaic_spw0.selfcal1. These gain corrections are then applied to the data to form a set of self-calibrated data, and a new image is then formed (3c391_ctm_spw0_IQUV_selfcal1.image). Note that in the clean before the self-cal, it is important that we only image I so that any cleaned polarization does not affect the gaincal. In CASA 4.0 we first use '''delmod''' on the MS to get rid of the previous polarized model.<br />
<source lang="python"><br />
#In CASA<br />
delmod('3c391_ctm_mosaic_spw0.ms')<br />
#<br />
clean(vis='3c391_ctm_mosaic_spw0.ms',imagename='3c391_ctm_spw0_ms_I',<br />
field='',spw='',<br />
mode='mfs',<br />
niter=25000,<br />
gain=0.1,threshold='1mJy',<br />
psfmode='clark',<br />
imagermode='mosaic',ftmachine='mosaic',<br />
multiscale=[0, 6, 18, 54],smallscalebias=0.9,<br />
interactive=True,<br />
imsize=[480,480],cell=['2.5arcsec','2.5arcsec'],<br />
stokes='I',<br />
weighting='briggs',robust=0.5,<br />
usescratch=False)<br />
</source><br />
You shouldn't have to clean extremely deep. You want to be sure to capture as much of the source total flux density as possible, but not include low-level questionable features or sub-structure (ripples) that might be due to calibration or clean artifacts. <br />
<br />
After you are happy with the image:<br />
<source lang="python"><br />
#In CASA<br />
gaincal(vis='3c391_ctm_mosaic_spw0.ms',caltable='3c391_ctm_mosaic_spw0.selfcal1',<br />
field='',spw='',selectdata=False,<br />
solint='30s',refant='ea21',minblperant=4,minsnr=3,<br />
gaintype='G',calmode='p',append=False)<br />
#<br />
applycal(vis='3c391_ctm_mosaic_spw0.ms',<br />
field='',spw='',selectdata=False,<br />
gaintable= ['3c391_ctm_mosaic_spw0.selfcal1'],gainfield=[''],interp=['nearest'],<br />
calwt=[False],applymode='calflag')<br />
</source><br />
The ''CORRECTED_DATA'' column of the MS now contains the self-calibrated visibilities, they will now be used by clean. Note that the <br />
'''{{gaincal}}''' step will report a number of solutions with insufficient SNR. By default, with parameter ''applymode='calflag' '', data with no good solutions will be flagged by '''{{applycal}}'''; in this case you will see it report the flagged fraction increasing from 28% to 43%. This may or may not be a good thing. You can control the action of '''{{applycal}}''' in this regard by changing the value of parameter ''applymode'' by setting ''applymode='calflagstrict' ''will be even more stringent about flagging things without valid calibration, while ''applymode='calonly' ''will calibrate those with solutions while passing through data without unchanged. Note that you can see ahead of time what applycal will do by running with ''applymode='trial' ''which will do the reporting but nothing else.<br />
<br />
{| style="background:#98FB98"<br />
|-<br />
| '''Questions for the Advanced Student:'''<br />
* Does allowing applycal to flag the data give better images?<br />
* Or, does using ''applymode='calonly' ''give improved results?<br />
|-<br />
|}<br />
<br />
If you planned on doing multiple iterations of self-cal, you would do another I-only image (e.g., ''3c391_ctm_spw0_ms_I_selfcal1'') as that is what is needed for the next step. If you want to just go ahead and see what this selfcal has done, make a full IQUV cube:<br />
<source lang="python"><br />
#In CASA<br />
clean(vis='3c391_ctm_mosaic_spw0.ms',imagename='3c391_ctm_spw0_IQUV_selfcal1',<br />
field='',spw='',<br />
mode='mfs',<br />
niter=25000,<br />
gain=0.1,threshold='1mJy',<br />
psfmode='clarkstokes',<br />
imagermode='mosaic',ftmachine='mosaic',<br />
multiscale=[0, 6, 18, 54],smallscalebias=0.9,<br />
interactive=True,<br />
imsize=[480,480],cell=['2.5arcsec','2.5arcsec'],<br />
stokes='IQUV',<br />
weighting='briggs',robust=0.5,<br />
usescratch=False)<br />
</source><br />
<br />
{| style="background:#98FB98"<br />
|-<br />
| '''Questions for the Advanced Student:'''<br />
* Is this better than the original IQUV image cube? By how much?<br />
* Can you make a difference image (between the original and selfcal1 images) using '''{{immath}}'''?<br />
* How big were the phase changes made by the calibration? Were there specific antennas with larger errors?<br />
|-<br />
|}<br />
<br />
Commonly, this self-cal procedure is applied multiple times.<br />
The number of iterations is determined by a combination of the data quality and number of antennas in the array, the structure of the source, the extent to which the original self-calibration assumptions are valid, and the user's patience. With reference to the original self-calibration equation above, if the observed visibility data cannot be modeled well by this equation, no amount of self-calibration will help. A not-uncommon limitation for moderately high dynamic range imaging is that there may be ''>baseline-based'' factors that modify the true visibility. If the corruptions to the true visibility cannot be modeled as antenna-based, as they are above, self-calibration won't help.<br />
<br />
Self-calibration requires experimentation. Do not be afraid to dump an image, or even a set of gain corrections, <br />
change something and try again. Having said that, here are several general comments or guidelines:<br />
<br />
* Bookkeeping is important! Suppose one conducts 9 iterations of self-calibration. Will it be possible to remember one month later (or maybe even one week later!) which set of gain corrections and images are which? In the example above, the descriptor'' 'selfcal1' ''is attached to various files to help keep straight which is what. Successive iterations of self-cal could then be'' 'selfcal2' '','' 'selfcal3' '', etc.<br />
<br />
* Care is required in the setting of ''imagename''. If one has an image that already exists, CASA will continue cleaning it (if it can), which is almost certainly not what one wants during self-calibration. Rather one wants a unique ''imagename'' for each pass of self-calibration.<br />
<br />
* A common metric for self-calibration is whether the image ''dynamic range'' (= max/rms) has improved. An improvement of 10% is quite acceptable.<br />
<br />
* Be careful when making images and setting clean regions or masks. Self-calibration assumes that the model is perfect. If one cleans a noise bump, self-calibration will quite happily try to adjust the gains so that the CORRECTED_DATA describe a source at the location of the noise bump. As the author demonstrated to himself during the writing of his thesis, it is quite possible to take completely noisy data and manufacture a source. It is far better to exclude some feature of a source or a weak source from initial cleaning and conduct another round of self-calibration than to create an artificial source. If a real source is excluded from initial cleaning, it will continue to be present in subsequent iterations of self-calibration; if it's not a real source, one probably isn't interested in it anyway.<br />
<br />
* Start self-calibration with phase-only solutions (parameter ''calmode='p' ''in '''{{gaincal}}'''). As discussed in the [http://adsabs.harvard.edu/abs/1989ASPC....6..287P High Dynamic Range Imaging] lecture, a phase error of 20 deg is as bad as an amplitude error of 10%.<br />
<br />
* In initial rounds of self-calibration, consider solution intervals longer than the nominal sampling time (parameter ''solint'' in '''{{gaincal}}''') and/or lower signal-to-noise ratio thresholds (parameter ''minsnr'' in '''{{gaincal}}'''). Depending upon the frequency and configuration and fidelity of the model image, it can be quite reasonable to start with ''solint='30s' ''or'' solint='60s' ''and/or ''minsnr=3 (''or even lower). One might also want to consider specifying a uvrange, if, for example, the field has structure on large scales (small <math>u</math>-<math>v</math>) that is not well represented by the current image.<br />
<br />
* The task '''{{applycal}}''' will flag data with no good calibration solutions. During the initial self-calibration steps, this flagging may be excessive. If so, one can restore the flags to the state right before running applycal by using the task '''{{flagmanager}}'''.<br />
<br />
* You can track the agreement between the DATA, CORRECTED_DATA, and MODEL in '''{{plotms}}'''. The options in Axes tab allows one to select which column is to be plotted. If the MODEL agrees well with the CORRECTED_DATA, one can use shorter solint and/or higher minsnr values.<br />
<br />
* You should consider examining the solutions from '''{{gaincal}}''' by using '''{{plotcal}}''' in order to assure that the corrections are sensible. Smoothly varying phases are good, jumps are usually not. (However, because the phases are often plotted &plusmn;180 degrees, there can be apparent jumps if the phases are very near &#043;180 deg or &minus;180 deg.)<br />
<br />
* In the case of a mosaic, such as here, one should also verify that the solutions are of equal quality for all of the fields.<br />
<br />
== On Your Own: 3C391 second frequency and G93.3+6.9 ==<br />
<br />
Now that you have run through spw 0 of 3C391, you are ready to strike off on your own with other datasets. We have provided two options here, described below. The first option is simplest as it is the same object using a different spectral window; for a more rewarding challenge try the L-band dataset on G93.3+6.9.<br />
<br />
You can find the data in the [http://casa.nrao.edu/Data/Synth2010/AdvancedEVLAcont.tgz CASA repository]. Both datasets&#151;3C391 spw 1 (at 7.5 GHz) and Supernova Remnant G93.3+6.9 at L-band&#151;are contained in this tarball. To keep their sizes small, these MSs do not have the scratch columns pre-made, so you can do an initial '''{{clearcal}}''' to force the creation of the scratch colums or wait until your first calibration task does it for you.<br />
<br />
1. 3C391 spw 1 (at 7.5 GHz)<br />
<br />
This is the second spectral window split off from the 3C391 dataset. You can process this as you did the first time, but beware of RFI in this band. You will have to avoid it through channel ranges and/or edit it out. Once you have processed this data and imaged it, you can combine those images in '''{{immath}}''' to make a spectral index image (see above) or combine the two calibrated MSs in '''{{clean}}''' to make a deeper MFS image (this might be tricky). You can also look for signs of Faraday Rotation by searching for a polarization angle change between the two spw. Can you derive the rotation measure (RM)?<br />
<br />
2. Supernova Remnant G93.3+6.9 at L-band<br />
<br />
This is data taken at L-band of an entirely different Supernova Remnant, centered near 1400 MHz. You should be able to process this data in a very similar manner to the C-band data on 3C391. Note that we are not telling you what you will see in the image ahead of time. Here are some data reduction hints to help you along:<br />
<br />
* There is strong RFI in this spectral window of the original 2 spw dataset. You will need to find it (e.g., using '''{{plotms}}''') and avoid it in imaging. You can also flag those channels using '''{{flagdata}}''', but this is not necessary. Note that there is a single baseline that shows very strong interference, see if you can find it. You can flag it using the baseline syntax in '''{{flagdata}}''' (e.g., parameter ''antenna='ea0x&ea0y' '').<br />
<br />
* We have not edited out bad or dead antennas for you (unlike in 3C391). You will need to find these using '''{{plotms}}''' and then '''{{flagdata}}''' them. One helpful '''{{plotms}}''' trick is to set parameter ''antenna='ea01' ''and pick a few channels (like'' spw='0:30~33' '') and a single scan (e.g., ''scan='2~3' '') and plot the amp versus Antenna2 on the X-axis. You should see the bad antennas (the low ones). As a check set'' antenna='ea02' ''and repeat. Is it the same?<br />
<br />
* In spite of RFI, the antenna-based calibration is remarkably resilient to moderate to low RFI contamination (which tends to be baseline-based). So rather than flagging channels with RFI, you might try going ahead with calibration and seeing if the solutions make sense. We were able to calibrate this data without flagging channels (only getting the bad baseline noted above).<br />
<br />
* There is no observation of a flux or polarization angle calibrator like J1331+3030. You need to use '''{{setjy}}''' to set the I flux of the gain calibrator. We use the approximate flux density of 5.8 Jy for J2038+5119.<br />
<br />
* When it comes time to calibrate the polarization leakage, we are in good shape since J2038+5119 was observed through a range of parallactic angle (use '''{{plotms}}''' to plot versus ParAngle). Use parameter'' poltype='Df+QU' ''to solve for leakage and the unknown polarization of this source. We do not know the true polarization angle of this source, so before doing parameter'' poltype='Xf' '',use '''{{setjy}}''' to set the Q flux to 5.8Jy * fractional pol (determined in leakage polcal run). This will at least align the polarization when you image it.<br />
<br />
* The L-band field of view is much larger than at C-band. From the [https://science.nrao.edu/facilities/vla/docs/manuals/oss VLA Observational Status Summary (OSS)] the resolution should be around 45" in D-config. Use a cellsize of 15" or smaller. What is the primary beam of the JVLA at 1.4MHz? How big should you make your image? <br />
<br />
* As you clean you will see faint sources all over the field; welcome to L-band imaging. This SNR has lots of structure&#151;try both standard and multi-scale clean.<br />
<br />
<br />
<!--<br />
== 3C391 Line studies ==<br />
<br />
A second data set on 3C391 was taken, this time in OSRO-2 mode, centered on the formaldehyde line at 4829.66 MHz, to search for absorption against the supernova remnant. Again, we made a 7-pointing mosaic, with the same pointing centers as the continuum data set. If you have also already gone through the [[EVLA high frequency Spectral Line tutorial - IRC+10216]] tutorial, then having reduced the continuum data in this tutorial, you should be able to combine what you have learned from these two tutorials to reduce this spectral line study of 3C391. Should you wish to do so, a 10-s averaged data set, with some pre-flagging done, is available from the [http://casa.nrao.edu/Data/EVLA/3C391/3c391_line_10s_summerschool.ms.tgz CASA repository].<br />
--><br />
<br />
Questions about this tutorial? Please contact the [https://science.nrao.edu/observing/helpdesk NRAO Helpdesk].<br />
<br />
[[Main Page | &#8629; '''CASAguides''']]<br />
<br />
<!-- --content update: [[User: Tperreau| Tony Perreault]] (4.5.2, 2016/03/08)<br />
--content update: [[User: HMedlin| Heidi Medlin]] (4.5.2, 2016/03/10)<br />
--><br />
<br />
<br />
{{Checked 4.5.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Imaging-CASA4.5.2&diff=19052VLA CASA Imaging-CASA4.5.22016-04-21T00:04:04Z<p>Emomjian: /* Dirty Image */</p>
<hr />
<div><br />
== Imaging ==<br />
<br />
This tutorial provides guidance on imaging procedures in CASA. <br />
<br />
We will be utilizing data taken with the Karl G. Jansky Very Large Array, of a supernova remnant G055.7+3.4.. The data were taken on August 23, 2010, in the first D-configuration for which the new wide-band capabilities of the WIDAR (Wideband Interferometric Digital ARchitecture) correlator were available. The 8-hour-long observation includes all available 1 GHz of bandwidth in L-band, from 1-2 GHz in frequency. <br />
<br />
We will skip the calibration process in this guide, as examples of calibration can be found in several other guides, including the <br />
[https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391 EVLA Continuum Tutorial 3C391] and <br />
[https://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 EVLA high frequency Spectral Line tutorial - IRC+10216] guides.<br />
<br />
A copy of the calibrated data <font color=green>(1.2GB)</font> can be downloaded from [http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.calib.tar.gz http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.calib.tar.gz] <br />
<br />
Your first step will be to unzip and untar the file in a terminal (before you start CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf SNR_G55_10s.calib.tar.gz<br />
</source><br />
<br />
Then start casa as usual via the '''casa''' command, which will bring up the ipython interface and launches the logger.<br />
<br />
== The CLEAN Algorithm ==<br />
<br />
[[Image:CLEAN_Cycle.png|500px|thumb|right|Fig. 1: The CLEAN major and minor cycles, indicating the steps undertaken during gridding, projection algorithms, and creation of images.]]<br />
<br />
The CLEAN algorithm, developed by J. Högbom (1974) enabled the synthesis of complex objects, even if they have relatively poor Fourier uv-plane coverage. Poor coverage occurs with partial earth rotation synthesis, or with arrays composed of few antennas. The "dirty" image is formed by a simple Fourier inversion of the sampled visibility data, with each point on the sky being represented by a suitably scaled and centered PSF (Point Spread Function, or dirty beam, which is the Fourier inversion of the visibility (u,v) coverage). This algorithm attempts to interpolate from the measured (u,v) points across gaps in the (u,v) coverage. It, in short, provides solutions to the convolution equation by representing radio sources by a number of point sources in an empty field. The brightest points are found by performing a cross-correlation between the dirty image, and the PSF. The brightest parts are subtracted, and the process is repeated again for the next brighter sources. Variants of CLEAN, such as multi-scale CLEAN, take into account extended kernels which may be better suited for extended objects.<br />
<br />
For single pointings, CASA uses the Cotton-Schwab cleaning algorithm in the task {{clean}} (''imagermode='csclean'''), which breaks the process into major and minor cycles (see Fig. 1). To start with, the visibilities are gridded, weighted, and Fourier transformed to create a (dirty) image. The minor cycle then operates in the image domain to find the clean components that are added to the clean model. The image is Fourier transformed back to the visibility domain, degridded, and the clean model is subtracted. This creates a new residual that is then gridded, weighted, and FFT'ed again to the image domain for the next iteration. The gridding, FFT, degridding, and subtraction processes form the major cycle. <br />
<br />
This iterative process is continued until a stopping criterion is reached, such as a maximum number of clean components, or a flux threshold in the residual image.<br />
<br />
In CASA {{clean}}, two versions of the PSF can be used (parameter ''psfmode''): ''hogbom'' uses the full sized PSF for subtraction. This is a thorough but slow method. ''psfmode='clark''' uses a smaller beam patch, which increases the speed. The patch size and length of the minor cycle are internally chosen such that clean converges well without giving up the speed improvement. It is thus the default option in {{clean}}. <br />
<br />
<!--<br />
''' 1. Högbom Algorithm '''<br /><br />
This algorithm will initially find the strength and position of a peak in a dirty image, subtract it from the dirty image, record this position and maginitude, and repeat for further peaks. The remainder of the dirty image is known as the residuals. <br />
<br />
The accumulated point sources, now residing in a model, is convolved with an idealized CLEAN beam (usually a Gaussian fitted to the central lobe of the dirty beam), creating a CLEAN image. As the final step, the residuals of the dirty image are then added to the CLEAN image. <br />
<br />
''' 2. Clark Algorithm '''<br /><br />
Clark (1980), developed a FFT-based CLEAN algorithm, which more efficiently shifts and scales the dirty beam by approximating the position and strength of components using a small patch of the dirty beam. This algorithm is the default within the {{clean}} task, which involves major and minor cycles. <br />
<br />
The algorithm will first select a beam patch, which will include the highest exterior sidelobes. Points are then selected from the dirty image, which are up to a fraction of the image peak, and are greater than the highest exterior sidelobe of the beam. It will then conduct a list-based Högbom CLEAN, creating a model and convolution with an idealized CLEAN beam. This process is the minor cycle. <br />
<br />
The major cycle involves transforming the point source model via a FFT (Fast-Fourier Transform), mutiplying this by the weight sampling function (more on this below), and transformed back. This is then subtracted from the dirty image, creating your CLEAN image. The process is then repeated with subsequent minor cycles. <br />
<br />
''' 3. Cotton-Schwab Algorithm '''<br /><br />
This is the default imager mode (''csclean''), and is a variant of the Clark algorith in which the major cycle involves the subtraction of CLEAN components of ungridded visibility data. This allows the removal of gridding errors, as well as noise. One advantage is its ability to image and clean many seperate fields simultaneously. Fields are cleaned independently in the minor cycle, and components from all fields cleaned together in the major cycles.<br />
<br />
This algorithm is faster than the Clark algorithm, except when dealing with a large number of visibility samples, due to the re-gridding process it undergoes. It is most useful in cleaning sensitive high-resolution images at lower frequencies where a number of confusing sources are within the primary beam. <br />
--> <br />
In a final step, {{clean}} derives a Gaussian fit to the inner part of the PSF, which defines the clean beam. The clean model is then convolved with the clean beam and added to the last residual image to create the final image. <br />
<br />
Note that the CASA team currently develops a refactored clean task, called [http://casa.nrao.edu/docs/TaskRef/tclean-task.html tclean]. It has a better interface and provides new algorithms as well as more combinations between imaging algorithms. tclean also includes software to parallelize the computations in a multi-processor environment. Eventually, tclean will replace the current {{clean}} task. For this guide, however, we will stick with the original {{clean}} task, as tclean is still in the development and testing phase. Nevertheless, the reader is encouraged to try tclean and send us feedback through the [https://help.nrao.edu/ NRAO helpdesk].<br />
<br />
For more details on imaging and deconvolution, we refer to the Astronomical Society of the Pacific Conference Series book entitled [http://www.aspbooks.org/a/volumes/table_of_contents/?book_id=292 Synthesis Imaging in Radio Astronomy II]. The chapter on [http://www.aspbooks.org/a/volumes/article_details/?paper_id=17942 Deconvolution] may prove helpful. In addition, imaging presentations are available on the [https://science.nrao.edu/science/meetings/2014/14th-synthesis-imaging-workshop/lectures Synthesis Imaging Workshop] and [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/program VLA Data Reduction Workshop] webpages. The [https://casa.nrao.edu/docs/cookbook/index.html CASA cookbook] chapter on [https://casa.nrao.edu/docs/cookbook/casa_cookbook006.html Synthesis Imaging] provides a wealth of information on the CASA implementation of {{clean}} and related tasks.<br />
<br />
Finally, we like to refer to the [https://science.nrao.edu/facilities/vla/docs/manuals/oss VLA Observational Status Summary] and the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide Guide to Observing with the VLA] for information on the VLA capabilities and observing strategies.<br />
<br />
== Weights and Tapering ==<br />
<br />
[[Image:SNR-G55-uvcoverage-CASA4.5.2.png|400px|thumb|right|Fig. 2: u,v coverage for the 8-hour observation of the supernova remnant G055.7+3.4]]<br />
<br />
When imaging data, a map is created associating the visibilities with the image. <br />
The sampling function, which is a function of the visibilities, is modified by a weight function that defines the shape and size of the PSF. Weighting therefore provides some control over the spatial resolution and the surface brightness sensitivity of the map, where either direction can be emphasized.<br />
<!-- <math> S(u,v) \to S(u,v)W(u,v) </math>. <br />
Due to the fact that each VLA antenna performs slightly differently, different weights should be applied to each antenna. Therefore, the weight column in the data table reflects how much weight each corrected data sample should receive. <br />
--><br />
<br />
There are three main weighting schemes (see also Table 1): <br />
<br />
1) '''Natural''' weighting: The natural weights data based on their rms only. All visibility weights in a cell are being summed. More visibilities in a cell will thus increase the cell's weight, which will emphasize the shorter baselines. Natural weighting therefore results in a larger PSF, better surface brightness sensitivity but also a degraded resolution. <br />
<br />
2) '''Uniform''' weighting: The weights are first gridded as in natural weighting but then each cell is corrected such that the weights are independent of the number of visibilities inside. Compared to natural weighting, uniform weighting emphasizes the longer baselines. Consequently, the PSF is smaller, which results in a better spatial resolution of the image. At the same time, however, the surface brightness sensitivity is reduced compared to natural weighting. The 'uniform' weighting of the baselines is a better representation of the uv-coverage and sidelobes are more suppressed. <br />
<br />
3) '''Briggs''' weighting: This scheme provides a compromise between natural and uniform weighting. The transition can be controlled with the ''robust'' parameter where ''robust=-2'' is close to uniform and ''robust=2'' close to natural weighting. <br />
Briggs weighting therefore offers a compromise for between spatial resolution and surface brightness sensitivity, and ''robust'' values near zero are typically being used.<br />
<br />
<!--<br />
For a brief intro to the different clean algorithms, as well as other deconvolution and imaging information, please see the website kept by Urvashi R.V. [http://www.aoc.nrao.edu/~rurvashi/ImagingAlgorithmsInCasa/node2.html#SECTION00223000000000000000 here].<br />
<br />
<br />
There are different options for <math> W(u,v) </math> (parameter ''weighting'' in {{clean}}):<br />
<br />
'''1. Natural''': For the CASA parameter ''weighting=’natural’'', visibilities are weighted only by the data weights, which are calculated during filling and calibration and should be equal to the inverse noise variance on that visibility. Imaging weight <math>W_i</math> of sample <math>i</math> is given by <math>W_i = \omega_i = 1/\sigma_i^2</math><br />
<br />
where the data weight <math>\omega_i</math> is determined from <math>\sigma_i</math>, the rms noise on visibility <math>i</math>. When data is gridded into the same uv-cell for imaging, the weights are summed, and thus a higher uv density results in higher imaging weights.<br />
<br />
The weight function can be described as <math> W(u,v) = 1/ \sigma^2 </math>, where <math> \sigma^2 </math> is the noise variance. Natural weighting will maximize point source sensitivity, and provide the lowest rms noise within an image, as well as the highest signal-to-noise. It will also generally give more weight to short baselines, thus the synthesized beam will be large. This form of weighting is the default within the clean task. <br />
<br />
'''2. Uniform''': For ''weighting = ’uniform’'', the data weights are calculated as in ’natural’ weighting. The data is then gridded to a number of cells in the uv-plane, and after all data is gridded the uv-cells are re-weighted to have 'uniform' imaging weights. This pumps up the influence on the image of data with low weights (they are multiplied up to be the same as for the highest weighted data), which sharpens resolution and reduces the sidelobe level in the field-of-view, but increases the rms image noise. <br />
<br />
For uniform weighting, we first grid the inverse variance <math>\omega_i</math> for all selected data onto a grid with uv cell-size given by <math>2/FOV</math> where <math>FOV</math> is the specified field of view (defaults to the image field of view). This forms the gridded weights <math>W_k</math>. The weight of the <math>i</math>-th sample is then:<math>W_i=\omega_i/W_k</math><br />
<br />
The weight function can be described as <math> W(u,v) = W(u,v) / W_k </math>, where <math> W_k </math> represents the local density of (u,v) points, otherwise known as the gridded weights. This form of weighting will increase the influence of data with lower weight, filling the (u,v) plane more uniformly, thereby reducing sidelobe levels in the field-of-view, but increasing the rms image noise. More weight is given to long baselines, therefore increasing angular resolution. Point source sensitivity is degraded due to the downweighting of some data. <br />
<br />
'''3. Briggs''': A flexible weighting scheme, that is a variant of uniform, and avoids giving too much weight to (u,v) points with a low natural weight. Weight function can be described as <br />
<math> W(u,v) = 1/ \sqrt{1+S_N^2/S_{thresh}^2} </math>, where <math> S_N </math> is the natural weight of the cell, <math> S_{thresh} </math> is a <br />
threshold. <br />
A high threshold will go to a natural weight, where as a low threshold will go to a uniform weight. This form of weighting also has adjustable parameters. The ''robust'' parameter will give variation between resolution and maximum point source sensitivity. It's value can range from -2.0 (close to uniform weight) to 2.0 (close to natural weight). By default, the parameter is set to 0.0, which gives a good trade-off. <br />
<br /> <br />
--><br />
<br />
[[Image:Weight_Tapering_table.png|450px|thumb|right|Table 1: Table summarizing the effects of using weights and tapering.]]<br />
'''Tapering''': In conjunction with the above weighting schemes, one can specify the ''uvtaper'' parameter within {{clean}}, which will control the radial weighting of visibilities, in the uv-plane. This in effect, reduces the visibilities, with weights decreasing as a function of uv-radius. The taper in {{clean}} is an elliptical Gaussian function. <br />
<!--<br />
The tapering will apodize, or filter/change the shape of the weight function (which is itself a Gaussian), which can be expressed as: <br /><br />
<math> W(u,v) = e^{-(u^2+v^2)/t^2} </math>, where t is the adjustable tapering parameter. <br />
--><br />
Tapering can smooth the image plane, give more weight to short baselines, but in turn degrade angular resolution. This process will increase the surface brightness sensitivity of the data. Aggressive tapering, however, can downweight some of the antennas and baselines to a degree where they are essentially not being considered anymore. The point source sensitivity will thus be decreased and in extreme cases the surface brightness sensitivity will suffer, too. <br />
<!--<br />
Due to the downweight of some data, point source sensitivity can be degraded, as some antennas and baselines will suppressed and, in the extreme case, effectively removed from the data averaging. If your observation was sampled by short baselines, tapering may improve sensitivity to extended structures. <br />
--><br />
<br />
We refer to the [https://casa.nrao.edu/docs/cookbook/casa_cookbook006.html CASA Cookbook Synthesis Imaging] chapter for the details of the weighting implementation in CASA's {{clean}}.<br />
<br />
== Primary and Synthesized Beam ==<br />
<br />
The primary beam of a single antenna defines the sensitivity across the field of view. For the VLA antennas, the main beam can be approximated by a Gaussian with a FWHM equal to <math>90*\lambda_{cm}</math> or <math>45/ \nu_{GHz}</math>. But note that there are sidelobes beyond the Gaussian kernel that are sensitive to bright sources (see below). Taking our observed frequency to be the middle of the band, 1.5GHz, our primary beam will be around 30 arcmin. <br />
<br />
If your science goal is to image a source, or field of view that is significantly larger than the FWHM of the VLA primary beam, then creating a mosaic from a number of pointings is usually the preferred method. For a tutorial on mosaicing, see the [https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391 3C391 tutorial]. In the following, however, we will discuss methods to image large maps from single pointing data. <br />
<br />
Since our observation was taken in D-configuration and we can check the [https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/resolution Observational Status Summary]'s section on VLA resolution to find that the synthesized beam will be around 46 arcsec. We want to oversample the synthesized beam by a factor of around five, so we will use a cell size of 8 arcsec. <br />
<br />
Our field contains bright point sources significantly outside the primary beam. We therefore create images that are 170 arcminutes on a side ([Image Size * Cell Size]*[1arcmin / 60arsec]), or almost 6x the size of the primary beam, catching the first and second sidelobe. This is ideal for showcasing both the problems inherent in such wide-band, wide-field imaging, as well as some of the solutions currently available in CASA to deal with these issues. Note that the execution time of {{clean}} depends on the image sizes. Large images generally take more computing time. There are some values, however, that are computationally not advisable. The logger output will then show a recommendation for the next larger but faster image size. As a rule of thumb we recommend image sizes <math>2^n*10</math>, e.g. 160, 1280, etc. <br />
<br />
It's worth considering why we are even interested in sources which are far outside the primary beam. This is mainly due to the fact that the VLA, with its wide bandwidth capabilities, is quite sensitive even far from phase center -- for example, at our observing frequencies in L-band, the primary beam gain is as much as 10% around 1 degree away. That means that any imaging errors for these far-away sources will have a significant impact on the image rms at phase center. The error associated with sidelobes emerging from a source at distance R can be parametrized as:<br />
<br />
<math><br />
\Delta(S) = S(R) \times PB(R) \times PSF(R)<br />
</math><br />
<br />
So, for R = 1 degree, source flux S(R) = 1 Jy, <math>\Delta(S)</math> = 1 mJy − 100 <math>{\mu}</math>Jy. Clearly, this will be a source of significant error.<br />
<br />
Cleaning far away from the phase center may thus become important in the vicinity of strong sources, in particular at the lower frequency bands where non-thermal emission dominates.<br />
<br />
== Clean Output Images ==<br />
<br />
As a result of the CLEAN algorithm, {{clean}} will create a number of output images. For an ''imagename='<imagename>''', this would be:<br />
<br />
'''<imagename>.image''' the residual + the model convolved with the clean beam. This is the final image (unit: Jy/clean beam). <br><br />
'''<imagename>.residual''' the residual after subtracting the clean model (unit: Jy/dirty beam).<br><br />
'''<imagename>.model''' the clean model, not convolved (unit: Jy/pixel).<br><br />
'''<imagename>.psf''' the point-spread function aka dirty beam<br><br />
'''<imagename>.flux''' the normalized sensitivity map. For single pointings this corresponds to the primary beam. <br><br />
<br />
Additional images will be created for specific algorithms like multi-term frequency synthesis or mosaicking. <br />
<br />
Note: If an image file is present in the working directory and the same name is provided in ''imagename'', {{clean}} will use that image (in particular the residual and model image) as a starting point for further cleaning. If you want a fresh run of {{clean}}, first remove all images of that name using 'rmtables()':<br />
<br />
<source lang="python"><br />
# In CASA<br />
rmtables('<imagename>.*')<br />
</source><br />
<br />
This method is preferable over 'rm -rf' as it also clears the cache.<br />
<br />
'''Note that interrupting {{clean}} by Ctrl+C may corrupt your visibilities - you may be better off choosing to let {{clean}} finish. We are currently implementing a command that will nicely exit to prevent this from happening, but for the moment try to avoid Ctrl+C.'''<br />
<br />
== Dirty Image ==<br />
<br />
[[Image:SN_G55_10s.dirty.image.psf.png|400px|thumb|right|Fig. 3: A dirty image of the supernova remnant G55.7+3.4 in greyscale, with apparent sidelobes (left), and the point spread function (PSF), also known as the dirty beam (right).]]<br />
<br />
First, we will create a dirty image (Fig. 3) to see the improvements as we step through several cleaning algorithms and parameters. The dirty image is the true image on the sky, convolved with the dirty beam (PSF). We will do this by running {{clean}} with niter=0, which will not perform any minor cycles. <br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.dirty', <br />
imsize=1280, cell='8arcsec', interactive=False, niter=0, <br />
stokes='I', usescratch=F)<br />
<br />
viewer('SNR_G55_10s.dirty.image')<br />
</source><br />
<br />
* ''imagermode='csclean''': use the Cotton-Schwab clean algorithm<br />
<br />
* ''imagename='SNR_G55_10s.dirty''': the root filename used for the various {{clean}} outputs. <br />
<br />
* ''imsize=1280'': the image size in number of pixels. A single value will result in a square image.<br />
<br />
* ''cell='8arcsec''': the size of one pixel; again, entering a single value will result in a square pixel size.<br />
<br />
* ''niter=0'': this controls the number of iterations {{clean}} will do in the minor cycle. Since this is a dirty image, we set it to 0.<br />
<br />
* ''interactive=False'': we will let {{clean}} use the entire field for placing model components. Alternatively, you could try using interactive=True, and create regions to constrain where components will be placed. However, this is a very complex field, and creating a region for every bit of diffuse emission as well as each point source can quickly become tedious. For a tutorial that covers more of an interactive clean, please see [https://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 tutorial.]<br />
<br />
* ''usescratch=F'': do not write the model visibilities to the model data column (only needed for self-calibration)<br />
<br />
* ''stokes='I''': since we have not done any polarization calibration, we only create a total-intensity image. For using CLEAN while including various Stoke's Parameters, please see the [https://casaguides.nrao.edu/index.php/EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 CASA guide.]<br />
Note that the clean beam is only defined after some clean iterations. The dirty image has therefore no beam size specified in the header and the PSF image is the representation of the response of the array to a point source.<br />
<br />
== Regular CLEAN & RMS Noise ==<br />
[[Image:SNR_G55_10s.niter.1K_vs_10K.png|400px|thumb|right|Fig. 4: Regular run of CLEAN, with niter=1000 (left), compared to niter=10000 (right).]]<br />
<br />
Now, we will create a regular clean image which uses mostly default values to see how deconvolution improves the image quality. The first run of {{clean}} will use a fixed number of minor cycle iterations of''niter=1000'' (default is 500), the second will have ''niter=10000''. Note that you may have to play with the image color map and brightness/contrast to get a better view of the image details. <br />
<br />
<source lang="python"><br />
# In CASA. Create default clean image.<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.Reg.Clean.niter1K', <br />
imsize=1280, cell='8arcsec', niter=1000, interactive=False)<br />
<br />
viewer('SNR_G55_10s.Reg.Clean.niter1K.image')<br />
</source><br />
<br />
The logger indicates that the image was obtained in two major cycles and some improvements over the dirty image are visible. But clearly we have not cleaned deep enough yet; the image still has many sidelobes, and an inspection of the residual image shows that it still contains source flux and structure. So let's increase the ''niter'' value to 10,000 and compare the images.<br />
<br />
<source lang="python"><br />
# In CASA. Create default clean image with niter = 10000<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.Reg.Clean.niter10K', <br />
imsize=1280, cell='8arcsec', niter=10000, interactive=False)<br />
<br />
viewer('SNR_G55_10s.Reg.Clean.niter10K.image')<br />
</source><br />
<br />
[[Image:SNR_G55_10s.rms.screen.png|400px|thumb|right|Fig. 5: Attempting to find the lowest rms value within the CLEAN'ed image using niter=1000, in order to calculate our threshold.]]<br />
<br />
As we can see from the resulting images, increasing the niter values (minor cycles) improves our image by reducing prominent sidelobes significantly. One could now further increase the ''niter'' parameter until the residuals are down to an acceptable level. To determine the number of iterations, one needs to keep in mind that {{clean}} will fail to converge once it starts cleaning too deeply into the noise. At that point, the cleaned flux and the peak residual flux values will start to oscillate as the number of iterations increase. This effect can be monitored on the CASA logger output. To avoid cleaning too deeply, we will set a ''threshold'' parameter that will stop minor cycle clean iterations once a peak residual value is being reached. <br />
<br />
First, will utilize the ''SNR_G55_10s.Reg.Clean.niter1K.image'' image to give us an idea of the rms noise (your sigma value).<br />
With the image open within the viewer, click on the 'Rectangle Drawing' button (Rectangle with R) and draw a square on the image at a position with little source or sidelobe contamination. Doing this should open up a "Regions" dock, which holds information about the region, including the selected pixel statistics in the aptly named "Statistics" tab. Take notice of the rms values as you click/drag the box around empty image locations, or by drawing additional boxes at suitable positions. <br />
<br />
The lowest rms value that we found was about 4E-5 Jy/beam, which we will use to calculate our threshold. There really is no set standard, but fairly good threshold values can vary anywhere between 2.0-4.0*sigma; using clean boxes (see the section on interactive cleaning) allows one to go to lower thresholds. For our purposes, we will choose a threshold of 2.5*sigma. Doing the math results in a value of 10E-5 or equivalently 0.10mJy/beam. Therefore, for future calls to the {{clean}} task, we will set ''threshold='0.1mJy'''. The clean cycle will be stopped when the residual peak flux equals or is less than the ''threshold'' value, or when the maximum number of iterations ''niter'' is reached. To ensure that the stopping criterion is indeed ''threshold'', ''niter'' should be set to a very high number. '''In the following, we nevertheless will use ''niter=1000'' to keep the execution times of {{clean}} on the low end as we focus on explaining different imaging methods.'''<br />
<br />
An alternative method to determine the approximate rms of an image is to use the [https://obs.vla.nrao.edu/ect/ VLA Exposure Calculator] and to enter the observing conditions.<br />
<br />
== CLEAN with Weights ==<br />
<br />
To see the effects of using different weighting schemes to the image, let's change the ''weighting'' parameter within {{clean}} and inspect the resulting images. We will be using the ''Natural, Uniform, and Briggs'' weighting algorithms. Here, we have chosen a smaller image size to mainly focus on our science target. <br />
<br />
<source lang="python"><br />
# In CASA. Natural weighting<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.natural', weighting='natural',<br />
imsize=540, cell='8arcsec', niter=1000, interactive=False, threshold='0.1mJy')<br />
<br />
# In CASA. Uniform weighting<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.uniform', weighting='uniform',<br />
imsize=540, cell='8arcsec', niter=1000, interactive=False, threshold='0.1mJy')<br />
<br />
# In CASA. Briggs weighting, with robust set to default of 0.0<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.briggs', weighting='briggs',<br />
robust=0, imsize=540, cell='8arcsec', niter=1000, interactive=False, threshold='0.1mJy')<br />
<br />
viewer()<br />
</source><br />
<br />
* ''weighting'': specification of the weighting scheme. For Briggs weighting, the robust parameter will be used.<br />
<br />
* ''threshold='0.1mJy''': threshold at which the cleaning process will halt.<br />
<br />
[[Image:SNR_G55_11s.nat.uni.briggs.png|1000px|thumb|center|Fig. 6: CLEAN images created with different weighting algorithms, including natural (left), uniform (center), and briggs (right).]]<br />
<br />
In Fig. 6 we see that the natural weighted image is most sensitive to extended emission (beam size of 46"x41"). The negative values around the extended emission (often referred to as a negative 'bowl') is a typical signature of missing short spacings, extended emission that even the minimum baseline is not able to detect. Uniform weighted data shows the highest resolution (26"x25") and Briggs 'robust=0' (default value) is a compromise with a beam of 29"x29". To be more sensitive to the extended emission, the 'robust' parameter could be tweaked further toward more positive values.<br />
<br />
== Multi-Scale CLEAN ==<br />
<br />
Since G55.7+3.4 is an extended source with many spatial scales, a more advanced form of imaging involves the use of multiple scales. MS-CLEAN is an extension of the classical CLEAN algorithm for handling extended sources. It works by assuming the sky is composed of emission at different spatial scales and works on them simultaneously, thereby creating a linear combination of images at different spatials scales. For a more detailed description of Multi Scale CLEAN, see the paper by J.T. Cornwell entitled [http://arxiv.org/abs/0806.2228 Multi-Scale CLEAN deconvolution of radio synthesis images].<br />
<br />
We will use a set of scales (which are expressed in units of the requested pixel, or cell, size) which are representative of the scales that are present in the data, including a zero-scale for point sources. <br />
<br />
[[Image:SN_G55.MultiScale.image.png|200px|thumb|right|Fig. 7: G55.7+3.4 Multi-Scale Clean]]<br />
[[Image:SN_G55_MultiScale.artifacts.png|200px|thumb|right|Fig. 8: Artifacts around point sources]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.MultiScale', <br />
imsize=1280, cell='8arcsec', multiscale=[0,6,10,30,60], smallscalebias=0.9, <br />
interactive=False, niter=1000, weighting='briggs', stokes='I', <br />
threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.MultiScale.image')<br />
</source><br />
<br />
<br />
<br />
* ''multiscale=[0,6,10,30,60]'': a set of scales on which to clean. A good rule of thumb when using multiscale is 0, 2xbeam, 5xbeam (where beam is the synthesized beam), and larger scales up to about half the minor axis maximum scale of the mapped structure. Since these are in units of the pixel size, our chosen values will be multiplied by the requested cell size. Thus, we are requesting scales of 0 (a point source), 48, 80, 240, and 480 arcseconds (8 arcminutes). Note that 16 arcminutes (960 arcseconds) roughly corresponds to the size of G55.7+3.4.<br />
<br />
* ''smallscalebias=0.9'': This parameter is known as the '''small scale bias''', and helps with faint extended structure, by balancing the weight given to smaller structures which tend to be brighter, but have less flux density. Increasing this value gives more weight to smaller scales. A value of 1.0 weighs the largest scale to zero, and a value of less than 0.2 weighs all scales nearly equally. The default value is 0.6.<br />
<br />
The logger will show how much cleaning is performed on the individual scales.<br />
<br />
This is the fastest of the imaging techniques described here, but it's easy to see that there are artifacts in the resulting image (Fig. 7). We can use the {{viewer}} to explore the point sources near the edge of the field by zooming in on them (Fig. 8). Click on the "Zooming" button on the top left corner and highlight an area by making a square around the portion where you would like to zoom-in. Double clicking within the square will zoom-in to the selected area. The square can be resized by clicking/dragging the corners, or removed by pressing the "Esc" key. After zooming in on the area, we can see some radio sources have prominent arcs, as well as spots with a six-pointed pattern surrounding them.<br />
<br />
Next we will explore some more advanced imaging techniques to mitigate the artifacts seen towards the edge of the image.<br />
<br />
== Multi-Scale, Wide-Field CLEAN (w-projection) ==<br />
<br />
[[Image:Faceting.png|250px|thumb|right|Fig. 9: Faceting when using widefield gridmode, which can be used in conjunction with w-projection. ]]<br />
[[Image:SNR_G55_MS_vs_MS.wProj.png|500px|thumb|right|Fig. 10: Multi-Scale image of arcs around point sources far from the phase center, versus MS with w-projection (right). We can see the that combining the w-projection algorithm with the multiscale algorithm improves the resulting image by removing prominent artifacts.]]<br />
<br />
The next {{clean}} algorithm we will employ is w-projection, which is a wide-field imaging technique that takes into account the non-coplanarity of the baselines as a function of distance from the phase center (Fig. 9). For wide-field imaging, the sky curvature and non-coplanar baselines results in a non-zero w-term. The w-term introduced by the sky and array curvature introduces a phase term that will limit the dynamic range of the resulting image. Applying 2-D imaging to such data will result in artifacts around sources away from the phase center, as we saw in running MS-CLEAN. Note that this affects mostly the lower frequency bands, especially for the more extended configurations, due to the field of view decreasing with higher frequencies. <br />
<br />
The w-term can be corrected by faceting (describe the sky curvature by many smaller planes) in either the image or uv-plane, or by employing w-projection. A combination of the two can also be employed within {{clean}} by setting the parameter ''gridmode='widefield'''. If w-projection is employed, it will be done for each facet. Note that w-projections is an order of magnitude faster than the faceting algorithm, but will require more memory. <br />
<br />
For more details on w-projection, as well as the algorithm itself, see [http://adsabs.harvard.edu/abs/2008ISTSP...2..647C "The Noncoplanar Baselines Effect in Radio Interferometry: The W-Projection Algorithm"]. Also, the chapter on [http://www.aspbooks.org/a/volumes/article_details/?paper_id=17953 Imaging with Non-Coplanar Arrays] may be helpful. <br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.wProj',<br />
gridmode='widefield', imsize=1280, cell='8arcsec',<br />
wprojplanes=-1, multiscale=[0,6,10,30,60], <br />
interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.ms.wProj.image')<br />
</source><br />
<br />
* ''gridmode='widefield''': Use the w-projection algorithm.<br />
<br />
* ''wprojplanes=-1'': The number of w-projection planes to use for deconvolution. Setting to -1 forces CLEAN to utilize an acceptable number of planes for the given data.<br />
<br />
This will take slightly longer than the previous imaging round; however, the resulting image (Fig. 10) has noticeably fewer artifacts. In particular, compare the same outlier source in the Multi-Scale w-projected image with the Multi-Scale-only image: note that the swept-back arcs have disappeared. There are still some obvious imaging artifacts remaining, though.<br />
<br />
== Multi-Scale, Multi-Term Frequency Synthesis ==<br />
[[Image:MultiFrequency_Synthesis_snapshot.png|250px|thumb|right|Fig. 11: Multi-Frequency Synthesis snapshot of (u,v) coverage. We can see from the image on the right, using this algorithm can greatly improve coverage, thereby improving image fidelity.]]<br />
[[Image:SNR_G55.ms_to_mfs.gif|200px|thumb|right|Fig. 12: Multi-Scale image artifacts versus MS-MFS artifacts near SNR, with nterms=2. We can see artifacts around point sources diminish, improving our image.]]<br />
[[Image:SN_G55_MS.MFS.alpha.png|200px|thumb|right|Fig. 13: Spectral Index image]]<br />
<br />
A consequence of simultaneously imaging the wide fractional bandwidths available with the VLA is that the primary and synthesized beams have substantial frequency-dependent variation over the observing band. If this is not accounted for, it will lead to imaging artifacts and compromise the achievable image rms.<br />
<br />
The dimensions of the (u,v) plane are measured in wavelengths, and therefore observing at several frequencies, a baseline can sample several ellipses in the (u,v) plane, each with different sizes. We can therefore fill in the gaps in the single frequency (u,v) coverage (Fig. 11) to achieve a much higher image fidelity. This method is called Multi-Frequency Synthesis (MFS). Also when observing in low-frequencies, it may prove beneficial to observe in small time-chunks, which are spread out in time. This will allow the coverage of more spatial-frequencies, allowing us to employ this algorithm more efficiently. <br />
<br />
The Multi-Scale Multi-Frequency-Synthesis algorithm (also known ad Multi-Term Frequency Synthesis) provides the ability to simultaneously image and fit for the intrinsic source spectrum. The spectrum is approximated using a polynomial Taylor Term expansion in frequency, with the degree of the polynomial as a user-controlled parameter. A least-squares approach is used, along with the standard clean-type iterations. <br />
<br />
For a more detailed explanation of the MS-MFS deconvolution algorithm, please see the paper by Urvashi Rau and Tim J. Cornwell entitled [http://arxiv.org/abs/1106.2745 A multi-scale multi-frequency deconvolution algorithm for synthesis imaging in radio interferometry]<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MFS',<br />
imsize=1280, cell='8arcsec', mode='mfs', nterms=2,<br />
multiscale=[0,6,10,30,60], <br />
interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.image.tt0')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.image.alpha')<br />
</source><br />
<br />
* ''nterms=2'': the number of Taylor terms to be used to model the frequency dependence of the sky emission. Note that the speed of the algorithm will depend on the value used here (more terms will be slower). nterms=2 will fit a spectral index, and nterms=3 a spectral index and curvature.<br />
<br />
This will take much longer than the two previous methods, so it would probably be a good time to have coffee or chat about VLA data reduction with your neighbor at this point. <br />
<br />
When clean is done <imagename>.image.tt0 will contain a total intensity image (Fig. 12), where tt* is a suffix to indicate the Taylor term; <imagename>.image.tt0 is the total intensity image and <imagename>.image.alpha will contain an image of the spectral index in regions where there is sufficient signal-to-noise (Fig. 13). Having this spectral index image can help convey information about the emission mechanism involved within the supernova remnant. It can also give information on the optical depth of the source. <br />
<br />
For more information on the multi-term, multi-frequency synthesis mode and its outputs, see section 5.2.5.1 in the [https://casa.nrao.edu/docs/cookbook/casa_cookbook006.html CASA cookbook]. <br />
<br />
Inspect the brighter point sources in the field near the supernova remnant. You will notice that some of the artifacts, which had been symmetric around the sources themselves, are now gone; however, since we did not use W-Projection this time, there are still strong features related to the non-coplanar baseline effects still apparent for sources further away.<br />
<br />
At this point, {{clean}} takes into account the frequency variation of the synthesized beam but not the frequency variation of the primary beam. For low frequencies and large bandwidths, this can be substantial. E.g. 1-2GHz L-band observations result in a variation of a factor of 2. One effect of such a large fractional bandwidth is that in multi-frequency synthesis primary beam nulls will be blurred and the interferometer is sensitive everywhere in the field of view. For spectral slopes, however, a frequency-dependent primary beam causes a steepening as the higher frequencies are less sensitive at a given point away from the phase center. A correction for this effect should be made with the task {{widebandpbcor}} (see below).<br />
<br />
== Multi-Scale, Multi-Term Frequency, Widefield CLEAN ==<br />
<br />
Finally, we will combine the W-Projection and MS-MFS algorithms. Be forewarned -- these imaging runs will take a while, and it's best to start them running and then move on to other things.<br />
<br />
Using the same parameters for the individual-algorithm images above, but combined into a single {{clean}} run, we have:<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MFS.wProj',<br />
gridmode='widefield', imsize=1280, cell='8arcsec', mode='mfs',<br />
nterms=2, wprojplanes=-1, multiscale=[0,6,10,30,60], <br />
interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.wProj.image.tt0')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.wProj.image.alpha')<br />
</source><br />
<br />
[[Image:SNR_G55_MS.MFS.wProj.png|1250px|thumb|center|Fig. 14: Here we see the differences as the images progress through the different algorithms used: MS -> MS-MFS -> MS-wProjection -> MS-MFS-wProjection.]]<br />
<br />
Again, looking at the same outlier source, we can see that the major sources of error have been removed, although there are still some residual artifacts (Fig. 14). One possible source of error is the time-dependent variation of the primary beam; another is the fact that we have only used ''nterms=2'', which may not be sufficient to model the spectra of some of the point sources. Some weak RFI may also show up that may need additional flagging.<br />
<br />
== Imaging Outlier Fields ==<br />
<br />
When strong sources are far away from the main target but still strong enough to produce sidelobes in the main image, they should be cleaned. Sometimes, however, it is not practical to image very large images for this purpose. An alternative is to use outlier fields. This mode will allow the user to specify a few locations that are then being cleaned along with the main image. The outlier fields should be centered on strong sources that, e.g., are known from sky catalogs or are identified by other means. <br />
<br />
Outlier fields are specified in an outlier file (here: 'outliers.txt') that contains the names, sizes, and positions (see Sect. 5.3.18.1 of the CASA cookbook): <br />
<br />
<pre><br />
#content of outliers.txt<br />
#<br />
#outlier field1<br />
imagename= 'Outlier1.MS.MFS'<br />
imsize=[512,512]<br />
phasecenter = 'J2000 19h23m27.693 22d37m37.180'<br />
#<br />
#outlier field2<br />
imagename='Outlier2.MS.MFS'<br />
imsize=[512,512]<br />
phasecenter = 'J2000 19h25m46.888 21d22m03.365'<br />
<br />
</pre><br />
<br />
{{clean}} will then be executed like the following:<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR.MS.MFS-Main',<br />
outlierfile='outliers.txt',<br />
imsize=[512,512], cell='8arcsec', mode='mfs',<br />
multiscale=[0,6,10,30,60], interactive=False, niter=1000, weighting='briggs',<br />
robust=0, stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
</source><br />
<br />
* ''outlierfile='outliers.txt''': the name of the outlier file.<br />
<br />
<!--<br />
Now we will image the supernova remnant, as well as the bright sources (outliers) towards the edges of the image, creating a small image for each one. For this, we will be utilizing {{clean}} without widefield mode (try it to see the effects) as we will be specifying a phase center, and the images will not be too big, therefore we will not have sources very far from the phase center. We will specify an outlier file with the the names, sizes, and positions of each outlier image. This form of imaging is useful for when you have several outlier fields you'd like to image in one go. It is particularly useful to clean out far away sources that throw sidelobes into the main image and need to be cleaned out. (see Sect. 5.3.18.1 of the CASA cookbook).<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename=['SNR.MS.MFS', 'Outlier1.MS.MFS', 'Outlier2.MS.MFS'],<br />
imsize=[[512,512],[512,512],[512,512]], cell='8arcsec', mode='mfs',<br />
multiscale=[0,6,10,30,60], interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean',<br />
phasecenter=['J2000 19h21m38.271 21d45m48.288', 'J2000 19h23m27.693 22d37m37.180', 'J2000 19h25m46.888 21d22m03.365'])<br />
</source><br />
--> <br />
<br />
[[Image:Outlier_Fields.png|1000px|thumb|center|Fig. 15: Images of the supernova remnant and bright outlying sources. ]]<br />
<br />
== Primary Beam Correction ==<br />
<br />
In interferometry, the images formed via deconvolution are representations of the sky, multiplied by the primary beam response of the antenna. The primary beam can be described by a Gaussian with a size depending on the observing frequency (see above). Images produced via {{clean}} are by default not corrected for the primary beam pattern (important for mosaics), and therefore do not have the correct flux away from the phase center. <br />
<br />
Correcting for the primary beam, however, can be done during {{clean}} by using the ''pbcor'' parameter. It can also be done after imaging using the task [https://casa.nrao.edu/docs/TaskRef/impbcor-task.html impbcor] for regular data sets, and {{widebandpbcorr}} for those that use Taylor-term expansion (nterms > 1). A third possibility, is utilizing the task {{immath}} to manually divide the ''<imagename>.image'' by the ''<imagename>.flux'' images (''<imagename>.flux.pbcoverage'' for mosaics). <br />
<br />
Flux corrected images usually don't look pretty, due to the noise at the edges being increased. Flux densities, however, should only be calculated from primary beam corrected images. Let's run the {{impbcor}} task to correct our multiscale image.<br />
<br />
<source lang="python"><br />
# In CASA<br />
impbcor(imagename='SNR_G55_10s.MultiScale.image', pbimage='SNR_G55_10s.MultiScale.flux', <br />
outfile='SNR_G55_10s.MS.pbcorr.image', mode='divide')<br />
</source><br />
* ''imagename'': the image to be corrected<br />
<br />
* ''pbimage'': the <imagename>.flux image as a representation of the primary beam (<imagename>.flux.pbcoverage for mosaics)<br />
<br />
Let us now use the {{widebandpbcor}} task for wideband (''nterms''>1) images. Note that for this task, we will be supplying the image name that is the prefix for the Taylor expansion images, tt0 and tt1, which must be on disk. Such files were created, e.g., during the last Multi-Scale, Multi-Frequency, Widefield run of CLEAN. <br />
<br />
[[Image:SNR_G55_10s.MS.MFS.wProj.pbcor.image.png|200px|thumb|right|Fig. 16: Primary beam corrected image using the MS.MFS.wProj image created during the clean process.]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
widebandpbcor(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MFS.wProj', <br />
nterms=2, action='pbcor', pbmin=0.2, spwlist=[0,1,2,3], <br />
weightlist=[0.5,1.0,0,1.0], chanlist=[63,63,63,63], threshold='0.1Jy')<br />
</source><br />
<br />
* ''spwlist=[0,1,2,3]'': We want to apply this correction to all spectral windows in our calibrated measurement set. <br />
<br />
* ''weightlist=[0.5,1.0,0,1.0]'': Since we did not specify reference frequencies during {{clean}}, the {{widebandpbcor}} task will pick them from the provided image. Running the task, the logger reports the multiple frequencies used for the primary beam, which are 1.256, 1.429, 1.602, and 1.775 GHz. Having created an amplitude vs. frequency plot of the calibrated measurement set with colorized spectral windows using {{plotms}}, we notice that the first chosen frequency lies within spectral window 0, which we know had lots of flagged data due to lots of RFI being present. This ''weightlist'' parameter allows us to give this chosen frequency less weight. The primary beam at 1.6GHz lies in an area with no data, therefore we will give a weight value of zero for this frequency. The remaining frequencies 1.429 and 1.775 GHz lie within spectral windows which contained less RFI, therefore we provide a larger weight percentage. <br />
<br />
* ''pbmin=0.2'': Gain level below which not to compute Taylor-coefficients or apply a primary beam correction.<br />
<br />
* ''chanlist=[63,63,63,63]'': Our measurement set contains 64 channels, including zero.<br />
<br />
* ''threshold='0.1Jy''': Threshold in the intensity map, below which not to recalculate the spectral index.<br />
<br />
It's important to note that the image will cut off at about 20% of the HPBW, as we are confident of the accuracy within this percentage. Anything outside becomes less accurate, thus there is a mask associated with the creation of the corrected primary beam image (Fig. 16). <br />
<br />
It would be a good exercise to use the {{viewer}} to plot both the primary beam corrected image, and the original cleaned image and compare the intensity (Jy/beam) values, which should differ slightly.<br />
<br />
== Imaging Spectral Cubes ==<br />
<br />
For spectral line imaging, CASA {{clean}} can be used in either ''mode='frequency' '' or ''mode='velocity' ''. Both will create a spectral axis in frequency. The velocity mode adds an additional velocity label to it. <br />
<br />
The following keywords are important for spectral modes (velocity in this example):<br />
<br />
<source lang="python"><br />
# In CASA<br />
mode = 'velocity' # Spectral gridding type (mfs, channel, velocity, frequency)<br />
nchan = -1 # Number of channels (planes) in output image; -1 = all<br />
start = '' # Velocity of first channel: e.g '0.0km/s'(''=first channel in first SpW of MS)<br />
width = '' # Channel width e.g '-1.0km/s' (''=width of first channel in first SpW of MS)<br />
<snip><br />
outframe = '' # spectral reference frame of output image; '' =input<br />
veltype = 'radio' # Velocity definition of output image<br />
<br />
<snip><br />
restfreq = '' # Rest frequency to assign to image (see help)<br />
</source><br />
<br />
The spectral dimension of the output cube will be defined by these parameters and {{clean}} will regrid the visibilities to it. Note that invoking {{cvel}} before imaging is in most cases not necessary, even when two or more measurement sets are being provided at the same time. <br />
<br />
The cube is specified by a ''start'' velocity in km/s, the ''nchan'' number of channels and a channel ''width'' (where the latter can also be negative for decreasing velocity cubes). <br />
<br />
To correct Doppler motions, {{clean}} also requires a velocity frame, where ''LSRK'' and ''BARY'' are the most popular Local Standard of Rest (kinematic) and sun-earth Barycenter references. ''doppler'' defines whether the data will be gridded via the optical or radio velocity definition. A description of the available options and definitions can be found in the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/line VLA observing guide] and the [https://casa.nrao.edu/docs/cookbook/casa_cookbook014.html CASA cookbook]. By default, {{clean}} will produce a cube in LRSK and radio. <br />
<br />
Note that {{clean}} will always work on the entire cube when searching for the highest residual fluxes in the minor cycle. To do this per channel, one can set ''chaniter=T''.<br />
<br />
CASA also offers ''mode='channel' ''. But we do not recommend this mode as the VLA does not observe with Doppler tracking. In other words, the fixed frequency observing of the VLA will let any spectral feature move through frequency space in time. The Doppler correction will then be in {{clean}} via ''mode='velocity''' or '' 'frequency' '' and not on the telescope itself. ''mode='channel' '' can also be confusing when imaging some shifted, but overlapping spectral windows.<br />
<br />
== Beam per Plane ==<br />
For large spectral cubes, the synthesized beam can vary substantially across the channels. To account for this, CASA will calculate separate beams for each channel when the difference is more than half a pixel across the cube. All CASA image analysis tasks are capable to handle such cubes. <br />
<br />
If it is desired to have a cube with a single synthesized beam, two options are available. Best is to use {{imsmooth}} with ''kernel='commonbeam' ''. This task will smooth each plane to that of the lowest resolution. It will also clean up all header variables such that only a single beam appears in the data. The second option is to define ''resmooth=T'' directly in {{clean}}.<br />
<br />
== Image Header ==<br />
<br />
The image header holds meta data associated with your CASA image. The task {{imhead}} will display this data within the casalog. We will first run {{imhead}} with ''mode='summary' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0', mode='summary')<br />
</source><br />
<br />
* ''mode='summary' '': gives general information about the image, including the object name, sky coordinates, image units, the telescope the data was taken with, and more. <br />
<br />
For further information about the image, let's now run it with ''mode='list' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0', mode='list')<br />
</source><br />
<br />
* ''mode='list' '': gives more detailed information, including beam major/minor axes, beam primary angle, and the location of the max/min intensity, and lots more. Essentially this mode displays the FITS header variables.<br />
<br />
<br />
Next, we convert our image from intensity to main beam brightness temperature.<br />
<br />
We will use the standard equation <br />
<br />
<math>T=1.222\times 10^{6} \frac{S}{\nu^{2} \theta_{maj} \theta_{min}} </math><br />
<br />
where the main beam brightness temperature <math>T</math> is given in K, the intensity <math>S</math> in Jy/beam, the reference frequency <math>\nu</math> in GHz, and the major and minor beam sizes <math>\theta</math> in arcseconds. <br />
<br />
For a beam of 29.30"x29.03" and a reference frequency of 1.579GHz (as taken from the previous {{imhead}} run) we calculate the brightness temperature using {{immath}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
immath(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0', mode='evalexpr', <br />
expr='1.222e6*IM0/1.579^2/(29.30*29.03)', <br />
outfile='SNR_G55_10s.ms.MFS.wProj.image.tt0-Tb')<br />
</source><br />
* ''mode='evalexpr' '': {{immath}} is used to calcuate with images<br />
<br />
* ''expr'': the mathematical expression to be evaluated. The images are abbreviated as IM0, IM1, ... in the sequence of their appearance in the ''imagename'' parameter.<br />
<br />
Finally, we will change the unit of the new image header to 'K'. <br />
To do this, we will run {{imhead}} with ''mode='put' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0-Tb', mode='put', hdkey='bunit', hdvalue='K')<br />
</source><br />
* ''hdkey'': the header keyword that will be changed<br />
<br />
* ''hdvalue'': the new value for header keyword<br />
<br />
Launching the viewer will now show our image in brightness temperature units: <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MFS.wProj.image.tt0-Tb')<br />
</source><br />
<br />
<br />
[[Main Page | &#8629; '''CASAguides''']]<br />
<br />
<!--<br />
-- Original: Miriam Hartman <br /><br />
-- Modifications: Lorant Sjouwerman (4.4.0, 2015/07/07) <br /><br />
-- Modifications: Juergen Ott (4.5.2, 2016/04/14) <br /><br />
-- Topical Guide: Jose Salcido (4.5.2, 2016/04/18) <br /><br />
-- Modifications: Juergen Ott (4.5.2, 2016/04/20) <br /><br />
--><br />
{{Checked 4.5.2}}</div>Emomjianhttps://casaguides.nrao.edu/index.php?title=VLA_CASA_Imaging-CASA4.5.2&diff=19051VLA CASA Imaging-CASA4.5.22016-04-21T00:03:29Z<p>Emomjian: </p>
<hr />
<div><br />
== Imaging ==<br />
<br />
This tutorial provides guidance on imaging procedures in CASA. <br />
<br />
We will be utilizing data taken with the Karl G. Jansky Very Large Array, of a supernova remnant G055.7+3.4.. The data were taken on August 23, 2010, in the first D-configuration for which the new wide-band capabilities of the WIDAR (Wideband Interferometric Digital ARchitecture) correlator were available. The 8-hour-long observation includes all available 1 GHz of bandwidth in L-band, from 1-2 GHz in frequency. <br />
<br />
We will skip the calibration process in this guide, as examples of calibration can be found in several other guides, including the <br />
[https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391 EVLA Continuum Tutorial 3C391] and <br />
[https://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 EVLA high frequency Spectral Line tutorial - IRC+10216] guides.<br />
<br />
A copy of the calibrated data <font color=green>(1.2GB)</font> can be downloaded from [http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.calib.tar.gz http://casa.nrao.edu/Data/EVLA/SNRG55/SNR_G55_10s.calib.tar.gz] <br />
<br />
Your first step will be to unzip and untar the file in a terminal (before you start CASA):<br />
<br />
<source lang="bash"><br />
tar -xzvf SNR_G55_10s.calib.tar.gz<br />
</source><br />
<br />
Then start casa as usual via the '''casa''' command, which will bring up the ipython interface and launches the logger.<br />
<br />
== The CLEAN Algorithm ==<br />
<br />
[[Image:CLEAN_Cycle.png|500px|thumb|right|Fig. 1: The CLEAN major and minor cycles, indicating the steps undertaken during gridding, projection algorithms, and creation of images.]]<br />
<br />
The CLEAN algorithm, developed by J. Högbom (1974) enabled the synthesis of complex objects, even if they have relatively poor Fourier uv-plane coverage. Poor coverage occurs with partial earth rotation synthesis, or with arrays composed of few antennas. The "dirty" image is formed by a simple Fourier inversion of the sampled visibility data, with each point on the sky being represented by a suitably scaled and centered PSF (Point Spread Function, or dirty beam, which is the Fourier inversion of the visibility (u,v) coverage). This algorithm attempts to interpolate from the measured (u,v) points across gaps in the (u,v) coverage. It, in short, provides solutions to the convolution equation by representing radio sources by a number of point sources in an empty field. The brightest points are found by performing a cross-correlation between the dirty image, and the PSF. The brightest parts are subtracted, and the process is repeated again for the next brighter sources. Variants of CLEAN, such as multi-scale CLEAN, take into account extended kernels which may be better suited for extended objects.<br />
<br />
For single pointings, CASA uses the Cotton-Schwab cleaning algorithm in the task {{clean}} (''imagermode='csclean'''), which breaks the process into major and minor cycles (see Fig. 1). To start with, the visibilities are gridded, weighted, and Fourier transformed to create a (dirty) image. The minor cycle then operates in the image domain to find the clean components that are added to the clean model. The image is Fourier transformed back to the visibility domain, degridded, and the clean model is subtracted. This creates a new residual that is then gridded, weighted, and FFT'ed again to the image domain for the next iteration. The gridding, FFT, degridding, and subtraction processes form the major cycle. <br />
<br />
This iterative process is continued until a stopping criterion is reached, such as a maximum number of clean components, or a flux threshold in the residual image.<br />
<br />
In CASA {{clean}}, two versions of the PSF can be used (parameter ''psfmode''): ''hogbom'' uses the full sized PSF for subtraction. This is a thorough but slow method. ''psfmode='clark''' uses a smaller beam patch, which increases the speed. The patch size and length of the minor cycle are internally chosen such that clean converges well without giving up the speed improvement. It is thus the default option in {{clean}}. <br />
<br />
<!--<br />
''' 1. Högbom Algorithm '''<br /><br />
This algorithm will initially find the strength and position of a peak in a dirty image, subtract it from the dirty image, record this position and maginitude, and repeat for further peaks. The remainder of the dirty image is known as the residuals. <br />
<br />
The accumulated point sources, now residing in a model, is convolved with an idealized CLEAN beam (usually a Gaussian fitted to the central lobe of the dirty beam), creating a CLEAN image. As the final step, the residuals of the dirty image are then added to the CLEAN image. <br />
<br />
''' 2. Clark Algorithm '''<br /><br />
Clark (1980), developed a FFT-based CLEAN algorithm, which more efficiently shifts and scales the dirty beam by approximating the position and strength of components using a small patch of the dirty beam. This algorithm is the default within the {{clean}} task, which involves major and minor cycles. <br />
<br />
The algorithm will first select a beam patch, which will include the highest exterior sidelobes. Points are then selected from the dirty image, which are up to a fraction of the image peak, and are greater than the highest exterior sidelobe of the beam. It will then conduct a list-based Högbom CLEAN, creating a model and convolution with an idealized CLEAN beam. This process is the minor cycle. <br />
<br />
The major cycle involves transforming the point source model via a FFT (Fast-Fourier Transform), mutiplying this by the weight sampling function (more on this below), and transformed back. This is then subtracted from the dirty image, creating your CLEAN image. The process is then repeated with subsequent minor cycles. <br />
<br />
''' 3. Cotton-Schwab Algorithm '''<br /><br />
This is the default imager mode (''csclean''), and is a variant of the Clark algorith in which the major cycle involves the subtraction of CLEAN components of ungridded visibility data. This allows the removal of gridding errors, as well as noise. One advantage is its ability to image and clean many seperate fields simultaneously. Fields are cleaned independently in the minor cycle, and components from all fields cleaned together in the major cycles.<br />
<br />
This algorithm is faster than the Clark algorithm, except when dealing with a large number of visibility samples, due to the re-gridding process it undergoes. It is most useful in cleaning sensitive high-resolution images at lower frequencies where a number of confusing sources are within the primary beam. <br />
--> <br />
In a final step, {{clean}} derives a Gaussian fit to the inner part of the PSF, which defines the clean beam. The clean model is then convolved with the clean beam and added to the last residual image to create the final image. <br />
<br />
Note that the CASA team currently develops a refactored clean task, called [http://casa.nrao.edu/docs/TaskRef/tclean-task.html tclean]. It has a better interface and provides new algorithms as well as more combinations between imaging algorithms. tclean also includes software to parallelize the computations in a multi-processor environment. Eventually, tclean will replace the current {{clean}} task. For this guide, however, we will stick with the original {{clean}} task, as tclean is still in the development and testing phase. Nevertheless, the reader is encouraged to try tclean and send us feedback through the [https://help.nrao.edu/ NRAO helpdesk].<br />
<br />
For more details on imaging and deconvolution, we refer to the Astronomical Society of the Pacific Conference Series book entitled [http://www.aspbooks.org/a/volumes/table_of_contents/?book_id=292 Synthesis Imaging in Radio Astronomy II]. The chapter on [http://www.aspbooks.org/a/volumes/article_details/?paper_id=17942 Deconvolution] may prove helpful. In addition, imaging presentations are available on the [https://science.nrao.edu/science/meetings/2014/14th-synthesis-imaging-workshop/lectures Synthesis Imaging Workshop] and [https://science.nrao.edu/science/meetings/2016/vla-data-reduction/program VLA Data Reduction Workshop] webpages. The [https://casa.nrao.edu/docs/cookbook/index.html CASA cookbook] chapter on [https://casa.nrao.edu/docs/cookbook/casa_cookbook006.html Synthesis Imaging] provides a wealth of information on the CASA implementation of {{clean}} and related tasks.<br />
<br />
Finally, we like to refer to the [https://science.nrao.edu/facilities/vla/docs/manuals/oss VLA Observational Status Summary] and the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide Guide to Observing with the VLA] for information on the VLA capabilities and observing strategies.<br />
<br />
== Weights and Tapering ==<br />
<br />
[[Image:SNR-G55-uvcoverage-CASA4.5.2.png|400px|thumb|right|Fig. 2: u,v coverage for the 8-hour observation of the supernova remnant G055.7+3.4]]<br />
<br />
When imaging data, a map is created associating the visibilities with the image. <br />
The sampling function, which is a function of the visibilities, is modified by a weight function that defines the shape and size of the PSF. Weighting therefore provides some control over the spatial resolution and the surface brightness sensitivity of the map, where either direction can be emphasized.<br />
<!-- <math> S(u,v) \to S(u,v)W(u,v) </math>. <br />
Due to the fact that each VLA antenna performs slightly differently, different weights should be applied to each antenna. Therefore, the weight column in the data table reflects how much weight each corrected data sample should receive. <br />
--><br />
<br />
There are three main weighting schemes (see also Table 1): <br />
<br />
1) '''Natural''' weighting: The natural weights data based on their rms only. All visibility weights in a cell are being summed. More visibilities in a cell will thus increase the cell's weight, which will emphasize the shorter baselines. Natural weighting therefore results in a larger PSF, better surface brightness sensitivity but also a degraded resolution. <br />
<br />
2) '''Uniform''' weighting: The weights are first gridded as in natural weighting but then each cell is corrected such that the weights are independent of the number of visibilities inside. Compared to natural weighting, uniform weighting emphasizes the longer baselines. Consequently, the PSF is smaller, which results in a better spatial resolution of the image. At the same time, however, the surface brightness sensitivity is reduced compared to natural weighting. The 'uniform' weighting of the baselines is a better representation of the uv-coverage and sidelobes are more suppressed. <br />
<br />
3) '''Briggs''' weighting: This scheme provides a compromise between natural and uniform weighting. The transition can be controlled with the ''robust'' parameter where ''robust=-2'' is close to uniform and ''robust=2'' close to natural weighting. <br />
Briggs weighting therefore offers a compromise for between spatial resolution and surface brightness sensitivity, and ''robust'' values near zero are typically being used.<br />
<br />
<!--<br />
For a brief intro to the different clean algorithms, as well as other deconvolution and imaging information, please see the website kept by Urvashi R.V. [http://www.aoc.nrao.edu/~rurvashi/ImagingAlgorithmsInCasa/node2.html#SECTION00223000000000000000 here].<br />
<br />
<br />
There are different options for <math> W(u,v) </math> (parameter ''weighting'' in {{clean}}):<br />
<br />
'''1. Natural''': For the CASA parameter ''weighting=’natural’'', visibilities are weighted only by the data weights, which are calculated during filling and calibration and should be equal to the inverse noise variance on that visibility. Imaging weight <math>W_i</math> of sample <math>i</math> is given by <math>W_i = \omega_i = 1/\sigma_i^2</math><br />
<br />
where the data weight <math>\omega_i</math> is determined from <math>\sigma_i</math>, the rms noise on visibility <math>i</math>. When data is gridded into the same uv-cell for imaging, the weights are summed, and thus a higher uv density results in higher imaging weights.<br />
<br />
The weight function can be described as <math> W(u,v) = 1/ \sigma^2 </math>, where <math> \sigma^2 </math> is the noise variance. Natural weighting will maximize point source sensitivity, and provide the lowest rms noise within an image, as well as the highest signal-to-noise. It will also generally give more weight to short baselines, thus the synthesized beam will be large. This form of weighting is the default within the clean task. <br />
<br />
'''2. Uniform''': For ''weighting = ’uniform’'', the data weights are calculated as in ’natural’ weighting. The data is then gridded to a number of cells in the uv-plane, and after all data is gridded the uv-cells are re-weighted to have 'uniform' imaging weights. This pumps up the influence on the image of data with low weights (they are multiplied up to be the same as for the highest weighted data), which sharpens resolution and reduces the sidelobe level in the field-of-view, but increases the rms image noise. <br />
<br />
For uniform weighting, we first grid the inverse variance <math>\omega_i</math> for all selected data onto a grid with uv cell-size given by <math>2/FOV</math> where <math>FOV</math> is the specified field of view (defaults to the image field of view). This forms the gridded weights <math>W_k</math>. The weight of the <math>i</math>-th sample is then:<math>W_i=\omega_i/W_k</math><br />
<br />
The weight function can be described as <math> W(u,v) = W(u,v) / W_k </math>, where <math> W_k </math> represents the local density of (u,v) points, otherwise known as the gridded weights. This form of weighting will increase the influence of data with lower weight, filling the (u,v) plane more uniformly, thereby reducing sidelobe levels in the field-of-view, but increasing the rms image noise. More weight is given to long baselines, therefore increasing angular resolution. Point source sensitivity is degraded due to the downweighting of some data. <br />
<br />
'''3. Briggs''': A flexible weighting scheme, that is a variant of uniform, and avoids giving too much weight to (u,v) points with a low natural weight. Weight function can be described as <br />
<math> W(u,v) = 1/ \sqrt{1+S_N^2/S_{thresh}^2} </math>, where <math> S_N </math> is the natural weight of the cell, <math> S_{thresh} </math> is a <br />
threshold. <br />
A high threshold will go to a natural weight, where as a low threshold will go to a uniform weight. This form of weighting also has adjustable parameters. The ''robust'' parameter will give variation between resolution and maximum point source sensitivity. It's value can range from -2.0 (close to uniform weight) to 2.0 (close to natural weight). By default, the parameter is set to 0.0, which gives a good trade-off. <br />
<br /> <br />
--><br />
<br />
[[Image:Weight_Tapering_table.png|450px|thumb|right|Table 1: Table summarizing the effects of using weights and tapering.]]<br />
'''Tapering''': In conjunction with the above weighting schemes, one can specify the ''uvtaper'' parameter within {{clean}}, which will control the radial weighting of visibilities, in the uv-plane. This in effect, reduces the visibilities, with weights decreasing as a function of uv-radius. The taper in {{clean}} is an elliptical Gaussian function. <br />
<!--<br />
The tapering will apodize, or filter/change the shape of the weight function (which is itself a Gaussian), which can be expressed as: <br /><br />
<math> W(u,v) = e^{-(u^2+v^2)/t^2} </math>, where t is the adjustable tapering parameter. <br />
--><br />
Tapering can smooth the image plane, give more weight to short baselines, but in turn degrade angular resolution. This process will increase the surface brightness sensitivity of the data. Aggressive tapering, however, can downweight some of the antennas and baselines to a degree where they are essentially not being considered anymore. The point source sensitivity will thus be decreased and in extreme cases the surface brightness sensitivity will suffer, too. <br />
<!--<br />
Due to the downweight of some data, point source sensitivity can be degraded, as some antennas and baselines will suppressed and, in the extreme case, effectively removed from the data averaging. If your observation was sampled by short baselines, tapering may improve sensitivity to extended structures. <br />
--><br />
<br />
We refer to the [https://casa.nrao.edu/docs/cookbook/casa_cookbook006.html CASA Cookbook Synthesis Imaging] chapter for the details of the weighting implementation in CASA's {{clean}}.<br />
<br />
== Primary and Synthesized Beam ==<br />
<br />
The primary beam of a single antenna defines the sensitivity across the field of view. For the VLA antennas, the main beam can be approximated by a Gaussian with a FWHM equal to <math>90*\lambda_{cm}</math> or <math>45/ \nu_{GHz}</math>. But note that there are sidelobes beyond the Gaussian kernel that are sensitive to bright sources (see below). Taking our observed frequency to be the middle of the band, 1.5GHz, our primary beam will be around 30 arcmin. <br />
<br />
If your science goal is to image a source, or field of view that is significantly larger than the FWHM of the VLA primary beam, then creating a mosaic from a number of pointings is usually the preferred method. For a tutorial on mosaicing, see the [https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391 3C391 tutorial]. In the following, however, we will discuss methods to image large maps from single pointing data. <br />
<br />
Since our observation was taken in D-configuration and we can check the [https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/resolution Observational Status Summary]'s section on VLA resolution to find that the synthesized beam will be around 46 arcsec. We want to oversample the synthesized beam by a factor of around five, so we will use a cell size of 8 arcsec. <br />
<br />
Our field contains bright point sources significantly outside the primary beam. We therefore create images that are 170 arcminutes on a side ([Image Size * Cell Size]*[1arcmin / 60arsec]), or almost 6x the size of the primary beam, catching the first and second sidelobe. This is ideal for showcasing both the problems inherent in such wide-band, wide-field imaging, as well as some of the solutions currently available in CASA to deal with these issues. Note that the execution time of {{clean}} depends on the image sizes. Large images generally take more computing time. There are some values, however, that are computationally not advisable. The logger output will then show a recommendation for the next larger but faster image size. As a rule of thumb we recommend image sizes <math>2^n*10</math>, e.g. 160, 1280, etc. <br />
<br />
It's worth considering why we are even interested in sources which are far outside the primary beam. This is mainly due to the fact that the VLA, with its wide bandwidth capabilities, is quite sensitive even far from phase center -- for example, at our observing frequencies in L-band, the primary beam gain is as much as 10% around 1 degree away. That means that any imaging errors for these far-away sources will have a significant impact on the image rms at phase center. The error associated with sidelobes emerging from a source at distance R can be parametrized as:<br />
<br />
<math><br />
\Delta(S) = S(R) \times PB(R) \times PSF(R)<br />
</math><br />
<br />
So, for R = 1 degree, source flux S(R) = 1 Jy, <math>\Delta(S)</math> = 1 mJy − 100 <math>{\mu}</math>Jy. Clearly, this will be a source of significant error.<br />
<br />
Cleaning far away from the phase center may thus become important in the vicinity of strong sources, in particular at the lower frequency bands where non-thermal emission dominates.<br />
<br />
== Clean Output Images ==<br />
<br />
As a result of the CLEAN algorithm, {{clean}} will create a number of output images. For an ''imagename='<imagename>''', this would be:<br />
<br />
'''<imagename>.image''' the residual + the model convolved with the clean beam. This is the final image (unit: Jy/clean beam). <br><br />
'''<imagename>.residual''' the residual after subtracting the clean model (unit: Jy/dirty beam).<br><br />
'''<imagename>.model''' the clean model, not convolved (unit: Jy/pixel).<br><br />
'''<imagename>.psf''' the point-spread function aka dirty beam<br><br />
'''<imagename>.flux''' the normalized sensitivity map. For single pointings this corresponds to the primary beam. <br><br />
<br />
Additional images will be created for specific algorithms like multi-term frequency synthesis or mosaicking. <br />
<br />
Note: If an image file is present in the working directory and the same name is provided in ''imagename'', {{clean}} will use that image (in particular the residual and model image) as a starting point for further cleaning. If you want a fresh run of {{clean}}, first remove all images of that name using 'rmtables()':<br />
<br />
<source lang="python"><br />
# In CASA<br />
rmtables('<imagename>.*')<br />
</source><br />
<br />
This method is preferable over 'rm -rf' as it also clears the cache.<br />
<br />
'''Note that interrupting {{clean}} by Ctrl+C may corrupt your visibilities - you may be better off choosing to let {{clean}} finish. We are currently implementing a command that will nicely exit to prevent this from happening, but for the moment try to avoid Ctrl+C.'''<br />
<br />
== Dirty Image ==<br />
<br />
[[Image:SN_G55_10s.dirty.image.psf.png|400px|thumb|right|Fig. 3: A dirty image of the supernova remnant G55.7+3.4 in greyscale, with apparent sidelobes (left), and the point spread function (PSF), also known as the dirty beam (right).]]<br />
<br />
First, we will create a dirty image (Fig. 3) to see the improvements as we step through several cleaning algorithms and parameters. The dirty image is the true image on the sky, convolved with the dirty beam (PSF). We will do this by running {{clean}} with niter=0, which will not perform any minor cycles. <br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.dirty', <br />
imsize=1280, cell='8arcsec', interactive=False, niter=0, <br />
stokes='I', usescratch=F)<br />
<br />
viewer('SNR_G55_10s.dirty.image')<br />
</source><br />
<br />
* ''imagermode='csclean''': use the Cotton-Schwab clean algorithm<br />
<br />
* ''imagename='SNR_G55_10s.dirty''': the root filename used for the various {{clean}} outputs. <br />
<br />
* ''imsize=1280'': the image size in number of pixels. A single value will result in a square image.<br />
<br />
* ''cell='8arcsec''': the size of one pixel; again, entering a single value will result in a square pixel size.<br />
<br />
* ''niter=0'': this controls the number of iterations {{clean}} will do in the minor cycle. Since this is a dirty image, we set it to 0.<br />
<br />
* ''interactive=False'': we will let {{clean}} use the entire field for placing model components. Alternatively, you could try using interactive=True, and create regions to constrain where components will be placed. However, this is a very complex field, and creating a region for every bit of diffuse emission as well as each point source can quickly become tedious. For a tutorial that covers more of an interactive clean, please see [https://casaguides.nrao.edu/index.php?title=EVLA_high_frequency_Spectral_Line_tutorial_-_IRC%2B10216 IRC+10216 tutorial.].<br />
<br />
* ''usescratch=F'': do not write the model visibilities to the model data column (only needed for self-calibration)<br />
<br />
* ''stokes='I''': since we have not done any polarization calibration, we only create a total-intensity image. For using CLEAN while including various Stoke's Parameters, please see the [https://casaguides.nrao.edu/index.php/EVLA_Continuum_Tutorial_3C391-CASA4.5 3C391 CASA guide.]<br />
Note that the clean beam is only defined after some clean iterations. The dirty image has therefore no beam size specified in the header and the PSF image is the representation of the response of the array to a point source.<br />
<br />
== Regular CLEAN & RMS Noise ==<br />
[[Image:SNR_G55_10s.niter.1K_vs_10K.png|400px|thumb|right|Fig. 4: Regular run of CLEAN, with niter=1000 (left), compared to niter=10000 (right).]]<br />
<br />
Now, we will create a regular clean image which uses mostly default values to see how deconvolution improves the image quality. The first run of {{clean}} will use a fixed number of minor cycle iterations of''niter=1000'' (default is 500), the second will have ''niter=10000''. Note that you may have to play with the image color map and brightness/contrast to get a better view of the image details. <br />
<br />
<source lang="python"><br />
# In CASA. Create default clean image.<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.Reg.Clean.niter1K', <br />
imsize=1280, cell='8arcsec', niter=1000, interactive=False)<br />
<br />
viewer('SNR_G55_10s.Reg.Clean.niter1K.image')<br />
</source><br />
<br />
The logger indicates that the image was obtained in two major cycles and some improvements over the dirty image are visible. But clearly we have not cleaned deep enough yet; the image still has many sidelobes, and an inspection of the residual image shows that it still contains source flux and structure. So let's increase the ''niter'' value to 10,000 and compare the images.<br />
<br />
<source lang="python"><br />
# In CASA. Create default clean image with niter = 10000<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.Reg.Clean.niter10K', <br />
imsize=1280, cell='8arcsec', niter=10000, interactive=False)<br />
<br />
viewer('SNR_G55_10s.Reg.Clean.niter10K.image')<br />
</source><br />
<br />
[[Image:SNR_G55_10s.rms.screen.png|400px|thumb|right|Fig. 5: Attempting to find the lowest rms value within the CLEAN'ed image using niter=1000, in order to calculate our threshold.]]<br />
<br />
As we can see from the resulting images, increasing the niter values (minor cycles) improves our image by reducing prominent sidelobes significantly. One could now further increase the ''niter'' parameter until the residuals are down to an acceptable level. To determine the number of iterations, one needs to keep in mind that {{clean}} will fail to converge once it starts cleaning too deeply into the noise. At that point, the cleaned flux and the peak residual flux values will start to oscillate as the number of iterations increase. This effect can be monitored on the CASA logger output. To avoid cleaning too deeply, we will set a ''threshold'' parameter that will stop minor cycle clean iterations once a peak residual value is being reached. <br />
<br />
First, will utilize the ''SNR_G55_10s.Reg.Clean.niter1K.image'' image to give us an idea of the rms noise (your sigma value).<br />
With the image open within the viewer, click on the 'Rectangle Drawing' button (Rectangle with R) and draw a square on the image at a position with little source or sidelobe contamination. Doing this should open up a "Regions" dock, which holds information about the region, including the selected pixel statistics in the aptly named "Statistics" tab. Take notice of the rms values as you click/drag the box around empty image locations, or by drawing additional boxes at suitable positions. <br />
<br />
The lowest rms value that we found was about 4E-5 Jy/beam, which we will use to calculate our threshold. There really is no set standard, but fairly good threshold values can vary anywhere between 2.0-4.0*sigma; using clean boxes (see the section on interactive cleaning) allows one to go to lower thresholds. For our purposes, we will choose a threshold of 2.5*sigma. Doing the math results in a value of 10E-5 or equivalently 0.10mJy/beam. Therefore, for future calls to the {{clean}} task, we will set ''threshold='0.1mJy'''. The clean cycle will be stopped when the residual peak flux equals or is less than the ''threshold'' value, or when the maximum number of iterations ''niter'' is reached. To ensure that the stopping criterion is indeed ''threshold'', ''niter'' should be set to a very high number. '''In the following, we nevertheless will use ''niter=1000'' to keep the execution times of {{clean}} on the low end as we focus on explaining different imaging methods.'''<br />
<br />
An alternative method to determine the approximate rms of an image is to use the [https://obs.vla.nrao.edu/ect/ VLA Exposure Calculator] and to enter the observing conditions.<br />
<br />
== CLEAN with Weights ==<br />
<br />
To see the effects of using different weighting schemes to the image, let's change the ''weighting'' parameter within {{clean}} and inspect the resulting images. We will be using the ''Natural, Uniform, and Briggs'' weighting algorithms. Here, we have chosen a smaller image size to mainly focus on our science target. <br />
<br />
<source lang="python"><br />
# In CASA. Natural weighting<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.natural', weighting='natural',<br />
imsize=540, cell='8arcsec', niter=1000, interactive=False, threshold='0.1mJy')<br />
<br />
# In CASA. Uniform weighting<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.uniform', weighting='uniform',<br />
imsize=540, cell='8arcsec', niter=1000, interactive=False, threshold='0.1mJy')<br />
<br />
# In CASA. Briggs weighting, with robust set to default of 0.0<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.briggs', weighting='briggs',<br />
robust=0, imsize=540, cell='8arcsec', niter=1000, interactive=False, threshold='0.1mJy')<br />
<br />
viewer()<br />
</source><br />
<br />
* ''weighting'': specification of the weighting scheme. For Briggs weighting, the robust parameter will be used.<br />
<br />
* ''threshold='0.1mJy''': threshold at which the cleaning process will halt.<br />
<br />
[[Image:SNR_G55_11s.nat.uni.briggs.png|1000px|thumb|center|Fig. 6: CLEAN images created with different weighting algorithms, including natural (left), uniform (center), and briggs (right).]]<br />
<br />
In Fig. 6 we see that the natural weighted image is most sensitive to extended emission (beam size of 46"x41"). The negative values around the extended emission (often referred to as a negative 'bowl') is a typical signature of missing short spacings, extended emission that even the minimum baseline is not able to detect. Uniform weighted data shows the highest resolution (26"x25") and Briggs 'robust=0' (default value) is a compromise with a beam of 29"x29". To be more sensitive to the extended emission, the 'robust' parameter could be tweaked further toward more positive values.<br />
<br />
== Multi-Scale CLEAN ==<br />
<br />
Since G55.7+3.4 is an extended source with many spatial scales, a more advanced form of imaging involves the use of multiple scales. MS-CLEAN is an extension of the classical CLEAN algorithm for handling extended sources. It works by assuming the sky is composed of emission at different spatial scales and works on them simultaneously, thereby creating a linear combination of images at different spatials scales. For a more detailed description of Multi Scale CLEAN, see the paper by J.T. Cornwell entitled [http://arxiv.org/abs/0806.2228 Multi-Scale CLEAN deconvolution of radio synthesis images].<br />
<br />
We will use a set of scales (which are expressed in units of the requested pixel, or cell, size) which are representative of the scales that are present in the data, including a zero-scale for point sources. <br />
<br />
[[Image:SN_G55.MultiScale.image.png|200px|thumb|right|Fig. 7: G55.7+3.4 Multi-Scale Clean]]<br />
[[Image:SN_G55_MultiScale.artifacts.png|200px|thumb|right|Fig. 8: Artifacts around point sources]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.MultiScale', <br />
imsize=1280, cell='8arcsec', multiscale=[0,6,10,30,60], smallscalebias=0.9, <br />
interactive=False, niter=1000, weighting='briggs', stokes='I', <br />
threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.MultiScale.image')<br />
</source><br />
<br />
<br />
<br />
* ''multiscale=[0,6,10,30,60]'': a set of scales on which to clean. A good rule of thumb when using multiscale is 0, 2xbeam, 5xbeam (where beam is the synthesized beam), and larger scales up to about half the minor axis maximum scale of the mapped structure. Since these are in units of the pixel size, our chosen values will be multiplied by the requested cell size. Thus, we are requesting scales of 0 (a point source), 48, 80, 240, and 480 arcseconds (8 arcminutes). Note that 16 arcminutes (960 arcseconds) roughly corresponds to the size of G55.7+3.4.<br />
<br />
* ''smallscalebias=0.9'': This parameter is known as the '''small scale bias''', and helps with faint extended structure, by balancing the weight given to smaller structures which tend to be brighter, but have less flux density. Increasing this value gives more weight to smaller scales. A value of 1.0 weighs the largest scale to zero, and a value of less than 0.2 weighs all scales nearly equally. The default value is 0.6.<br />
<br />
The logger will show how much cleaning is performed on the individual scales.<br />
<br />
This is the fastest of the imaging techniques described here, but it's easy to see that there are artifacts in the resulting image (Fig. 7). We can use the {{viewer}} to explore the point sources near the edge of the field by zooming in on them (Fig. 8). Click on the "Zooming" button on the top left corner and highlight an area by making a square around the portion where you would like to zoom-in. Double clicking within the square will zoom-in to the selected area. The square can be resized by clicking/dragging the corners, or removed by pressing the "Esc" key. After zooming in on the area, we can see some radio sources have prominent arcs, as well as spots with a six-pointed pattern surrounding them.<br />
<br />
Next we will explore some more advanced imaging techniques to mitigate the artifacts seen towards the edge of the image.<br />
<br />
== Multi-Scale, Wide-Field CLEAN (w-projection) ==<br />
<br />
[[Image:Faceting.png|250px|thumb|right|Fig. 9: Faceting when using widefield gridmode, which can be used in conjunction with w-projection. ]]<br />
[[Image:SNR_G55_MS_vs_MS.wProj.png|500px|thumb|right|Fig. 10: Multi-Scale image of arcs around point sources far from the phase center, versus MS with w-projection (right). We can see the that combining the w-projection algorithm with the multiscale algorithm improves the resulting image by removing prominent artifacts.]]<br />
<br />
The next {{clean}} algorithm we will employ is w-projection, which is a wide-field imaging technique that takes into account the non-coplanarity of the baselines as a function of distance from the phase center (Fig. 9). For wide-field imaging, the sky curvature and non-coplanar baselines results in a non-zero w-term. The w-term introduced by the sky and array curvature introduces a phase term that will limit the dynamic range of the resulting image. Applying 2-D imaging to such data will result in artifacts around sources away from the phase center, as we saw in running MS-CLEAN. Note that this affects mostly the lower frequency bands, especially for the more extended configurations, due to the field of view decreasing with higher frequencies. <br />
<br />
The w-term can be corrected by faceting (describe the sky curvature by many smaller planes) in either the image or uv-plane, or by employing w-projection. A combination of the two can also be employed within {{clean}} by setting the parameter ''gridmode='widefield'''. If w-projection is employed, it will be done for each facet. Note that w-projections is an order of magnitude faster than the faceting algorithm, but will require more memory. <br />
<br />
For more details on w-projection, as well as the algorithm itself, see [http://adsabs.harvard.edu/abs/2008ISTSP...2..647C "The Noncoplanar Baselines Effect in Radio Interferometry: The W-Projection Algorithm"]. Also, the chapter on [http://www.aspbooks.org/a/volumes/article_details/?paper_id=17953 Imaging with Non-Coplanar Arrays] may be helpful. <br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.wProj',<br />
gridmode='widefield', imsize=1280, cell='8arcsec',<br />
wprojplanes=-1, multiscale=[0,6,10,30,60], <br />
interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.ms.wProj.image')<br />
</source><br />
<br />
* ''gridmode='widefield''': Use the w-projection algorithm.<br />
<br />
* ''wprojplanes=-1'': The number of w-projection planes to use for deconvolution. Setting to -1 forces CLEAN to utilize an acceptable number of planes for the given data.<br />
<br />
This will take slightly longer than the previous imaging round; however, the resulting image (Fig. 10) has noticeably fewer artifacts. In particular, compare the same outlier source in the Multi-Scale w-projected image with the Multi-Scale-only image: note that the swept-back arcs have disappeared. There are still some obvious imaging artifacts remaining, though.<br />
<br />
== Multi-Scale, Multi-Term Frequency Synthesis ==<br />
[[Image:MultiFrequency_Synthesis_snapshot.png|250px|thumb|right|Fig. 11: Multi-Frequency Synthesis snapshot of (u,v) coverage. We can see from the image on the right, using this algorithm can greatly improve coverage, thereby improving image fidelity.]]<br />
[[Image:SNR_G55.ms_to_mfs.gif|200px|thumb|right|Fig. 12: Multi-Scale image artifacts versus MS-MFS artifacts near SNR, with nterms=2. We can see artifacts around point sources diminish, improving our image.]]<br />
[[Image:SN_G55_MS.MFS.alpha.png|200px|thumb|right|Fig. 13: Spectral Index image]]<br />
<br />
A consequence of simultaneously imaging the wide fractional bandwidths available with the VLA is that the primary and synthesized beams have substantial frequency-dependent variation over the observing band. If this is not accounted for, it will lead to imaging artifacts and compromise the achievable image rms.<br />
<br />
The dimensions of the (u,v) plane are measured in wavelengths, and therefore observing at several frequencies, a baseline can sample several ellipses in the (u,v) plane, each with different sizes. We can therefore fill in the gaps in the single frequency (u,v) coverage (Fig. 11) to achieve a much higher image fidelity. This method is called Multi-Frequency Synthesis (MFS). Also when observing in low-frequencies, it may prove beneficial to observe in small time-chunks, which are spread out in time. This will allow the coverage of more spatial-frequencies, allowing us to employ this algorithm more efficiently. <br />
<br />
The Multi-Scale Multi-Frequency-Synthesis algorithm (also known ad Multi-Term Frequency Synthesis) provides the ability to simultaneously image and fit for the intrinsic source spectrum. The spectrum is approximated using a polynomial Taylor Term expansion in frequency, with the degree of the polynomial as a user-controlled parameter. A least-squares approach is used, along with the standard clean-type iterations. <br />
<br />
For a more detailed explanation of the MS-MFS deconvolution algorithm, please see the paper by Urvashi Rau and Tim J. Cornwell entitled [http://arxiv.org/abs/1106.2745 A multi-scale multi-frequency deconvolution algorithm for synthesis imaging in radio interferometry]<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MFS',<br />
imsize=1280, cell='8arcsec', mode='mfs', nterms=2,<br />
multiscale=[0,6,10,30,60], <br />
interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.image.tt0')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.image.alpha')<br />
</source><br />
<br />
* ''nterms=2'': the number of Taylor terms to be used to model the frequency dependence of the sky emission. Note that the speed of the algorithm will depend on the value used here (more terms will be slower). nterms=2 will fit a spectral index, and nterms=3 a spectral index and curvature.<br />
<br />
This will take much longer than the two previous methods, so it would probably be a good time to have coffee or chat about VLA data reduction with your neighbor at this point. <br />
<br />
When clean is done <imagename>.image.tt0 will contain a total intensity image (Fig. 12), where tt* is a suffix to indicate the Taylor term; <imagename>.image.tt0 is the total intensity image and <imagename>.image.alpha will contain an image of the spectral index in regions where there is sufficient signal-to-noise (Fig. 13). Having this spectral index image can help convey information about the emission mechanism involved within the supernova remnant. It can also give information on the optical depth of the source. <br />
<br />
For more information on the multi-term, multi-frequency synthesis mode and its outputs, see section 5.2.5.1 in the [https://casa.nrao.edu/docs/cookbook/casa_cookbook006.html CASA cookbook]. <br />
<br />
Inspect the brighter point sources in the field near the supernova remnant. You will notice that some of the artifacts, which had been symmetric around the sources themselves, are now gone; however, since we did not use W-Projection this time, there are still strong features related to the non-coplanar baseline effects still apparent for sources further away.<br />
<br />
At this point, {{clean}} takes into account the frequency variation of the synthesized beam but not the frequency variation of the primary beam. For low frequencies and large bandwidths, this can be substantial. E.g. 1-2GHz L-band observations result in a variation of a factor of 2. One effect of such a large fractional bandwidth is that in multi-frequency synthesis primary beam nulls will be blurred and the interferometer is sensitive everywhere in the field of view. For spectral slopes, however, a frequency-dependent primary beam causes a steepening as the higher frequencies are less sensitive at a given point away from the phase center. A correction for this effect should be made with the task {{widebandpbcor}} (see below).<br />
<br />
== Multi-Scale, Multi-Term Frequency, Widefield CLEAN ==<br />
<br />
Finally, we will combine the W-Projection and MS-MFS algorithms. Be forewarned -- these imaging runs will take a while, and it's best to start them running and then move on to other things.<br />
<br />
Using the same parameters for the individual-algorithm images above, but combined into a single {{clean}} run, we have:<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MFS.wProj',<br />
gridmode='widefield', imsize=1280, cell='8arcsec', mode='mfs',<br />
nterms=2, wprojplanes=-1, multiscale=[0,6,10,30,60], <br />
interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.wProj.image.tt0')<br />
<br />
viewer('SNR_G55_10s.ms.MFS.wProj.image.alpha')<br />
</source><br />
<br />
[[Image:SNR_G55_MS.MFS.wProj.png|1250px|thumb|center|Fig. 14: Here we see the differences as the images progress through the different algorithms used: MS -> MS-MFS -> MS-wProjection -> MS-MFS-wProjection.]]<br />
<br />
Again, looking at the same outlier source, we can see that the major sources of error have been removed, although there are still some residual artifacts (Fig. 14). One possible source of error is the time-dependent variation of the primary beam; another is the fact that we have only used ''nterms=2'', which may not be sufficient to model the spectra of some of the point sources. Some weak RFI may also show up that may need additional flagging.<br />
<br />
== Imaging Outlier Fields ==<br />
<br />
When strong sources are far away from the main target but still strong enough to produce sidelobes in the main image, they should be cleaned. Sometimes, however, it is not practical to image very large images for this purpose. An alternative is to use outlier fields. This mode will allow the user to specify a few locations that are then being cleaned along with the main image. The outlier fields should be centered on strong sources that, e.g., are known from sky catalogs or are identified by other means. <br />
<br />
Outlier fields are specified in an outlier file (here: 'outliers.txt') that contains the names, sizes, and positions (see Sect. 5.3.18.1 of the CASA cookbook): <br />
<br />
<pre><br />
#content of outliers.txt<br />
#<br />
#outlier field1<br />
imagename= 'Outlier1.MS.MFS'<br />
imsize=[512,512]<br />
phasecenter = 'J2000 19h23m27.693 22d37m37.180'<br />
#<br />
#outlier field2<br />
imagename='Outlier2.MS.MFS'<br />
imsize=[512,512]<br />
phasecenter = 'J2000 19h25m46.888 21d22m03.365'<br />
<br />
</pre><br />
<br />
{{clean}} will then be executed like the following:<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename='SNR.MS.MFS-Main',<br />
outlierfile='outliers.txt',<br />
imsize=[512,512], cell='8arcsec', mode='mfs',<br />
multiscale=[0,6,10,30,60], interactive=False, niter=1000, weighting='briggs',<br />
robust=0, stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean')<br />
</source><br />
<br />
* ''outlierfile='outliers.txt''': the name of the outlier file.<br />
<br />
<!--<br />
Now we will image the supernova remnant, as well as the bright sources (outliers) towards the edges of the image, creating a small image for each one. For this, we will be utilizing {{clean}} without widefield mode (try it to see the effects) as we will be specifying a phase center, and the images will not be too big, therefore we will not have sources very far from the phase center. We will specify an outlier file with the the names, sizes, and positions of each outlier image. This form of imaging is useful for when you have several outlier fields you'd like to image in one go. It is particularly useful to clean out far away sources that throw sidelobes into the main image and need to be cleaned out. (see Sect. 5.3.18.1 of the CASA cookbook).<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
clean(vis='SNR_G55_10s.calib.ms', imagename=['SNR.MS.MFS', 'Outlier1.MS.MFS', 'Outlier2.MS.MFS'],<br />
imsize=[[512,512],[512,512],[512,512]], cell='8arcsec', mode='mfs',<br />
multiscale=[0,6,10,30,60], interactive=False, niter=1000, weighting='briggs',<br />
stokes='I', threshold='0.1mJy', usescratch=F, imagermode='csclean',<br />
phasecenter=['J2000 19h21m38.271 21d45m48.288', 'J2000 19h23m27.693 22d37m37.180', 'J2000 19h25m46.888 21d22m03.365'])<br />
</source><br />
--> <br />
<br />
[[Image:Outlier_Fields.png|1000px|thumb|center|Fig. 15: Images of the supernova remnant and bright outlying sources. ]]<br />
<br />
== Primary Beam Correction ==<br />
<br />
In interferometry, the images formed via deconvolution are representations of the sky, multiplied by the primary beam response of the antenna. The primary beam can be described by a Gaussian with a size depending on the observing frequency (see above). Images produced via {{clean}} are by default not corrected for the primary beam pattern (important for mosaics), and therefore do not have the correct flux away from the phase center. <br />
<br />
Correcting for the primary beam, however, can be done during {{clean}} by using the ''pbcor'' parameter. It can also be done after imaging using the task [https://casa.nrao.edu/docs/TaskRef/impbcor-task.html impbcor] for regular data sets, and {{widebandpbcorr}} for those that use Taylor-term expansion (nterms > 1). A third possibility, is utilizing the task {{immath}} to manually divide the ''<imagename>.image'' by the ''<imagename>.flux'' images (''<imagename>.flux.pbcoverage'' for mosaics). <br />
<br />
Flux corrected images usually don't look pretty, due to the noise at the edges being increased. Flux densities, however, should only be calculated from primary beam corrected images. Let's run the {{impbcor}} task to correct our multiscale image.<br />
<br />
<source lang="python"><br />
# In CASA<br />
impbcor(imagename='SNR_G55_10s.MultiScale.image', pbimage='SNR_G55_10s.MultiScale.flux', <br />
outfile='SNR_G55_10s.MS.pbcorr.image', mode='divide')<br />
</source><br />
* ''imagename'': the image to be corrected<br />
<br />
* ''pbimage'': the <imagename>.flux image as a representation of the primary beam (<imagename>.flux.pbcoverage for mosaics)<br />
<br />
Let us now use the {{widebandpbcor}} task for wideband (''nterms''>1) images. Note that for this task, we will be supplying the image name that is the prefix for the Taylor expansion images, tt0 and tt1, which must be on disk. Such files were created, e.g., during the last Multi-Scale, Multi-Frequency, Widefield run of CLEAN. <br />
<br />
[[Image:SNR_G55_10s.MS.MFS.wProj.pbcor.image.png|200px|thumb|right|Fig. 16: Primary beam corrected image using the MS.MFS.wProj image created during the clean process.]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
widebandpbcor(vis='SNR_G55_10s.calib.ms', imagename='SNR_G55_10s.ms.MFS.wProj', <br />
nterms=2, action='pbcor', pbmin=0.2, spwlist=[0,1,2,3], <br />
weightlist=[0.5,1.0,0,1.0], chanlist=[63,63,63,63], threshold='0.1Jy')<br />
</source><br />
<br />
* ''spwlist=[0,1,2,3]'': We want to apply this correction to all spectral windows in our calibrated measurement set. <br />
<br />
* ''weightlist=[0.5,1.0,0,1.0]'': Since we did not specify reference frequencies during {{clean}}, the {{widebandpbcor}} task will pick them from the provided image. Running the task, the logger reports the multiple frequencies used for the primary beam, which are 1.256, 1.429, 1.602, and 1.775 GHz. Having created an amplitude vs. frequency plot of the calibrated measurement set with colorized spectral windows using {{plotms}}, we notice that the first chosen frequency lies within spectral window 0, which we know had lots of flagged data due to lots of RFI being present. This ''weightlist'' parameter allows us to give this chosen frequency less weight. The primary beam at 1.6GHz lies in an area with no data, therefore we will give a weight value of zero for this frequency. The remaining frequencies 1.429 and 1.775 GHz lie within spectral windows which contained less RFI, therefore we provide a larger weight percentage. <br />
<br />
* ''pbmin=0.2'': Gain level below which not to compute Taylor-coefficients or apply a primary beam correction.<br />
<br />
* ''chanlist=[63,63,63,63]'': Our measurement set contains 64 channels, including zero.<br />
<br />
* ''threshold='0.1Jy''': Threshold in the intensity map, below which not to recalculate the spectral index.<br />
<br />
It's important to note that the image will cut off at about 20% of the HPBW, as we are confident of the accuracy within this percentage. Anything outside becomes less accurate, thus there is a mask associated with the creation of the corrected primary beam image (Fig. 16). <br />
<br />
It would be a good exercise to use the {{viewer}} to plot both the primary beam corrected image, and the original cleaned image and compare the intensity (Jy/beam) values, which should differ slightly.<br />
<br />
== Imaging Spectral Cubes ==<br />
<br />
For spectral line imaging, CASA {{clean}} can be used in either ''mode='frequency' '' or ''mode='velocity' ''. Both will create a spectral axis in frequency. The velocity mode adds an additional velocity label to it. <br />
<br />
The following keywords are important for spectral modes (velocity in this example):<br />
<br />
<source lang="python"><br />
# In CASA<br />
mode = 'velocity' # Spectral gridding type (mfs, channel, velocity, frequency)<br />
nchan = -1 # Number of channels (planes) in output image; -1 = all<br />
start = '' # Velocity of first channel: e.g '0.0km/s'(''=first channel in first SpW of MS)<br />
width = '' # Channel width e.g '-1.0km/s' (''=width of first channel in first SpW of MS)<br />
<snip><br />
outframe = '' # spectral reference frame of output image; '' =input<br />
veltype = 'radio' # Velocity definition of output image<br />
<br />
<snip><br />
restfreq = '' # Rest frequency to assign to image (see help)<br />
</source><br />
<br />
The spectral dimension of the output cube will be defined by these parameters and {{clean}} will regrid the visibilities to it. Note that invoking {{cvel}} before imaging is in most cases not necessary, even when two or more measurement sets are being provided at the same time. <br />
<br />
The cube is specified by a ''start'' velocity in km/s, the ''nchan'' number of channels and a channel ''width'' (where the latter can also be negative for decreasing velocity cubes). <br />
<br />
To correct Doppler motions, {{clean}} also requires a velocity frame, where ''LSRK'' and ''BARY'' are the most popular Local Standard of Rest (kinematic) and sun-earth Barycenter references. ''doppler'' defines whether the data will be gridded via the optical or radio velocity definition. A description of the available options and definitions can be found in the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/line VLA observing guide] and the [https://casa.nrao.edu/docs/cookbook/casa_cookbook014.html CASA cookbook]. By default, {{clean}} will produce a cube in LRSK and radio. <br />
<br />
Note that {{clean}} will always work on the entire cube when searching for the highest residual fluxes in the minor cycle. To do this per channel, one can set ''chaniter=T''.<br />
<br />
CASA also offers ''mode='channel' ''. But we do not recommend this mode as the VLA does not observe with Doppler tracking. In other words, the fixed frequency observing of the VLA will let any spectral feature move through frequency space in time. The Doppler correction will then be in {{clean}} via ''mode='velocity''' or '' 'frequency' '' and not on the telescope itself. ''mode='channel' '' can also be confusing when imaging some shifted, but overlapping spectral windows.<br />
<br />
== Beam per Plane ==<br />
For large spectral cubes, the synthesized beam can vary substantially across the channels. To account for this, CASA will calculate separate beams for each channel when the difference is more than half a pixel across the cube. All CASA image analysis tasks are capable to handle such cubes. <br />
<br />
If it is desired to have a cube with a single synthesized beam, two options are available. Best is to use {{imsmooth}} with ''kernel='commonbeam' ''. This task will smooth each plane to that of the lowest resolution. It will also clean up all header variables such that only a single beam appears in the data. The second option is to define ''resmooth=T'' directly in {{clean}}.<br />
<br />
== Image Header ==<br />
<br />
The image header holds meta data associated with your CASA image. The task {{imhead}} will display this data within the casalog. We will first run {{imhead}} with ''mode='summary' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0', mode='summary')<br />
</source><br />
<br />
* ''mode='summary' '': gives general information about the image, including the object name, sky coordinates, image units, the telescope the data was taken with, and more. <br />
<br />
For further information about the image, let's now run it with ''mode='list' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0', mode='list')<br />
</source><br />
<br />
* ''mode='list' '': gives more detailed information, including beam major/minor axes, beam primary angle, and the location of the max/min intensity, and lots more. Essentially this mode displays the FITS header variables.<br />
<br />
<br />
Next, we convert our image from intensity to main beam brightness temperature.<br />
<br />
We will use the standard equation <br />
<br />
<math>T=1.222\times 10^{6} \frac{S}{\nu^{2} \theta_{maj} \theta_{min}} </math><br />
<br />
where the main beam brightness temperature <math>T</math> is given in K, the intensity <math>S</math> in Jy/beam, the reference frequency <math>\nu</math> in GHz, and the major and minor beam sizes <math>\theta</math> in arcseconds. <br />
<br />
For a beam of 29.30"x29.03" and a reference frequency of 1.579GHz (as taken from the previous {{imhead}} run) we calculate the brightness temperature using {{immath}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
immath(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0', mode='evalexpr', <br />
expr='1.222e6*IM0/1.579^2/(29.30*29.03)', <br />
outfile='SNR_G55_10s.ms.MFS.wProj.image.tt0-Tb')<br />
</source><br />
* ''mode='evalexpr' '': {{immath}} is used to calcuate with images<br />
<br />
* ''expr'': the mathematical expression to be evaluated. The images are abbreviated as IM0, IM1, ... in the sequence of their appearance in the ''imagename'' parameter.<br />
<br />
Finally, we will change the unit of the new image header to 'K'. <br />
To do this, we will run {{imhead}} with ''mode='put' '':<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead(imagename='SNR_G55_10s.ms.MFS.wProj.image.tt0-Tb', mode='put', hdkey='bunit', hdvalue='K')<br />
</source><br />
* ''hdkey'': the header keyword that will be changed<br />
<br />
* ''hdvalue'': the new value for header keyword<br />
<br />
Launching the viewer will now show our image in brightness temperature units: <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('SNR_G55_10s.ms.MFS.wProj.image.tt0-Tb')<br />
</source><br />
<br />
<br />
[[Main Page | &#8629; '''CASAguides''']]<br />
<br />
<!--<br />
-- Original: Miriam Hartman <br /><br />
-- Modifications: Lorant Sjouwerman (4.4.0, 2015/07/07) <br /><br />
-- Modifications: Juergen Ott (4.5.2, 2016/04/14) <br /><br />
-- Topical Guide: Jose Salcido (4.5.2, 2016/04/18) <br /><br />
-- Modifications: Juergen Ott (4.5.2, 2016/04/20) <br /><br />
--><br />
{{Checked 4.5.2}}</div>Emomjian