https://casaguides.nrao.edu/api.php?action=feedcontributions&user=Jbertone&feedformat=atomCASA Guides - User contributions [en]2024-03-29T13:57:23ZUser contributionsMediaWiki 1.38.6https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2&diff=31244M100 Band3 SingleDish 6.22021-10-18T20:14:23Z<p>Jbertone: /* Imaging the Amplitude Calibrator and Measure the Value of Jy/K */</p>
<hr />
<div><div style="margin-left:8%; margin-right:8%; margin-top:4%; margin-bottom:4%"><br />
<br />
[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
*'''This guide is written for CASA 6.2 and uses Python 3'''<br />
<br />
*'''This guide assumes that you have downloaded the Total Power data in M100_Band3_TP_UncalibratedData.tgz as described in [[M100_Band3#Obtaining_the_Data]]'''<br />
<br />
*'''The processes described below WILL NOT OPERATE in CASA versions earlier than 6.2.0'''<br />
<br />
*'''Details of the ALMA observations are described at [[M100_Band3]]<br />
<br />
*'''This portion of the guide covers calibration and imaging starting from the raw visibility data.'''<br />
<br />
<br />
= Overview =<br />
<br />
This portion of CASA Guide will cover the data reduction of the Total Power (TP) array observations of M100. The objective is to generate from the TP dataset, a position-position-velocity cube suitable for merging with 7m and 12m datasets.<br />
<br />
The data consist of "amplitude calibrator" datasets (containing observational data of the quasar 3C279) and "science" datasets containing data for the science target M100. To be clear - the observations of 3C279 and M100 both follow the same general process - i.e. they are themselves calibrated with frequent observations of ''blank sky'' - the so-called 'OFF', which is used to determine the System temperature as a function of frequency, and form a ''referenced'' bandpass, in units of Antenna Temperature (Ta*) [K].<br />
<br />
Ultimately the quasar data are compared with recent flux measurements (in Jy/beam), obtained from archived 12m data, to determine the scaling factor between Ta* and Jy/beam, and that scaling factor is then applied to the science data, to bootstrap to Jy/beam.<br />
<br />
The combination of the resultant image with the interferometric (12-m array and 7-m array) data is explained in a separate page, valid for CASA versions 6.2 and later [[M100_Band3_Combine_6.2]].<br />
<br />
<br />
<br />
==Confirm Your Version of CASA==<br />
This guide has been written for CASA release 6.2.0. Please confirm your version before proceeding.<br />
The value returned from ''casalith.version_string()'' should be 6.2.0. If not, please obtain at least version 6.2.0 (or later), from [https://casa.nrao.edu/casa_obtaining.shtml]<br />
<br />
<source lang="python" style="display:inline-block"><br />
#IN CASA<br />
version = casalith.version_string()<br />
print("You are using " + version)<br />
if (version < '6.2.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
== Summary of Datasets ==<br />
<br />
The observations were made on the 1st, 5th, 7th, and 17th July 2014, using two or three 12-m antennas and the ACA correlator.<br />
The table below indicates the ID's of the Execution Blocks, their start and end times, and the antennas in the array.<br />
There are nine science datasets (i.e., two or three per day).<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Science<br />
uid___A002_X85c183_X36f Observed from 2014-07-01T21:51:26.2 to 2014-07-01T22:40:28.4 (UTC) DA61, PM03, PM04<br />
uid___A002_X85c183_X60b Observed from 2014-07-01T22:43:50.0 to 2014-07-01T23:32:39.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_X2ab Observed from 2014-07-05T23:58:03.6 to 2014-07-06T00:46:52.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X8602fa_X577 Observed from 2014-07-06T00:55:17.8 to 2014-07-06T01:44:07.3 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_X2d4 Observed from 2014-07-07T23:03:48.1 to 2014-07-07T23:53:47.9 (UTC) PM03, PM04<br />
uid___A002_X864236_X693 Observed from 2014-07-07T23:56:09.6 to 2014-07-08T00:46:07.1 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_Xd9 Observed from 2014-07-17T20:55:15.5 to 2014-07-17T21:44:06.1 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X664 Observed from 2014-07-17T22:24:17.3 to 2014-07-17T23:13:08.0 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X96c Observed from 2014-07-17T23:23:37.0 to 2014-07-18T00:12:25.3 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
Similarly, the concomitant ''Calibration'' datasets (used to convert from Antenna temperature units into Jy/beam units, for combination), are:<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Amplitude Calibrator<br />
uid___A002_X85c183_X895 Observed from 2014-07-01T23:35:23.1 to 2014-07-02T00:07:54.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_Xc3 Observed from 2014-07-05T23:21:25.6 to 2014-07-05T23:53:41.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_Xe1 Observed from 2014-07-07T22:27:35.4 to 2014-07-07T23:01:05.7 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_X3ae Observed from 2014-07-17T21:48:30.0 to 2014-07-17T22:20:52.2 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
= Script Calibration =<br />
The scripts listed below process all the single-dish science and calibrator data, using a few loops. <br />
<br />
# (optional) The calibrator (i.e. quasar) data are processed (see the section [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2#Imaging_the_Amplitude_Calibrator_and_Measure_the_Value_of_Jy.2FK Imaging the Amplitude Calibrator and Measure the Value of Jy/K] below)<br />
# The science data are processed with - [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]] <br />
# and the (calibrated) science data are imaged with - [[media:M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]<br />
<br />
Executing each of these scripts in order, will be sufficient to calibrate and image the science data (although the calibrator processing script can be omitted, as the relevant parameters are actually already explicitly written into the script).<br />
<br />
They can be invoked from the command line using the execfile command.<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
execfile('M100.casa6p1.calib.py')<br />
</pre><br />
<br />
= Manual Calibration =<br />
<br />
By way of description, and as a means to introduce and demonstrate some of the interactive and diagnostic, tools here we describe in more detail the steps taken by the scripts - although in this case we demonstrate the process for only '''one''' of the ASDM datasets. In the ''Imaging'' section, we assume '''all''' the ASDMs are appropriately reduced either by the manual process, or by using the script. <br />
<br />
The entire process takes less than 30 minutes.<br />
<br />
There are three basic steps to the Calibration of the Science data:<br />
# Import, inspection and flagging<br />
# Calibration<br />
#* Sky and bandpass calibration (i.e. Tsys and frequency sampling function)<br />
#* Adjustment for Correlator non-linearity (this step is necessary only for data taken in ALMA Cycle 3 and earlier)<br />
#* Conversion to Jy/beam<br />
# Imaging<br />
<br />
=== Import "Analysis Utilities" ===<br />
<br />
The [[Analysis_Utilities]] package will be used most critically for the imaging processes. While it is not necessary for completing the calibration stages, it is used to check data quality, and should be loaded.<br />
<br />
Import the package from https://casaguides.nrao.edu/index.php/Analysis_Utilities, and apply the class therein.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
</source><br />
<br />
== Import, inspection and flagging ==<br />
<br />
=== Import ===<br />
The first thing to do is to convert the dataset into the CASA Measurement Set (MS) format. The raw data have been provided to you in the ASDM (ALMA Science Data Model), which is the native format of the data produced by the observatory but cannot be processed by CASA. The conversion from ASDM to MS is done with the task {{importasdm}}.<br />
<br />
<source lang="python" style="overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=True)<br />
</source><br />
<br />
The converted dataset (with a suffix ".ms") is created and then flags embedded in the ASDM dataset (so-called "BDF flags") are transferred to MS using the bdfflags parameter.<br />
<br />
'''Note:''' The above code should be used for Cycle 3 data or later. Cycle 2 data and earlier should use:<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=False)<br />
os.system(os.environ['CASAPATH'].split()[0] + '/bin/bdflags2MS -f "COR DELA INT MIS SIG SYN TFB WVR ZER" uid___A002_X85c183_X36f uid___A002_X85c183_X36f.ms')<br />
</pre><br />
<br />
<br />
=== Inspection - listobs ===<br />
After import, the usual first step is then to get some basic information about the data.<br />
We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied.<br />
<br />
<source lang="python" style="display:inline-block><br />
# In CASA<br />
listobs(vis='uid___A002_X85c183_X36f.ms',<br />
listfile='uid___A002_X85c183_X36f.ms.listobs')<br />
</source><br />
<br />
The output will be written in a file named uid___A002_X85c183_X36f.ms.listobs which you can (and should) view at your leisure.<br />
{{listobs}} displays information about the targets, dates, frequencies, etc. Below is an excerpt from listobs for uid___A002_X85c183_X36f:<br />
<br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1215+1654 12:15:03.979140 +16.54.37.95680 J2000 0 14652<br />
1 none M100 12:22:54.360000 +15.48.50.60000 J2000 1 61509<br />
<br />
Spectral Windows: (25 unique spectral windows and 2 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_03#BB_1#SW-01#FULL_RES 124 TOPO 91955.512 -15625.000 1937500.0 90994.5750 1 XX YY<br />
2 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 90978.950 1734375.000 1734375.0 90978.9500 1 XX YY<br />
3 ALMA_RB_03#BB_2#SW-01#FULL_RES 124 TOPO 93893.012 -15625.000 1937500.0 92932.0750 2 XX YY<br />
4 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 92924.262 1937500.000 1937500.0 92924.2625 2 XX YY<br />
5 ALMA_RB_03#BB_3#SW-01#FULL_RES 124 TOPO 102033.637 15625.000 1937500.0 102994.5750 3 XX YY<br />
6 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 102986.762 1937500.000 1937500.0 102986.7625 3 XX YY<br />
7 ALMA_RB_03#BB_4#SW-01#FULL_RES 124 TOPO 104033.637 15625.000 1937500.0 104994.5750 4 XX YY<br />
8 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 104986.762 1937500.000 1937500.0 104986.7625 4 XX YY<br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
10 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100926.562 1781250.000 1781250.0 100926.5625 1 XX YY<br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
12 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102741.712 1781250.000 1781250.0 102741.7125 2 XX YY<br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
14 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112783.712 1781250.000 1781250.0 112783.7125 3 XX YY<br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
16 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114658.712 1781250.000 1781250.0 114658.7125 4 XX YY<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
18 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100949.756 1992187.500 1992187.5 100949.7559 1 XX YY<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
20 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102764.906 1992187.500 1992187.5 102764.9059 2 XX YY<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
22 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112806.906 1992187.500 1992187.5 112806.9059 3 XX YY<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
24 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114681.906 1992187.500 1992187.5 114681.9059 4 XX YY<br />
<br />
Sources: 48<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1215+1654 0 - - <br />
0 J1215+1654 25 - - <br />
0 J1215+1654 26 - - <br />
0 J1215+1654 27 - - <br />
0 J1215+1654 1 - - <br />
0 J1215+1654 2 - - <br />
0 J1215+1654 3 - - <br />
0 J1215+1654 4 - - <br />
0 J1215+1654 5 - - <br />
0 J1215+1654 6 - - <br />
0 J1215+1654 7 - - <br />
0 J1215+1654 8 - - <br />
0 J1215+1654 9 - - <br />
0 J1215+1654 10 - - <br />
0 J1215+1654 11 - - <br />
0 J1215+1654 12 - - <br />
0 J1215+1654 13 - - <br />
0 J1215+1654 14 - - <br />
0 J1215+1654 15 - - <br />
0 J1215+1654 16 - - <br />
0 J1215+1654 17 100950 0 <br />
0 J1215+1654 18 100950 0 <br />
0 J1215+1654 19 102794.1 0 <br />
0 J1215+1654 20 102794.1 0 <br />
0 J1215+1654 21 112794.1 0 <br />
0 J1215+1654 22 112794.1 0 <br />
0 J1215+1654 23 114669.1 0 <br />
0 J1215+1654 24 114669.1 0 <br />
1 M100 0 - - <br />
1 M100 25 - - <br />
1 M100 26 - - <br />
1 M100 27 - - <br />
1 M100 9 - - <br />
1 M100 10 - - <br />
1 M100 11 - - <br />
1 M100 12 - - <br />
1 M100 13 - - <br />
1 M100 14 - - <br />
1 M100 15 - - <br />
1 M100 16 - - <br />
1 M100 17 100950 0 <br />
1 M100 18 100950 0 <br />
1 M100 19 102794.1 0 <br />
1 M100 20 102794.1 0 <br />
1 M100 21 112794.1 0 <br />
1 M100 22 112794.1 0 <br />
1 M100 23 114669.1 0 <br />
1 M100 24 114669.1 0 <br />
<br />
Antennas: 3:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA61 A075 12.0 m -067.45.17.9 -22.53.21.4 -4.5609 -499.7012 23.0322 2225072.419944 -5440148.858968 -2481499.171703<br />
1 PM03 T701 12.0 m -067.45.18.8 -22.53.22.2 -29.1265 -522.7875 22.2052 2225045.995589 -5440149.141967 -2481520.118569<br />
2 PM04 T703 12.0 m -067.45.16.2 -22.53.23.9 42.8797 -575.6910 21.7763 2225104.700870 -5440102.471978 -2481568.689518<br />
</pre><br />
<br />
From this output you can for example see the followings.<br />
<br />
* From "Data records" section (not shown here): The execution consists of 15 scans with various scan intents.<br />
** Scans 1 and 2 are pointing and sideband gain ratio calibrations (done interferometrically), which need to be done prior to observing the target, on the quasar J1215+1654.<br />
** Scans 3 and 4 are interferometric delay and system noise temperature (Tsys) measurements also on J1215+1654 (they are in principle unnecessary; just a hack to make things happen on the telescope control software).<br />
** Scans 6, 7, 9, 10, 12, 13, and 15, whose scan intents contain "OBSERVE_TARGET", are for raster mapping of the target M100. The associated spectral window (SPW) ID's are 0 and 17-24.<br />
** Scans 5, 8, 11, and 14 with scan intents "CALIBRATE_ATMOSPHERE" are Tsys measurements for M100. The SPW ID's for Tsys scans are 0 and 9-16.<br />
* From "Spectral Windows" section:<br />
** SPW 0 contains the Water Vapor Radiometer (WVR) data, which are taken for all scans, but which are not used here.<br />
** SPW's 1, 3, 5, and 7 are placed on standard continuum frequencies and are used for the pointing scan only.<br />
** SPW's 9, 11, 13, and 15 cover the same frequency range as the science SPW's (see the next bullet) and are used to measure the Tsys used for calibration. They have 128 spectral channels (15.6 MHz spacing) in 2000 MHz bandwidth.<br />
** SPW's 17, 19, 21, and 23 cover the frequency ranges used for the actual science observations. They have 4080 spectral channels (488 kHz spacing) in 1992 MHz bandwidth. Hereafter these SPW's are referred to as "science" SPW's. The target line, CO J=1-0, is placed in SPW 23, the corresponding Tsys is taken from SPW 15.<br />
** The remaining, even-number SPW's contain channel-averages of the actually used SPW's (e.g., SPW 24 corresponds to SPW 23 averaged into one channel).<br />
* From "Antennas" section: Three 12-m antennas (DA61, PM03, and PM04) were used for the observations.<br />
<br />
=== Inspection - scanpattern ===<br />
The scan pattern of the raster mapping can be visualized by issuing the following command (after import of Analysis Utils - see the top of this page for information).<br />
A plot will be shown in a window and also saved as a PNG file uid___A002_X85c183_X36f.ms.sampling.png.<br />
<br />
<br />
[[File:X36f.TPSampling-6.1.png|200px|thumb|right|<caption>The scan pattern of the raster mapping for uid___A002_X85c183_X36f.</caption>]]<br />
<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms',<br />
showplot=True,<br />
plotfile='uid___A002_X85c183_X36f.ms.sampling.png')<br />
</source><br />
<br />
=== Inspection - System Noise Temperature ===<br />
Let's start by checking the Tsys.<br />
We use the task {{gencal}} to extract the Tsys into a CASA calibration table. This table is only used for inspection of Tsys quality, and not in further processing.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
gencal(vis='uid___A002_X85c183_X36f.ms',<br />
caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
caltype='tsys')<br />
</source><br />
<br />
The generated Tsys calibration table can be plotted using the task {{plotbandpass}}. The generated plots are saved in the directory<br />
uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime.<br />
<br />
<br />
[[File:X36f.Tsys.overlayTime-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for DA61 in uid___A002_X85c183_X36f. Colors represent time.</caption>]]<br />
<br />
<br />
<br />
[[File:X36f.Tsys.overlayAntenna-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for SPW 15 in uid___A002_X85c183_X36f, using overlay='antenna'. Colors represent antennas.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotbandpass(caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
overlay='time',<br />
xaxis='freq',<br />
yaxis='amp',<br />
subplot=22,<br />
buildpdf=False,<br />
interactive=False,<br />
showatm=True,<br />
pwv='auto',<br />
chanrange='5~123',<br />
showfdm=True,<br />
field='',<br />
figfile='uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime/uid___A002_X85c183_X36f.ms.tsys')<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
Although the "science" SPW's have 1992 MHz bandwidths with 4080 spectral channels, their edge channels can be very noisy because each intermediate frequency signal path (baseband) is equipped with a bandpass filter of ~1.8 GHz width. Note that this is not necessarily always the case, it is in general, safest to flag out the edge channels, unless a spectral line of interest falls very close to the edge of the SPW band-edge.<br />
We flag 120 channels on each side of the "science" SPW's.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis = 'uid___A002_X85c183_X36f.ms',<br />
mode = 'manual',<br />
spw = '17:0~119;3960~4079,19:0~119;3960~4079,21:0~119;3960~4079,23:0~119;3960~4079',<br />
action = 'apply',<br />
flagbackup = True)<br />
</source><br />
<br />
==Calibration - sky, bandpass, nonlinearity and baseline ==<br />
<br />
=== Tsys Calibration ===<br />
We calibrate the data into Antenna temperature in units of K, using the equation Ta* = Tsys*(ON-OFF)/OFF, where ON and OFF are the data on-source (i.e., during the raster scanning) and off-source (on a reference position where only background emission exists), respectively.<br />
The calibration is done by the task {{sdcal}} which requires matching of list of the "science" and Tsys spectral windows. <br />
Tsys observations will always have 128 channels over 2GHz bandwidth, and as described in {{listobs}}, we already know which spectral windows match the science spectral windows most closely, in frequency.<br />
for example, spectral window 9, with 128 channels matches well with spectral window 17, 11 with 19, 13 with 21 and 15 with 2. Thus, we form the Tsys and Science spectral window mapping input parameters, with the syntax: {9:[17], 11:[19], 13:[21],15:[23]}<br />
<br />
<pre style="background-color: #fffacd; overflow-x:auto; white-space: pre"><br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
.<br />
.<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
.<br />
.<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
.<br />
.<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
.<br />
.<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
</pre><br />
<br />
And the Tsys calibration command is formed like the following:<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdcal(infile = 'uid___A002_X85c183_X36f.ms',<br />
calmode = 'ps,tsys,apply',<br />
spwmap = {9:[17], 11:[19], 13:[21],15:[23]},<br />
spw = '17,19,21,23')<br />
</source><br />
<br />
'''Note:''' in this step, the calibrations are applied in situ. That is: the calibrated data are written into the original data - however it is written in such a way that any ''additional'' calls to sdcal will not apply any further calibrations. We will examine the quality of the bandpass and Tsys calibration after the correction for non-linearity, next.<br />
<br />
<br />
<br />
[[File:Vis_Scan6_7_9_10_12_13_15-6.1.png|200px|thumb|right|<caption>Output of plotms, showing spectra in Ta* [K]</caption>]]<br />
<br />
<br />
The calibrated bandpasses, averaged over each scan, can be plotted with:<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='23',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis.png',<br />
showgui=True)<br />
</source><br />
<br />
'''Note:''' For ''amplitude calibrator datasets'', (as opposed to science target maps) the calmode option for sdcal should be 'otfraster,tsys,apply' ('otfraster' instead of 'ps').<br />
'otfraster' tells the task to use both ends of each raster row as "OFF" data (as opposed to a dedicated "OFF" sky measurement).<br />
This is because temporal fluctuation of atmospheric emission is the dominant source of noise in radio continuum observations, and hence the "OFF" needs to be as close as possible to the "ON" in time (and spatial location).<br />
<br />
=== Application of Non-Linearity Correction Factor ===<br />
<br />
In Cycle 1 and 2, single-dish data taken with the ACA correlator suffer from non-linearity which originates in the digital signal processing.<br />
Its impact was thoroughly studied both experimentally and theoretically, and it was concluded that multiplication by a correction factor of 1.25 yields an amplitude accuracy of +/-5%.<br />
<br />
To preserve the data history, we use here the tasks {{split}} to split out the science ''corrected'' data columns - note that from here on, the spectral windows are '''renumbered: [0,1,2,3]'''. We then generate a calibration table, using a defined function ''to_amp_factor'', with {{gencal}}. Finally, we apply the calibration with {{applycal}}.<br />
We apply the correction here using tasks gencal and applycal - this is different to the methods used in the past, and are available now because of the migration to MS-only format of Single-dish CASA.<br />
<br />
'''Note:''' The ACA online software has been equipped with this correction for data from cycle 3 and later (it should be applied for data from cycle 2 and earlier).<br />
<br />
Also note that we need to import the math module in order to use the sqrt command in Python 3.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import math<br />
<br />
to_amp_factor = lambda x: 1. / math.sqrt(x)<br />
<br />
split(vis='uid___A002_X85c183_X36f.ms',<br />
outputvis='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='17,19,21,23')<br />
<br />
gencal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
caltable='uid___A002_X85c183_X36f.ms.nlc',<br />
caltype='amp', spw='',<br />
parameter=[to_amp_factor(1.25)])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
gaintable='uid___A002_X85c183_X36f.ms.nlc')<br />
</source><br />
<br />
'''Note:''' the Python command ''lambda'' simply defines a function. Thus the line above : ''to_amp_factor = lambda x: 1. / math.sqrt(x)'' defines a function where the call: ''to_amp_factor(4)'' will return: ''0.5''<br />
<br />
=== Baseline Subtraction (only for Science Datasets) ===<br />
<br />
'''Note:''' this step is appropriate for spectral line observations (science datasets) but ''should not be done for the continuum observations (amplitude calibrator datasets) -- it would eliminate the continuum emission!''<br />
'''Note:''' this step is optional and need not be completed to produce an image - however it is recommended for the majority of cases (esp. where the bandpass is affected by low-order modulations, and where the line-emission does not completely dominate the bandpass.)<br />
<br />
We will subtract low-order spectral baselines using the task {{sdbaseline}}. Typically these manifest from structure in the distribution of water in the atmosphere: the ON and OFF positions are slightly different, and the difference manifests as - typically - a low frequency variation in the spectra. The situation is exacerbated for observations closer to water lines and also for higher frequencies.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdbaseline(infile='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='',<br />
blfunc='poly',<br />
order=1,<br />
outfile='uid___A002_X85c183_X36f.ms.bl',<br />
overwrite=True)<br />
</source><br />
<br />
<br />
<br />
[[File: vis_bl_Scan6_7_9_10_12_13_15-6.1.png |200px|thumb|right|<caption>Output of plotms, showing result of baseline step, units in Ta* [K]</caption>]]<br />
<br />
<br />
The spectra can be checked using {{plotms}}, which we have already used in a previous step.<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms.bl',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='3',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis_bl.png',<br />
showgui=True)<br />
</source><br />
<br />
We do not concatenate separate Execution Blocks, because we need to determine and apply day-by-day Jy/K conversion factor.<br />
<br />
At this point in the guide, we have finished the calibration for ASDM uid___A002_X85c183_X36f (except for the calculation of the Jy/K conversion factors). In order to finish the calibration for the entire M100 dataset, the above steps should be repeated with each of the ADSMs shown in [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2#Summary_of_Datasets Summary of Datasets]. Once each ASDM has been calibrated, the user can move on to the flagging, Jy/K conversion, and imaging processes below.<br />
<br />
Note that one of the antennas, PM02, had a problem in the first baseband (SPW 17) on 2014-07-05, and its impact may not be specific to this one SPW (e.g., it may have resulted in poor pointing calibration).<br />
Therefore the PM02 data should be flagged from the affected datasets: uid___A002_X8602fa_X2ab (science), and uid___A002_X8602fa_X577 (science). The calibrator set uid___A002_X8602fa_Xc3 will also be affected, and will be flagged out in the subsequent processing of the calibrators.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis='uid___A002_X8602fa_X2ab.ms.bl',<br />
antenna='PM02&&&')<br />
flagdata(vis='uid___A002_X8602fa_X577.ms.bl',<br />
antenna='PM02&&&')<br />
</source><br />
<br />
<br />
== Convert science data from Kelvin to Jansky ==<br />
<br />
At this stage the data have been calibrated into units of antenna temperature, in Kelvins.<br />
<br />
It now remains to convert the science data into Jy/beam units - the ''Jy per K'' factors. This is important for combination of the data, with 7m and 12m sets. Later in this guide we show how to determine the correct factors from the calibrator datasets, for conversion (the process is essentially identical to processing the science datasets, with the exception that the baseline step is omitted from the calibration dataset processing). However to determine the factors, the calibration data must also be imaged and then interrogated to determine the appropriate values. <br />
<br />
Applying the calibration factors to the science data is the last step in the script [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]]. <br />
<br />
We continue to use only one ASDM as an example; uid___A002_X85c183_X36f. The script should be consulted for treatment of all the M100 Single-dish ASDMs (and the correct application of the Jy per K factors). Again, we apply the factors by making use of the {{gencal}} and {{applycal}} tasks. Also as before, we make use of the python function defined earlier: ''to_amp_factor''<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
<br />
factors = [41.52, 42.47, 43.58, 42.98]<br />
gencal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
caltable='uid___A002_X85c183_X36f.ms.jy',<br />
caltype='amp',<br />
spw='0,1,2,3',<br />
parameter=[ to_amp_factor(v) for v in factors ])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
gaintable='uid___A002_X85c183_X36f.ms.jy')<br />
</source><br />
<br />
= Image the Science Target =<br />
<br />
The final stage is to image the science data. This step is done by the script "[[media: M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]" which is invoked with ''execfile('M100.casa6p1.imaging.py') '' in CASA.<br />
This step uses a number of helper utilities to automatically determine the properties of the image data - the analysis Utilities library must be loaded (see the text above).<br />
<br />
== Determining grid parameters, gridding and obtaining grid metadata ==<br />
Define the list of the calibrated science datasets, which the CASA task {{sdimaging}} accepts. Note that this step is the same for science and calibrator data.<br />
This section is a little more python-heavy than the calibration step.<br />
<br />
In this section, we will simply process all the calibrated science data. The calibrator data should never be combined. There is no need to ''concatenate'' the calibrated data, as the imaging task can explicitly accept multiple datasets as input.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
asdmlist = ['uid___A002_X85c183_X36f',<br />
'uid___A002_X85c183_X60b',<br />
'uid___A002_X8602fa_X2ab',<br />
'uid___A002_X8602fa_X577',<br />
'uid___A002_X864236_X2d4',<br />
'uid___A002_X864236_X693',<br />
'uid___A002_X86fcfa_X664',<br />
'uid___A002_X86fcfa_X96c',<br />
'uid___A002_X86fcfa_Xd9']<br />
vislist = map(lambda x: x + '.ms.bl', asdmlist)<br />
</source><br />
<br />
<br />
We first have to define and obtain some basic parameters. As all the ASDMs are ostensibly of the same kinds of observations, we refer to the first science ASDM to determine the basic properties of the resulting data cube. The cube size, pixel size, theoretical beam size, central frequency and other basic parameters are extracted using some CASA-functions and the Analysis Utils library (see the top of this page for information on Analysis Utils).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
spw = 3<br />
imagename = 'M100_TP_CO_cube.spw%s.image'%(spw)<br />
blimagename = imagename + '.bl'<br />
integimagename = imagename + '.integ'<br />
fwhmfactor = 1.13<br />
diameter = 12<br />
<br />
xSampling, ySampling, maxsize = aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms', showplot=False)<br />
<br />
msmd.open(asdmlist[0]+'.ms.bl')<br />
freq = msmd.meanfreq(spw)<br />
msmd.close()<br />
print("SPW %d: %.3f GHz" % (spw, freq*1e-9))<br />
<br />
theorybeam = aU.primaryBeamArcsec(frequency=freq*1e-9,<br />
fwhmfactor=fwhmfactor,<br />
diameter=diameter)<br />
<br />
cell = theorybeam/9.0<br />
imsize = int(round(maxsize/cell)*2)<br />
</source><br />
And now we learn the central frequency of spectral window 3, is 114.682 GHz.<br />
Our cube will have a total length of 253.0 arcsec ('''maxsize'''), and with a cell side of 5.64 arcsec ('''cell''') -which samples the theoretical beam (''theory beam'') with 9x9 pixels, we will need 90 pixels along the image side ('''imsize''').<br />
<br />
'''Note:''' The Jy/K value depends on the parameters of gridding convolution (i.e., "gridfunction", "cell", and function-specific parameters ["convsupport" for gridfunction='SF']). That is, the same gridding parameters should be used for both amplitude calibrator and science images -- otherwise the calibration into Jy unit becomes invalid. Please refer to the CASA cookbook for information on the gridding kernels.<br />
<br />
<br />
Now the data are imaged.<br />
The velocity channel maps of the CO J=1-0 line (restfreq='115.271201800GHz') are created as a data cube which covers the velocity range of 1400-1700 km/s at a spacing of 5 km/s (start='1400km/s', width='5km/s', nchan=70).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA <br />
sdimaging(infiles=vislist,<br />
field='M100',<br />
spw='%s'%(spw),<br />
nchan=70,<br />
mode='velocity',<br />
start='1400km/s',<br />
width='5km/s',<br />
veltype='radio',<br />
outframe='lsrk',<br />
restfreq='115.271204GHz',<br />
gridfunction='SF',<br />
convsupport=6,<br />
stokes='I',<br />
phasecenter='J2000 12h22m54.9 +15d49m15',<br />
ephemsrcname='',<br />
imsize=imsize,<br />
cell='%sarcsec'%(cell),<br />
brightnessunit='Jy/beam',<br />
overwrite=True,<br />
outfile=imagename)<br />
</source><br />
<br />
Now you can browse the data cube using the viewer.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
viewer(imagename)<br />
</source><br />
<br />
== Subtract a Residual Background from the Image ==<br />
<br />
If you plot the line profile using the viewer, you may notice that the background (i.e., line-free channels) level is slightly negative.<br />
To correct this, spectral baselines are subtracted from the image using the task {{imcontsub}}.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
imcontsub(imagename=imagename,<br />
linefile=blimagename,<br />
contfile='M100.residualbaseline.image',<br />
fitorder=1,<br />
chans='0~7;62~69')<br />
os.system('rm -rf M100.residualbaseline.image')<br />
</source><br />
<br />
This task has two outputs - the baselined data is returned in the same input file (''image name - M100_TP_CO_cube.spw3.image.bl''), while the subtracted component is returned in ''M100.residual baseline.image"". We have no further need for the subtracted component, and it can be discarded.<br />
<br />
== Basic Analysis - Moments ==<br />
<br />
Make a 0th moment (integrated intensity) map using the task {{immoments}} and browse it using the viewer.<br />
<br />
<br />
[[File:M100.CO.cube.bl.image.mom0-6.1.png|200px|thumb|right|<caption>The integrated intensity map of M100.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
immoments(imagename=blimagename,<br />
moments=[0],<br />
axis='spectral',<br />
chans='8~61',<br />
outfile=integimagename)<br />
<br />
viewer(integimagename)<br />
</source><br />
<br />
= Imaging the Amplitude Calibrator and Measure the Value of Jy/K =<br />
<br />
Determination of the Jy/K conversion factor is achieved by imaging a source whose continuum flux is known and measuring the observed brightness temperature. One observation of this source - the amplitude calibrator - was made per day, giving a total of four datasets (see top).<br />
<br />
The true size of the beam (point spread function), on which the Jy/K value depends, in a map is determined by several ingredients.<br />
* The intrinsic beam size is determined by the antenna diameter and optics (and of course wavelength). Here we assume the intrinsic beam size of 1.13*lambda/D, where D=12 [m] is the diameter of the antennas, and the factor 1.13 is derived from the optics design of the ALMA 12-m antennas.<br />
* The effective beam size is broadened by the following causes:<br />
** The scanning pattern used to observe the map (i.e., sample spacing in both directions).<br />
** The method used to grid the individual spectra into a map/cube.<br />
The Analysis Utilities provides several tools to get estimates of these.<br />
<br />
It is not necessary, but If you wish to try this out for yourself, you will first need to download the raw amplitude calibrator data file M100_Band3_TP_ampcal_UncalibratedData.tgz. How to obtain the data is described at [[M100_Band3#Obtaining the Data]]. Once you have unpacked the data you will have a directory called M100_Band3_TP_ampcal_UncalibratedData. In this directory, execute the scripts for individual datasets:<br />
<br />
*[[media:A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py|A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py]]<br />
*[[media:A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py|A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py]]<br />
*[[media:A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py|A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py]]<br />
*[[media:A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py|A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py]]<br />
<br />
to calibrate, image and derive Jy/K factors of all four calibrator datasets. The general approach is almost identical to that of calibrating the science target data, with the exception of the baselining step. Then, Jy/K values are derived from the peak of temperature in images and stored in text files (with the suffix, '.jyperk.txt').<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
es = aU.stuffForScienceDataReduction()<br />
execfile('A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py')<br />
</pre><br />
<br />
The Calibration Jy/K values are determined as follows. Note that the values for individual antennas are averaged for each day, each SPW, because the antennas used for the Science observations were not necessarily available for the corresponding amplitude calibrator observations.<br />
<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
#Date Amp_Cal_Dataset SPW17 SPW19 SPW21 SPW23<br />
2014-07-01 uid___A002_X85c183_X895 41.52 42.47 43.58 42.98<br />
2014-07-05 uid___A002_X8602fa_Xc3 41.38 42.59 42.08 42.53<br />
2014-07-07 uid___A002_X864236_Xe1 40.88 42.48 41.47 41.94<br />
2014-07-17 uid___A002_X86fcfa_X3ae 41.71 42.88 41.69 42.18<br />
</pre><br />
<br />
= Combination with 12m and 7m Array Data =<br />
<br />
If you wish to learn how to combine the Total Power data with interferometric 12m and 7m data, see the guide for Data Combination [[M100_Band3_Combine_6.2]]<br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2&diff=31243M100 Band3 SingleDish 6.22021-10-18T19:43:20Z<p>Jbertone: </p>
<hr />
<div><div style="margin-left:8%; margin-right:8%; margin-top:4%; margin-bottom:4%"><br />
<br />
[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
*'''This guide is written for CASA 6.2 and uses Python 3'''<br />
<br />
*'''This guide assumes that you have downloaded the Total Power data in M100_Band3_TP_UncalibratedData.tgz as described in [[M100_Band3#Obtaining_the_Data]]'''<br />
<br />
*'''The processes described below WILL NOT OPERATE in CASA versions earlier than 6.2.0'''<br />
<br />
*'''Details of the ALMA observations are described at [[M100_Band3]]<br />
<br />
*'''This portion of the guide covers calibration and imaging starting from the raw visibility data.'''<br />
<br />
<br />
= Overview =<br />
<br />
This portion of CASA Guide will cover the data reduction of the Total Power (TP) array observations of M100. The objective is to generate from the TP dataset, a position-position-velocity cube suitable for merging with 7m and 12m datasets.<br />
<br />
The data consist of "amplitude calibrator" datasets (containing observational data of the quasar 3C279) and "science" datasets containing data for the science target M100. To be clear - the observations of 3C279 and M100 both follow the same general process - i.e. they are themselves calibrated with frequent observations of ''blank sky'' - the so-called 'OFF', which is used to determine the System temperature as a function of frequency, and form a ''referenced'' bandpass, in units of Antenna Temperature (Ta*) [K].<br />
<br />
Ultimately the quasar data are compared with recent flux measurements (in Jy/beam), obtained from archived 12m data, to determine the scaling factor between Ta* and Jy/beam, and that scaling factor is then applied to the science data, to bootstrap to Jy/beam.<br />
<br />
The combination of the resultant image with the interferometric (12-m array and 7-m array) data is explained in a separate page, valid for CASA versions 6.2 and later [[M100_Band3_Combine_6.2]].<br />
<br />
<br />
<br />
==Confirm Your Version of CASA==<br />
This guide has been written for CASA release 6.2.0. Please confirm your version before proceeding.<br />
The value returned from ''casalith.version_string()'' should be 6.2.0. If not, please obtain at least version 6.2.0 (or later), from [https://casa.nrao.edu/casa_obtaining.shtml]<br />
<br />
<source lang="python" style="display:inline-block"><br />
#IN CASA<br />
version = casalith.version_string()<br />
print("You are using " + version)<br />
if (version < '6.2.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
== Summary of Datasets ==<br />
<br />
The observations were made on the 1st, 5th, 7th, and 17th July 2014, using two or three 12-m antennas and the ACA correlator.<br />
The table below indicates the ID's of the Execution Blocks, their start and end times, and the antennas in the array.<br />
There are nine science datasets (i.e., two or three per day).<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Science<br />
uid___A002_X85c183_X36f Observed from 2014-07-01T21:51:26.2 to 2014-07-01T22:40:28.4 (UTC) DA61, PM03, PM04<br />
uid___A002_X85c183_X60b Observed from 2014-07-01T22:43:50.0 to 2014-07-01T23:32:39.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_X2ab Observed from 2014-07-05T23:58:03.6 to 2014-07-06T00:46:52.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X8602fa_X577 Observed from 2014-07-06T00:55:17.8 to 2014-07-06T01:44:07.3 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_X2d4 Observed from 2014-07-07T23:03:48.1 to 2014-07-07T23:53:47.9 (UTC) PM03, PM04<br />
uid___A002_X864236_X693 Observed from 2014-07-07T23:56:09.6 to 2014-07-08T00:46:07.1 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_Xd9 Observed from 2014-07-17T20:55:15.5 to 2014-07-17T21:44:06.1 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X664 Observed from 2014-07-17T22:24:17.3 to 2014-07-17T23:13:08.0 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X96c Observed from 2014-07-17T23:23:37.0 to 2014-07-18T00:12:25.3 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
Similarly, the concomitant ''Calibration'' datasets (used to convert from Antenna temperature units into Jy/beam units, for combination), are:<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Amplitude Calibrator<br />
uid___A002_X85c183_X895 Observed from 2014-07-01T23:35:23.1 to 2014-07-02T00:07:54.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_Xc3 Observed from 2014-07-05T23:21:25.6 to 2014-07-05T23:53:41.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_Xe1 Observed from 2014-07-07T22:27:35.4 to 2014-07-07T23:01:05.7 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_X3ae Observed from 2014-07-17T21:48:30.0 to 2014-07-17T22:20:52.2 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
= Script Calibration =<br />
The scripts listed below process all the single-dish science and calibrator data, using a few loops. <br />
<br />
# (optional) The calibrator (i.e. quasar) data are processed (see the section [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2#Imaging_the_Amplitude_Calibrator_and_Measure_the_Value_of_Jy.2FK Imaging the Amplitude Calibrator and Measure the Value of Jy/K] below)<br />
# The science data are processed with - [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]] <br />
# and the (calibrated) science data are imaged with - [[media:M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]<br />
<br />
Executing each of these scripts in order, will be sufficient to calibrate and image the science data (although the calibrator processing script can be omitted, as the relevant parameters are actually already explicitly written into the script).<br />
<br />
They can be invoked from the command line using the execfile command.<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
execfile('M100.casa6p1.calib.py')<br />
</pre><br />
<br />
= Manual Calibration =<br />
<br />
By way of description, and as a means to introduce and demonstrate some of the interactive and diagnostic, tools here we describe in more detail the steps taken by the scripts - although in this case we demonstrate the process for only '''one''' of the ASDM datasets. In the ''Imaging'' section, we assume '''all''' the ASDMs are appropriately reduced either by the manual process, or by using the script. <br />
<br />
The entire process takes less than 30 minutes.<br />
<br />
There are three basic steps to the Calibration of the Science data:<br />
# Import, inspection and flagging<br />
# Calibration<br />
#* Sky and bandpass calibration (i.e. Tsys and frequency sampling function)<br />
#* Adjustment for Correlator non-linearity (this step is necessary only for data taken in ALMA Cycle 3 and earlier)<br />
#* Conversion to Jy/beam<br />
# Imaging<br />
<br />
=== Import "Analysis Utilities" ===<br />
<br />
The [[Analysis_Utilities]] package will be used most critically for the imaging processes. While it is not necessary for completing the calibration stages, it is used to check data quality, and should be loaded.<br />
<br />
Import the package from https://casaguides.nrao.edu/index.php/Analysis_Utilities, and apply the class therein.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
</source><br />
<br />
== Import, inspection and flagging ==<br />
<br />
=== Import ===<br />
The first thing to do is to convert the dataset into the CASA Measurement Set (MS) format. The raw data have been provided to you in the ASDM (ALMA Science Data Model), which is the native format of the data produced by the observatory but cannot be processed by CASA. The conversion from ASDM to MS is done with the task {{importasdm}}.<br />
<br />
<source lang="python" style="overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=True)<br />
</source><br />
<br />
The converted dataset (with a suffix ".ms") is created and then flags embedded in the ASDM dataset (so-called "BDF flags") are transferred to MS using the bdfflags parameter.<br />
<br />
'''Note:''' The above code should be used for Cycle 3 data or later. Cycle 2 data and earlier should use:<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=False)<br />
os.system(os.environ['CASAPATH'].split()[0] + '/bin/bdflags2MS -f "COR DELA INT MIS SIG SYN TFB WVR ZER" uid___A002_X85c183_X36f uid___A002_X85c183_X36f.ms')<br />
</pre><br />
<br />
<br />
=== Inspection - listobs ===<br />
After import, the usual first step is then to get some basic information about the data.<br />
We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied.<br />
<br />
<source lang="python" style="display:inline-block><br />
# In CASA<br />
listobs(vis='uid___A002_X85c183_X36f.ms',<br />
listfile='uid___A002_X85c183_X36f.ms.listobs')<br />
</source><br />
<br />
The output will be written in a file named uid___A002_X85c183_X36f.ms.listobs which you can (and should) view at your leisure.<br />
{{listobs}} displays information about the targets, dates, frequencies, etc. Below is an excerpt from listobs for uid___A002_X85c183_X36f:<br />
<br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1215+1654 12:15:03.979140 +16.54.37.95680 J2000 0 14652<br />
1 none M100 12:22:54.360000 +15.48.50.60000 J2000 1 61509<br />
<br />
Spectral Windows: (25 unique spectral windows and 2 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_03#BB_1#SW-01#FULL_RES 124 TOPO 91955.512 -15625.000 1937500.0 90994.5750 1 XX YY<br />
2 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 90978.950 1734375.000 1734375.0 90978.9500 1 XX YY<br />
3 ALMA_RB_03#BB_2#SW-01#FULL_RES 124 TOPO 93893.012 -15625.000 1937500.0 92932.0750 2 XX YY<br />
4 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 92924.262 1937500.000 1937500.0 92924.2625 2 XX YY<br />
5 ALMA_RB_03#BB_3#SW-01#FULL_RES 124 TOPO 102033.637 15625.000 1937500.0 102994.5750 3 XX YY<br />
6 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 102986.762 1937500.000 1937500.0 102986.7625 3 XX YY<br />
7 ALMA_RB_03#BB_4#SW-01#FULL_RES 124 TOPO 104033.637 15625.000 1937500.0 104994.5750 4 XX YY<br />
8 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 104986.762 1937500.000 1937500.0 104986.7625 4 XX YY<br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
10 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100926.562 1781250.000 1781250.0 100926.5625 1 XX YY<br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
12 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102741.712 1781250.000 1781250.0 102741.7125 2 XX YY<br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
14 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112783.712 1781250.000 1781250.0 112783.7125 3 XX YY<br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
16 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114658.712 1781250.000 1781250.0 114658.7125 4 XX YY<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
18 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100949.756 1992187.500 1992187.5 100949.7559 1 XX YY<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
20 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102764.906 1992187.500 1992187.5 102764.9059 2 XX YY<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
22 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112806.906 1992187.500 1992187.5 112806.9059 3 XX YY<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
24 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114681.906 1992187.500 1992187.5 114681.9059 4 XX YY<br />
<br />
Sources: 48<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1215+1654 0 - - <br />
0 J1215+1654 25 - - <br />
0 J1215+1654 26 - - <br />
0 J1215+1654 27 - - <br />
0 J1215+1654 1 - - <br />
0 J1215+1654 2 - - <br />
0 J1215+1654 3 - - <br />
0 J1215+1654 4 - - <br />
0 J1215+1654 5 - - <br />
0 J1215+1654 6 - - <br />
0 J1215+1654 7 - - <br />
0 J1215+1654 8 - - <br />
0 J1215+1654 9 - - <br />
0 J1215+1654 10 - - <br />
0 J1215+1654 11 - - <br />
0 J1215+1654 12 - - <br />
0 J1215+1654 13 - - <br />
0 J1215+1654 14 - - <br />
0 J1215+1654 15 - - <br />
0 J1215+1654 16 - - <br />
0 J1215+1654 17 100950 0 <br />
0 J1215+1654 18 100950 0 <br />
0 J1215+1654 19 102794.1 0 <br />
0 J1215+1654 20 102794.1 0 <br />
0 J1215+1654 21 112794.1 0 <br />
0 J1215+1654 22 112794.1 0 <br />
0 J1215+1654 23 114669.1 0 <br />
0 J1215+1654 24 114669.1 0 <br />
1 M100 0 - - <br />
1 M100 25 - - <br />
1 M100 26 - - <br />
1 M100 27 - - <br />
1 M100 9 - - <br />
1 M100 10 - - <br />
1 M100 11 - - <br />
1 M100 12 - - <br />
1 M100 13 - - <br />
1 M100 14 - - <br />
1 M100 15 - - <br />
1 M100 16 - - <br />
1 M100 17 100950 0 <br />
1 M100 18 100950 0 <br />
1 M100 19 102794.1 0 <br />
1 M100 20 102794.1 0 <br />
1 M100 21 112794.1 0 <br />
1 M100 22 112794.1 0 <br />
1 M100 23 114669.1 0 <br />
1 M100 24 114669.1 0 <br />
<br />
Antennas: 3:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA61 A075 12.0 m -067.45.17.9 -22.53.21.4 -4.5609 -499.7012 23.0322 2225072.419944 -5440148.858968 -2481499.171703<br />
1 PM03 T701 12.0 m -067.45.18.8 -22.53.22.2 -29.1265 -522.7875 22.2052 2225045.995589 -5440149.141967 -2481520.118569<br />
2 PM04 T703 12.0 m -067.45.16.2 -22.53.23.9 42.8797 -575.6910 21.7763 2225104.700870 -5440102.471978 -2481568.689518<br />
</pre><br />
<br />
From this output you can for example see the followings.<br />
<br />
* From "Data records" section (not shown here): The execution consists of 15 scans with various scan intents.<br />
** Scans 1 and 2 are pointing and sideband gain ratio calibrations (done interferometrically), which need to be done prior to observing the target, on the quasar J1215+1654.<br />
** Scans 3 and 4 are interferometric delay and system noise temperature (Tsys) measurements also on J1215+1654 (they are in principle unnecessary; just a hack to make things happen on the telescope control software).<br />
** Scans 6, 7, 9, 10, 12, 13, and 15, whose scan intents contain "OBSERVE_TARGET", are for raster mapping of the target M100. The associated spectral window (SPW) ID's are 0 and 17-24.<br />
** Scans 5, 8, 11, and 14 with scan intents "CALIBRATE_ATMOSPHERE" are Tsys measurements for M100. The SPW ID's for Tsys scans are 0 and 9-16.<br />
* From "Spectral Windows" section:<br />
** SPW 0 contains the Water Vapor Radiometer (WVR) data, which are taken for all scans, but which are not used here.<br />
** SPW's 1, 3, 5, and 7 are placed on standard continuum frequencies and are used for the pointing scan only.<br />
** SPW's 9, 11, 13, and 15 cover the same frequency range as the science SPW's (see the next bullet) and are used to measure the Tsys used for calibration. They have 128 spectral channels (15.6 MHz spacing) in 2000 MHz bandwidth.<br />
** SPW's 17, 19, 21, and 23 cover the frequency ranges used for the actual science observations. They have 4080 spectral channels (488 kHz spacing) in 1992 MHz bandwidth. Hereafter these SPW's are referred to as "science" SPW's. The target line, CO J=1-0, is placed in SPW 23, the corresponding Tsys is taken from SPW 15.<br />
** The remaining, even-number SPW's contain channel-averages of the actually used SPW's (e.g., SPW 24 corresponds to SPW 23 averaged into one channel).<br />
* From "Antennas" section: Three 12-m antennas (DA61, PM03, and PM04) were used for the observations.<br />
<br />
=== Inspection - scanpattern ===<br />
The scan pattern of the raster mapping can be visualized by issuing the following command (after import of Analysis Utils - see the top of this page for information).<br />
A plot will be shown in a window and also saved as a PNG file uid___A002_X85c183_X36f.ms.sampling.png.<br />
<br />
<br />
[[File:X36f.TPSampling-6.1.png|200px|thumb|right|<caption>The scan pattern of the raster mapping for uid___A002_X85c183_X36f.</caption>]]<br />
<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms',<br />
showplot=True,<br />
plotfile='uid___A002_X85c183_X36f.ms.sampling.png')<br />
</source><br />
<br />
=== Inspection - System Noise Temperature ===<br />
Let's start by checking the Tsys.<br />
We use the task {{gencal}} to extract the Tsys into a CASA calibration table. This table is only used for inspection of Tsys quality, and not in further processing.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
gencal(vis='uid___A002_X85c183_X36f.ms',<br />
caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
caltype='tsys')<br />
</source><br />
<br />
The generated Tsys calibration table can be plotted using the task {{plotbandpass}}. The generated plots are saved in the directory<br />
uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime.<br />
<br />
<br />
[[File:X36f.Tsys.overlayTime-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for DA61 in uid___A002_X85c183_X36f. Colors represent time.</caption>]]<br />
<br />
<br />
<br />
[[File:X36f.Tsys.overlayAntenna-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for SPW 15 in uid___A002_X85c183_X36f, using overlay='antenna'. Colors represent antennas.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotbandpass(caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
overlay='time',<br />
xaxis='freq',<br />
yaxis='amp',<br />
subplot=22,<br />
buildpdf=False,<br />
interactive=False,<br />
showatm=True,<br />
pwv='auto',<br />
chanrange='5~123',<br />
showfdm=True,<br />
field='',<br />
figfile='uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime/uid___A002_X85c183_X36f.ms.tsys')<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
Although the "science" SPW's have 1992 MHz bandwidths with 4080 spectral channels, their edge channels can be very noisy because each intermediate frequency signal path (baseband) is equipped with a bandpass filter of ~1.8 GHz width. Note that this is not necessarily always the case, it is in general, safest to flag out the edge channels, unless a spectral line of interest falls very close to the edge of the SPW band-edge.<br />
We flag 120 channels on each side of the "science" SPW's.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis = 'uid___A002_X85c183_X36f.ms',<br />
mode = 'manual',<br />
spw = '17:0~119;3960~4079,19:0~119;3960~4079,21:0~119;3960~4079,23:0~119;3960~4079',<br />
action = 'apply',<br />
flagbackup = True)<br />
</source><br />
<br />
==Calibration - sky, bandpass, nonlinearity and baseline ==<br />
<br />
=== Tsys Calibration ===<br />
We calibrate the data into Antenna temperature in units of K, using the equation Ta* = Tsys*(ON-OFF)/OFF, where ON and OFF are the data on-source (i.e., during the raster scanning) and off-source (on a reference position where only background emission exists), respectively.<br />
The calibration is done by the task {{sdcal}} which requires matching of list of the "science" and Tsys spectral windows. <br />
Tsys observations will always have 128 channels over 2GHz bandwidth, and as described in {{listobs}}, we already know which spectral windows match the science spectral windows most closely, in frequency.<br />
for example, spectral window 9, with 128 channels matches well with spectral window 17, 11 with 19, 13 with 21 and 15 with 2. Thus, we form the Tsys and Science spectral window mapping input parameters, with the syntax: {9:[17], 11:[19], 13:[21],15:[23]}<br />
<br />
<pre style="background-color: #fffacd; overflow-x:auto; white-space: pre"><br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
.<br />
.<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
.<br />
.<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
.<br />
.<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
.<br />
.<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
</pre><br />
<br />
And the Tsys calibration command is formed like the following:<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdcal(infile = 'uid___A002_X85c183_X36f.ms',<br />
calmode = 'ps,tsys,apply',<br />
spwmap = {9:[17], 11:[19], 13:[21],15:[23]},<br />
spw = '17,19,21,23')<br />
</source><br />
<br />
'''Note:''' in this step, the calibrations are applied in situ. That is: the calibrated data are written into the original data - however it is written in such a way that any ''additional'' calls to sdcal will not apply any further calibrations. We will examine the quality of the bandpass and Tsys calibration after the correction for non-linearity, next.<br />
<br />
<br />
<br />
[[File:Vis_Scan6_7_9_10_12_13_15-6.1.png|200px|thumb|right|<caption>Output of plotms, showing spectra in Ta* [K]</caption>]]<br />
<br />
<br />
The calibrated bandpasses, averaged over each scan, can be plotted with:<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='23',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis.png',<br />
showgui=True)<br />
</source><br />
<br />
'''Note:''' For ''amplitude calibrator datasets'', (as opposed to science target maps) the calmode option for sdcal should be 'otfraster,tsys,apply' ('otfraster' instead of 'ps').<br />
'otfraster' tells the task to use both ends of each raster row as "OFF" data (as opposed to a dedicated "OFF" sky measurement).<br />
This is because temporal fluctuation of atmospheric emission is the dominant source of noise in radio continuum observations, and hence the "OFF" needs to be as close as possible to the "ON" in time (and spatial location).<br />
<br />
=== Application of Non-Linearity Correction Factor ===<br />
<br />
In Cycle 1 and 2, single-dish data taken with the ACA correlator suffer from non-linearity which originates in the digital signal processing.<br />
Its impact was thoroughly studied both experimentally and theoretically, and it was concluded that multiplication by a correction factor of 1.25 yields an amplitude accuracy of +/-5%.<br />
<br />
To preserve the data history, we use here the tasks {{split}} to split out the science ''corrected'' data columns - note that from here on, the spectral windows are '''renumbered: [0,1,2,3]'''. We then generate a calibration table, using a defined function ''to_amp_factor'', with {{gencal}}. Finally, we apply the calibration with {{applycal}}.<br />
We apply the correction here using tasks gencal and applycal - this is different to the methods used in the past, and are available now because of the migration to MS-only format of Single-dish CASA.<br />
<br />
'''Note:''' The ACA online software has been equipped with this correction for data from cycle 3 and later (it should be applied for data from cycle 2 and earlier).<br />
<br />
Also note that we need to import the math module in order to use the sqrt command in Python 3.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import math<br />
<br />
to_amp_factor = lambda x: 1. / math.sqrt(x)<br />
<br />
split(vis='uid___A002_X85c183_X36f.ms',<br />
outputvis='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='17,19,21,23')<br />
<br />
gencal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
caltable='uid___A002_X85c183_X36f.ms.nlc',<br />
caltype='amp', spw='',<br />
parameter=[to_amp_factor(1.25)])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
gaintable='uid___A002_X85c183_X36f.ms.nlc')<br />
</source><br />
<br />
'''Note:''' the Python command ''lambda'' simply defines a function. Thus the line above : ''to_amp_factor = lambda x: 1. / math.sqrt(x)'' defines a function where the call: ''to_amp_factor(4)'' will return: ''0.5''<br />
<br />
=== Baseline Subtraction (only for Science Datasets) ===<br />
<br />
'''Note:''' this step is appropriate for spectral line observations (science datasets) but ''should not be done for the continuum observations (amplitude calibrator datasets) -- it would eliminate the continuum emission!''<br />
'''Note:''' this step is optional and need not be completed to produce an image - however it is recommended for the majority of cases (esp. where the bandpass is affected by low-order modulations, and where the line-emission does not completely dominate the bandpass.)<br />
<br />
We will subtract low-order spectral baselines using the task {{sdbaseline}}. Typically these manifest from structure in the distribution of water in the atmosphere: the ON and OFF positions are slightly different, and the difference manifests as - typically - a low frequency variation in the spectra. The situation is exacerbated for observations closer to water lines and also for higher frequencies.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdbaseline(infile='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='',<br />
blfunc='poly',<br />
order=1,<br />
outfile='uid___A002_X85c183_X36f.ms.bl',<br />
overwrite=True)<br />
</source><br />
<br />
<br />
<br />
[[File: vis_bl_Scan6_7_9_10_12_13_15-6.1.png |200px|thumb|right|<caption>Output of plotms, showing result of baseline step, units in Ta* [K]</caption>]]<br />
<br />
<br />
The spectra can be checked using {{plotms}}, which we have already used in a previous step.<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms.bl',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='3',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis_bl.png',<br />
showgui=True)<br />
</source><br />
<br />
We do not concatenate separate Execution Blocks, because we need to determine and apply day-by-day Jy/K conversion factor.<br />
<br />
At this point in the guide, we have finished the calibration for ASDM uid___A002_X85c183_X36f (except for the calculation of the Jy/K conversion factors). In order to finish the calibration for the entire M100 dataset, the above steps should be repeated with each of the ADSMs shown in [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2#Summary_of_Datasets Summary of Datasets]. Once each ASDM has been calibrated, the user can move on to the flagging, Jy/K conversion, and imaging processes below.<br />
<br />
Note that one of the antennas, PM02, had a problem in the first baseband (SPW 17) on 2014-07-05, and its impact may not be specific to this one SPW (e.g., it may have resulted in poor pointing calibration).<br />
Therefore the PM02 data should be flagged from the affected datasets: uid___A002_X8602fa_X2ab (science), and uid___A002_X8602fa_X577 (science). The calibrator set uid___A002_X8602fa_Xc3 will also be affected, and will be flagged out in the subsequent processing of the calibrators.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis='uid___A002_X8602fa_X2ab.ms.bl',<br />
antenna='PM02&&&')<br />
flagdata(vis='uid___A002_X8602fa_X577.ms.bl',<br />
antenna='PM02&&&')<br />
</source><br />
<br />
<br />
== Convert science data from Kelvin to Jansky ==<br />
<br />
At this stage the data have been calibrated into units of antenna temperature, in Kelvins.<br />
<br />
It now remains to convert the science data into Jy/beam units - the ''Jy per K'' factors. This is important for combination of the data, with 7m and 12m sets. Later in this guide we show how to determine the correct factors from the calibrator datasets, for conversion (the process is essentially identical to processing the science datasets, with the exception that the baseline step is omitted from the calibration dataset processing). However to determine the factors, the calibration data must also be imaged and then interrogated to determine the appropriate values. <br />
<br />
Applying the calibration factors to the science data is the last step in the script [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]]. <br />
<br />
We continue to use only one ASDM as an example; uid___A002_X85c183_X36f. The script should be consulted for treatment of all the M100 Single-dish ASDMs (and the correct application of the Jy per K factors). Again, we apply the factors by making use of the {{gencal}} and {{applycal}} tasks. Also as before, we make use of the python function defined earlier: ''to_amp_factor''<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
<br />
factors = [41.52, 42.47, 43.58, 42.98]<br />
gencal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
caltable='uid___A002_X85c183_X36f.ms.jy',<br />
caltype='amp',<br />
spw='0,1,2,3',<br />
parameter=[ to_amp_factor(v) for v in factors ])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
gaintable='uid___A002_X85c183_X36f.ms.jy')<br />
</source><br />
<br />
= Image the Science Target =<br />
<br />
The final stage is to image the science data. This step is done by the script "[[media: M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]" which is invoked with ''execfile('M100.casa6p1.imaging.py') '' in CASA.<br />
This step uses a number of helper utilities to automatically determine the properties of the image data - the analysis Utilities library must be loaded (see the text above).<br />
<br />
== Determining grid parameters, gridding and obtaining grid metadata ==<br />
Define the list of the calibrated science datasets, which the CASA task {{sdimaging}} accepts. Note that this step is the same for science and calibrator data.<br />
This section is a little more python-heavy than the calibration step.<br />
<br />
In this section, we will simply process all the calibrated science data. The calibrator data should never be combined. There is no need to ''concatenate'' the calibrated data, as the imaging task can explicitly accept multiple datasets as input.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
asdmlist = ['uid___A002_X85c183_X36f',<br />
'uid___A002_X85c183_X60b',<br />
'uid___A002_X8602fa_X2ab',<br />
'uid___A002_X8602fa_X577',<br />
'uid___A002_X864236_X2d4',<br />
'uid___A002_X864236_X693',<br />
'uid___A002_X86fcfa_X664',<br />
'uid___A002_X86fcfa_X96c',<br />
'uid___A002_X86fcfa_Xd9']<br />
vislist = map(lambda x: x + '.ms.bl', asdmlist)<br />
</source><br />
<br />
<br />
We first have to define and obtain some basic parameters. As all the ASDMs are ostensibly of the same kinds of observations, we refer to the first science ASDM to determine the basic properties of the resulting data cube. The cube size, pixel size, theoretical beam size, central frequency and other basic parameters are extracted using some CASA-functions and the Analysis Utils library (see the top of this page for information on Analysis Utils).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
spw = 3<br />
imagename = 'M100_TP_CO_cube.spw%s.image'%(spw)<br />
blimagename = imagename + '.bl'<br />
integimagename = imagename + '.integ'<br />
fwhmfactor = 1.13<br />
diameter = 12<br />
<br />
xSampling, ySampling, maxsize = aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms', showplot=False)<br />
<br />
msmd.open(asdmlist[0]+'.ms.bl')<br />
freq = msmd.meanfreq(spw)<br />
msmd.close()<br />
print("SPW %d: %.3f GHz" % (spw, freq*1e-9))<br />
<br />
theorybeam = aU.primaryBeamArcsec(frequency=freq*1e-9,<br />
fwhmfactor=fwhmfactor,<br />
diameter=diameter)<br />
<br />
cell = theorybeam/9.0<br />
imsize = int(round(maxsize/cell)*2)<br />
</source><br />
And now we learn the central frequency of spectral window 3, is 114.682 GHz.<br />
Our cube will have a total length of 253.0 arcsec ('''maxsize'''), and with a cell side of 5.64 arcsec ('''cell''') -which samples the theoretical beam (''theory beam'') with 9x9 pixels, we will need 90 pixels along the image side ('''imsize''').<br />
<br />
'''Note:''' The Jy/K value depends on the parameters of gridding convolution (i.e., "gridfunction", "cell", and function-specific parameters ["convsupport" for gridfunction='SF']). That is, the same gridding parameters should be used for both amplitude calibrator and science images -- otherwise the calibration into Jy unit becomes invalid. Please refer to the CASA cookbook for information on the gridding kernels.<br />
<br />
<br />
Now the data are imaged.<br />
The velocity channel maps of the CO J=1-0 line (restfreq='115.271201800GHz') are created as a data cube which covers the velocity range of 1400-1700 km/s at a spacing of 5 km/s (start='1400km/s', width='5km/s', nchan=70).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA <br />
sdimaging(infiles=vislist,<br />
field='M100',<br />
spw='%s'%(spw),<br />
nchan=70,<br />
mode='velocity',<br />
start='1400km/s',<br />
width='5km/s',<br />
veltype='radio',<br />
outframe='lsrk',<br />
restfreq='115.271204GHz',<br />
gridfunction='SF',<br />
convsupport=6,<br />
stokes='I',<br />
phasecenter='J2000 12h22m54.9 +15d49m15',<br />
ephemsrcname='',<br />
imsize=imsize,<br />
cell='%sarcsec'%(cell),<br />
brightnessunit='Jy/beam',<br />
overwrite=True,<br />
outfile=imagename)<br />
</source><br />
<br />
Now you can browse the data cube using the viewer.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
viewer(imagename)<br />
</source><br />
<br />
== Subtract a Residual Background from the Image ==<br />
<br />
If you plot the line profile using the viewer, you may notice that the background (i.e., line-free channels) level is slightly negative.<br />
To correct this, spectral baselines are subtracted from the image using the task {{imcontsub}}.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
imcontsub(imagename=imagename,<br />
linefile=blimagename,<br />
contfile='M100.residualbaseline.image',<br />
fitorder=1,<br />
chans='0~7;62~69')<br />
os.system('rm -rf M100.residualbaseline.image')<br />
</source><br />
<br />
This task has two outputs - the baselined data is returned in the same input file (''image name - M100_TP_CO_cube.spw3.image.bl''), while the subtracted component is returned in ''M100.residual baseline.image"". We have no further need for the subtracted component, and it can be discarded.<br />
<br />
== Basic Analysis - Moments ==<br />
<br />
Make a 0th moment (integrated intensity) map using the task {{immoments}} and browse it using the viewer.<br />
<br />
<br />
[[File:M100.CO.cube.bl.image.mom0-6.1.png|200px|thumb|right|<caption>The integrated intensity map of M100.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
immoments(imagename=blimagename,<br />
moments=[0],<br />
axis='spectral',<br />
chans='8~61',<br />
outfile=integimagename)<br />
<br />
viewer(integimagename)<br />
</source><br />
<br />
= Imaging the Amplitude Calibrator and Measure the Value of Jy/K =<br />
<br />
Determination of the Jy/K conversion factor is achieved by imaging a source whose continuum flux is known and measuring the observed brightness temperature. One observation of this source - the amplitude calibrator - was made per day, giving a total of four datasets (see top).<br />
<br />
The true size of the beam (point spread function), on which the Jy/K value depends, in a map is determined by several ingredients.<br />
* The intrinsic beam size is determined by the antenna diameter and optics (and of course wavelength). Here we assume the intrinsic beam size of 1.13*lambda/D, where D=12 [m] is the diameter of the antennas, and the factor 1.13 is derived from the optics design of the ALMA 12-m antennas.<br />
* The effective beam size is broadened by the following causes:<br />
** The scanning pattern used to observe the map (i.e., sample spacing in both directions).<br />
** The method used to grid the individual spectra into a map/cube.<br />
The Analysis Utilities provides several tools to get estimates of these.<br />
<br />
It is not necessary, but If you wish to try this out for yourself, you will first need to download the raw amplitude calibrator data file M100_Band3_TP_ampcal_UncalibratedData.tgz. How to obtain the data is described at [[M100_Band3#Obtaining the Data]]. Once you have unpacked the data you will have a directory called M100_Band3_TP_ampcal_UncalibratedData. In this directory, execute the scripts for individual datasets ([[media:A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py|A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py|A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py|A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py]], and [[media:A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py|A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py]]) to calibrate, image and derive Jy/K factors of all four calibrator datasets. The general approach is almost identical to that of calibrating the science target data, with the exception of the baselining step. Then, Jy/K values are derived from the peak of temperature in images and stored in text files (with the suffix, '.jyperk.txt').<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
es = aU.stuffForScienceDataReduction()<br />
execfile('A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py')<br />
</pre><br />
<br />
The Calibration Jy/K values are determined as follows. Note that the values for individual antennas are averaged for each day, each SPW, because the antennas used for the Science observations were not necessarily available for the corresponding amplitude calibrator observations.<br />
<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
#Date Amp_Cal_Dataset SPW17 SPW19 SPW21 SPW23<br />
2014-07-01 uid___A002_X85c183_X895 41.52 42.47 43.58 42.98<br />
2014-07-05 uid___A002_X8602fa_Xc3 41.38 42.59 42.08 42.53<br />
2014-07-07 uid___A002_X864236_Xe1 40.88 42.48 41.47 41.94<br />
2014-07-17 uid___A002_X86fcfa_X3ae 41.71 42.88 41.69 42.18<br />
</pre><br />
<br />
= Combination with 12m and 7m Array Data =<br />
<br />
If you wish to learn how to combine the Total Power data with interferometric 12m and 7m data, see the guide for Data Combination [[M100_Band3_Combine_6.2]]<br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2&diff=31242M100 Band3 SingleDish 6.22021-10-18T18:43:15Z<p>Jbertone: </p>
<hr />
<div><div style="margin-left:8%; margin-right:8%; margin-top:4%; margin-bottom:4%"><br />
<br />
[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
*'''This guide is written for CASA 6.2 and uses Python 3. The same functionality is implemented in Python 2 using CASA 5.7 at [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_5.7 M100_Band3_SingleDish_5.7]'''<br />
<br />
*'''This guide assumes that you have downloaded the Total Power data in M100_Band3_TP_UncalibratedData.tgz as described in [[M100_Band3#Obtaining_the_Data]]'''<br />
<br />
*'''The processes described below WILL NOT OPERATE in CASA versions earlier than 6.1.0'''<br />
<br />
*'''Details of the ALMA observations are described at [[M100_Band3]]<br />
<br />
*'''This portion of the guide covers calibration and imaging starting from the raw visibility data.'''<br />
<br />
<br />
= Overview =<br />
<br />
This portion of CASA Guide will cover the data reduction of the Total Power (TP) array observations of M100. The objective is to generate from the TP dataset, a position-position-velocity cube suitable for merging with 7m and 12m datasets.<br />
<br />
The data consist of "amplitude calibrator" datasets (containing observational data of the quasar 3C279) and "science" datasets containing data for the science target M100. To be clear - the observations of 3C279 and M100 both follow the same general process - i.e. they are themselves calibrated with frequent observations of ''blank sky'' - the so-called 'OFF', which is used to determine the System temperature as a function of frequency, and form a ''referenced'' bandpass, in units of Antenna Temperature (Ta*) [K].<br />
<br />
Ultimately the quasar data are compared with recent flux measurements (in Jy/beam), obtained from archived 12m data, to determine the scaling factor between Ta* and Jy/beam, and that scaling factor is then applied to the science data, to bootstrap to Jy/beam.<br />
<br />
The combination of the resultant image with the interferometric (12-m array and 7-m array) data is explained in a separate page, valid for CASA versions 6.1 and later [[M100_Band3_Combine_6.1]].<br />
<br />
This guide is designed for CASA 6.1.0.<br />
<br />
<br />
==Confirm Your Version of CASA==<br />
This guide has been written for CASA release 6.1.0. Please confirm your version before proceeding.<br />
The value returned from ''casalith.version_string()'' should be 6.1.0. If not, please obtain at least version 6.1.0 (or later), from [https://casa.nrao.edu/casa_obtaining.shtml]<br />
<br />
<source lang="python" style="display:inline-block"><br />
#IN CASA<br />
version = casalith.version_string()<br />
print("You are using " + version)<br />
if (version < '6.1.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
== Summary of Datasets ==<br />
<br />
The observations were made on the 1st, 5th, 7th, and 17th July 2014, using two or three 12-m antennas and the ACA correlator.<br />
The table below indicates the ID's of the Execution Blocks, their start and end times, and the antennas in the array.<br />
There are nine science datasets (i.e., two or three per day).<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Science<br />
uid___A002_X85c183_X36f Observed from 2014-07-01T21:51:26.2 to 2014-07-01T22:40:28.4 (UTC) DA61, PM03, PM04<br />
uid___A002_X85c183_X60b Observed from 2014-07-01T22:43:50.0 to 2014-07-01T23:32:39.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_X2ab Observed from 2014-07-05T23:58:03.6 to 2014-07-06T00:46:52.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X8602fa_X577 Observed from 2014-07-06T00:55:17.8 to 2014-07-06T01:44:07.3 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_X2d4 Observed from 2014-07-07T23:03:48.1 to 2014-07-07T23:53:47.9 (UTC) PM03, PM04<br />
uid___A002_X864236_X693 Observed from 2014-07-07T23:56:09.6 to 2014-07-08T00:46:07.1 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_Xd9 Observed from 2014-07-17T20:55:15.5 to 2014-07-17T21:44:06.1 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X664 Observed from 2014-07-17T22:24:17.3 to 2014-07-17T23:13:08.0 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X96c Observed from 2014-07-17T23:23:37.0 to 2014-07-18T00:12:25.3 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
Similarly, the concomitant ''Calibration'' datasets (used to convert from Antenna temperature units into Jy/beam units, for combination), are:<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Amplitude Calibrator<br />
uid___A002_X85c183_X895 Observed from 2014-07-01T23:35:23.1 to 2014-07-02T00:07:54.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_Xc3 Observed from 2014-07-05T23:21:25.6 to 2014-07-05T23:53:41.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_Xe1 Observed from 2014-07-07T22:27:35.4 to 2014-07-07T23:01:05.7 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_X3ae Observed from 2014-07-17T21:48:30.0 to 2014-07-17T22:20:52.2 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
= Script Calibration =<br />
The scripts listed below process all the single-dish science and calibrator data, using a few loops. <br />
<br />
# (optional) The calibrator (i.e. quasar) data are processed (see the section [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Imaging_the_Amplitude_Calibrator_and_Measure_the_Value_of_Jy.2FK Imaging the Amplitude Calibrator and Measure the Value of Jy/K] below)<br />
# The science data are processed with - [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]] <br />
# and the (calibrated) science data are imaged with - [[media:M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]<br />
<br />
Executing each of these scripts in order, will be sufficient to calibrate and image the science data (although the calibrator processing script can be omitted, as the relevant parameters are actually already explicitly written into the script).<br />
<br />
They can be invoked from the command line using the execfile command.<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
execfile('M100.casa6p1.calib.py')<br />
</pre><br />
<br />
= Manual Calibration =<br />
<br />
By way of description, and as a means to introduce and demonstrate some of the interactive and diagnostic, tools here we describe in more detail the steps taken by the scripts - although in this case we demonstrate the process for only '''one''' of the ASDM datasets. In the ''Imaging'' section, we assume '''all''' the ASDMs are appropriately reduced either by the manual process, or by using the script. <br />
<br />
The entire process takes less than 30 minutes.<br />
<br />
There are three basic steps to the Calibration of the Science data:<br />
# Import, inspection and flagging<br />
# Calibration<br />
#* Sky and bandpass calibration (i.e. Tsys and frequency sampling function)<br />
#* Adjustment for Correlator non-linearity (this step is necessary only for data taken in ALMA Cycle 3 and earlier)<br />
#* Conversion to Jy/beam<br />
# Imaging<br />
<br />
=== Import "Analysis Utilities" ===<br />
<br />
The [[Analysis_Utilities]] package will be used most critically for the imaging processes. While it is not necessary for completing the calibration stages, it is used to check data quality, and should be loaded.<br />
<br />
Import the package from https://casaguides.nrao.edu/index.php/Analysis_Utilities, and apply the class therein.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
</source><br />
<br />
== Import, inspection and flagging ==<br />
<br />
=== Import ===<br />
The first thing to do is to convert the dataset into the CASA Measurement Set (MS) format. The raw data have been provided to you in the ASDM (ALMA Science Data Model), which is the native format of the data produced by the observatory but cannot be processed by CASA. The conversion from ASDM to MS is done with the task {{importasdm}}.<br />
<br />
<source lang="python" style="overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=True)<br />
</source><br />
<br />
The converted dataset (with a suffix ".ms") is created and then flags embedded in the ASDM dataset (so-called "BDF flags") are transferred to MS using the bdfflags parameter.<br />
<br />
'''Note:''' The above code should be used for Cycle 3 data or later. Cycle 2 data and earlier should use:<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=False)<br />
os.system(os.environ['CASAPATH'].split()[0] + '/bin/bdflags2MS -f "COR DELA INT MIS SIG SYN TFB WVR ZER" uid___A002_X85c183_X36f uid___A002_X85c183_X36f.ms')<br />
</pre><br />
<br />
<br />
=== Inspection - listobs ===<br />
After import, the usual first step is then to get some basic information about the data.<br />
We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied.<br />
<br />
<source lang="python" style="display:inline-block><br />
# In CASA<br />
listobs(vis='uid___A002_X85c183_X36f.ms',<br />
listfile='uid___A002_X85c183_X36f.ms.listobs')<br />
</source><br />
<br />
The output will be written in a file named uid___A002_X85c183_X36f.ms.listobs which you can (and should) view at your leisure.<br />
{{listobs}} displays information about the targets, dates, frequencies, etc. Below is an excerpt from listobs for uid___A002_X85c183_X36f:<br />
<br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1215+1654 12:15:03.979140 +16.54.37.95680 J2000 0 14652<br />
1 none M100 12:22:54.360000 +15.48.50.60000 J2000 1 61509<br />
<br />
Spectral Windows: (25 unique spectral windows and 2 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_03#BB_1#SW-01#FULL_RES 124 TOPO 91955.512 -15625.000 1937500.0 90994.5750 1 XX YY<br />
2 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 90978.950 1734375.000 1734375.0 90978.9500 1 XX YY<br />
3 ALMA_RB_03#BB_2#SW-01#FULL_RES 124 TOPO 93893.012 -15625.000 1937500.0 92932.0750 2 XX YY<br />
4 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 92924.262 1937500.000 1937500.0 92924.2625 2 XX YY<br />
5 ALMA_RB_03#BB_3#SW-01#FULL_RES 124 TOPO 102033.637 15625.000 1937500.0 102994.5750 3 XX YY<br />
6 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 102986.762 1937500.000 1937500.0 102986.7625 3 XX YY<br />
7 ALMA_RB_03#BB_4#SW-01#FULL_RES 124 TOPO 104033.637 15625.000 1937500.0 104994.5750 4 XX YY<br />
8 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 104986.762 1937500.000 1937500.0 104986.7625 4 XX YY<br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
10 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100926.562 1781250.000 1781250.0 100926.5625 1 XX YY<br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
12 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102741.712 1781250.000 1781250.0 102741.7125 2 XX YY<br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
14 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112783.712 1781250.000 1781250.0 112783.7125 3 XX YY<br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
16 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114658.712 1781250.000 1781250.0 114658.7125 4 XX YY<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
18 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100949.756 1992187.500 1992187.5 100949.7559 1 XX YY<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
20 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102764.906 1992187.500 1992187.5 102764.9059 2 XX YY<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
22 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112806.906 1992187.500 1992187.5 112806.9059 3 XX YY<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
24 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114681.906 1992187.500 1992187.5 114681.9059 4 XX YY<br />
<br />
Sources: 48<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1215+1654 0 - - <br />
0 J1215+1654 25 - - <br />
0 J1215+1654 26 - - <br />
0 J1215+1654 27 - - <br />
0 J1215+1654 1 - - <br />
0 J1215+1654 2 - - <br />
0 J1215+1654 3 - - <br />
0 J1215+1654 4 - - <br />
0 J1215+1654 5 - - <br />
0 J1215+1654 6 - - <br />
0 J1215+1654 7 - - <br />
0 J1215+1654 8 - - <br />
0 J1215+1654 9 - - <br />
0 J1215+1654 10 - - <br />
0 J1215+1654 11 - - <br />
0 J1215+1654 12 - - <br />
0 J1215+1654 13 - - <br />
0 J1215+1654 14 - - <br />
0 J1215+1654 15 - - <br />
0 J1215+1654 16 - - <br />
0 J1215+1654 17 100950 0 <br />
0 J1215+1654 18 100950 0 <br />
0 J1215+1654 19 102794.1 0 <br />
0 J1215+1654 20 102794.1 0 <br />
0 J1215+1654 21 112794.1 0 <br />
0 J1215+1654 22 112794.1 0 <br />
0 J1215+1654 23 114669.1 0 <br />
0 J1215+1654 24 114669.1 0 <br />
1 M100 0 - - <br />
1 M100 25 - - <br />
1 M100 26 - - <br />
1 M100 27 - - <br />
1 M100 9 - - <br />
1 M100 10 - - <br />
1 M100 11 - - <br />
1 M100 12 - - <br />
1 M100 13 - - <br />
1 M100 14 - - <br />
1 M100 15 - - <br />
1 M100 16 - - <br />
1 M100 17 100950 0 <br />
1 M100 18 100950 0 <br />
1 M100 19 102794.1 0 <br />
1 M100 20 102794.1 0 <br />
1 M100 21 112794.1 0 <br />
1 M100 22 112794.1 0 <br />
1 M100 23 114669.1 0 <br />
1 M100 24 114669.1 0 <br />
<br />
Antennas: 3:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA61 A075 12.0 m -067.45.17.9 -22.53.21.4 -4.5609 -499.7012 23.0322 2225072.419944 -5440148.858968 -2481499.171703<br />
1 PM03 T701 12.0 m -067.45.18.8 -22.53.22.2 -29.1265 -522.7875 22.2052 2225045.995589 -5440149.141967 -2481520.118569<br />
2 PM04 T703 12.0 m -067.45.16.2 -22.53.23.9 42.8797 -575.6910 21.7763 2225104.700870 -5440102.471978 -2481568.689518<br />
</pre><br />
<br />
From this output you can for example see the followings.<br />
<br />
* From "Data records" section (not shown here): The execution consists of 15 scans with various scan intents.<br />
** Scans 1 and 2 are pointing and sideband gain ratio calibrations (done interferometrically), which need to be done prior to observing the target, on the quasar J1215+1654.<br />
** Scans 3 and 4 are interferometric delay and system noise temperature (Tsys) measurements also on J1215+1654 (they are in principle unnecessary; just a hack to make things happen on the telescope control software).<br />
** Scans 6, 7, 9, 10, 12, 13, and 15, whose scan intents contain "OBSERVE_TARGET", are for raster mapping of the target M100. The associated spectral window (SPW) ID's are 0 and 17-24.<br />
** Scans 5, 8, 11, and 14 with scan intents "CALIBRATE_ATMOSPHERE" are Tsys measurements for M100. The SPW ID's for Tsys scans are 0 and 9-16.<br />
* From "Spectral Windows" section:<br />
** SPW 0 contains the Water Vapor Radiometer (WVR) data, which are taken for all scans, but which are not used here.<br />
** SPW's 1, 3, 5, and 7 are placed on standard continuum frequencies and are used for the pointing scan only.<br />
** SPW's 9, 11, 13, and 15 cover the same frequency range as the science SPW's (see the next bullet) and are used to measure the Tsys used for calibration. They have 128 spectral channels (15.6 MHz spacing) in 2000 MHz bandwidth.<br />
** SPW's 17, 19, 21, and 23 cover the frequency ranges used for the actual science observations. They have 4080 spectral channels (488 kHz spacing) in 1992 MHz bandwidth. Hereafter these SPW's are referred to as "science" SPW's. The target line, CO J=1-0, is placed in SPW 23, the corresponding Tsys is taken from SPW 15.<br />
** The remaining, even-number SPW's contain channel-averages of the actually used SPW's (e.g., SPW 24 corresponds to SPW 23 averaged into one channel).<br />
* From "Antennas" section: Three 12-m antennas (DA61, PM03, and PM04) were used for the observations.<br />
<br />
=== Inspection - scanpattern ===<br />
The scan pattern of the raster mapping can be visualized by issuing the following command (after import of Analysis Utils - see the top of this page for information).<br />
A plot will be shown in a window and also saved as a PNG file uid___A002_X85c183_X36f.ms.sampling.png.<br />
<br />
<br />
[[File:X36f.TPSampling-6.1.png|200px|thumb|right|<caption>The scan pattern of the raster mapping for uid___A002_X85c183_X36f.</caption>]]<br />
<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms',<br />
showplot=True,<br />
plotfile='uid___A002_X85c183_X36f.ms.sampling.png')<br />
</source><br />
<br />
=== Inspection - System Noise Temperature ===<br />
Let's start by checking the Tsys.<br />
We use the task {{gencal}} to extract the Tsys into a CASA calibration table. This table is only used for inspection of Tsys quality, and not in further processing.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
gencal(vis='uid___A002_X85c183_X36f.ms',<br />
caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
caltype='tsys')<br />
</source><br />
<br />
The generated Tsys calibration table can be plotted using the task {{plotbandpass}}. The generated plots are saved in the directory<br />
uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime.<br />
<br />
<br />
[[File:X36f.Tsys.overlayTime-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for DA61 in uid___A002_X85c183_X36f. Colors represent time.</caption>]]<br />
<br />
<br />
<br />
[[File:X36f.Tsys.overlayAntenna-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for SPW 15 in uid___A002_X85c183_X36f, using overlay='antenna'. Colors represent antennas.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotbandpass(caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
overlay='time',<br />
xaxis='freq',<br />
yaxis='amp',<br />
subplot=22,<br />
buildpdf=False,<br />
interactive=False,<br />
showatm=True,<br />
pwv='auto',<br />
chanrange='5~123',<br />
showfdm=True,<br />
field='',<br />
figfile='uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime/uid___A002_X85c183_X36f.ms.tsys')<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
Although the "science" SPW's have 1992 MHz bandwidths with 4080 spectral channels, their edge channels can be very noisy because each intermediate frequency signal path (baseband) is equipped with a bandpass filter of ~1.8 GHz width. Note that this is not necessarily always the case, it is in general, safest to flag out the edge channels, unless a spectral line of interest falls very close to the edge of the SPW band-edge.<br />
We flag 120 channels on each side of the "science" SPW's.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis = 'uid___A002_X85c183_X36f.ms',<br />
mode = 'manual',<br />
spw = '17:0~119;3960~4079,19:0~119;3960~4079,21:0~119;3960~4079,23:0~119;3960~4079',<br />
action = 'apply',<br />
flagbackup = True)<br />
</source><br />
<br />
==Calibration - sky, bandpass, nonlinearity and baseline ==<br />
<br />
=== Tsys Calibration ===<br />
We calibrate the data into Antenna temperature in units of K, using the equation Ta* = Tsys*(ON-OFF)/OFF, where ON and OFF are the data on-source (i.e., during the raster scanning) and off-source (on a reference position where only background emission exists), respectively.<br />
The calibration is done by the task {{sdcal}} which requires matching of list of the "science" and Tsys spectral windows. <br />
Tsys observations will always have 128 channels over 2GHz bandwidth, and as described in {{listobs}}, we already know which spectral windows match the science spectral windows most closely, in frequency.<br />
for example, spectral window 9, with 128 channels matches well with spectral window 17, 11 with 19, 13 with 21 and 15 with 2. Thus, we form the Tsys and Science spectral window mapping input parameters, with the syntax: {9:[17], 11:[19], 13:[21],15:[23]}<br />
<br />
<pre style="background-color: #fffacd; overflow-x:auto; white-space: pre"><br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
.<br />
.<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
.<br />
.<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
.<br />
.<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
.<br />
.<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
</pre><br />
<br />
And the Tsys calibration command is formed like the following:<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdcal(infile = 'uid___A002_X85c183_X36f.ms',<br />
calmode = 'ps,tsys,apply',<br />
spwmap = {9:[17], 11:[19], 13:[21],15:[23]},<br />
spw = '17,19,21,23')<br />
</source><br />
<br />
'''Note:''' in this step, the calibrations are applied in situ. That is: the calibrated data are written into the original data - however it is written in such a way that any ''additional'' calls to sdcal will not apply any further calibrations. We will examine the quality of the bandpass and Tsys calibration after the correction for non-linearity, next.<br />
<br />
<br />
<br />
[[File:Vis_Scan6_7_9_10_12_13_15-6.1.png|200px|thumb|right|<caption>Output of plotms, showing spectra in Ta* [K]</caption>]]<br />
<br />
<br />
The calibrated bandpasses, averaged over each scan, can be plotted with:<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='23',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis.png',<br />
showgui=True)<br />
</source><br />
<br />
'''Note:''' For ''amplitude calibrator datasets'', (as opposed to science target maps) the calmode option for sdcal should be 'otfraster,tsys,apply' ('otfraster' instead of 'ps').<br />
'otfraster' tells the task to use both ends of each raster row as "OFF" data (as opposed to a dedicated "OFF" sky measurement).<br />
This is because temporal fluctuation of atmospheric emission is the dominant source of noise in radio continuum observations, and hence the "OFF" needs to be as close as possible to the "ON" in time (and spatial location).<br />
<br />
=== Application of Non-Linearity Correction Factor ===<br />
<br />
In Cycle 1 and 2, single-dish data taken with the ACA correlator suffer from non-linearity which originates in the digital signal processing.<br />
Its impact was thoroughly studied both experimentally and theoretically, and it was concluded that multiplication by a correction factor of 1.25 yields an amplitude accuracy of +/-5%.<br />
<br />
To preserve the data history, we use here the tasks {{split}} to split out the science ''corrected'' data columns - note that from here on, the spectral windows are '''renumbered: [0,1,2,3]'''. We then generate a calibration table, using a defined function ''to_amp_factor'', with {{gencal}}. Finally, we apply the calibration with {{applycal}}.<br />
We apply the correction here using tasks gencal and applycal - this is different to the methods used in the past, and are available now because of the migration to MS-only format of Single-dish CASA.<br />
<br />
'''Note:''' The ACA online software has been equipped with this correction for data from cycle 3 and later (it should be applied for data from cycle 2 and earlier).<br />
<br />
Also note that we need to import the math module in order to use the sqrt command in Python 3.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import math<br />
<br />
to_amp_factor = lambda x: 1. / math.sqrt(x)<br />
<br />
split(vis='uid___A002_X85c183_X36f.ms',<br />
outputvis='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='17,19,21,23')<br />
<br />
gencal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
caltable='uid___A002_X85c183_X36f.ms.nlc',<br />
caltype='amp', spw='',<br />
parameter=[to_amp_factor(1.25)])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
gaintable='uid___A002_X85c183_X36f.ms.nlc')<br />
</source><br />
<br />
'''Note:''' the Python command ''lambda'' simply defines a function. Thus the line above : ''to_amp_factor = lambda x: 1. / math.sqrt(x)'' defines a function where the call: ''to_amp_factor(4)'' will return: ''0.5''<br />
<br />
=== Baseline Subtraction (only for Science Datasets) ===<br />
<br />
'''Note:''' this step is appropriate for spectral line observations (science datasets) but ''should not be done for the continuum observations (amplitude calibrator datasets) -- it would eliminate the continuum emission!''<br />
'''Note:''' this step is optional and need not be completed to produce an image - however it is recommended for the majority of cases (esp. where the bandpass is affected by low-order modulations, and where the line-emission does not completely dominate the bandpass.)<br />
<br />
We will subtract low-order spectral baselines using the task {{sdbaseline}}. Typically these manifest from structure in the distribution of water in the atmosphere: the ON and OFF positions are slightly different, and the difference manifests as - typically - a low frequency variation in the spectra. The situation is exacerbated for observations closer to water lines and also for higher frequencies.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdbaseline(infile='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='',<br />
blfunc='poly',<br />
order=1,<br />
outfile='uid___A002_X85c183_X36f.ms.bl',<br />
overwrite=True)<br />
</source><br />
<br />
<br />
<br />
[[File: vis_bl_Scan6_7_9_10_12_13_15-6.1.png |200px|thumb|right|<caption>Output of plotms, showing result of baseline step, units in Ta* [K]</caption>]]<br />
<br />
<br />
The spectra can be checked using {{plotms}}, which we have already used in a previous step.<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms.bl',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='3',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis_bl.png',<br />
showgui=True)<br />
</source><br />
<br />
We do not concatenate separate Execution Blocks, because we need to determine and apply day-by-day Jy/K conversion factor.<br />
<br />
At this point in the guide, we have finished the calibration for ASDM uid___A002_X85c183_X36f (except for the calculation of the Jy/K conversion factors). In order to finish the calibration for the entire M100 dataset, the above steps should be repeated with each of the ADSMs shown in [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Summary_of_Datasets Summary of Datasets]. Once each ASDM has been calibrated, the user can move on to the flagging, Jy/K conversion, and imaging processes below.<br />
<br />
Note that one of the antennas, PM02, had a problem in the first baseband (SPW 17) on 2014-07-05, and its impact may not be specific to this one SPW (e.g., it may have resulted in poor pointing calibration).<br />
Therefore the PM02 data should be flagged from the affected datasets: uid___A002_X8602fa_X2ab (science), and uid___A002_X8602fa_X577 (science). The calibrator set uid___A002_X8602fa_Xc3 will also be affected, and will be flagged out in the subsequent processing of the calibrators.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis='uid___A002_X8602fa_X2ab.ms.bl',<br />
antenna='PM02&&&')<br />
flagdata(vis='uid___A002_X8602fa_X577.ms.bl',<br />
antenna='PM02&&&')<br />
</source><br />
<br />
<br />
== Convert science data from Kelvin to Jansky ==<br />
<br />
At this stage the data have been calibrated into units of antenna temperature, in Kelvins.<br />
<br />
It now remains to convert the science data into Jy/beam units - the ''Jy per K'' factors. This is important for combination of the data, with 7m and 12m sets. Later in this guide we show how to determine the correct factors from the calibrator datasets, for conversion (the process is essentially identical to processing the science datasets, with the exception that the baseline step is omitted from the calibration dataset processing). However to determine the factors, the calibration data must also be imaged and then interrogated to determine the appropriate values. <br />
<br />
Applying the calibration factors to the science data is the last step in the script [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]]. <br />
<br />
We continue to use only one ASDM as an example; uid___A002_X85c183_X36f. The script should be consulted for treatment of all the M100 Single-dish ASDMs (and the correct application of the Jy per K factors). Again, we apply the factors by making use of the {{gencal}} and {{applycal}} tasks. Also as before, we make use of the python function defined earlier: ''to_amp_factor''<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
<br />
factors = [41.52, 42.47, 43.58, 42.98]<br />
gencal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
caltable='uid___A002_X85c183_X36f.ms.jy',<br />
caltype='amp',<br />
spw='0,1,2,3',<br />
parameter=[ to_amp_factor(v) for v in factors ])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
gaintable='uid___A002_X85c183_X36f.ms.jy')<br />
</source><br />
<br />
= Image the Science Target =<br />
<br />
The final stage is to image the science data. This step is done by the script "[[media: M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]" which is invoked with ''execfile('M100.casa6p1.imaging.py') '' in CASA.<br />
This step uses a number of helper utilities to automatically determine the properties of the image data - the analysis Utilities library must be loaded (see the text above).<br />
<br />
== Determining grid parameters, gridding and obtaining grid metadata ==<br />
Define the list of the calibrated science datasets, which the CASA task {{sdimaging}} accepts. Note that this step is the same for science and calibrator data.<br />
This section is a little more python-heavy than the calibration step. CASA 6.1.0 has benefited from significant effort in streamlining the calibration steps, and the imaging steps are to be more automated in coming cycles.<br />
<br />
In this section, we will simply process all the calibrated science data. The calibrator data should never be combined. There is no need to ''concatenate'' the calibrated data, as the imaging task can explicitly accept multiple datasets as input.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
asdmlist = ['uid___A002_X85c183_X36f',<br />
'uid___A002_X85c183_X60b',<br />
'uid___A002_X8602fa_X2ab',<br />
'uid___A002_X8602fa_X577',<br />
'uid___A002_X864236_X2d4',<br />
'uid___A002_X864236_X693',<br />
'uid___A002_X86fcfa_X664',<br />
'uid___A002_X86fcfa_X96c',<br />
'uid___A002_X86fcfa_Xd9']<br />
vislist = map(lambda x: x + '.ms.bl', asdmlist)<br />
</source><br />
<br />
<br />
We first have to define and obtain some basic parameters. As all the ASDMs are ostensibly of the same kinds of observations, we refer to the first science ASDM to determine the basic properties of the resulting data cube. The cube size, pixel size, theoretical beam size, central frequency and other basic parameters are extracted using some CASA-functions and the Analysis Utils library (see the top of this page for information on Analysis Utils).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
spw = 3<br />
imagename = 'M100_TP_CO_cube.spw%s.image'%(spw)<br />
blimagename = imagename + '.bl'<br />
integimagename = imagename + '.integ'<br />
fwhmfactor = 1.13<br />
diameter = 12<br />
<br />
xSampling, ySampling, maxsize = aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms', showplot=False)<br />
<br />
msmd.open(asdmlist[0]+'.ms.bl')<br />
freq = msmd.meanfreq(spw)<br />
msmd.close()<br />
print("SPW %d: %.3f GHz" % (spw, freq*1e-9))<br />
<br />
theorybeam = aU.primaryBeamArcsec(frequency=freq*1e-9,<br />
fwhmfactor=fwhmfactor,<br />
diameter=diameter)<br />
<br />
cell = theorybeam/9.0<br />
imsize = int(round(maxsize/cell)*2)<br />
</source><br />
And now we learn the central frequency of spectral window 3, is 114.682 GHz.<br />
Our cube will have a total length of 253.0 arcsec ('''maxsize'''), and with a cell side of 5.64 arcsec ('''cell''') -which samples the theoretical beam (''theory beam'') with 9x9 pixels, we will need 90 pixels along the image side ('''imsize''').<br />
<br />
'''Note:''' The Jy/K value depends on the parameters of gridding convolution (i.e., "gridfunction", "cell", and function-specific parameters ["convsupport" for gridfunction='SF']). That is, the same gridding parameters should be used for both amplitude calibrator and science images -- otherwise the calibration into Jy unit becomes invalid. Please refer to the CASA cookbook for information on the gridding kernels.<br />
<br />
<br />
Now the data are imaged.<br />
The velocity channel maps of the CO J=1-0 line (restfreq='115.271201800GHz') are created as a data cube which covers the velocity range of 1400-1700 km/s at a spacing of 5 km/s (start='1400km/s', width='5km/s', nchan=70).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA <br />
sdimaging(infiles=vislist,<br />
field='M100',<br />
spw='%s'%(spw),<br />
nchan=70,<br />
mode='velocity',<br />
start='1400km/s',<br />
width='5km/s',<br />
veltype='radio',<br />
outframe='lsrk',<br />
restfreq='115.271204GHz',<br />
gridfunction='SF',<br />
convsupport=6,<br />
stokes='I',<br />
phasecenter='J2000 12h22m54.9 +15d49m15',<br />
ephemsrcname='',<br />
imsize=imsize,<br />
cell='%sarcsec'%(cell),<br />
brightnessunit='Jy/beam',<br />
overwrite=True,<br />
outfile=imagename)<br />
</source><br />
<br />
Now you can browse the data cube using the viewer.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
viewer(imagename)<br />
</source><br />
<br />
== Subtract a Residual Background from the Image ==<br />
<br />
If you plot the line profile using the viewer, you may notice that the background (i.e., line-free channels) level is slightly negative.<br />
To correct this, spectral baselines are subtracted from the image using the task {{imcontsub}}.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
imcontsub(imagename=imagename,<br />
linefile=blimagename,<br />
contfile='M100.residualbaseline.image',<br />
fitorder=1,<br />
chans='0~7;62~69')<br />
os.system('rm -rf M100.residualbaseline.image')<br />
</source><br />
<br />
This task has two outputs - the baselined data is returned in the same input file (''image name - M100_TP_CO_cube.spw3.image.bl''), while the subtracted component is returned in ''M100.residual baseline.image"". We have no further need for the subtracted component, and it can be discarded.<br />
<br />
== Basic Analysis - Moments ==<br />
<br />
Make a 0th moment (integrated intensity) map using the task {{immoments}} and browse it using the viewer.<br />
<br />
<br />
[[File:M100.CO.cube.bl.image.mom0-6.1.png|200px|thumb|right|<caption>The integrated intensity map of M100.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
immoments(imagename=blimagename,<br />
moments=[0],<br />
axis='spectral',<br />
chans='8~61',<br />
outfile=integimagename)<br />
<br />
viewer(integimagename)<br />
</source><br />
<br />
= Imaging the Amplitude Calibrator and Measure the Value of Jy/K =<br />
<br />
Determination of the Jy/K conversion factor is achieved by imaging a source whose continuum flux is known and measuring the observed brightness temperature. One observation of this source - the amplitude calibrator - was made per day, giving a total of four datasets (see top).<br />
<br />
The true size of the beam (point spread function), on which the Jy/K value depends, in a map is determined by several ingredients.<br />
* The intrinsic beam size is determined by the antenna diameter and optics (and of course wavelength). Here we assume the intrinsic beam size of 1.13*lambda/D, where D=12 [m] is the diameter of the antennas, and the factor 1.13 is derived from the optics design of the ALMA 12-m antennas.<br />
* The effective beam size is broadened by the following causes:<br />
** The scanning pattern used to observe the map (i.e., sample spacing in both directions).<br />
** The method used to grid the individual spectra into a map/cube.<br />
The Analysis Utilities provides several tools to get estimates of these.<br />
<br />
It is not necessary, but If you wish to try this out for yourself, you will first need to download the raw amplitude calibrator data file M100_Band3_TP_ampcal_UncalibratedData.tgz. How to obtain the data is described at [[M100_Band3#Obtaining the Data]]. Once you have unpacked the data you will have a directory called M100_Band3_TP_ampcal_UncalibratedData. In this directory, execute the scripts for individual datasets ([[media:A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py|A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py|A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py|A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py]], and [[media:A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py|A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py]]) to calibrate, image and derive Jy/K factors of all four calibrator datasets. The general approach is almost identical to that of calibrating the science target data, with the exception of the baselining step. Then, Jy/K values are derived from the peak of temperature in images and stored in text files (with the suffix, '.jyperk.txt').<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
es = aU.stuffForScienceDataReduction()<br />
execfile('A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py')<br />
</pre><br />
<br />
The Calibration Jy/K values are determined as follows. Note that the values for individual antennas are averaged for each day, each SPW, because the antennas used for the Science observations were not necessarily available for the corresponding amplitude calibrator observations.<br />
<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
#Date Amp_Cal_Dataset SPW17 SPW19 SPW21 SPW23<br />
2014-07-01 uid___A002_X85c183_X895 41.52 42.47 43.58 42.98<br />
2014-07-05 uid___A002_X8602fa_Xc3 41.38 42.59 42.08 42.53<br />
2014-07-07 uid___A002_X864236_Xe1 40.88 42.48 41.47 41.94<br />
2014-07-17 uid___A002_X86fcfa_X3ae 41.71 42.88 41.69 42.18<br />
</pre><br />
<br />
= Combination with 12m and 7m Array Data =<br />
<br />
If you wish to learn how to combine the Total Power data with interferometric 12m and 7m data, see the guide for Data Combination [[M100_Band3_Combine_6.1]]<br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2&diff=31241M100 Band3 SingleDish 6.22021-10-18T18:40:01Z<p>Jbertone: /* Overview */</p>
<hr />
<div><div style="margin-left:8%; margin-right:8%"><br />
<br />
[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
*'''This guide is written for CASA 6.1 and uses Python 3. The same functionality is implemented in Python 2 using CASA 5.7 at [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_5.7 M100_Band3_SingleDish_5.7]'''<br />
<br />
*'''This guide assumes that you have downloaded the Total Power data in M100_Band3_TP_UncalibratedData.tgz as described in [[M100_Band3#Obtaining_the_Data]]'''<br />
<br />
*'''The processes described below WILL NOT OPERATE in CASA versions earlier than 6.1.0'''<br />
<br />
*'''Details of the ALMA observations are described at [[M100_Band3]]<br />
<br />
*'''This portion of the guide covers calibration and imaging starting from the raw visibility data.'''<br />
<br />
<br />
= Overview =<br />
<br />
This portion of CASA Guide will cover the data reduction of the Total Power (TP) array observations of M100. The objective is to generate from the TP dataset, a position-position-velocity cube suitable for merging with 7m and 12m datasets.<br />
<br />
The data consist of "amplitude calibrator" datasets (containing observational data of the quasar 3C279) and "science" datasets containing data for the science target M100. To be clear - the observations of 3C279 and M100 both follow the same general process - i.e. they are themselves calibrated with frequent observations of ''blank sky'' - the so-called 'OFF', which is used to determine the System temperature as a function of frequency, and form a ''referenced'' bandpass, in units of Antenna Temperature (Ta*) [K].<br />
<br />
Ultimately the quasar data are compared with recent flux measurements (in Jy/beam), obtained from archived 12m data, to determine the scaling factor between Ta* and Jy/beam, and that scaling factor is then applied to the science data, to bootstrap to Jy/beam.<br />
<br />
The combination of the resultant image with the interferometric (12-m array and 7-m array) data is explained in a separate page, valid for CASA versions 6.1 and later [[M100_Band3_Combine_6.1]].<br />
<br />
This guide is designed for CASA 6.1.0.<br />
<br />
<br />
==Confirm Your Version of CASA==<br />
This guide has been written for CASA release 6.1.0. Please confirm your version before proceeding.<br />
The value returned from ''casalith.version_string()'' should be 6.1.0. If not, please obtain at least version 6.1.0 (or later), from [https://casa.nrao.edu/casa_obtaining.shtml]<br />
<br />
<source lang="python" style="display:inline-block"><br />
#IN CASA<br />
version = casalith.version_string()<br />
print("You are using " + version)<br />
if (version < '6.1.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
== Summary of Datasets ==<br />
<br />
The observations were made on the 1st, 5th, 7th, and 17th July 2014, using two or three 12-m antennas and the ACA correlator.<br />
The table below indicates the ID's of the Execution Blocks, their start and end times, and the antennas in the array.<br />
There are nine science datasets (i.e., two or three per day).<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Science<br />
uid___A002_X85c183_X36f Observed from 2014-07-01T21:51:26.2 to 2014-07-01T22:40:28.4 (UTC) DA61, PM03, PM04<br />
uid___A002_X85c183_X60b Observed from 2014-07-01T22:43:50.0 to 2014-07-01T23:32:39.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_X2ab Observed from 2014-07-05T23:58:03.6 to 2014-07-06T00:46:52.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X8602fa_X577 Observed from 2014-07-06T00:55:17.8 to 2014-07-06T01:44:07.3 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_X2d4 Observed from 2014-07-07T23:03:48.1 to 2014-07-07T23:53:47.9 (UTC) PM03, PM04<br />
uid___A002_X864236_X693 Observed from 2014-07-07T23:56:09.6 to 2014-07-08T00:46:07.1 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_Xd9 Observed from 2014-07-17T20:55:15.5 to 2014-07-17T21:44:06.1 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X664 Observed from 2014-07-17T22:24:17.3 to 2014-07-17T23:13:08.0 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X96c Observed from 2014-07-17T23:23:37.0 to 2014-07-18T00:12:25.3 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
Similarly, the concomitant ''Calibration'' datasets (used to convert from Antenna temperature units into Jy/beam units, for combination), are:<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Amplitude Calibrator<br />
uid___A002_X85c183_X895 Observed from 2014-07-01T23:35:23.1 to 2014-07-02T00:07:54.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_Xc3 Observed from 2014-07-05T23:21:25.6 to 2014-07-05T23:53:41.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_Xe1 Observed from 2014-07-07T22:27:35.4 to 2014-07-07T23:01:05.7 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_X3ae Observed from 2014-07-17T21:48:30.0 to 2014-07-17T22:20:52.2 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
= Script Calibration =<br />
The scripts listed below process all the single-dish science and calibrator data, using a few loops. <br />
<br />
# (optional) The calibrator (i.e. quasar) data are processed (see the section [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Imaging_the_Amplitude_Calibrator_and_Measure_the_Value_of_Jy.2FK Imaging the Amplitude Calibrator and Measure the Value of Jy/K] below)<br />
# The science data are processed with - [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]] <br />
# and the (calibrated) science data are imaged with - [[media:M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]<br />
<br />
Executing each of these scripts in order, will be sufficient to calibrate and image the science data (although the calibrator processing script can be omitted, as the relevant parameters are actually already explicitly written into the script).<br />
<br />
They can be invoked from the command line using the execfile command.<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
execfile('M100.casa6p1.calib.py')<br />
</pre><br />
<br />
= Manual Calibration =<br />
<br />
By way of description, and as a means to introduce and demonstrate some of the interactive and diagnostic, tools here we describe in more detail the steps taken by the scripts - although in this case we demonstrate the process for only '''one''' of the ASDM datasets. In the ''Imaging'' section, we assume '''all''' the ASDMs are appropriately reduced either by the manual process, or by using the script. <br />
<br />
The entire process takes less than 30 minutes.<br />
<br />
There are three basic steps to the Calibration of the Science data:<br />
# Import, inspection and flagging<br />
# Calibration<br />
#* Sky and bandpass calibration (i.e. Tsys and frequency sampling function)<br />
#* Adjustment for Correlator non-linearity (this step is necessary only for data taken in ALMA Cycle 3 and earlier)<br />
#* Conversion to Jy/beam<br />
# Imaging<br />
<br />
=== Import "Analysis Utilities" ===<br />
<br />
The [[Analysis_Utilities]] package will be used most critically for the imaging processes. While it is not necessary for completing the calibration stages, it is used to check data quality, and should be loaded.<br />
<br />
Import the package from https://casaguides.nrao.edu/index.php/Analysis_Utilities, and apply the class therein.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
</source><br />
<br />
== Import, inspection and flagging ==<br />
<br />
=== Import ===<br />
The first thing to do is to convert the dataset into the CASA Measurement Set (MS) format. The raw data have been provided to you in the ASDM (ALMA Science Data Model), which is the native format of the data produced by the observatory but cannot be processed by CASA. The conversion from ASDM to MS is done with the task {{importasdm}}.<br />
<br />
<source lang="python" style="overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=True)<br />
</source><br />
<br />
The converted dataset (with a suffix ".ms") is created and then flags embedded in the ASDM dataset (so-called "BDF flags") are transferred to MS using the bdfflags parameter.<br />
<br />
'''Note:''' The above code should be used for Cycle 3 data or later. Cycle 2 data and earlier should use:<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=False)<br />
os.system(os.environ['CASAPATH'].split()[0] + '/bin/bdflags2MS -f "COR DELA INT MIS SIG SYN TFB WVR ZER" uid___A002_X85c183_X36f uid___A002_X85c183_X36f.ms')<br />
</pre><br />
<br />
<br />
=== Inspection - listobs ===<br />
After import, the usual first step is then to get some basic information about the data.<br />
We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied.<br />
<br />
<source lang="python" style="display:inline-block><br />
# In CASA<br />
listobs(vis='uid___A002_X85c183_X36f.ms',<br />
listfile='uid___A002_X85c183_X36f.ms.listobs')<br />
</source><br />
<br />
The output will be written in a file named uid___A002_X85c183_X36f.ms.listobs which you can (and should) view at your leisure.<br />
{{listobs}} displays information about the targets, dates, frequencies, etc. Below is an excerpt from listobs for uid___A002_X85c183_X36f:<br />
<br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1215+1654 12:15:03.979140 +16.54.37.95680 J2000 0 14652<br />
1 none M100 12:22:54.360000 +15.48.50.60000 J2000 1 61509<br />
<br />
Spectral Windows: (25 unique spectral windows and 2 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_03#BB_1#SW-01#FULL_RES 124 TOPO 91955.512 -15625.000 1937500.0 90994.5750 1 XX YY<br />
2 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 90978.950 1734375.000 1734375.0 90978.9500 1 XX YY<br />
3 ALMA_RB_03#BB_2#SW-01#FULL_RES 124 TOPO 93893.012 -15625.000 1937500.0 92932.0750 2 XX YY<br />
4 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 92924.262 1937500.000 1937500.0 92924.2625 2 XX YY<br />
5 ALMA_RB_03#BB_3#SW-01#FULL_RES 124 TOPO 102033.637 15625.000 1937500.0 102994.5750 3 XX YY<br />
6 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 102986.762 1937500.000 1937500.0 102986.7625 3 XX YY<br />
7 ALMA_RB_03#BB_4#SW-01#FULL_RES 124 TOPO 104033.637 15625.000 1937500.0 104994.5750 4 XX YY<br />
8 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 104986.762 1937500.000 1937500.0 104986.7625 4 XX YY<br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
10 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100926.562 1781250.000 1781250.0 100926.5625 1 XX YY<br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
12 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102741.712 1781250.000 1781250.0 102741.7125 2 XX YY<br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
14 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112783.712 1781250.000 1781250.0 112783.7125 3 XX YY<br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
16 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114658.712 1781250.000 1781250.0 114658.7125 4 XX YY<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
18 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100949.756 1992187.500 1992187.5 100949.7559 1 XX YY<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
20 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102764.906 1992187.500 1992187.5 102764.9059 2 XX YY<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
22 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112806.906 1992187.500 1992187.5 112806.9059 3 XX YY<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
24 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114681.906 1992187.500 1992187.5 114681.9059 4 XX YY<br />
<br />
Sources: 48<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1215+1654 0 - - <br />
0 J1215+1654 25 - - <br />
0 J1215+1654 26 - - <br />
0 J1215+1654 27 - - <br />
0 J1215+1654 1 - - <br />
0 J1215+1654 2 - - <br />
0 J1215+1654 3 - - <br />
0 J1215+1654 4 - - <br />
0 J1215+1654 5 - - <br />
0 J1215+1654 6 - - <br />
0 J1215+1654 7 - - <br />
0 J1215+1654 8 - - <br />
0 J1215+1654 9 - - <br />
0 J1215+1654 10 - - <br />
0 J1215+1654 11 - - <br />
0 J1215+1654 12 - - <br />
0 J1215+1654 13 - - <br />
0 J1215+1654 14 - - <br />
0 J1215+1654 15 - - <br />
0 J1215+1654 16 - - <br />
0 J1215+1654 17 100950 0 <br />
0 J1215+1654 18 100950 0 <br />
0 J1215+1654 19 102794.1 0 <br />
0 J1215+1654 20 102794.1 0 <br />
0 J1215+1654 21 112794.1 0 <br />
0 J1215+1654 22 112794.1 0 <br />
0 J1215+1654 23 114669.1 0 <br />
0 J1215+1654 24 114669.1 0 <br />
1 M100 0 - - <br />
1 M100 25 - - <br />
1 M100 26 - - <br />
1 M100 27 - - <br />
1 M100 9 - - <br />
1 M100 10 - - <br />
1 M100 11 - - <br />
1 M100 12 - - <br />
1 M100 13 - - <br />
1 M100 14 - - <br />
1 M100 15 - - <br />
1 M100 16 - - <br />
1 M100 17 100950 0 <br />
1 M100 18 100950 0 <br />
1 M100 19 102794.1 0 <br />
1 M100 20 102794.1 0 <br />
1 M100 21 112794.1 0 <br />
1 M100 22 112794.1 0 <br />
1 M100 23 114669.1 0 <br />
1 M100 24 114669.1 0 <br />
<br />
Antennas: 3:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA61 A075 12.0 m -067.45.17.9 -22.53.21.4 -4.5609 -499.7012 23.0322 2225072.419944 -5440148.858968 -2481499.171703<br />
1 PM03 T701 12.0 m -067.45.18.8 -22.53.22.2 -29.1265 -522.7875 22.2052 2225045.995589 -5440149.141967 -2481520.118569<br />
2 PM04 T703 12.0 m -067.45.16.2 -22.53.23.9 42.8797 -575.6910 21.7763 2225104.700870 -5440102.471978 -2481568.689518<br />
</pre><br />
<br />
From this output you can for example see the followings.<br />
<br />
* From "Data records" section (not shown here): The execution consists of 15 scans with various scan intents.<br />
** Scans 1 and 2 are pointing and sideband gain ratio calibrations (done interferometrically), which need to be done prior to observing the target, on the quasar J1215+1654.<br />
** Scans 3 and 4 are interferometric delay and system noise temperature (Tsys) measurements also on J1215+1654 (they are in principle unnecessary; just a hack to make things happen on the telescope control software).<br />
** Scans 6, 7, 9, 10, 12, 13, and 15, whose scan intents contain "OBSERVE_TARGET", are for raster mapping of the target M100. The associated spectral window (SPW) ID's are 0 and 17-24.<br />
** Scans 5, 8, 11, and 14 with scan intents "CALIBRATE_ATMOSPHERE" are Tsys measurements for M100. The SPW ID's for Tsys scans are 0 and 9-16.<br />
* From "Spectral Windows" section:<br />
** SPW 0 contains the Water Vapor Radiometer (WVR) data, which are taken for all scans, but which are not used here.<br />
** SPW's 1, 3, 5, and 7 are placed on standard continuum frequencies and are used for the pointing scan only.<br />
** SPW's 9, 11, 13, and 15 cover the same frequency range as the science SPW's (see the next bullet) and are used to measure the Tsys used for calibration. They have 128 spectral channels (15.6 MHz spacing) in 2000 MHz bandwidth.<br />
** SPW's 17, 19, 21, and 23 cover the frequency ranges used for the actual science observations. They have 4080 spectral channels (488 kHz spacing) in 1992 MHz bandwidth. Hereafter these SPW's are referred to as "science" SPW's. The target line, CO J=1-0, is placed in SPW 23, the corresponding Tsys is taken from SPW 15.<br />
** The remaining, even-number SPW's contain channel-averages of the actually used SPW's (e.g., SPW 24 corresponds to SPW 23 averaged into one channel).<br />
* From "Antennas" section: Three 12-m antennas (DA61, PM03, and PM04) were used for the observations.<br />
<br />
=== Inspection - scanpattern ===<br />
The scan pattern of the raster mapping can be visualized by issuing the following command (after import of Analysis Utils - see the top of this page for information).<br />
A plot will be shown in a window and also saved as a PNG file uid___A002_X85c183_X36f.ms.sampling.png.<br />
<br />
<br />
[[File:X36f.TPSampling-6.1.png|200px|thumb|right|<caption>The scan pattern of the raster mapping for uid___A002_X85c183_X36f.</caption>]]<br />
<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms',<br />
showplot=True,<br />
plotfile='uid___A002_X85c183_X36f.ms.sampling.png')<br />
</source><br />
<br />
=== Inspection - System Noise Temperature ===<br />
Let's start by checking the Tsys.<br />
We use the task {{gencal}} to extract the Tsys into a CASA calibration table. This table is only used for inspection of Tsys quality, and not in further processing.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
gencal(vis='uid___A002_X85c183_X36f.ms',<br />
caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
caltype='tsys')<br />
</source><br />
<br />
The generated Tsys calibration table can be plotted using the task {{plotbandpass}}. The generated plots are saved in the directory<br />
uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime.<br />
<br />
<br />
[[File:X36f.Tsys.overlayTime-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for DA61 in uid___A002_X85c183_X36f. Colors represent time.</caption>]]<br />
<br />
<br />
<br />
[[File:X36f.Tsys.overlayAntenna-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for SPW 15 in uid___A002_X85c183_X36f, using overlay='antenna'. Colors represent antennas.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotbandpass(caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
overlay='time',<br />
xaxis='freq',<br />
yaxis='amp',<br />
subplot=22,<br />
buildpdf=False,<br />
interactive=False,<br />
showatm=True,<br />
pwv='auto',<br />
chanrange='5~123',<br />
showfdm=True,<br />
field='',<br />
figfile='uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime/uid___A002_X85c183_X36f.ms.tsys')<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
Although the "science" SPW's have 1992 MHz bandwidths with 4080 spectral channels, their edge channels can be very noisy because each intermediate frequency signal path (baseband) is equipped with a bandpass filter of ~1.8 GHz width. Note that this is not necessarily always the case, it is in general, safest to flag out the edge channels, unless a spectral line of interest falls very close to the edge of the SPW band-edge.<br />
We flag 120 channels on each side of the "science" SPW's.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis = 'uid___A002_X85c183_X36f.ms',<br />
mode = 'manual',<br />
spw = '17:0~119;3960~4079,19:0~119;3960~4079,21:0~119;3960~4079,23:0~119;3960~4079',<br />
action = 'apply',<br />
flagbackup = True)<br />
</source><br />
<br />
==Calibration - sky, bandpass, nonlinearity and baseline ==<br />
<br />
=== Tsys Calibration ===<br />
We calibrate the data into Antenna temperature in units of K, using the equation Ta* = Tsys*(ON-OFF)/OFF, where ON and OFF are the data on-source (i.e., during the raster scanning) and off-source (on a reference position where only background emission exists), respectively.<br />
The calibration is done by the task {{sdcal}} which requires matching of list of the "science" and Tsys spectral windows. <br />
Tsys observations will always have 128 channels over 2GHz bandwidth, and as described in {{listobs}}, we already know which spectral windows match the science spectral windows most closely, in frequency.<br />
for example, spectral window 9, with 128 channels matches well with spectral window 17, 11 with 19, 13 with 21 and 15 with 2. Thus, we form the Tsys and Science spectral window mapping input parameters, with the syntax: {9:[17], 11:[19], 13:[21],15:[23]}<br />
<br />
<pre style="background-color: #fffacd; overflow-x:auto; white-space: pre"><br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
.<br />
.<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
.<br />
.<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
.<br />
.<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
.<br />
.<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
</pre><br />
<br />
And the Tsys calibration command is formed like the following:<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdcal(infile = 'uid___A002_X85c183_X36f.ms',<br />
calmode = 'ps,tsys,apply',<br />
spwmap = {9:[17], 11:[19], 13:[21],15:[23]},<br />
spw = '17,19,21,23')<br />
</source><br />
<br />
'''Note:''' in this step, the calibrations are applied in situ. That is: the calibrated data are written into the original data - however it is written in such a way that any ''additional'' calls to sdcal will not apply any further calibrations. We will examine the quality of the bandpass and Tsys calibration after the correction for non-linearity, next.<br />
<br />
<br />
<br />
[[File:Vis_Scan6_7_9_10_12_13_15-6.1.png|200px|thumb|right|<caption>Output of plotms, showing spectra in Ta* [K]</caption>]]<br />
<br />
<br />
The calibrated bandpasses, averaged over each scan, can be plotted with:<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='23',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis.png',<br />
showgui=True)<br />
</source><br />
<br />
'''Note:''' For ''amplitude calibrator datasets'', (as opposed to science target maps) the calmode option for sdcal should be 'otfraster,tsys,apply' ('otfraster' instead of 'ps').<br />
'otfraster' tells the task to use both ends of each raster row as "OFF" data (as opposed to a dedicated "OFF" sky measurement).<br />
This is because temporal fluctuation of atmospheric emission is the dominant source of noise in radio continuum observations, and hence the "OFF" needs to be as close as possible to the "ON" in time (and spatial location).<br />
<br />
=== Application of Non-Linearity Correction Factor ===<br />
<br />
In Cycle 1 and 2, single-dish data taken with the ACA correlator suffer from non-linearity which originates in the digital signal processing.<br />
Its impact was thoroughly studied both experimentally and theoretically, and it was concluded that multiplication by a correction factor of 1.25 yields an amplitude accuracy of +/-5%.<br />
<br />
To preserve the data history, we use here the tasks {{split}} to split out the science ''corrected'' data columns - note that from here on, the spectral windows are '''renumbered: [0,1,2,3]'''. We then generate a calibration table, using a defined function ''to_amp_factor'', with {{gencal}}. Finally, we apply the calibration with {{applycal}}.<br />
We apply the correction here using tasks gencal and applycal - this is different to the methods used in the past, and are available now because of the migration to MS-only format of Single-dish CASA.<br />
<br />
'''Note:''' The ACA online software has been equipped with this correction for data from cycle 3 and later (it should be applied for data from cycle 2 and earlier).<br />
<br />
Also note that we need to import the math module in order to use the sqrt command in Python 3.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import math<br />
<br />
to_amp_factor = lambda x: 1. / math.sqrt(x)<br />
<br />
split(vis='uid___A002_X85c183_X36f.ms',<br />
outputvis='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='17,19,21,23')<br />
<br />
gencal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
caltable='uid___A002_X85c183_X36f.ms.nlc',<br />
caltype='amp', spw='',<br />
parameter=[to_amp_factor(1.25)])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
gaintable='uid___A002_X85c183_X36f.ms.nlc')<br />
</source><br />
<br />
'''Note:''' the Python command ''lambda'' simply defines a function. Thus the line above : ''to_amp_factor = lambda x: 1. / math.sqrt(x)'' defines a function where the call: ''to_amp_factor(4)'' will return: ''0.5''<br />
<br />
=== Baseline Subtraction (only for Science Datasets) ===<br />
<br />
'''Note:''' this step is appropriate for spectral line observations (science datasets) but ''should not be done for the continuum observations (amplitude calibrator datasets) -- it would eliminate the continuum emission!''<br />
'''Note:''' this step is optional and need not be completed to produce an image - however it is recommended for the majority of cases (esp. where the bandpass is affected by low-order modulations, and where the line-emission does not completely dominate the bandpass.)<br />
<br />
We will subtract low-order spectral baselines using the task {{sdbaseline}}. Typically these manifest from structure in the distribution of water in the atmosphere: the ON and OFF positions are slightly different, and the difference manifests as - typically - a low frequency variation in the spectra. The situation is exacerbated for observations closer to water lines and also for higher frequencies.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdbaseline(infile='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='',<br />
blfunc='poly',<br />
order=1,<br />
outfile='uid___A002_X85c183_X36f.ms.bl',<br />
overwrite=True)<br />
</source><br />
<br />
<br />
<br />
[[File: vis_bl_Scan6_7_9_10_12_13_15-6.1.png |200px|thumb|right|<caption>Output of plotms, showing result of baseline step, units in Ta* [K]</caption>]]<br />
<br />
<br />
The spectra can be checked using {{plotms}}, which we have already used in a previous step.<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms.bl',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='3',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis_bl.png',<br />
showgui=True)<br />
</source><br />
<br />
We do not concatenate separate Execution Blocks, because we need to determine and apply day-by-day Jy/K conversion factor.<br />
<br />
At this point in the guide, we have finished the calibration for ASDM uid___A002_X85c183_X36f (except for the calculation of the Jy/K conversion factors). In order to finish the calibration for the entire M100 dataset, the above steps should be repeated with each of the ADSMs shown in [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Summary_of_Datasets Summary of Datasets]. Once each ASDM has been calibrated, the user can move on to the flagging, Jy/K conversion, and imaging processes below.<br />
<br />
Note that one of the antennas, PM02, had a problem in the first baseband (SPW 17) on 2014-07-05, and its impact may not be specific to this one SPW (e.g., it may have resulted in poor pointing calibration).<br />
Therefore the PM02 data should be flagged from the affected datasets: uid___A002_X8602fa_X2ab (science), and uid___A002_X8602fa_X577 (science). The calibrator set uid___A002_X8602fa_Xc3 will also be affected, and will be flagged out in the subsequent processing of the calibrators.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis='uid___A002_X8602fa_X2ab.ms.bl',<br />
antenna='PM02&&&')<br />
flagdata(vis='uid___A002_X8602fa_X577.ms.bl',<br />
antenna='PM02&&&')<br />
</source><br />
<br />
<br />
== Convert science data from Kelvin to Jansky ==<br />
<br />
At this stage the data have been calibrated into units of antenna temperature, in Kelvins.<br />
<br />
It now remains to convert the science data into Jy/beam units - the ''Jy per K'' factors. This is important for combination of the data, with 7m and 12m sets. Later in this guide we show how to determine the correct factors from the calibrator datasets, for conversion (the process is essentially identical to processing the science datasets, with the exception that the baseline step is omitted from the calibration dataset processing). However to determine the factors, the calibration data must also be imaged and then interrogated to determine the appropriate values. <br />
<br />
Applying the calibration factors to the science data is the last step in the script [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]]. <br />
<br />
We continue to use only one ASDM as an example; uid___A002_X85c183_X36f. The script should be consulted for treatment of all the M100 Single-dish ASDMs (and the correct application of the Jy per K factors). Again, we apply the factors by making use of the {{gencal}} and {{applycal}} tasks. Also as before, we make use of the python function defined earlier: ''to_amp_factor''<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
<br />
factors = [41.52, 42.47, 43.58, 42.98]<br />
gencal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
caltable='uid___A002_X85c183_X36f.ms.jy',<br />
caltype='amp',<br />
spw='0,1,2,3',<br />
parameter=[ to_amp_factor(v) for v in factors ])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
gaintable='uid___A002_X85c183_X36f.ms.jy')<br />
</source><br />
<br />
= Image the Science Target =<br />
<br />
The final stage is to image the science data. This step is done by the script "[[media: M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]" which is invoked with ''execfile('M100.casa6p1.imaging.py') '' in CASA.<br />
This step uses a number of helper utilities to automatically determine the properties of the image data - the analysis Utilities library must be loaded (see the text above).<br />
<br />
== Determining grid parameters, gridding and obtaining grid metadata ==<br />
Define the list of the calibrated science datasets, which the CASA task {{sdimaging}} accepts. Note that this step is the same for science and calibrator data.<br />
This section is a little more python-heavy than the calibration step. CASA 6.1.0 has benefited from significant effort in streamlining the calibration steps, and the imaging steps are to be more automated in coming cycles.<br />
<br />
In this section, we will simply process all the calibrated science data. The calibrator data should never be combined. There is no need to ''concatenate'' the calibrated data, as the imaging task can explicitly accept multiple datasets as input.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
asdmlist = ['uid___A002_X85c183_X36f',<br />
'uid___A002_X85c183_X60b',<br />
'uid___A002_X8602fa_X2ab',<br />
'uid___A002_X8602fa_X577',<br />
'uid___A002_X864236_X2d4',<br />
'uid___A002_X864236_X693',<br />
'uid___A002_X86fcfa_X664',<br />
'uid___A002_X86fcfa_X96c',<br />
'uid___A002_X86fcfa_Xd9']<br />
vislist = map(lambda x: x + '.ms.bl', asdmlist)<br />
</source><br />
<br />
<br />
We first have to define and obtain some basic parameters. As all the ASDMs are ostensibly of the same kinds of observations, we refer to the first science ASDM to determine the basic properties of the resulting data cube. The cube size, pixel size, theoretical beam size, central frequency and other basic parameters are extracted using some CASA-functions and the Analysis Utils library (see the top of this page for information on Analysis Utils).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
spw = 3<br />
imagename = 'M100_TP_CO_cube.spw%s.image'%(spw)<br />
blimagename = imagename + '.bl'<br />
integimagename = imagename + '.integ'<br />
fwhmfactor = 1.13<br />
diameter = 12<br />
<br />
xSampling, ySampling, maxsize = aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms', showplot=False)<br />
<br />
msmd.open(asdmlist[0]+'.ms.bl')<br />
freq = msmd.meanfreq(spw)<br />
msmd.close()<br />
print("SPW %d: %.3f GHz" % (spw, freq*1e-9))<br />
<br />
theorybeam = aU.primaryBeamArcsec(frequency=freq*1e-9,<br />
fwhmfactor=fwhmfactor,<br />
diameter=diameter)<br />
<br />
cell = theorybeam/9.0<br />
imsize = int(round(maxsize/cell)*2)<br />
</source><br />
And now we learn the central frequency of spectral window 3, is 114.682 GHz.<br />
Our cube will have a total length of 253.0 arcsec ('''maxsize'''), and with a cell side of 5.64 arcsec ('''cell''') -which samples the theoretical beam (''theory beam'') with 9x9 pixels, we will need 90 pixels along the image side ('''imsize''').<br />
<br />
'''Note:''' The Jy/K value depends on the parameters of gridding convolution (i.e., "gridfunction", "cell", and function-specific parameters ["convsupport" for gridfunction='SF']). That is, the same gridding parameters should be used for both amplitude calibrator and science images -- otherwise the calibration into Jy unit becomes invalid. Please refer to the CASA cookbook for information on the gridding kernels.<br />
<br />
<br />
Now the data are imaged.<br />
The velocity channel maps of the CO J=1-0 line (restfreq='115.271201800GHz') are created as a data cube which covers the velocity range of 1400-1700 km/s at a spacing of 5 km/s (start='1400km/s', width='5km/s', nchan=70).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA <br />
sdimaging(infiles=vislist,<br />
field='M100',<br />
spw='%s'%(spw),<br />
nchan=70,<br />
mode='velocity',<br />
start='1400km/s',<br />
width='5km/s',<br />
veltype='radio',<br />
outframe='lsrk',<br />
restfreq='115.271204GHz',<br />
gridfunction='SF',<br />
convsupport=6,<br />
stokes='I',<br />
phasecenter='J2000 12h22m54.9 +15d49m15',<br />
ephemsrcname='',<br />
imsize=imsize,<br />
cell='%sarcsec'%(cell),<br />
brightnessunit='Jy/beam',<br />
overwrite=True,<br />
outfile=imagename)<br />
</source><br />
<br />
Now you can browse the data cube using the viewer.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
viewer(imagename)<br />
</source><br />
<br />
== Subtract a Residual Background from the Image ==<br />
<br />
If you plot the line profile using the viewer, you may notice that the background (i.e., line-free channels) level is slightly negative.<br />
To correct this, spectral baselines are subtracted from the image using the task {{imcontsub}}.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
imcontsub(imagename=imagename,<br />
linefile=blimagename,<br />
contfile='M100.residualbaseline.image',<br />
fitorder=1,<br />
chans='0~7;62~69')<br />
os.system('rm -rf M100.residualbaseline.image')<br />
</source><br />
<br />
This task has two outputs - the baselined data is returned in the same input file (''image name - M100_TP_CO_cube.spw3.image.bl''), while the subtracted component is returned in ''M100.residual baseline.image"". We have no further need for the subtracted component, and it can be discarded.<br />
<br />
== Basic Analysis - Moments ==<br />
<br />
Make a 0th moment (integrated intensity) map using the task {{immoments}} and browse it using the viewer.<br />
<br />
<br />
[[File:M100.CO.cube.bl.image.mom0-6.1.png|200px|thumb|right|<caption>The integrated intensity map of M100.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
immoments(imagename=blimagename,<br />
moments=[0],<br />
axis='spectral',<br />
chans='8~61',<br />
outfile=integimagename)<br />
<br />
viewer(integimagename)<br />
</source><br />
<br />
= Imaging the Amplitude Calibrator and Measure the Value of Jy/K =<br />
<br />
Determination of the Jy/K conversion factor is achieved by imaging a source whose continuum flux is known and measuring the observed brightness temperature. One observation of this source - the amplitude calibrator - was made per day, giving a total of four datasets (see top).<br />
<br />
The true size of the beam (point spread function), on which the Jy/K value depends, in a map is determined by several ingredients.<br />
* The intrinsic beam size is determined by the antenna diameter and optics (and of course wavelength). Here we assume the intrinsic beam size of 1.13*lambda/D, where D=12 [m] is the diameter of the antennas, and the factor 1.13 is derived from the optics design of the ALMA 12-m antennas.<br />
* The effective beam size is broadened by the following causes:<br />
** The scanning pattern used to observe the map (i.e., sample spacing in both directions).<br />
** The method used to grid the individual spectra into a map/cube.<br />
The Analysis Utilities provides several tools to get estimates of these.<br />
<br />
It is not necessary, but If you wish to try this out for yourself, you will first need to download the raw amplitude calibrator data file M100_Band3_TP_ampcal_UncalibratedData.tgz. How to obtain the data is described at [[M100_Band3#Obtaining the Data]]. Once you have unpacked the data you will have a directory called M100_Band3_TP_ampcal_UncalibratedData. In this directory, execute the scripts for individual datasets ([[media:A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py|A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py|A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py|A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py]], and [[media:A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py|A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py]]) to calibrate, image and derive Jy/K factors of all four calibrator datasets. The general approach is almost identical to that of calibrating the science target data, with the exception of the baselining step. Then, Jy/K values are derived from the peak of temperature in images and stored in text files (with the suffix, '.jyperk.txt').<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
es = aU.stuffForScienceDataReduction()<br />
execfile('A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py')<br />
</pre><br />
<br />
The Calibration Jy/K values are determined as follows. Note that the values for individual antennas are averaged for each day, each SPW, because the antennas used for the Science observations were not necessarily available for the corresponding amplitude calibrator observations.<br />
<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
#Date Amp_Cal_Dataset SPW17 SPW19 SPW21 SPW23<br />
2014-07-01 uid___A002_X85c183_X895 41.52 42.47 43.58 42.98<br />
2014-07-05 uid___A002_X8602fa_Xc3 41.38 42.59 42.08 42.53<br />
2014-07-07 uid___A002_X864236_Xe1 40.88 42.48 41.47 41.94<br />
2014-07-17 uid___A002_X86fcfa_X3ae 41.71 42.88 41.69 42.18<br />
</pre><br />
<br />
= Combination with 12m and 7m Array Data =<br />
<br />
If you wish to learn how to combine the Total Power data with interferometric 12m and 7m data, see the guide for Data Combination [[M100_Band3_Combine_6.1]]<br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2&diff=31240M100 Band3 SingleDish 6.22021-10-18T18:39:20Z<p>Jbertone: </p>
<hr />
<div><div style="margin-left:8%; margin-right:8%"><br />
<br />
[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
*'''This guide is written for CASA 6.1 and uses Python 3. The same functionality is implemented in Python 2 using CASA 5.7 at [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_5.7 M100_Band3_SingleDish_5.7]'''<br />
<br />
*'''This guide assumes that you have downloaded the Total Power data in M100_Band3_TP_UncalibratedData.tgz as described in [[M100_Band3#Obtaining_the_Data]]'''<br />
<br />
*'''The processes described below WILL NOT OPERATE in CASA versions earlier than 6.1.0'''<br />
<br />
*'''Details of the ALMA observations are described at [[M100_Band3]]<br />
<br />
*'''This portion of the guide covers calibration and imaging starting from the raw visibility data.'''<br />
<br />
<br />
= Overview =<br />
<br />
This portion of CASA Guide will cover the data reduction of the Total Power (TP) array observations of M100. The objective is to generate from the TP dataset, a position-position-velocity cube suitable for merging with 7m and 12m datasets.<br />
<br />
The data consist of "amplitude calibrator" datasets (containing observational data of the quasar 3C279) and "science" datasets containing data for the science target M100. To be clear - the observations of 3C279 and M100 both follow the same general process - i.e. they are themselves calibrated with frequent observations of ''blank sky'' - the so-called 'OFF', which is used to determine the System temperature as a function of frequency, and form a ''referenced'' bandpass, in units of Antenna Temperature (Ta*) [K].<br />
<br />
Ultimately the quasar data are compared with recent flux measurements (in Jy/beam), obtained from archived 12m data, to determine the scaling factor between Ta* and Jy/beam, and that scaling factor is then applied to the science data, to bootstrap to Jy/beam.<br />
<br />
The combination of the resultant image with the interferometric (12-m array and 7-m array) data is explained in a separate page, valid for CASA versions 6.1 and later [[M100_Band3_Combine_6.1]].<br />
<br />
This guide is designed for CASA 6.1.0.<br />
<br />
----<br />
<br />
==Confirm Your Version of CASA==<br />
This guide has been written for CASA release 6.1.0. Please confirm your version before proceeding.<br />
The value returned from ''casalith.version_string()'' should be 6.1.0. If not, please obtain at least version 6.1.0 (or later), from [https://casa.nrao.edu/casa_obtaining.shtml]<br />
<br />
<source lang="python" style="display:inline-block"><br />
#IN CASA<br />
version = casalith.version_string()<br />
print("You are using " + version)<br />
if (version < '6.1.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
== Summary of Datasets ==<br />
<br />
The observations were made on the 1st, 5th, 7th, and 17th July 2014, using two or three 12-m antennas and the ACA correlator.<br />
The table below indicates the ID's of the Execution Blocks, their start and end times, and the antennas in the array.<br />
There are nine science datasets (i.e., two or three per day).<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Science<br />
uid___A002_X85c183_X36f Observed from 2014-07-01T21:51:26.2 to 2014-07-01T22:40:28.4 (UTC) DA61, PM03, PM04<br />
uid___A002_X85c183_X60b Observed from 2014-07-01T22:43:50.0 to 2014-07-01T23:32:39.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_X2ab Observed from 2014-07-05T23:58:03.6 to 2014-07-06T00:46:52.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X8602fa_X577 Observed from 2014-07-06T00:55:17.8 to 2014-07-06T01:44:07.3 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_X2d4 Observed from 2014-07-07T23:03:48.1 to 2014-07-07T23:53:47.9 (UTC) PM03, PM04<br />
uid___A002_X864236_X693 Observed from 2014-07-07T23:56:09.6 to 2014-07-08T00:46:07.1 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_Xd9 Observed from 2014-07-17T20:55:15.5 to 2014-07-17T21:44:06.1 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X664 Observed from 2014-07-17T22:24:17.3 to 2014-07-17T23:13:08.0 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X96c Observed from 2014-07-17T23:23:37.0 to 2014-07-18T00:12:25.3 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
Similarly, the concomitant ''Calibration'' datasets (used to convert from Antenna temperature units into Jy/beam units, for combination), are:<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Amplitude Calibrator<br />
uid___A002_X85c183_X895 Observed from 2014-07-01T23:35:23.1 to 2014-07-02T00:07:54.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_Xc3 Observed from 2014-07-05T23:21:25.6 to 2014-07-05T23:53:41.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_Xe1 Observed from 2014-07-07T22:27:35.4 to 2014-07-07T23:01:05.7 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_X3ae Observed from 2014-07-17T21:48:30.0 to 2014-07-17T22:20:52.2 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
= Script Calibration =<br />
The scripts listed below process all the single-dish science and calibrator data, using a few loops. <br />
<br />
# (optional) The calibrator (i.e. quasar) data are processed (see the section [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Imaging_the_Amplitude_Calibrator_and_Measure_the_Value_of_Jy.2FK Imaging the Amplitude Calibrator and Measure the Value of Jy/K] below)<br />
# The science data are processed with - [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]] <br />
# and the (calibrated) science data are imaged with - [[media:M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]<br />
<br />
Executing each of these scripts in order, will be sufficient to calibrate and image the science data (although the calibrator processing script can be omitted, as the relevant parameters are actually already explicitly written into the script).<br />
<br />
They can be invoked from the command line using the execfile command.<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
execfile('M100.casa6p1.calib.py')<br />
</pre><br />
<br />
= Manual Calibration =<br />
<br />
By way of description, and as a means to introduce and demonstrate some of the interactive and diagnostic, tools here we describe in more detail the steps taken by the scripts - although in this case we demonstrate the process for only '''one''' of the ASDM datasets. In the ''Imaging'' section, we assume '''all''' the ASDMs are appropriately reduced either by the manual process, or by using the script. <br />
<br />
The entire process takes less than 30 minutes.<br />
<br />
There are three basic steps to the Calibration of the Science data:<br />
# Import, inspection and flagging<br />
# Calibration<br />
#* Sky and bandpass calibration (i.e. Tsys and frequency sampling function)<br />
#* Adjustment for Correlator non-linearity (this step is necessary only for data taken in ALMA Cycle 3 and earlier)<br />
#* Conversion to Jy/beam<br />
# Imaging<br />
<br />
=== Import "Analysis Utilities" ===<br />
<br />
The [[Analysis_Utilities]] package will be used most critically for the imaging processes. While it is not necessary for completing the calibration stages, it is used to check data quality, and should be loaded.<br />
<br />
Import the package from https://casaguides.nrao.edu/index.php/Analysis_Utilities, and apply the class therein.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
</source><br />
<br />
== Import, inspection and flagging ==<br />
<br />
=== Import ===<br />
The first thing to do is to convert the dataset into the CASA Measurement Set (MS) format. The raw data have been provided to you in the ASDM (ALMA Science Data Model), which is the native format of the data produced by the observatory but cannot be processed by CASA. The conversion from ASDM to MS is done with the task {{importasdm}}.<br />
<br />
<source lang="python" style="overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=True)<br />
</source><br />
<br />
The converted dataset (with a suffix ".ms") is created and then flags embedded in the ASDM dataset (so-called "BDF flags") are transferred to MS using the bdfflags parameter.<br />
<br />
'''Note:''' The above code should be used for Cycle 3 data or later. Cycle 2 data and earlier should use:<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=False)<br />
os.system(os.environ['CASAPATH'].split()[0] + '/bin/bdflags2MS -f "COR DELA INT MIS SIG SYN TFB WVR ZER" uid___A002_X85c183_X36f uid___A002_X85c183_X36f.ms')<br />
</pre><br />
<br />
<br />
=== Inspection - listobs ===<br />
After import, the usual first step is then to get some basic information about the data.<br />
We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied.<br />
<br />
<source lang="python" style="display:inline-block><br />
# In CASA<br />
listobs(vis='uid___A002_X85c183_X36f.ms',<br />
listfile='uid___A002_X85c183_X36f.ms.listobs')<br />
</source><br />
<br />
The output will be written in a file named uid___A002_X85c183_X36f.ms.listobs which you can (and should) view at your leisure.<br />
{{listobs}} displays information about the targets, dates, frequencies, etc. Below is an excerpt from listobs for uid___A002_X85c183_X36f:<br />
<br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1215+1654 12:15:03.979140 +16.54.37.95680 J2000 0 14652<br />
1 none M100 12:22:54.360000 +15.48.50.60000 J2000 1 61509<br />
<br />
Spectral Windows: (25 unique spectral windows and 2 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_03#BB_1#SW-01#FULL_RES 124 TOPO 91955.512 -15625.000 1937500.0 90994.5750 1 XX YY<br />
2 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 90978.950 1734375.000 1734375.0 90978.9500 1 XX YY<br />
3 ALMA_RB_03#BB_2#SW-01#FULL_RES 124 TOPO 93893.012 -15625.000 1937500.0 92932.0750 2 XX YY<br />
4 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 92924.262 1937500.000 1937500.0 92924.2625 2 XX YY<br />
5 ALMA_RB_03#BB_3#SW-01#FULL_RES 124 TOPO 102033.637 15625.000 1937500.0 102994.5750 3 XX YY<br />
6 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 102986.762 1937500.000 1937500.0 102986.7625 3 XX YY<br />
7 ALMA_RB_03#BB_4#SW-01#FULL_RES 124 TOPO 104033.637 15625.000 1937500.0 104994.5750 4 XX YY<br />
8 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 104986.762 1937500.000 1937500.0 104986.7625 4 XX YY<br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
10 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100926.562 1781250.000 1781250.0 100926.5625 1 XX YY<br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
12 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102741.712 1781250.000 1781250.0 102741.7125 2 XX YY<br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
14 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112783.712 1781250.000 1781250.0 112783.7125 3 XX YY<br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
16 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114658.712 1781250.000 1781250.0 114658.7125 4 XX YY<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
18 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100949.756 1992187.500 1992187.5 100949.7559 1 XX YY<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
20 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102764.906 1992187.500 1992187.5 102764.9059 2 XX YY<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
22 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112806.906 1992187.500 1992187.5 112806.9059 3 XX YY<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
24 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114681.906 1992187.500 1992187.5 114681.9059 4 XX YY<br />
<br />
Sources: 48<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1215+1654 0 - - <br />
0 J1215+1654 25 - - <br />
0 J1215+1654 26 - - <br />
0 J1215+1654 27 - - <br />
0 J1215+1654 1 - - <br />
0 J1215+1654 2 - - <br />
0 J1215+1654 3 - - <br />
0 J1215+1654 4 - - <br />
0 J1215+1654 5 - - <br />
0 J1215+1654 6 - - <br />
0 J1215+1654 7 - - <br />
0 J1215+1654 8 - - <br />
0 J1215+1654 9 - - <br />
0 J1215+1654 10 - - <br />
0 J1215+1654 11 - - <br />
0 J1215+1654 12 - - <br />
0 J1215+1654 13 - - <br />
0 J1215+1654 14 - - <br />
0 J1215+1654 15 - - <br />
0 J1215+1654 16 - - <br />
0 J1215+1654 17 100950 0 <br />
0 J1215+1654 18 100950 0 <br />
0 J1215+1654 19 102794.1 0 <br />
0 J1215+1654 20 102794.1 0 <br />
0 J1215+1654 21 112794.1 0 <br />
0 J1215+1654 22 112794.1 0 <br />
0 J1215+1654 23 114669.1 0 <br />
0 J1215+1654 24 114669.1 0 <br />
1 M100 0 - - <br />
1 M100 25 - - <br />
1 M100 26 - - <br />
1 M100 27 - - <br />
1 M100 9 - - <br />
1 M100 10 - - <br />
1 M100 11 - - <br />
1 M100 12 - - <br />
1 M100 13 - - <br />
1 M100 14 - - <br />
1 M100 15 - - <br />
1 M100 16 - - <br />
1 M100 17 100950 0 <br />
1 M100 18 100950 0 <br />
1 M100 19 102794.1 0 <br />
1 M100 20 102794.1 0 <br />
1 M100 21 112794.1 0 <br />
1 M100 22 112794.1 0 <br />
1 M100 23 114669.1 0 <br />
1 M100 24 114669.1 0 <br />
<br />
Antennas: 3:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA61 A075 12.0 m -067.45.17.9 -22.53.21.4 -4.5609 -499.7012 23.0322 2225072.419944 -5440148.858968 -2481499.171703<br />
1 PM03 T701 12.0 m -067.45.18.8 -22.53.22.2 -29.1265 -522.7875 22.2052 2225045.995589 -5440149.141967 -2481520.118569<br />
2 PM04 T703 12.0 m -067.45.16.2 -22.53.23.9 42.8797 -575.6910 21.7763 2225104.700870 -5440102.471978 -2481568.689518<br />
</pre><br />
<br />
From this output you can for example see the followings.<br />
<br />
* From "Data records" section (not shown here): The execution consists of 15 scans with various scan intents.<br />
** Scans 1 and 2 are pointing and sideband gain ratio calibrations (done interferometrically), which need to be done prior to observing the target, on the quasar J1215+1654.<br />
** Scans 3 and 4 are interferometric delay and system noise temperature (Tsys) measurements also on J1215+1654 (they are in principle unnecessary; just a hack to make things happen on the telescope control software).<br />
** Scans 6, 7, 9, 10, 12, 13, and 15, whose scan intents contain "OBSERVE_TARGET", are for raster mapping of the target M100. The associated spectral window (SPW) ID's are 0 and 17-24.<br />
** Scans 5, 8, 11, and 14 with scan intents "CALIBRATE_ATMOSPHERE" are Tsys measurements for M100. The SPW ID's for Tsys scans are 0 and 9-16.<br />
* From "Spectral Windows" section:<br />
** SPW 0 contains the Water Vapor Radiometer (WVR) data, which are taken for all scans, but which are not used here.<br />
** SPW's 1, 3, 5, and 7 are placed on standard continuum frequencies and are used for the pointing scan only.<br />
** SPW's 9, 11, 13, and 15 cover the same frequency range as the science SPW's (see the next bullet) and are used to measure the Tsys used for calibration. They have 128 spectral channels (15.6 MHz spacing) in 2000 MHz bandwidth.<br />
** SPW's 17, 19, 21, and 23 cover the frequency ranges used for the actual science observations. They have 4080 spectral channels (488 kHz spacing) in 1992 MHz bandwidth. Hereafter these SPW's are referred to as "science" SPW's. The target line, CO J=1-0, is placed in SPW 23, the corresponding Tsys is taken from SPW 15.<br />
** The remaining, even-number SPW's contain channel-averages of the actually used SPW's (e.g., SPW 24 corresponds to SPW 23 averaged into one channel).<br />
* From "Antennas" section: Three 12-m antennas (DA61, PM03, and PM04) were used for the observations.<br />
<br />
=== Inspection - scanpattern ===<br />
The scan pattern of the raster mapping can be visualized by issuing the following command (after import of Analysis Utils - see the top of this page for information).<br />
A plot will be shown in a window and also saved as a PNG file uid___A002_X85c183_X36f.ms.sampling.png.<br />
<br />
<br />
[[File:X36f.TPSampling-6.1.png|200px|thumb|right|<caption>The scan pattern of the raster mapping for uid___A002_X85c183_X36f.</caption>]]<br />
<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms',<br />
showplot=True,<br />
plotfile='uid___A002_X85c183_X36f.ms.sampling.png')<br />
</source><br />
<br />
=== Inspection - System Noise Temperature ===<br />
Let's start by checking the Tsys.<br />
We use the task {{gencal}} to extract the Tsys into a CASA calibration table. This table is only used for inspection of Tsys quality, and not in further processing.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
gencal(vis='uid___A002_X85c183_X36f.ms',<br />
caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
caltype='tsys')<br />
</source><br />
<br />
The generated Tsys calibration table can be plotted using the task {{plotbandpass}}. The generated plots are saved in the directory<br />
uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime.<br />
<br />
<br />
[[File:X36f.Tsys.overlayTime-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for DA61 in uid___A002_X85c183_X36f. Colors represent time.</caption>]]<br />
<br />
<br />
<br />
[[File:X36f.Tsys.overlayAntenna-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for SPW 15 in uid___A002_X85c183_X36f, using overlay='antenna'. Colors represent antennas.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotbandpass(caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
overlay='time',<br />
xaxis='freq',<br />
yaxis='amp',<br />
subplot=22,<br />
buildpdf=False,<br />
interactive=False,<br />
showatm=True,<br />
pwv='auto',<br />
chanrange='5~123',<br />
showfdm=True,<br />
field='',<br />
figfile='uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime/uid___A002_X85c183_X36f.ms.tsys')<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
Although the "science" SPW's have 1992 MHz bandwidths with 4080 spectral channels, their edge channels can be very noisy because each intermediate frequency signal path (baseband) is equipped with a bandpass filter of ~1.8 GHz width. Note that this is not necessarily always the case, it is in general, safest to flag out the edge channels, unless a spectral line of interest falls very close to the edge of the SPW band-edge.<br />
We flag 120 channels on each side of the "science" SPW's.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis = 'uid___A002_X85c183_X36f.ms',<br />
mode = 'manual',<br />
spw = '17:0~119;3960~4079,19:0~119;3960~4079,21:0~119;3960~4079,23:0~119;3960~4079',<br />
action = 'apply',<br />
flagbackup = True)<br />
</source><br />
<br />
==Calibration - sky, bandpass, nonlinearity and baseline ==<br />
<br />
=== Tsys Calibration ===<br />
We calibrate the data into Antenna temperature in units of K, using the equation Ta* = Tsys*(ON-OFF)/OFF, where ON and OFF are the data on-source (i.e., during the raster scanning) and off-source (on a reference position where only background emission exists), respectively.<br />
The calibration is done by the task {{sdcal}} which requires matching of list of the "science" and Tsys spectral windows. <br />
Tsys observations will always have 128 channels over 2GHz bandwidth, and as described in {{listobs}}, we already know which spectral windows match the science spectral windows most closely, in frequency.<br />
for example, spectral window 9, with 128 channels matches well with spectral window 17, 11 with 19, 13 with 21 and 15 with 2. Thus, we form the Tsys and Science spectral window mapping input parameters, with the syntax: {9:[17], 11:[19], 13:[21],15:[23]}<br />
<br />
<pre style="background-color: #fffacd; overflow-x:auto; white-space: pre"><br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
.<br />
.<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
.<br />
.<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
.<br />
.<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
.<br />
.<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
</pre><br />
<br />
And the Tsys calibration command is formed like the following:<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdcal(infile = 'uid___A002_X85c183_X36f.ms',<br />
calmode = 'ps,tsys,apply',<br />
spwmap = {9:[17], 11:[19], 13:[21],15:[23]},<br />
spw = '17,19,21,23')<br />
</source><br />
<br />
'''Note:''' in this step, the calibrations are applied in situ. That is: the calibrated data are written into the original data - however it is written in such a way that any ''additional'' calls to sdcal will not apply any further calibrations. We will examine the quality of the bandpass and Tsys calibration after the correction for non-linearity, next.<br />
<br />
<br />
<br />
[[File:Vis_Scan6_7_9_10_12_13_15-6.1.png|200px|thumb|right|<caption>Output of plotms, showing spectra in Ta* [K]</caption>]]<br />
<br />
<br />
The calibrated bandpasses, averaged over each scan, can be plotted with:<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='23',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis.png',<br />
showgui=True)<br />
</source><br />
<br />
'''Note:''' For ''amplitude calibrator datasets'', (as opposed to science target maps) the calmode option for sdcal should be 'otfraster,tsys,apply' ('otfraster' instead of 'ps').<br />
'otfraster' tells the task to use both ends of each raster row as "OFF" data (as opposed to a dedicated "OFF" sky measurement).<br />
This is because temporal fluctuation of atmospheric emission is the dominant source of noise in radio continuum observations, and hence the "OFF" needs to be as close as possible to the "ON" in time (and spatial location).<br />
<br />
=== Application of Non-Linearity Correction Factor ===<br />
<br />
In Cycle 1 and 2, single-dish data taken with the ACA correlator suffer from non-linearity which originates in the digital signal processing.<br />
Its impact was thoroughly studied both experimentally and theoretically, and it was concluded that multiplication by a correction factor of 1.25 yields an amplitude accuracy of +/-5%.<br />
<br />
To preserve the data history, we use here the tasks {{split}} to split out the science ''corrected'' data columns - note that from here on, the spectral windows are '''renumbered: [0,1,2,3]'''. We then generate a calibration table, using a defined function ''to_amp_factor'', with {{gencal}}. Finally, we apply the calibration with {{applycal}}.<br />
We apply the correction here using tasks gencal and applycal - this is different to the methods used in the past, and are available now because of the migration to MS-only format of Single-dish CASA.<br />
<br />
'''Note:''' The ACA online software has been equipped with this correction for data from cycle 3 and later (it should be applied for data from cycle 2 and earlier).<br />
<br />
Also note that we need to import the math module in order to use the sqrt command in Python 3.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import math<br />
<br />
to_amp_factor = lambda x: 1. / math.sqrt(x)<br />
<br />
split(vis='uid___A002_X85c183_X36f.ms',<br />
outputvis='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='17,19,21,23')<br />
<br />
gencal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
caltable='uid___A002_X85c183_X36f.ms.nlc',<br />
caltype='amp', spw='',<br />
parameter=[to_amp_factor(1.25)])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
gaintable='uid___A002_X85c183_X36f.ms.nlc')<br />
</source><br />
<br />
'''Note:''' the Python command ''lambda'' simply defines a function. Thus the line above : ''to_amp_factor = lambda x: 1. / math.sqrt(x)'' defines a function where the call: ''to_amp_factor(4)'' will return: ''0.5''<br />
<br />
=== Baseline Subtraction (only for Science Datasets) ===<br />
<br />
'''Note:''' this step is appropriate for spectral line observations (science datasets) but ''should not be done for the continuum observations (amplitude calibrator datasets) -- it would eliminate the continuum emission!''<br />
'''Note:''' this step is optional and need not be completed to produce an image - however it is recommended for the majority of cases (esp. where the bandpass is affected by low-order modulations, and where the line-emission does not completely dominate the bandpass.)<br />
<br />
We will subtract low-order spectral baselines using the task {{sdbaseline}}. Typically these manifest from structure in the distribution of water in the atmosphere: the ON and OFF positions are slightly different, and the difference manifests as - typically - a low frequency variation in the spectra. The situation is exacerbated for observations closer to water lines and also for higher frequencies.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdbaseline(infile='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='',<br />
blfunc='poly',<br />
order=1,<br />
outfile='uid___A002_X85c183_X36f.ms.bl',<br />
overwrite=True)<br />
</source><br />
<br />
<br />
<br />
[[File: vis_bl_Scan6_7_9_10_12_13_15-6.1.png |200px|thumb|right|<caption>Output of plotms, showing result of baseline step, units in Ta* [K]</caption>]]<br />
<br />
<br />
The spectra can be checked using {{plotms}}, which we have already used in a previous step.<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms.bl',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='3',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis_bl.png',<br />
showgui=True)<br />
</source><br />
<br />
We do not concatenate separate Execution Blocks, because we need to determine and apply day-by-day Jy/K conversion factor.<br />
<br />
At this point in the guide, we have finished the calibration for ASDM uid___A002_X85c183_X36f (except for the calculation of the Jy/K conversion factors). In order to finish the calibration for the entire M100 dataset, the above steps should be repeated with each of the ADSMs shown in [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Summary_of_Datasets Summary of Datasets]. Once each ASDM has been calibrated, the user can move on to the flagging, Jy/K conversion, and imaging processes below.<br />
<br />
Note that one of the antennas, PM02, had a problem in the first baseband (SPW 17) on 2014-07-05, and its impact may not be specific to this one SPW (e.g., it may have resulted in poor pointing calibration).<br />
Therefore the PM02 data should be flagged from the affected datasets: uid___A002_X8602fa_X2ab (science), and uid___A002_X8602fa_X577 (science). The calibrator set uid___A002_X8602fa_Xc3 will also be affected, and will be flagged out in the subsequent processing of the calibrators.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis='uid___A002_X8602fa_X2ab.ms.bl',<br />
antenna='PM02&&&')<br />
flagdata(vis='uid___A002_X8602fa_X577.ms.bl',<br />
antenna='PM02&&&')<br />
</source><br />
<br />
<br />
== Convert science data from Kelvin to Jansky ==<br />
<br />
At this stage the data have been calibrated into units of antenna temperature, in Kelvins.<br />
<br />
It now remains to convert the science data into Jy/beam units - the ''Jy per K'' factors. This is important for combination of the data, with 7m and 12m sets. Later in this guide we show how to determine the correct factors from the calibrator datasets, for conversion (the process is essentially identical to processing the science datasets, with the exception that the baseline step is omitted from the calibration dataset processing). However to determine the factors, the calibration data must also be imaged and then interrogated to determine the appropriate values. <br />
<br />
Applying the calibration factors to the science data is the last step in the script [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]]. <br />
<br />
We continue to use only one ASDM as an example; uid___A002_X85c183_X36f. The script should be consulted for treatment of all the M100 Single-dish ASDMs (and the correct application of the Jy per K factors). Again, we apply the factors by making use of the {{gencal}} and {{applycal}} tasks. Also as before, we make use of the python function defined earlier: ''to_amp_factor''<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
<br />
factors = [41.52, 42.47, 43.58, 42.98]<br />
gencal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
caltable='uid___A002_X85c183_X36f.ms.jy',<br />
caltype='amp',<br />
spw='0,1,2,3',<br />
parameter=[ to_amp_factor(v) for v in factors ])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
gaintable='uid___A002_X85c183_X36f.ms.jy')<br />
</source><br />
<br />
= Image the Science Target =<br />
<br />
The final stage is to image the science data. This step is done by the script "[[media: M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]" which is invoked with ''execfile('M100.casa6p1.imaging.py') '' in CASA.<br />
This step uses a number of helper utilities to automatically determine the properties of the image data - the analysis Utilities library must be loaded (see the text above).<br />
<br />
== Determining grid parameters, gridding and obtaining grid metadata ==<br />
Define the list of the calibrated science datasets, which the CASA task {{sdimaging}} accepts. Note that this step is the same for science and calibrator data.<br />
This section is a little more python-heavy than the calibration step. CASA 6.1.0 has benefited from significant effort in streamlining the calibration steps, and the imaging steps are to be more automated in coming cycles.<br />
<br />
In this section, we will simply process all the calibrated science data. The calibrator data should never be combined. There is no need to ''concatenate'' the calibrated data, as the imaging task can explicitly accept multiple datasets as input.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
asdmlist = ['uid___A002_X85c183_X36f',<br />
'uid___A002_X85c183_X60b',<br />
'uid___A002_X8602fa_X2ab',<br />
'uid___A002_X8602fa_X577',<br />
'uid___A002_X864236_X2d4',<br />
'uid___A002_X864236_X693',<br />
'uid___A002_X86fcfa_X664',<br />
'uid___A002_X86fcfa_X96c',<br />
'uid___A002_X86fcfa_Xd9']<br />
vislist = map(lambda x: x + '.ms.bl', asdmlist)<br />
</source><br />
<br />
<br />
We first have to define and obtain some basic parameters. As all the ASDMs are ostensibly of the same kinds of observations, we refer to the first science ASDM to determine the basic properties of the resulting data cube. The cube size, pixel size, theoretical beam size, central frequency and other basic parameters are extracted using some CASA-functions and the Analysis Utils library (see the top of this page for information on Analysis Utils).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
spw = 3<br />
imagename = 'M100_TP_CO_cube.spw%s.image'%(spw)<br />
blimagename = imagename + '.bl'<br />
integimagename = imagename + '.integ'<br />
fwhmfactor = 1.13<br />
diameter = 12<br />
<br />
xSampling, ySampling, maxsize = aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms', showplot=False)<br />
<br />
msmd.open(asdmlist[0]+'.ms.bl')<br />
freq = msmd.meanfreq(spw)<br />
msmd.close()<br />
print("SPW %d: %.3f GHz" % (spw, freq*1e-9))<br />
<br />
theorybeam = aU.primaryBeamArcsec(frequency=freq*1e-9,<br />
fwhmfactor=fwhmfactor,<br />
diameter=diameter)<br />
<br />
cell = theorybeam/9.0<br />
imsize = int(round(maxsize/cell)*2)<br />
</source><br />
And now we learn the central frequency of spectral window 3, is 114.682 GHz.<br />
Our cube will have a total length of 253.0 arcsec ('''maxsize'''), and with a cell side of 5.64 arcsec ('''cell''') -which samples the theoretical beam (''theory beam'') with 9x9 pixels, we will need 90 pixels along the image side ('''imsize''').<br />
<br />
'''Note:''' The Jy/K value depends on the parameters of gridding convolution (i.e., "gridfunction", "cell", and function-specific parameters ["convsupport" for gridfunction='SF']). That is, the same gridding parameters should be used for both amplitude calibrator and science images -- otherwise the calibration into Jy unit becomes invalid. Please refer to the CASA cookbook for information on the gridding kernels.<br />
<br />
<br />
Now the data are imaged.<br />
The velocity channel maps of the CO J=1-0 line (restfreq='115.271201800GHz') are created as a data cube which covers the velocity range of 1400-1700 km/s at a spacing of 5 km/s (start='1400km/s', width='5km/s', nchan=70).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA <br />
sdimaging(infiles=vislist,<br />
field='M100',<br />
spw='%s'%(spw),<br />
nchan=70,<br />
mode='velocity',<br />
start='1400km/s',<br />
width='5km/s',<br />
veltype='radio',<br />
outframe='lsrk',<br />
restfreq='115.271204GHz',<br />
gridfunction='SF',<br />
convsupport=6,<br />
stokes='I',<br />
phasecenter='J2000 12h22m54.9 +15d49m15',<br />
ephemsrcname='',<br />
imsize=imsize,<br />
cell='%sarcsec'%(cell),<br />
brightnessunit='Jy/beam',<br />
overwrite=True,<br />
outfile=imagename)<br />
</source><br />
<br />
Now you can browse the data cube using the viewer.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
viewer(imagename)<br />
</source><br />
<br />
== Subtract a Residual Background from the Image ==<br />
<br />
If you plot the line profile using the viewer, you may notice that the background (i.e., line-free channels) level is slightly negative.<br />
To correct this, spectral baselines are subtracted from the image using the task {{imcontsub}}.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
imcontsub(imagename=imagename,<br />
linefile=blimagename,<br />
contfile='M100.residualbaseline.image',<br />
fitorder=1,<br />
chans='0~7;62~69')<br />
os.system('rm -rf M100.residualbaseline.image')<br />
</source><br />
<br />
This task has two outputs - the baselined data is returned in the same input file (''image name - M100_TP_CO_cube.spw3.image.bl''), while the subtracted component is returned in ''M100.residual baseline.image"". We have no further need for the subtracted component, and it can be discarded.<br />
<br />
== Basic Analysis - Moments ==<br />
<br />
Make a 0th moment (integrated intensity) map using the task {{immoments}} and browse it using the viewer.<br />
<br />
<br />
[[File:M100.CO.cube.bl.image.mom0-6.1.png|200px|thumb|right|<caption>The integrated intensity map of M100.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
immoments(imagename=blimagename,<br />
moments=[0],<br />
axis='spectral',<br />
chans='8~61',<br />
outfile=integimagename)<br />
<br />
viewer(integimagename)<br />
</source><br />
<br />
= Imaging the Amplitude Calibrator and Measure the Value of Jy/K =<br />
<br />
Determination of the Jy/K conversion factor is achieved by imaging a source whose continuum flux is known and measuring the observed brightness temperature. One observation of this source - the amplitude calibrator - was made per day, giving a total of four datasets (see top).<br />
<br />
The true size of the beam (point spread function), on which the Jy/K value depends, in a map is determined by several ingredients.<br />
* The intrinsic beam size is determined by the antenna diameter and optics (and of course wavelength). Here we assume the intrinsic beam size of 1.13*lambda/D, where D=12 [m] is the diameter of the antennas, and the factor 1.13 is derived from the optics design of the ALMA 12-m antennas.<br />
* The effective beam size is broadened by the following causes:<br />
** The scanning pattern used to observe the map (i.e., sample spacing in both directions).<br />
** The method used to grid the individual spectra into a map/cube.<br />
The Analysis Utilities provides several tools to get estimates of these.<br />
<br />
It is not necessary, but If you wish to try this out for yourself, you will first need to download the raw amplitude calibrator data file M100_Band3_TP_ampcal_UncalibratedData.tgz. How to obtain the data is described at [[M100_Band3#Obtaining the Data]]. Once you have unpacked the data you will have a directory called M100_Band3_TP_ampcal_UncalibratedData. In this directory, execute the scripts for individual datasets ([[media:A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py|A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py|A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py|A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py]], and [[media:A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py|A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py]]) to calibrate, image and derive Jy/K factors of all four calibrator datasets. The general approach is almost identical to that of calibrating the science target data, with the exception of the baselining step. Then, Jy/K values are derived from the peak of temperature in images and stored in text files (with the suffix, '.jyperk.txt').<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
es = aU.stuffForScienceDataReduction()<br />
execfile('A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py')<br />
</pre><br />
<br />
The Calibration Jy/K values are determined as follows. Note that the values for individual antennas are averaged for each day, each SPW, because the antennas used for the Science observations were not necessarily available for the corresponding amplitude calibrator observations.<br />
<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
#Date Amp_Cal_Dataset SPW17 SPW19 SPW21 SPW23<br />
2014-07-01 uid___A002_X85c183_X895 41.52 42.47 43.58 42.98<br />
2014-07-05 uid___A002_X8602fa_Xc3 41.38 42.59 42.08 42.53<br />
2014-07-07 uid___A002_X864236_Xe1 40.88 42.48 41.47 41.94<br />
2014-07-17 uid___A002_X86fcfa_X3ae 41.71 42.88 41.69 42.18<br />
</pre><br />
<br />
= Combination with 12m and 7m Array Data =<br />
<br />
If you wish to learn how to combine the Total Power data with interferometric 12m and 7m data, see the guide for Data Combination [[M100_Band3_Combine_6.1]]<br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.2&diff=31239M100 Band3 SingleDish 6.22021-10-18T18:20:32Z<p>Jbertone: </p>
<hr />
<div><div style="margin:8%"><br />
<br />
[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
*'''This guide is written for CASA 6.1 and uses Python 3. The same functionality is implemented in Python 2 using CASA 5.7 at [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_5.7 M100_Band3_SingleDish_5.7]'''<br />
<br />
*'''This guide assumes that you have downloaded the Total Power data in M100_Band3_TP_UncalibratedData.tgz as described in [[M100_Band3#Obtaining_the_Data]]'''<br />
<br />
*'''The processes described below WILL NOT OPERATE in CASA versions earlier than 6.1.0'''<br />
<br />
*'''Details of the ALMA observations are described at [[M100_Band3]]<br />
<br />
*'''This portion of the guide covers calibration and imaging starting from the raw visibility data.'''<br />
<br />
<br />
= Overview =<br />
<br />
This portion of CASA Guide will cover the data reduction of the Total Power (TP) array observations of M100. The objective is to generate from the TP dataset, a position-position-velocity cube suitable for merging with 7m and 12m datasets.<br />
<br />
The data consist of "amplitude calibrator" datasets (containing observational data of the quasar 3C279) and "science" datasets containing data for the science target M100. To be clear - the observations of 3C279 and M100 both follow the same general process - i.e. they are themselves calibrated with frequent observations of ''blank sky'' - the so-called 'OFF', which is used to determine the System temperature as a function of frequency, and form a ''referenced'' bandpass, in units of Antenna Temperature (Ta*) [K].<br />
<br />
Ultimately the quasar data are compared with recent flux measurements (in Jy/beam), obtained from archived 12m data, to determine the scaling factor between Ta* and Jy/beam, and that scaling factor is then applied to the science data, to bootstrap to Jy/beam.<br />
<br />
The combination of the resultant image with the interferometric (12-m array and 7-m array) data is explained in a separate page, valid for CASA versions 6.1 and later [[M100_Band3_Combine_6.1]].<br />
<br />
This guide is designed for CASA 6.1.0.<br />
<br />
----<br />
<br />
==Confirm Your Version of CASA==<br />
This guide has been written for CASA release 6.1.0. Please confirm your version before proceeding.<br />
The value returned from ''casalith.version_string()'' should be 6.1.0. If not, please obtain at least version 6.1.0 (or later), from [https://casa.nrao.edu/casa_obtaining.shtml]<br />
<br />
<source lang="python" style="display:inline-block"><br />
#IN CASA<br />
version = casalith.version_string()<br />
print("You are using " + version)<br />
if (version < '6.1.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
== Summary of Datasets ==<br />
<br />
The observations were made on the 1st, 5th, 7th, and 17th July 2014, using two or three 12-m antennas and the ACA correlator.<br />
The table below indicates the ID's of the Execution Blocks, their start and end times, and the antennas in the array.<br />
There are nine science datasets (i.e., two or three per day).<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Science<br />
uid___A002_X85c183_X36f Observed from 2014-07-01T21:51:26.2 to 2014-07-01T22:40:28.4 (UTC) DA61, PM03, PM04<br />
uid___A002_X85c183_X60b Observed from 2014-07-01T22:43:50.0 to 2014-07-01T23:32:39.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_X2ab Observed from 2014-07-05T23:58:03.6 to 2014-07-06T00:46:52.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X8602fa_X577 Observed from 2014-07-06T00:55:17.8 to 2014-07-06T01:44:07.3 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_X2d4 Observed from 2014-07-07T23:03:48.1 to 2014-07-07T23:53:47.9 (UTC) PM03, PM04<br />
uid___A002_X864236_X693 Observed from 2014-07-07T23:56:09.6 to 2014-07-08T00:46:07.1 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_Xd9 Observed from 2014-07-17T20:55:15.5 to 2014-07-17T21:44:06.1 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X664 Observed from 2014-07-17T22:24:17.3 to 2014-07-17T23:13:08.0 (UTC) DV10, PM03, PM04<br />
uid___A002_X86fcfa_X96c Observed from 2014-07-17T23:23:37.0 to 2014-07-18T00:12:25.3 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
Similarly, the concomitant ''Calibration'' datasets (used to convert from Antenna temperature units into Jy/beam units, for combination), are:<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
#Amplitude Calibrator<br />
uid___A002_X85c183_X895 Observed from 2014-07-01T23:35:23.1 to 2014-07-02T00:07:54.6 (UTC) DA61, PM03, PM04<br />
uid___A002_X8602fa_Xc3 Observed from 2014-07-05T23:21:25.6 to 2014-07-05T23:53:41.0 (UTC) PM02, PM03, PM04<br />
uid___A002_X864236_Xe1 Observed from 2014-07-07T22:27:35.4 to 2014-07-07T23:01:05.7 (UTC) PM03, PM04<br />
uid___A002_X86fcfa_X3ae Observed from 2014-07-17T21:48:30.0 to 2014-07-17T22:20:52.2 (UTC) DV10, PM03, PM04<br />
</pre><br />
<br />
= Script Calibration =<br />
The scripts listed below process all the single-dish science and calibrator data, using a few loops. <br />
<br />
# (optional) The calibrator (i.e. quasar) data are processed (see the section [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Imaging_the_Amplitude_Calibrator_and_Measure_the_Value_of_Jy.2FK Imaging the Amplitude Calibrator and Measure the Value of Jy/K] below)<br />
# The science data are processed with - [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]] <br />
# and the (calibrated) science data are imaged with - [[media:M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]<br />
<br />
Executing each of these scripts in order, will be sufficient to calibrate and image the science data (although the calibrator processing script can be omitted, as the relevant parameters are actually already explicitly written into the script).<br />
<br />
They can be invoked from the command line using the execfile command.<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
execfile('M100.casa6p1.calib.py')<br />
</pre><br />
<br />
= Manual Calibration =<br />
<br />
By way of description, and as a means to introduce and demonstrate some of the interactive and diagnostic, tools here we describe in more detail the steps taken by the scripts - although in this case we demonstrate the process for only '''one''' of the ASDM datasets. In the ''Imaging'' section, we assume '''all''' the ASDMs are appropriately reduced either by the manual process, or by using the script. <br />
<br />
The entire process takes less than 30 minutes.<br />
<br />
There are three basic steps to the Calibration of the Science data:<br />
# Import, inspection and flagging<br />
# Calibration<br />
#* Sky and bandpass calibration (i.e. Tsys and frequency sampling function)<br />
#* Adjustment for Correlator non-linearity (this step is necessary only for data taken in ALMA Cycle 3 and earlier)<br />
#* Conversion to Jy/beam<br />
# Imaging<br />
<br />
=== Import "Analysis Utilities" ===<br />
<br />
The [[Analysis_Utilities]] package will be used most critically for the imaging processes. While it is not necessary for completing the calibration stages, it is used to check data quality, and should be loaded.<br />
<br />
Import the package from https://casaguides.nrao.edu/index.php/Analysis_Utilities, and apply the class therein.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
</source><br />
<br />
== Import, inspection and flagging ==<br />
<br />
=== Import ===<br />
The first thing to do is to convert the dataset into the CASA Measurement Set (MS) format. The raw data have been provided to you in the ASDM (ALMA Science Data Model), which is the native format of the data produced by the observatory but cannot be processed by CASA. The conversion from ASDM to MS is done with the task {{importasdm}}.<br />
<br />
<source lang="python" style="overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=True)<br />
</source><br />
<br />
The converted dataset (with a suffix ".ms") is created and then flags embedded in the ASDM dataset (so-called "BDF flags") are transferred to MS using the bdfflags parameter.<br />
<br />
'''Note:''' The above code should be used for Cycle 3 data or later. Cycle 2 data and earlier should use:<br />
<br />
<pre style="background-color: #E0FFFF; overflow-x:auto; white-space: pre"><br />
# In CASA<br />
importasdm('uid___A002_X85c183_X36f',<br />
asis='Antenna Station Receiver Source CalAtmosphere CalWVR', bdfflags=False)<br />
os.system(os.environ['CASAPATH'].split()[0] + '/bin/bdflags2MS -f "COR DELA INT MIS SIG SYN TFB WVR ZER" uid___A002_X85c183_X36f uid___A002_X85c183_X36f.ms')<br />
</pre><br />
<br />
<br />
=== Inspection - listobs ===<br />
After import, the usual first step is then to get some basic information about the data.<br />
We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied.<br />
<br />
<source lang="python" style="display:inline-block><br />
# In CASA<br />
listobs(vis='uid___A002_X85c183_X36f.ms',<br />
listfile='uid___A002_X85c183_X36f.ms.listobs')<br />
</source><br />
<br />
The output will be written in a file named uid___A002_X85c183_X36f.ms.listobs which you can (and should) view at your leisure.<br />
{{listobs}} displays information about the targets, dates, frequencies, etc. Below is an excerpt from listobs for uid___A002_X85c183_X36f:<br />
<br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
Fields: 2<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1215+1654 12:15:03.979140 +16.54.37.95680 J2000 0 14652<br />
1 none M100 12:22:54.360000 +15.48.50.60000 J2000 1 61509<br />
<br />
Spectral Windows: (25 unique spectral windows and 2 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_03#BB_1#SW-01#FULL_RES 124 TOPO 91955.512 -15625.000 1937500.0 90994.5750 1 XX YY<br />
2 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 90978.950 1734375.000 1734375.0 90978.9500 1 XX YY<br />
3 ALMA_RB_03#BB_2#SW-01#FULL_RES 124 TOPO 93893.012 -15625.000 1937500.0 92932.0750 2 XX YY<br />
4 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 92924.262 1937500.000 1937500.0 92924.2625 2 XX YY<br />
5 ALMA_RB_03#BB_3#SW-01#FULL_RES 124 TOPO 102033.637 15625.000 1937500.0 102994.5750 3 XX YY<br />
6 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 102986.762 1937500.000 1937500.0 102986.7625 3 XX YY<br />
7 ALMA_RB_03#BB_4#SW-01#FULL_RES 124 TOPO 104033.637 15625.000 1937500.0 104994.5750 4 XX YY<br />
8 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 104986.762 1937500.000 1937500.0 104986.7625 4 XX YY<br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
10 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100926.562 1781250.000 1781250.0 100926.5625 1 XX YY<br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
12 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102741.712 1781250.000 1781250.0 102741.7125 2 XX YY<br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
14 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112783.712 1781250.000 1781250.0 112783.7125 3 XX YY<br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
16 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114658.712 1781250.000 1781250.0 114658.7125 4 XX YY<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
18 ALMA_RB_03#BB_1#SW-01#CH_AVG 1 TOPO 100949.756 1992187.500 1992187.5 100949.7559 1 XX YY<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
20 ALMA_RB_03#BB_2#SW-01#CH_AVG 1 TOPO 102764.906 1992187.500 1992187.5 102764.9059 2 XX YY<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
22 ALMA_RB_03#BB_3#SW-01#CH_AVG 1 TOPO 112806.906 1992187.500 1992187.5 112806.9059 3 XX YY<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
24 ALMA_RB_03#BB_4#SW-01#CH_AVG 1 TOPO 114681.906 1992187.500 1992187.5 114681.9059 4 XX YY<br />
<br />
Sources: 48<br />
ID Name SpwId RestFreq(MHz) SysVel(km/s) <br />
0 J1215+1654 0 - - <br />
0 J1215+1654 25 - - <br />
0 J1215+1654 26 - - <br />
0 J1215+1654 27 - - <br />
0 J1215+1654 1 - - <br />
0 J1215+1654 2 - - <br />
0 J1215+1654 3 - - <br />
0 J1215+1654 4 - - <br />
0 J1215+1654 5 - - <br />
0 J1215+1654 6 - - <br />
0 J1215+1654 7 - - <br />
0 J1215+1654 8 - - <br />
0 J1215+1654 9 - - <br />
0 J1215+1654 10 - - <br />
0 J1215+1654 11 - - <br />
0 J1215+1654 12 - - <br />
0 J1215+1654 13 - - <br />
0 J1215+1654 14 - - <br />
0 J1215+1654 15 - - <br />
0 J1215+1654 16 - - <br />
0 J1215+1654 17 100950 0 <br />
0 J1215+1654 18 100950 0 <br />
0 J1215+1654 19 102794.1 0 <br />
0 J1215+1654 20 102794.1 0 <br />
0 J1215+1654 21 112794.1 0 <br />
0 J1215+1654 22 112794.1 0 <br />
0 J1215+1654 23 114669.1 0 <br />
0 J1215+1654 24 114669.1 0 <br />
1 M100 0 - - <br />
1 M100 25 - - <br />
1 M100 26 - - <br />
1 M100 27 - - <br />
1 M100 9 - - <br />
1 M100 10 - - <br />
1 M100 11 - - <br />
1 M100 12 - - <br />
1 M100 13 - - <br />
1 M100 14 - - <br />
1 M100 15 - - <br />
1 M100 16 - - <br />
1 M100 17 100950 0 <br />
1 M100 18 100950 0 <br />
1 M100 19 102794.1 0 <br />
1 M100 20 102794.1 0 <br />
1 M100 21 112794.1 0 <br />
1 M100 22 112794.1 0 <br />
1 M100 23 114669.1 0 <br />
1 M100 24 114669.1 0 <br />
<br />
Antennas: 3:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA61 A075 12.0 m -067.45.17.9 -22.53.21.4 -4.5609 -499.7012 23.0322 2225072.419944 -5440148.858968 -2481499.171703<br />
1 PM03 T701 12.0 m -067.45.18.8 -22.53.22.2 -29.1265 -522.7875 22.2052 2225045.995589 -5440149.141967 -2481520.118569<br />
2 PM04 T703 12.0 m -067.45.16.2 -22.53.23.9 42.8797 -575.6910 21.7763 2225104.700870 -5440102.471978 -2481568.689518<br />
</pre><br />
<br />
From this output you can for example see the followings.<br />
<br />
* From "Data records" section (not shown here): The execution consists of 15 scans with various scan intents.<br />
** Scans 1 and 2 are pointing and sideband gain ratio calibrations (done interferometrically), which need to be done prior to observing the target, on the quasar J1215+1654.<br />
** Scans 3 and 4 are interferometric delay and system noise temperature (Tsys) measurements also on J1215+1654 (they are in principle unnecessary; just a hack to make things happen on the telescope control software).<br />
** Scans 6, 7, 9, 10, 12, 13, and 15, whose scan intents contain "OBSERVE_TARGET", are for raster mapping of the target M100. The associated spectral window (SPW) ID's are 0 and 17-24.<br />
** Scans 5, 8, 11, and 14 with scan intents "CALIBRATE_ATMOSPHERE" are Tsys measurements for M100. The SPW ID's for Tsys scans are 0 and 9-16.<br />
* From "Spectral Windows" section:<br />
** SPW 0 contains the Water Vapor Radiometer (WVR) data, which are taken for all scans, but which are not used here.<br />
** SPW's 1, 3, 5, and 7 are placed on standard continuum frequencies and are used for the pointing scan only.<br />
** SPW's 9, 11, 13, and 15 cover the same frequency range as the science SPW's (see the next bullet) and are used to measure the Tsys used for calibration. They have 128 spectral channels (15.6 MHz spacing) in 2000 MHz bandwidth.<br />
** SPW's 17, 19, 21, and 23 cover the frequency ranges used for the actual science observations. They have 4080 spectral channels (488 kHz spacing) in 1992 MHz bandwidth. Hereafter these SPW's are referred to as "science" SPW's. The target line, CO J=1-0, is placed in SPW 23, the corresponding Tsys is taken from SPW 15.<br />
** The remaining, even-number SPW's contain channel-averages of the actually used SPW's (e.g., SPW 24 corresponds to SPW 23 averaged into one channel).<br />
* From "Antennas" section: Three 12-m antennas (DA61, PM03, and PM04) were used for the observations.<br />
<br />
=== Inspection - scanpattern ===<br />
The scan pattern of the raster mapping can be visualized by issuing the following command (after import of Analysis Utils - see the top of this page for information).<br />
A plot will be shown in a window and also saved as a PNG file uid___A002_X85c183_X36f.ms.sampling.png.<br />
<br />
<br />
[[File:X36f.TPSampling-6.1.png|200px|thumb|right|<caption>The scan pattern of the raster mapping for uid___A002_X85c183_X36f.</caption>]]<br />
<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms',<br />
showplot=True,<br />
plotfile='uid___A002_X85c183_X36f.ms.sampling.png')<br />
</source><br />
<br />
=== Inspection - System Noise Temperature ===<br />
Let's start by checking the Tsys.<br />
We use the task {{gencal}} to extract the Tsys into a CASA calibration table. This table is only used for inspection of Tsys quality, and not in further processing.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
gencal(vis='uid___A002_X85c183_X36f.ms',<br />
caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
caltype='tsys')<br />
</source><br />
<br />
The generated Tsys calibration table can be plotted using the task {{plotbandpass}}. The generated plots are saved in the directory<br />
uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime.<br />
<br />
<br />
[[File:X36f.Tsys.overlayTime-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for DA61 in uid___A002_X85c183_X36f. Colors represent time.</caption>]]<br />
<br />
<br />
<br />
[[File:X36f.Tsys.overlayAntenna-6.1.png|200px|thumb|right|<caption>The Tsys as a function of frequency for SPW 15 in uid___A002_X85c183_X36f, using overlay='antenna'. Colors represent antennas.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotbandpass(caltable='uid___A002_X85c183_X36f.ms.tsys',<br />
overlay='time',<br />
xaxis='freq',<br />
yaxis='amp',<br />
subplot=22,<br />
buildpdf=False,<br />
interactive=False,<br />
showatm=True,<br />
pwv='auto',<br />
chanrange='5~123',<br />
showfdm=True,<br />
field='',<br />
figfile='uid___A002_X85c183_X36f.ms.tsys.plots.overlayTime/uid___A002_X85c183_X36f.ms.tsys')<br />
</source><br />
<br />
=== Flagging ===<br />
<br />
Although the "science" SPW's have 1992 MHz bandwidths with 4080 spectral channels, their edge channels can be very noisy because each intermediate frequency signal path (baseband) is equipped with a bandpass filter of ~1.8 GHz width. Note that this is not necessarily always the case, it is in general, safest to flag out the edge channels, unless a spectral line of interest falls very close to the edge of the SPW band-edge.<br />
We flag 120 channels on each side of the "science" SPW's.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis = 'uid___A002_X85c183_X36f.ms',<br />
mode = 'manual',<br />
spw = '17:0~119;3960~4079,19:0~119;3960~4079,21:0~119;3960~4079,23:0~119;3960~4079',<br />
action = 'apply',<br />
flagbackup = True)<br />
</source><br />
<br />
==Calibration - sky, bandpass, nonlinearity and baseline ==<br />
<br />
=== Tsys Calibration ===<br />
We calibrate the data into Antenna temperature in units of K, using the equation Ta* = Tsys*(ON-OFF)/OFF, where ON and OFF are the data on-source (i.e., during the raster scanning) and off-source (on a reference position where only background emission exists), respectively.<br />
The calibration is done by the task {{sdcal}} which requires matching of list of the "science" and Tsys spectral windows. <br />
Tsys observations will always have 128 channels over 2GHz bandwidth, and as described in {{listobs}}, we already know which spectral windows match the science spectral windows most closely, in frequency.<br />
for example, spectral window 9, with 128 channels matches well with spectral window 17, 11 with 19, 13 with 21 and 15 with 2. Thus, we form the Tsys and Science spectral window mapping input parameters, with the syntax: {9:[17], 11:[19], 13:[21],15:[23]}<br />
<br />
<pre style="background-color: #fffacd; overflow-x:auto; white-space: pre"><br />
9 ALMA_RB_03#BB_1#SW-01#FULL_RES 128 TOPO 101942.187 -15625.000 2000000.0 100950.0000 1 XX YY<br />
.<br />
.<br />
17 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
11 ALMA_RB_03#BB_2#SW-01#FULL_RES 128 TOPO 103757.337 -15625.000 2000000.0 102765.1500 2 XX YY<br />
.<br />
.<br />
19 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
13 ALMA_RB_03#BB_3#SW-01#FULL_RES 128 TOPO 111814.962 15625.000 2000000.0 112807.1500 3 XX YY<br />
.<br />
.<br />
21 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
</pre><br />
<pre style="background-color: #fffacd;overflow-x:auto; white-space: pre"><br />
15 ALMA_RB_03#BB_4#SW-01#FULL_RES 128 TOPO 113689.962 15625.000 2000000.0 114682.1500 4 XX YY<br />
.<br />
.<br />
23 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
</pre><br />
<br />
And the Tsys calibration command is formed like the following:<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdcal(infile = 'uid___A002_X85c183_X36f.ms',<br />
calmode = 'ps,tsys,apply',<br />
spwmap = {9:[17], 11:[19], 13:[21],15:[23]},<br />
spw = '17,19,21,23')<br />
</source><br />
<br />
'''Note:''' in this step, the calibrations are applied in situ. That is: the calibrated data are written into the original data - however it is written in such a way that any ''additional'' calls to sdcal will not apply any further calibrations. We will examine the quality of the bandpass and Tsys calibration after the correction for non-linearity, next.<br />
<br />
<br />
<br />
[[File:Vis_Scan6_7_9_10_12_13_15-6.1.png|200px|thumb|right|<caption>Output of plotms, showing spectra in Ta* [K]</caption>]]<br />
<br />
<br />
The calibrated bandpasses, averaged over each scan, can be plotted with:<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='23',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis.png',<br />
showgui=True)<br />
</source><br />
<br />
'''Note:''' For ''amplitude calibrator datasets'', (as opposed to science target maps) the calmode option for sdcal should be 'otfraster,tsys,apply' ('otfraster' instead of 'ps').<br />
'otfraster' tells the task to use both ends of each raster row as "OFF" data (as opposed to a dedicated "OFF" sky measurement).<br />
This is because temporal fluctuation of atmospheric emission is the dominant source of noise in radio continuum observations, and hence the "OFF" needs to be as close as possible to the "ON" in time (and spatial location).<br />
<br />
=== Application of Non-Linearity Correction Factor ===<br />
<br />
In Cycle 1 and 2, single-dish data taken with the ACA correlator suffer from non-linearity which originates in the digital signal processing.<br />
Its impact was thoroughly studied both experimentally and theoretically, and it was concluded that multiplication by a correction factor of 1.25 yields an amplitude accuracy of +/-5%.<br />
<br />
To preserve the data history, we use here the tasks {{split}} to split out the science ''corrected'' data columns - note that from here on, the spectral windows are '''renumbered: [0,1,2,3]'''. We then generate a calibration table, using a defined function ''to_amp_factor'', with {{gencal}}. Finally, we apply the calibration with {{applycal}}.<br />
We apply the correction here using tasks gencal and applycal - this is different to the methods used in the past, and are available now because of the migration to MS-only format of Single-dish CASA.<br />
<br />
'''Note:''' The ACA online software has been equipped with this correction for data from cycle 3 and later (it should be applied for data from cycle 2 and earlier).<br />
<br />
Also note that we need to import the math module in order to use the sqrt command in Python 3.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
import math<br />
<br />
to_amp_factor = lambda x: 1. / math.sqrt(x)<br />
<br />
split(vis='uid___A002_X85c183_X36f.ms',<br />
outputvis='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='17,19,21,23')<br />
<br />
gencal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
caltable='uid___A002_X85c183_X36f.ms.nlc',<br />
caltype='amp', spw='',<br />
parameter=[to_amp_factor(1.25)])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.cal.split',<br />
gaintable='uid___A002_X85c183_X36f.ms.nlc')<br />
</source><br />
<br />
'''Note:''' the Python command ''lambda'' simply defines a function. Thus the line above : ''to_amp_factor = lambda x: 1. / math.sqrt(x)'' defines a function where the call: ''to_amp_factor(4)'' will return: ''0.5''<br />
<br />
=== Baseline Subtraction (only for Science Datasets) ===<br />
<br />
'''Note:''' this step is appropriate for spectral line observations (science datasets) but ''should not be done for the continuum observations (amplitude calibrator datasets) -- it would eliminate the continuum emission!''<br />
'''Note:''' this step is optional and need not be completed to produce an image - however it is recommended for the majority of cases (esp. where the bandpass is affected by low-order modulations, and where the line-emission does not completely dominate the bandpass.)<br />
<br />
We will subtract low-order spectral baselines using the task {{sdbaseline}}. Typically these manifest from structure in the distribution of water in the atmosphere: the ON and OFF positions are slightly different, and the difference manifests as - typically - a low frequency variation in the spectra. The situation is exacerbated for observations closer to water lines and also for higher frequencies.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
sdbaseline(infile='uid___A002_X85c183_X36f.ms.cal.split',<br />
datacolumn='corrected',<br />
spw='',<br />
blfunc='poly',<br />
order=1,<br />
outfile='uid___A002_X85c183_X36f.ms.bl',<br />
overwrite=True)<br />
</source><br />
<br />
<br />
<br />
[[File: vis_bl_Scan6_7_9_10_12_13_15-6.1.png |200px|thumb|right|<caption>Output of plotms, showing result of baseline step, units in Ta* [K]</caption>]]<br />
<br />
<br />
The spectra can be checked using {{plotms}}, which we have already used in a previous step.<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
plotms(vis='uid___A002_X85c183_X36f.ms.bl',<br />
xaxis='freq', yaxis='amp', ydatacolumn='corrected',<br />
field='M100',spw='3',<br />
scan='6,7,9,10,12,13,15',<br />
averagedata=True,avgtime='9999',<br />
gridcols=3,gridrows=3,iteraxis='scan',<br />
plotrange=[113.5, 116, -0.01, 0.08],<br />
plotfile='vis_bl.png',<br />
showgui=True)<br />
</source><br />
<br />
We do not concatenate separate Execution Blocks, because we need to determine and apply day-by-day Jy/K conversion factor.<br />
<br />
At this point in the guide, we have finished the calibration for ASDM uid___A002_X85c183_X36f (except for the calculation of the Jy/K conversion factors). In order to finish the calibration for the entire M100 dataset, the above steps should be repeated with each of the ADSMs shown in [https://casaguides.nrao.edu/index.php?title=M100_Band3_SingleDish_6.1#Summary_of_Datasets Summary of Datasets]. Once each ASDM has been calibrated, the user can move on to the flagging, Jy/K conversion, and imaging processes below.<br />
<br />
Note that one of the antennas, PM02, had a problem in the first baseband (SPW 17) on 2014-07-05, and its impact may not be specific to this one SPW (e.g., it may have resulted in poor pointing calibration).<br />
Therefore the PM02 data should be flagged from the affected datasets: uid___A002_X8602fa_X2ab (science), and uid___A002_X8602fa_X577 (science). The calibrator set uid___A002_X8602fa_Xc3 will also be affected, and will be flagged out in the subsequent processing of the calibrators.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
flagdata(vis='uid___A002_X8602fa_X2ab.ms.bl',<br />
antenna='PM02&&&')<br />
flagdata(vis='uid___A002_X8602fa_X577.ms.bl',<br />
antenna='PM02&&&')<br />
</source><br />
<br />
<br />
== Convert science data from Kelvin to Jansky ==<br />
<br />
At this stage the data have been calibrated into units of antenna temperature, in Kelvins.<br />
<br />
It now remains to convert the science data into Jy/beam units - the ''Jy per K'' factors. This is important for combination of the data, with 7m and 12m sets. Later in this guide we show how to determine the correct factors from the calibrator datasets, for conversion (the process is essentially identical to processing the science datasets, with the exception that the baseline step is omitted from the calibration dataset processing). However to determine the factors, the calibration data must also be imaged and then interrogated to determine the appropriate values. <br />
<br />
Applying the calibration factors to the science data is the last step in the script [[media:M100.casa6p1.calib.py|M100.casa6p1.calib.py]]. <br />
<br />
We continue to use only one ASDM as an example; uid___A002_X85c183_X36f. The script should be consulted for treatment of all the M100 Single-dish ASDMs (and the correct application of the Jy per K factors). Again, we apply the factors by making use of the {{gencal}} and {{applycal}} tasks. Also as before, we make use of the python function defined earlier: ''to_amp_factor''<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
<br />
factors = [41.52, 42.47, 43.58, 42.98]<br />
gencal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
caltable='uid___A002_X85c183_X36f.ms.jy',<br />
caltype='amp',<br />
spw='0,1,2,3',<br />
parameter=[ to_amp_factor(v) for v in factors ])<br />
<br />
applycal(vis='uid___A002_X85c183_X36f.ms.bl',<br />
gaintable='uid___A002_X85c183_X36f.ms.jy')<br />
</source><br />
<br />
= Image the Science Target =<br />
<br />
The final stage is to image the science data. This step is done by the script "[[media: M100.casa6p1.imaging.py|M100.casa6p1.imaging.py]]" which is invoked with ''execfile('M100.casa6p1.imaging.py') '' in CASA.<br />
This step uses a number of helper utilities to automatically determine the properties of the image data - the analysis Utilities library must be loaded (see the text above).<br />
<br />
== Determining grid parameters, gridding and obtaining grid metadata ==<br />
Define the list of the calibrated science datasets, which the CASA task {{sdimaging}} accepts. Note that this step is the same for science and calibrator data.<br />
This section is a little more python-heavy than the calibration step. CASA 6.1.0 has benefited from significant effort in streamlining the calibration steps, and the imaging steps are to be more automated in coming cycles.<br />
<br />
In this section, we will simply process all the calibrated science data. The calibrator data should never be combined. There is no need to ''concatenate'' the calibrated data, as the imaging task can explicitly accept multiple datasets as input.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
asdmlist = ['uid___A002_X85c183_X36f',<br />
'uid___A002_X85c183_X60b',<br />
'uid___A002_X8602fa_X2ab',<br />
'uid___A002_X8602fa_X577',<br />
'uid___A002_X864236_X2d4',<br />
'uid___A002_X864236_X693',<br />
'uid___A002_X86fcfa_X664',<br />
'uid___A002_X86fcfa_X96c',<br />
'uid___A002_X86fcfa_Xd9']<br />
vislist = map(lambda x: x + '.ms.bl', asdmlist)<br />
</source><br />
<br />
<br />
We first have to define and obtain some basic parameters. As all the ASDMs are ostensibly of the same kinds of observations, we refer to the first science ASDM to determine the basic properties of the resulting data cube. The cube size, pixel size, theoretical beam size, central frequency and other basic parameters are extracted using some CASA-functions and the Analysis Utils library (see the top of this page for information on Analysis Utils).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
spw = 3<br />
imagename = 'M100_TP_CO_cube.spw%s.image'%(spw)<br />
blimagename = imagename + '.bl'<br />
integimagename = imagename + '.integ'<br />
fwhmfactor = 1.13<br />
diameter = 12<br />
<br />
xSampling, ySampling, maxsize = aU.getTPSampling(vis='uid___A002_X85c183_X36f.ms', showplot=False)<br />
<br />
msmd.open(asdmlist[0]+'.ms.bl')<br />
freq = msmd.meanfreq(spw)<br />
msmd.close()<br />
print("SPW %d: %.3f GHz" % (spw, freq*1e-9))<br />
<br />
theorybeam = aU.primaryBeamArcsec(frequency=freq*1e-9,<br />
fwhmfactor=fwhmfactor,<br />
diameter=diameter)<br />
<br />
cell = theorybeam/9.0<br />
imsize = int(round(maxsize/cell)*2)<br />
</source><br />
And now we learn the central frequency of spectral window 3, is 114.682 GHz.<br />
Our cube will have a total length of 253.0 arcsec ('''maxsize'''), and with a cell side of 5.64 arcsec ('''cell''') -which samples the theoretical beam (''theory beam'') with 9x9 pixels, we will need 90 pixels along the image side ('''imsize''').<br />
<br />
'''Note:''' The Jy/K value depends on the parameters of gridding convolution (i.e., "gridfunction", "cell", and function-specific parameters ["convsupport" for gridfunction='SF']). That is, the same gridding parameters should be used for both amplitude calibrator and science images -- otherwise the calibration into Jy unit becomes invalid. Please refer to the CASA cookbook for information on the gridding kernels.<br />
<br />
<br />
Now the data are imaged.<br />
The velocity channel maps of the CO J=1-0 line (restfreq='115.271201800GHz') are created as a data cube which covers the velocity range of 1400-1700 km/s at a spacing of 5 km/s (start='1400km/s', width='5km/s', nchan=70).<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA <br />
sdimaging(infiles=vislist,<br />
field='M100',<br />
spw='%s'%(spw),<br />
nchan=70,<br />
mode='velocity',<br />
start='1400km/s',<br />
width='5km/s',<br />
veltype='radio',<br />
outframe='lsrk',<br />
restfreq='115.271204GHz',<br />
gridfunction='SF',<br />
convsupport=6,<br />
stokes='I',<br />
phasecenter='J2000 12h22m54.9 +15d49m15',<br />
ephemsrcname='',<br />
imsize=imsize,<br />
cell='%sarcsec'%(cell),<br />
brightnessunit='Jy/beam',<br />
overwrite=True,<br />
outfile=imagename)<br />
</source><br />
<br />
Now you can browse the data cube using the viewer.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
viewer(imagename)<br />
</source><br />
<br />
== Subtract a Residual Background from the Image ==<br />
<br />
If you plot the line profile using the viewer, you may notice that the background (i.e., line-free channels) level is slightly negative.<br />
To correct this, spectral baselines are subtracted from the image using the task {{imcontsub}}.<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
imcontsub(imagename=imagename,<br />
linefile=blimagename,<br />
contfile='M100.residualbaseline.image',<br />
fitorder=1,<br />
chans='0~7;62~69')<br />
os.system('rm -rf M100.residualbaseline.image')<br />
</source><br />
<br />
This task has two outputs - the baselined data is returned in the same input file (''image name - M100_TP_CO_cube.spw3.image.bl''), while the subtracted component is returned in ''M100.residual baseline.image"". We have no further need for the subtracted component, and it can be discarded.<br />
<br />
== Basic Analysis - Moments ==<br />
<br />
Make a 0th moment (integrated intensity) map using the task {{immoments}} and browse it using the viewer.<br />
<br />
<br />
[[File:M100.CO.cube.bl.image.mom0-6.1.png|200px|thumb|right|<caption>The integrated intensity map of M100.</caption>]]<br />
<br />
<br />
<source lang="python" style="display:inline-block"><br />
# In CASA<br />
immoments(imagename=blimagename,<br />
moments=[0],<br />
axis='spectral',<br />
chans='8~61',<br />
outfile=integimagename)<br />
<br />
viewer(integimagename)<br />
</source><br />
<br />
= Imaging the Amplitude Calibrator and Measure the Value of Jy/K =<br />
<br />
Determination of the Jy/K conversion factor is achieved by imaging a source whose continuum flux is known and measuring the observed brightness temperature. One observation of this source - the amplitude calibrator - was made per day, giving a total of four datasets (see top).<br />
<br />
The true size of the beam (point spread function), on which the Jy/K value depends, in a map is determined by several ingredients.<br />
* The intrinsic beam size is determined by the antenna diameter and optics (and of course wavelength). Here we assume the intrinsic beam size of 1.13*lambda/D, where D=12 [m] is the diameter of the antennas, and the factor 1.13 is derived from the optics design of the ALMA 12-m antennas.<br />
* The effective beam size is broadened by the following causes:<br />
** The scanning pattern used to observe the map (i.e., sample spacing in both directions).<br />
** The method used to grid the individual spectra into a map/cube.<br />
The Analysis Utilities provides several tools to get estimates of these.<br />
<br />
It is not necessary, but If you wish to try this out for yourself, you will first need to download the raw amplitude calibrator data file M100_Band3_TP_ampcal_UncalibratedData.tgz. How to obtain the data is described at [[M100_Band3#Obtaining the Data]]. Once you have unpacked the data you will have a directory called M100_Band3_TP_ampcal_UncalibratedData. In this directory, execute the scripts for individual datasets ([[media:A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py|A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py|A002_X8602fa_Xc3.ms.scriptForSDampcalReduction.casa6p1.py]], [[media:A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py|A002_X864236_Xe1.ms.scriptForSDampcalReduction.casa6p1.py]], and [[media:A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py|A002_X86fcfa_X3ae.ms.scriptForSDampcalReduction.casa6p1.py]]) to calibrate, image and derive Jy/K factors of all four calibrator datasets. The general approach is almost identical to that of calibrating the science target data, with the exception of the baselining step. Then, Jy/K values are derived from the peak of temperature in images and stored in text files (with the suffix, '.jyperk.txt').<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
# In CASA<br />
import analysisUtils as aU<br />
es = aU.stuffForScienceDataReduction()<br />
execfile('A002_X85c183_X895.ms.scriptForSDampcalReduction.casa6p1.py')<br />
</pre><br />
<br />
The Calibration Jy/K values are determined as follows. Note that the values for individual antennas are averaged for each day, each SPW, because the antennas used for the Science observations were not necessarily available for the corresponding amplitude calibrator observations.<br />
<br />
<pre style="background-color: #E0FFFF; display:inline-block"><br />
#Date Amp_Cal_Dataset SPW17 SPW19 SPW21 SPW23<br />
2014-07-01 uid___A002_X85c183_X895 41.52 42.47 43.58 42.98<br />
2014-07-05 uid___A002_X8602fa_Xc3 41.38 42.59 42.08 42.53<br />
2014-07-07 uid___A002_X864236_Xe1 40.88 42.48 41.47 41.94<br />
2014-07-17 uid___A002_X86fcfa_X3ae 41.71 42.88 41.69 42.18<br />
</pre><br />
<br />
= Combination with 12m and 7m Array Data =<br />
<br />
If you wish to learn how to combine the Total Power data with interferometric 12m and 7m data, see the guide for Data Combination [[M100_Band3_Combine_6.1]]<br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_6.1&diff=31230AntennaeBand7 Imaging 6.12021-10-13T17:22:39Z<p>Jbertone: </p>
<hr />
<div>*'''This guide is written for CASA 6.1 and uses Python 3. The same functionality is implemented in Python 2 using CASA 5.7 at [[AntennaeBand7 Imaging 5.7]]'''<br />
* '''This guide is designed for CASA 6.1. If you are using an older version of CASA please see [[AntennaeBand7]] for earlier versions of this CASAguide.<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration 6.1]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 6.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casalith<br />
version = casalith.version_string()<br />
print ("You are using {}".format(version))<br />
if (version < '6.1'):<br />
print ("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print ("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print ("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review, for example, https://science.nrao.edu/science/meetings/2016/15th-synthesis-imaging-workshop/documents/wilner_vla16.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{tclean}}. To invoke mosaic mode, you simply set the parameter '''gridder='mosaic''''. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.pb''.<br />
<br />
<!-- <br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''.<br />
--><br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{tclean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review, for example, https://science.nrao.edu/science/meetings/2016/15th-synthesis-imaging-workshop/documents/bsmMosaicking2016.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH_6.1.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH_6.1.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Figure 1), and from 70 to 100 in the northern mosaic (Figure 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_North-AMPvsCH.png', showgui = True)<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_South-AMPvsCH.png', showgui = True)<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{tclean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{tclean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.pb # the primary beam response<br />
** <imagename>.image.pbcor # the primary beam-corrected image (if '''pbcor=True''')<br />
** <imagename>.weight # the primary beam coverage ('''gridder=’mosaic’ or 'awproject' ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''specmode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.online/ splatalogue].<br />
*'''niter=0''': Maximum number of tclean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
In CASA 5.4 and later, {{tclean}} calls with '''gridder''' = ''''mosaic'''' have an additional parameter '''mosweight''' with a default of '''True'''. When '''mosweight''' = '''True''', the gridder weights each field in the mosaic independently. The '''mosweight''' parameter is particularly important for mosaics with non-uniform sensitivity, with rectangular shapes, or when using more uniform values of robust Briggs weighting. For more information on '''mosweight''', please see the {{tclean}} documentation. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
tclean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter=12,<br />
specmode='mfs',<br />
deconvolver='hogbom',<br />
restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
gridder='mosaic', mosweight=True,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=False,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.23" x 0.72", with a position angle (P.A.) of 85.4 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
<br />
[[Image:North_cont_clean_6.1.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes, there is definitely a detection in the vicinity of the Northern nucleus (see Figure 5 and Figure 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.5 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). We have included an example mask to use in Antennae_Band7_CalibratedData.tgz. If you would like to use your own mask, make sure to removed the mask file before running the command below. Otherwise, you can edit the mask within the clean GUI. See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=True''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_North.Cont.Clean'+ext)<br />
<br />
tclean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter=12,<br />
deconvolver='hogbom',<br />
specmode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
gridder='mosaic', mosweight=True,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only ~30 iterations (see Figure 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{tclean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{tclean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{tclean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean_6.1.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. Again, you can use the provided mask or make your own.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_South.Cont.Clean'+ext)<br />
<br />
tclean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter=15,<br />
specmode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
gridder='mosaic', mosweight=True, deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.13"x 0.67", and P.A.= 61 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after ~50 iterations (Figure 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.86 mJy/beam and the rms is 0.49 mJy/beam. For the Southern continuum we find a peak of 4.70 mJy/beam and an rms of 0.47 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Figure 5), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image_6.1.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image_6.1.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':True,'range':[-0.00049,0.00386]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview(raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':True,'range':[-0.00047,0.00470]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
In CASA 5.4, you will see a warning that uvcontsub is using the original Visibility Iterator, which is the underlying iteration and retrieval mechanism used when processing visibility data. <br />
A new version of the Visibility Iterator has been developed and deployed in several other calibration tasks in CASA 5.0 and later (see [https://science.nrao.edu/enews/casa_005/ CASA eNews] for more information). <br />
We can ignore this warning. <br />
<br />
<pre style="background-color: #fffacd;"><br />
WARN calibrater Forcing use of OLD VisibilityIterator.<br />
</pre><br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[Image:North_CO3_2_vel_6.1.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel_6.1.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''specmode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. <br />
When making spectral images you have three choices for the '''specmode''' parameter: '''mfs''', '''cube''', and '''cubedata'''. <br />
As noted above, setting '''specmode='mfs'''' gives an output image with only one channel, and is used for continuum imaging. The other '''specmode''' options create output data cubes with one or more channels.<br />
<br />
Data are taken using constant frequency channels. <br />
For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. <br />
For '''specmode='cube'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. <br />
The Doppler Shift is taken out during the regridding to the desired outframe in {{tclean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{tclean}}. <br />
<br />
To see what velocity parameters you want to set in {{tclean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png', showgui = True) <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we provide a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel. You can choose to use and edit the provided mask or create your own mask.<br />
<br />
Notable parameters included in the {{tclean}} call are:<br />
*'''specmode='cube' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'cube' mode to create an output data cube. We set the velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=20000''': Maximum number of clean iterations. With complex emission structure like that found in the Antennae CO data, we will need a large number of iterations to clean the emission. Note that you can increase the number of iterations allowed on-the-fly in the interactive {{tclean}} window. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
*'''savemodel='modelcolumn'''': This saves the clean model to the ms file. We will need this for self-calibration later. <br />
<br />
[[Image:North_interact_6.1.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_North.CO3_2Line.Clean'+ext)<br />
<br />
tclean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter=12,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic', mosweight=True,<br />
deconvolver='hogbom',<br />
imsize=500,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Cycle through the channels until you reach ~1650 km/s, where you can more easily see where to draw the clean box around the CO emission. Be sure the check the circle next to 'All Channels' when making your clean mask. Now clean using the green circle arrow. You can watch the progress in the logger. When the first ~650 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about ~7000 iterations (Red X), when the artifacts start to look as bright as the residual. Note again that we are not cleaning to the theoretical noise level. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_South.CO3_2Line.Clean'+ext)<br />
<br />
tclean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter=15,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
Clean about 8,000 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{tclean}} is run, it saves a uv-model of the resulting image in the MODEL column of the measurement set if ''savemodel'' = ''modelcolumn''. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan_6.1.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='data',plotfile='North_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
You will notice that while the model looks reasonable across the channels where we imaged the data, the model amplitudes are set to 1 elsewhere. When {{tclean}} initializes a model, it defaults to an amplitude of 1 and phase 0 (i.e., a point source with an amplitude of 1 in the center of the image). When we set '''savemodel = 'modelcolumn'''' in the {{tclean}} call, the default model was replaced ''only where {{tclean}} had a model to replace it with'', namely, over the channels we imaged. In future, CASA will set a flag to let users know that these model points are not valid for the updated model. In our case, we will perform self-calibration only on those channels with significant signal in the line emission, omitting the rest of the channels. <br />
<br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='model',plotfile='North_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
<br />
[[Image:north_pcal1_phase_6.1.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions. Note that when '''iteraxis''' is set, {{plotms}} appends the axes being iterated over to the output filename specified in '''plotfile'''; this can make for rather unwieldy filenames but is useful for ALMA pipeline purposes. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteraxis='antenna',gridrows=5,gridcols=3,plotrange=[0,0,-80,80],<br />
plotfile='north_pcal1_phase.png', showgui = True)<br />
<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{tclean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
tclean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter=12,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',mosweight=True,deconvolver='hogbom',<br />
imsize=500,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
After 3000 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 9,000 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}])<br />
<br />
</source><br />
<br />
<!--<br />
zoom={'channel': 43}) doesn't work with two images<br />
--><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, cycle to channel 43. Then you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you can use the tapedeck to move to the new channel of interest and see the changes in both cubes.<br />
<br />
To compare the statistics for each image, click the "Next" button in the "Statistics" tab of the "Region" sub-panel. <br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 25% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:south_selfchan_6.1.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=True,coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
<br />
[[Image:south_pcal1_phase_6.1.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteraxis='antenna',gridrows=5,gridcols=3,plotrange=[0,0,-80,80],<br />
plotfile='south_pcal1_phase.png', showgui = True)<br />
<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "All Channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
tclean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter=15,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',mosweight=True,deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=40000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
After 4000 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Stop after 25,000 or so iterations.<br />
<br />
As for the Southern mosaic, you can compare the before and after self-cal images (look, for example, at channel 31):<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}])<br />
</source><br />
<br />
<!--<br />
zoom={'channel': 31})<br />
doesn't work for multiple images<br />
--><br />
<br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 24% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0_6.1.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord_6.1.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord_6.1.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0_6.1.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord_6.1.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord_6.1.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':True,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':True,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=True''' and/or '''dropstokes=True'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
<div style="background-color: #CCFF99;"><br />
<p>Last checked on CASA Version 6.1.0.<br />
</p><br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Imaging_for_CASA_6.1&diff=312283C286 Band6Pol Imaging for CASA 6.12021-10-13T13:10:05Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Imaging]][[Category:Polarization]]<br />
<br />
==Overview==<br />
'''This guide is designed for CASA 6.1.'''<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the imaging of the <br />
total power and polarized emission. Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
This guide picks up where the '''[[3C286_Band6Pol_Calibration_for_CASA_5.4]]''' section left off: right after running the CASA task {{split}} to split off the science target from the rest of the measurement set following calibration. If you completed the Calibration section of the guide, then you can continue where you left off, with 3c286_Band6.pol.cal.ms. <br />
<br />
If you did not complete the Calibration portion of the guide, then you can instead download the file 3C286_Band6_CalibratedData.tgz (the calibrated uv data) from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Once the download has finished, unpack the file:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_CalibratedData.tgz<br />
<br />
cd 3C286_Band6_CalibratedData<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
After that, you should have 3c286_Band6.pol.cal.ms in your working directory.<br />
<br />
==Full polarization imaging of the target==<br />
<br />
We will start by making a continuum full Stokes image of 3C286 using {{tclean}}. <br />
We first define the dataset we will use in all the following steps:<br />
<br />
<source lang="python"><br />
# In CASA<br />
vis='3c286_Band6.pol.cal.ms'<br />
</source><br />
<br />
Before starting the cleaning, it makes sense to check if the emission is resolved or not by plotting amplitude as a function of uv-distance.<br />
The higher amplitudes at shorter uv-distances suggest that the source is slightly resolved.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis, spw='', xaxis='uvwave', yaxis='amp', correlation='XX,YY',<br />
avgtime='1e8', avgchannel='1000', coloraxis='baseline',showgui=True)<br />
</source><br />
<br />
[[File:Target_uvplot_5.4.png|200px|thumb|right|Amplitude vs. uv-distance. The source seems to be slightly resolved.]]<br />
<br />
This plot also helps us defining the cellsize needed for the task clean.<br />
From the plot we see that the longest baseline is 500 kilo wavelength. This means that the maximum resolution can be<br />
approximately 1/500000*206265 arcsec = 0.4 arcsec.<br />
We will use a '''cellsize=0.1''', to oversample the beam sufficiently. The FWHM of the primary beam of ALMA in Band 6 is about 25 arcsec, and we want to image out to at least that extent, so we will use '''imsize=250'''.<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf 3c286.cal.StokesIQUV_noselfcal.clean*')<br />
tclean(vis,<br />
imagename='3c286.cal.StokesIQUV_noselfcal.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
deconvolver='clarkstokes',<br />
stokes='IQUV',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
niter=500,<br />
pbcor=True,<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
Specifying '''interactive=True''' , the {{tclean}} task brings up a viewer to show the residual clean image, clean masks can be defined, and when the residuals are noise-like, the cleaning process can be interrupted, hiting the red X symbol. <br />
Please note that the cleaning will not start without any active mask (to activate a mask you need to double click inside the region). <br />
See [http://casaguides.nrao.edu/index.php?title=TWHydraBand7_Imaging_3.4 TW Hydra casaguide] for further details on the interactive use of clean.<br />
<br />
Finally, note that we delete any previous versions of the output images before proceeding with the clean command. This is important, because if images with the supplied root name already exist, CASA will clean those further instead of producing new output images.<br />
<br />
After around 500 iterations, using an elliptical clean box around the central source, the residuals are already noise-like, so you can stop the cleaning. <br />
You can use the viewer to look at your image. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('3c286.cal.StokesIQUV_noselfcal.clean.image',gui=True)<br />
</source><br />
<br />
[[File:3C286_Iimage_5.4.png|200px|thumb|right|Total intensity image of 3C286]]<br />
<br />
You will notice a warning at the end of the clean command. The model column may not have been saved. Use the following command to make sure the model column is saved. <br />
<source lang="python"><br />
# In CASA<br />
<br />
tclean(vis,<br />
imagename='3c286.cal.StokesIQUV_noselfcal.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
deconvolver='clarkstokes',<br />
stokes='IQUV',<br />
interactive=False, <br />
niter=0,<br />
weighting='briggs',<br />
robust=0.5,<br />
pbcor=True,<br />
calcres=False,<br />
calcpsf=False,<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
We will determine some statistics for the image using the task {{imstat}} (some details are given below):<br />
<source lang="python"><br />
# In CASA<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_noselfcal.clean.image', stokes='I', region='', box='10,10,90,35')<br />
rms=(calstat['rms'][0])<br />
print '>> rms in continuum image: '+str(rms)<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_noselfcal.clean.image', region='')<br />
peak=(calstat['max'][0])<br />
print '>> Peak in continuum image: '+str(peak)<br />
print '>> Dynamic range in continuum image: '+str(peak/rms)<br />
</source><br />
<br />
This tells us that the rms of this image is 1.1 mJy and the peak flux density is ~324 mJy. The dynamic range (ratio between peak and rms) is approximately 295.<br />
<br />
For future reference, we create a png file of the continuum image:<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286.cal.StokesIQUV_noselfcal.clean.png')<br />
imview(raster={'file': '3c286.cal.StokesIQUV_noselfcal.clean.image', 'colorwedge':True,<br />
'range':[-0.006, 0.4], 'scaling':-0.6, 'colormap':'Rainbow 2'},<br />
out='3c286.cal.StokesIQUV_noselfcal.clean.png')<br />
</source><br />
<br />
The mask you drew for the clean has been saved in your working directory as a file "3c286.cal.StokesIQUV_noselfcal.clean.mask". If you want to use it in the following self-calibration you should rename it. If you downloaded the calibrated data you have already a file mask, named 3c286_StokesIQUV_selfcal.mask.<br />
<br />
To rename the file:<br />
<source lang="python"><br />
# In CASA<br />
os.system('mv 3c286.cal.StokesIQUV_noselfcal.clean.mask 3c286_StokesIQUV_selfcal.mask')<br />
</source><br />
<br />
<br />
=== Self-calibration ===<br />
<br />
The quality of the image can be improved by applying the self-calibration to the uv data, <br />
see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.3 Antennae] and [http://casaguides.nrao.edu/index.php?title=TWHydraBand7_Imaging_3.4 TwHydra] guides for details on this technique.<br />
<br />
The task {{tclean}} has automatically saved a model in the header of the measurement set, we use it to self-calibrate <br />
through the task {{gaincal}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_IQUV.G1p')<br />
gaincal(vis,<br />
caltable='3c286_IQUV.G1p',<br />
solint='300s',<br />
refantmode='strict',<br />
refant='DV23',minblperant=4,<br />
calmode='p',minsnr=4)<br />
<br />
</source><br />
<br />
The critical parameters in this setup are '''solint''' and '''calmode'''.<br />
We will force the reference antenna to remain constant with the reantmode='strict'.<br />
We want to calculate phase solutions only so we use '''calmode=p''' and we start using '''solint=300s''', which means averaging 30 visibilities in the solution calculation.<br />
It is usually best to start with a large solint and then go down to the integration time if the S/N of the data permits. <br />
<br />
To check the quality of the solutions we obtained, we plot the gains for each antenna, as function of time:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_IQUV.G1p',xaxis='time',yaxis='phase',iteraxis='antenna',coloraxis='spw',showgui=True)<br />
</source><br />
<br />
<br />
Now we apply those solutions to the uv data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis,<br />
field='',<br />
gaintable=['3c286_IQUV.G1p'],calwt=False)<br />
<br />
</source><br />
<br />
and make a new image:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf 3c286.cal.StokesIQUV_selfcal1.clean*')<br />
tclean(vis,<br />
imagename='3c286.cal.StokesIQUV_selfcal1.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
deconvolver='clarkstokes',<br />
stokes='IQUV',<br />
mask= '3c286_StokesIQUV_selfcal.mask',<br />
weighting='briggs',<br />
robust=0.5,<br />
interactive=True,<br />
pbcor=True,<br />
niter=1000,<br />
savemodel='modelcolumn')<br />
<br />
# Save the model column<br />
tclean(vis,<br />
imagename='3c286.cal.StokesIQUV_selfcal1.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
deconvolver='clarkstokes',<br />
stokes='IQUV',<br />
interactive=False,<br />
weighting='briggs',<br />
robust=0.5,<br />
pbcor=True,<br />
niter=0,<br />
calcres=False,<br />
calcpsf=False,<br />
savemodel='modelcolumn')<br />
<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_selfcal1.clean.image', stokes='I', region='', box='10,10,90,35')<br />
rms=(calstat['rms'][0])<br />
print '>> rms in continuum image: '+str(rms)<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_selfcal1.clean.image', region='')<br />
peak=(calstat['max'][0])<br />
print '>> Peak in continuum image: '+str(peak)<br />
print '>> Dynamic range in continuum image: '+str(peak/rms)<br />
<br />
</source><br />
<br />
The rms of this new image is 0.31 mJy/beam, lower than the value measured in the first image (0.9 mJy/beam).<br />
We run a second self-calibration, decreasing to 30 sec the solint, apply it to the data, and image it again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_IQUV.G2p')<br />
gaincal(vis,<br />
caltable='3c286_IQUV.G2p',<br />
solint='30s',refantmode='strict',<br />
refant='DV23',minblperant=4,<br />
calmode='p',minsnr=4)<br />
<br />
plotms(vis='3c286_IQUV.G2p',xaxis='time',yaxis='phase',iteraxis='antenna',coloraxis='spw',showgui=True)<br />
<br />
applycal(vis,<br />
field='',<br />
gaintable=['3c286_IQUV.G2p'],calwt=False)<br />
<br />
os.system('rm -rf 3c286.cal.StokesIQUV_selfcal2.clean*')<br />
tclean(vis,<br />
imagename='3c286.cal.StokesIQUV_selfcal2.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
deconvolver='clarkstokes',<br />
stokes='IQUV',<br />
mask= '3c286_StokesIQUV_selfcal.mask',<br />
interactive=True,<br />
weighting='briggs',<br />
robust=0.5,<br />
pbcor=True,<br />
niter=1000,<br />
savemodel='modelcolumn')<br />
<br />
tclean(vis,<br />
imagename='3c286.cal.StokesIQUV_selfcal2.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
deconvolver='clarkstokes',<br />
stokes='IQUV',<br />
interactive=False,<br />
weighting='briggs',<br />
robust=0.5,<br />
pbcor=True,<br />
niter=0,<br />
calcres=False,<br />
calcpsf=False,<br />
savemodel='modelcolumn')<br />
<br />
</source><br />
<br />
You can notice when the first residual image appears that some residual central emission is evident. Continue the cleaning, by using the green continue button, until the residual image appears as much as possible noise like. <br />
In this example we stop the clean after 2 cycles, and we check the image statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_selfcal2.clean.image', stokes='I', region='', box='10,10,90,35')<br />
rms=(calstat['rms'][0])<br />
print '>> rms in continuum image: '+str(rms)<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_selfcal2.clean.image', region='')<br />
peak=(calstat['max'][0])<br />
print '>> Peak in continuum image: '+str(peak)<br />
print '>> Dynamic range in continuum image: '+str(peak/rms)<br />
</source><br />
<br />
<br />
The rms level in this new image is ~0.2 mJy/beam. This level can be improved by a last run of selfcalibration, this time <br />
solving for both amplitude and phase.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_IQUV.Gap')<br />
gaincal(vis,<br />
caltable='3c286_IQUV.Gap',<br />
solint='30s',gaintable=['3c286_IQUV.G2p'],<br />
refant='DV23',minblperant=4,refantmode='strict',<br />
calmode='ap',minsnr=4)<br />
<br />
plotms(vis='3c286_IQUV.Gap',xaxis='time',yaxis='amp',iteraxis='antenna',coloraxis='spw',showgui=True)<br />
<br />
applycal(vis,<br />
field='',<br />
gaintable=['3c286_IQUV.G2p','3c286_IQUV.Gap'],calwt=False)<br />
<br />
os.system('rm -rf 3c286.cal.StokesIQUV_selfcal_ap.clean*')<br />
tclean(vis,<br />
imagename='3c286.cal.StokesIQUV_selfcal_ap.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
deconvolver='clarkstokes',<br />
stokes='IQUV',<br />
mask= '3c286_StokesIQUV_selfcal.mask',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
pbcor=True,<br />
niter=1000,<br />
savemodel='modelcolumn')<br />
<br />
</source><br />
<br />
This time the first residual map shows a new bright extended component which was completely hidden by the noise in the previous images. We add a clean box around it and proceed with the clean. After 1000 cycles we stop and check the statistics of the image:<br />
<br />
<source lang="python"><br />
# In CASA<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_selfcal_ap.clean.image', stokes='I', region='', box='10,10,90,35')<br />
rms=(calstat['rms'][0])<br />
print '>> rms in continuum image: '+str(rms)<br />
calstat=imstat(imagename='3c286.cal.StokesIQUV_selfcal_ap.clean.image', region='')<br />
peak=(calstat['max'][0])<br />
print '>> Peak in continuum image: '+str(peak)<br />
print '>> Dynamic range in continuum image: '+str(peak/rms)<br />
</source><br />
<br />
The rms is 0.11 mJy and the peak of the image is 359 mJy, leading to a dynamic range of 3291.<br />
<br />
We can consider this result satisfying and stop the self-calibration.<br />
<br />
=== Stokes images ===<br />
<br />
We can extract the Stokes Q, U, and V images from the selfcalibrated image, using the taks immath:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286.cal.StokesI.clean.image*')<br />
immath(imagename='3c286.cal.StokesIQUV_selfcal_ap.clean.image',outfile='3c286.cal.StokesI.clean.image',expr='IM0',stokes='I')<br />
os.system('rm -rf 3c286.cal.StokesQ.clean.image*')<br />
immath(imagename='3c286.cal.StokesIQUV_selfcal_ap.clean.image',outfile='3c286.cal.StokesQ.clean.image',expr='IM0',stokes='Q')<br />
os.system('rm -rf 3c286.cal.StokesU.clean.image*')<br />
immath(imagename='3c286.cal.StokesIQUV_selfcal_ap.clean.image',outfile='3c286.cal.StokesU.clean.image',expr='IM0',stokes='U')<br />
os.system('rm -rf 3c286.cal.StokesV.clean.image*')<br />
immath(imagename='3c286.cal.StokesIQUV_selfcal_ap.clean.image',outfile='3c286.cal.StokesV.clean.image',expr='IM0',stokes='V')<br />
<br />
</source><br />
<br />
We measure the statistic in each emage, using the task imstat:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
I_rms=(imstat(imagename='3c286.cal.StokesI.clean.image', region='', box='10,10,90,35')['rms'][0])<br />
I_peak=(imstat(imagename='3c286.cal.StokesI.clean.image', region='')['max'][0])<br />
Q_rms=(imstat(imagename='3c286.cal.StokesQ.clean.image', region='', box='10,10,90,35')['rms'][0])<br />
Q_peak=(imstat(imagename='3c286.cal.StokesQ.clean.image', region='')['max'][0])<br />
U_rms=(imstat(imagename='3c286.cal.StokesU.clean.image', region='', box='10,10,90,35')['rms'][0])<br />
U_peak=(imstat(imagename='3c286.cal.StokesU.clean.image', region='')['max'][0])<br />
<br />
print I_rms, I_peak, Q_rms, Q_peak, U_rms, U_peak<br />
<br />
</source><br />
<br />
<br />
The results of the measurements for the I, Q, and U images are reported in the table below. All flux densities are reported in mJy/beam.<br />
<br />
The results obtained from the V image are not reported in the table, since '''at this time the quality and accuracy of the circular polarization data are still undergoing verification. Results from the V image are therefore not recommended for scientific purposes at this time'''.<br />
<br />
{| class="wikitable"<br />
| <br />
| style="text-align:center;"| I<br />
| style="text-align:center;"| Q<br />
| style="text-align:center;"| U<br />
|-<br />
| rms <br />
| style="text-align:center;"| 0.129<br />
| style="text-align:center;"| 0.032<br />
| style="text-align:center;"| 0.040<br />
|-<br />
|Peak<br />
|style="text-align:center;"|360 <br />
|style="text-align:center;"|12.1<br />
|style="text-align:center;"|58.0<br />
|-<br />
|Dyn range<br />
|style="text-align:center;"|2792<br />
|style="text-align:center;"|373<br />
|style="text-align:center;"|1463<br />
|}<br />
<br />
==Constructing Polarization Intensity and Angle Images==<br />
<br />
We have now produced the images for all the four Stokes parameters: I, Q, U, and V. Stokes Q and U describe the linear polarization and V describes the circular polarization. Specifically, Q describes the amount of linear polarization aligned with a given axis, and U describes the amount of linear polarization at a 45 deg angle to that axis. The V parameter describes the amount of circular polarization, with the sign (positive or negative) describing the sense of the circular polarization (right- or left-hand circularly polarized).<br />
<br />
In general, few celestial sources are expected to show circular polarization, with the notable exception of masers. Terrestrial and satellite sources are often highly circularly polarized. The V image is therefore often worth forming because any V emission could be indicative of unflagged RFI within the data (or problems with the calibration!).<br />
<br />
Because the Q and U images both describe the amount of linear polarization, it is more common to work with a linear polarization intensity image, <math>P = \sqrt{Q^2 +U^2}</math>. We can also calculate the polarization position angle <math>\chi = 0.5 arctan U/Q</math>.<br />
<br />
The relevant task is {{immath}}; for specific examples of polarization image processing see<br />
[http://casa.nrao.edu/Release4.2.2/docs/userman/UserMansu323.html Polarization Manipulation]. <br />
<br />
To form the linear polarization image, we combine the Q and U images using the mode='poli' option of {{immath}}.<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.POLI')<br />
immath(outfile='3c286_Band6.POLI',<br />
mode='poli',<br />
imagename=['3c286.cal.StokesQ.clean.image','3c286.cal.StokesU.clean.image'],<br />
sigma='0.0Jy/beam')<br />
</source><br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
calstat=imstat(imagename='3c286_Band6.POLI', region='', box='10,10,90,35')<br />
rms=(calstat['rms'][0])<br />
print rms<br />
</source><br />
The measured rms is 0.051 mJy/beam.<br />
<br />
To form the polarization position angle image we combine again the Q and U images using the mode='pola' option of {{immath}}. The Q and U images can be listed in either order. <br />
To avoid displaying the position angle of noise, we can use the polithresh parameter to set a threshold intensity of the linear polarization above which we wish to calculate the polarization angle. An appropriate level here might be the 5σ level of the polarization image, about 0.2 mJy/beam. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.POLA')<br />
immath(outfile='3c286_Band6.POLA',<br />
mode='pola',<br />
imagename=['3c286.cal.StokesQ.clean.image','3c286.cal.StokesU.clean.image'],<br />
polithresh='0.2mJy/beam')<br />
</source><br />
<br />
If desired, it is also possible to form the fractional linear polarization image, defined as P/I. <br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.RATIO')<br />
immath(outfile='3c286_Band6.RATIO',<br />
mode='evalexpr',<br />
imagename=['3c286.cal.StokesI.clean.image','3c286.cal.StokesQ.clean.image','3c286.cal.StokesU.clean.image'],<br />
expr='sqrt((IM1^2+IM2^2)/IM0[IM0>5e-3]^2)')<br />
</source><br />
Since the total intensity image can (and hopefully does) approach zero in regions free of source emission, dividing by the total intensity can produce very high pixel values in these regions. We therefore wish to restrict our fractional polarization image to regions containing real emission, which we do by setting a threshold in the total intensity image, which in this case corresponds to five times the noise level in the Stokes I image. The computation of the polarized intensity is specified in the previous command by <br />
<pre style="background-color: #fffacd;"><br />
expr='sqrt((IM1^2+IM2^2)/IM0[IM0>5e-3]^2)'<br />
</pre><br />
with the expression in square brackets setting the threshold in IM0 (the total intensity image). Note that IM0, IM1 and IM2 correspond to the three files listed in the ''imagename'' array, '''in that order'''. In this case, the order in which the different images are specified is critical.<br />
<br />
In order to have a dimensionless value for the ratio image, we now modify the keyword bunit in the header of the file:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead('3c286_Band6.RATIO', mode='put', hdkey='bunit', hdvalue='')<br />
</source><br />
<br />
==Image visualization and analysis==<br />
<br />
Now we can view these various images using {{viewer}}. It is instructive to display the I, P, and <math>\chi</math> images (total intensity, total linearly polarized intensity, and polarization position angle) together, to show how the polarized emission relates to the total intensity, and how the magnetic field is structured. We can do this using the viewer.<br />
* Begin by loading the linear polarization image in the viewer:<br />
<source lang="python"><br />
# In CASA<br />
viewer('3c286_Band6.POLI',gui=True)<br />
</source><br />
* Next, load the total intensity image as a contour image. In the viewer panel, hit the "Open" icon (the leftmost button in the top row of icons in the viewer). This will bring up a 'Load Data' GUI showing all images and MS in the current directory. Select the total intensity image (3c286.cal.StokesI.clean.image) and click the 'Contour Map' button on the right hand side.<br />
* Finally, load the polarization position angle image (3c286_Band6.POLA) as a vector map.<br />
[[File:Pol_cont_vect_5.4.png|200px|thumb|right|Full-polarization image of 3C286]]<br />
<br />
While we set the ''polithresh'' parameter when we created the position angle (<math>\chi</math>) image, a digression here is instructive in the use of LEL Expressions. Had we not set this parameter, the position angle would have been derived for all pixels within the full IQUV image cube. There is only polarized emission from a limited subset of pixels within this image. Therefore, to avoid plotting vectors corresponding to the position angle of pure noise, we now wish to select only the regions where the polarized intensity is brighter than some threshold value. To do this, we use an LEL (Lattice Expression Language) Expression in the 'Load Data' GUI. For our chosen threshold of 0.2 mJy/beam (the 5 sigma level in the P image), we paste the expression <br />
<pre style="background-color: #fffacd;"><br />
'3c286_Band6.POLA'['3c286_Band6.POLI'>0.001]<br />
</pre><br />
into the LEL Expression box in the GUI, and click the 'Vector Map' button. This would load the vectors only for regions where P>1 mJy/beam.<br />
<br />
To optimize the display for ease of interpretation we click the wrench icon to open a 'Data Display Options' GUI. This will have 3 tabs, corresponding to the three images loaded. <br />
We can change the image color map and transfer function, the contour levels and color, and the the vector spacing and color.<br />
<br />
For the polarization angle vector it is also possible to add a rotation.<br />
The polarization position angle as calculated is the electric vector position angle (EVPA). If we are interested in the orientation of the magnetic field, then for an optically thin source the magnetic field orientation is perpendicular to the EVPA, so we must rotate the vectors by 90 degrees. Select the vector image tab in the 'Data Display Options' GUI (labeled as the LEL expression we entered in the Load Data GUI) and enter ''90'' in the ''Extra rotation'' box. If the vectors appear too densely packed on the image, change the spacing of the vectors by setting ''X-increment'' and ''Y-increment'' to a larger value. <br />
<br />
To get quantitative information from the images, we can either use the task {{imstat}} (we already used it above) or the <br />
task {{imfit}}. <br />
The task {{imstat}} returns the statistics in a Python dictionary, from which we can read what we need in a Python variable. In this example we extract the peak value:<br />
<source lang="python"><br />
# In CASA<br />
calstat=imstat(imagename='3c286_Band6.POLI', region='')<br />
print calstat<br />
calstat['max'][0]<br />
</source><br />
<br />
The output saved in '''calstat''' is pasted below.<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'blc': array([0, 0, 0, 0], dtype=int32),<br />
'blcf': '13:31:09.255, +30.30.20.460, Plinear, 2.3301e+11Hz',<br />
'flux': array([ 0.14658273]),<br />
'max': array([ 0.059297]),<br />
'maxpos': array([125, 126, 0, 0], dtype=int32),<br />
'maxposf': '13:31:08.288, +30.30.33.060, Plinear, 2.3301e+11Hz',<br />
'mean': array([ 8.15708938e-05]),<br />
'medabsdevmed': array([ 1.72724358e-05]),<br />
'median': array([ 4.41299599e-05]),<br />
'min': array([ 1.71590003e-07]),<br />
'minpos': array([179, 222, 0, 0], dtype=int32),<br />
'minposf': '13:31:07.870, +30.30.42.660, Plinear, 2.3301e+11Hz',<br />
'npts': array([ 62500.]),<br />
'q1': array([ 2.82123328e-05]),<br />
'q3': array([ 6.32808587e-05]),<br />
'quartile': array([ 3.50685259e-05]),<br />
'rms': array([ 0.00101149]),<br />
'sigma': array([ 0.00100821]),<br />
'sum': array([ 5.09818086]),<br />
'sumsq': array([ 0.06394478]),<br />
'trc': array([249, 249, 0, 0], dtype=int32),<br />
'trcf': '13:31:07.329, +30.30.45.360, Plinear, 2.3301e+11Hz'}<br />
<br />
calstat['max'][0]=0.059296995401382446<br />
</pre><br />
<br />
The task {{imfit}} finds one or more elliptical gaussian components on an image region.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fit_res=imfit(imagename='3c286_Band6.POLI', region='')<br />
print fit_res<br />
</source><br />
<br />
The output saved in '''fit_res''' is pasted below.<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'converged': array([ True], dtype=bool),<br />
'deconvolved': {'component0': {'flux': {'error': array([ 9.60832825e-05, 0.00000000e+00, 0.00000000e+00,<br />
0.00000000e+00]),<br />
'polarisation': 'Stokes',<br />
'unit': 'Jy',<br />
'value': array([ 0.06067279, 0. , 0. , 0. ])},<br />
'ispoint': True,<br />
'label': '',<br />
'shape': {'direction': {'error': {'latitude': {'unit': 'arcsec',<br />
'value': 0.0003836272403833159},<br />
'longitude': {'unit': 'arcsec', 'value': 0.0001171027199620003}},<br />
'm0': {'unit': 'rad', 'value': -2.74392768000858},<br />
'm1': {'unit': 'rad', 'value': 0.5324855389285916},<br />
'refer': 'J2000',<br />
'type': 'direction'},<br />
'type': 'Point'},<br />
'spectrum': {'channel': 0,<br />
'frequency': {'m0': {'unit': 'GHz', 'value': 233.00995157526424},<br />
'refer': 'LSRK',<br />
'type': 'frequency'},<br />
'type': 'Constant'},<br />
'sum': {'unit': 'Jy/beam', 'value': 1.0485037714242935}},<br />
'nelements': 1},<br />
'pixelsperarcsec': array([10., 10.]),<br />
'results': {'component0': {'beam': {'beamarcsec': {'major': {'unit': 'arcsec',<br />
'value': 0.7654748558998108},<br />
'minor': {'unit': 'arcsec', 'value': 0.4009932577610016},<br />
'positionangle': {'unit': 'deg', 'value': 7.937343597412109}},<br />
'beampixels': 34.78022766827157,<br />
'beamster': 8.174894453628999e-12},<br />
'flux': {'error': array([ 9.60832825e-05, 0.00000000e+00, 0.00000000e+00,<br />
0.00000000e+00]),<br />
'polarisation': 'Stokes',<br />
'unit': 'Jy',<br />
'value': array([ 0.06067279, 0. , 0. , 0. ])},<br />
'ispoint': True,<br />
'label': '',<br />
'peak': {'error': 5.1664878617355715e-05,<br />
'unit': 'Jy/beam',<br />
'value': 0.060420386484414515},<br />
'pixelcoords': array([ 125.28235521, 125.66516488]),<br />
'shape': {'direction': {'error': {'latitude': {'unit': 'arcsec',<br />
'value': 0.0003836272403833159},<br />
'longitude': {'unit': 'arcsec', 'value': 0.0001171027199620003}},<br />
'm0': {'unit': 'rad', 'value': -2.74392768000858},<br />
'm1': {'unit': 'rad', 'value': 0.5324855389285916},<br />
'refer': 'J2000',<br />
'type': 'direction'},<br />
'majoraxis': {'unit': 'arcsec', 'value': 0.7695855012711231},<br />
'majoraxiserror': {'unit': 'arcsec', 'value': 0.0009115928541202604},<br />
'minoraxis': {'unit': 'arcsec', 'value': 0.4005175850839287},<br />
'minoraxiserror': {'unit': 'arcsec', 'value': 0.0002472301217960687},<br />
'positionangle': {'unit': 'deg', 'value': 8.001794332665543},<br />
'positionangleerror': {'unit': 'deg', 'value': 0.035699774475669716},<br />
'type': 'Gaussian'},<br />
'spectrum': {'channel': 0,<br />
'frequency': {'m0': {'unit': 'GHz', 'value': 233.00995157526424},<br />
'refer': 'LSRK',<br />
'type': 'frequency'},<br />
'type': 'Constant'},<br />
'sum': {'unit': 'Jy/beam', 'value': 1.0485037714242935}},<br />
'nelements': 1}}<br />
</pre><br />
<br />
To extract the flux and its error, we can read them from CASA in Python variables:<br />
<source lang="python"><br />
# In CASA<br />
fluxPI=fit_res['results']['component0']['flux']['value'][0]<br />
errorPI=fit_res['results']['component0']['flux']['error'][0]<br />
</source><br />
<br />
The flux measured in the fitted gaussian is 60.7 mJy with an error of 0.096 mJy.<br />
<br />
We now use this method to estimate the fluxes in all the Stokes images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# First we run imfit <br />
resI=imfit(imagename = '3c286.cal.StokesI.clean.image', box = '110,110,145,145')<br />
resQ=imfit(imagename = '3c286.cal.StokesQ.clean.image', box = '110,110,145,145')<br />
resU=imfit(imagename = '3c286.cal.StokesU.clean.image', box = '110,110,145,145')<br />
resV=imfit(imagename = '3c286.cal.StokesV.clean.image', box = '110,110,145,145')<br />
<br />
# and then we extract the flux and error values for each Stokes<br />
<br />
fluxI=resI['results']['component0']['flux']['value'][0]<br />
errorI=resI['results']['component0']['flux']['error'][0]<br />
<br />
fluxQ=resQ['results']['component0']['flux']['value'][1]<br />
errorQ=resQ['results']['component0']['flux']['error'][1]<br />
<br />
fluxU=resU['results']['component0']['flux']['value'][2]<br />
errorU=resU['results']['component0']['flux']['error'][2]<br />
<br />
fluxV=resV['results']['component0']['flux']['value'][3]<br />
errorV=resV['results']['component0']['flux']['error'][3]<br />
<br />
</source> <br />
<br />
Now we use these values to compute polarization intensity, angle and ratio, and their errors:<br />
<br />
<source lang="python"><br />
# In CASA <br />
fluxPI = sqrt( fluxQ**2 + fluxU**2 )<br />
errorPI = sqrt( (fluxQ*errorQ)**2 + (fluxU*errorU)**2 ) / fluxPI<br />
<br />
fluxPImjy = 1000*fluxPI<br />
errPImjy = 1000*errorPI<br />
<br />
polRatio = fluxPI / fluxI<br />
errPRat = polRatio * sqrt( (errorPI/fluxPI)**2 + (errorI/fluxI)**2 )<br />
<br />
polAngle = 0.5 * degrees( atan2(fluxU,fluxQ) )<br />
errPA = 0.5 * degrees( sqrt( (fluxQ*errorU)**2 + (fluxU*errorQ)**2 ) / fluxPI**2 )<br />
<br />
</source><br />
<br />
The results are collected in the following table:<br />
<br />
<center><br />
{| class="wikitable" <br />
| style="text-align:center;"| <br />
| style="text-align:center;"| Flux <br />
| style="text-align:center;"| err <br />
|-<br />
| style="text-align:center;"| I (mJy) <br />
| style="text-align:center;"| 370<br />
| style="text-align:center;"| 0.74<br />
|-<br />
| style="text-align:center;"| Q (mJy)<br />
| style="text-align:center;"| 12.4<br />
| style="text-align:center;"| 0.053<br />
|-<br />
| style="text-align:center;"| U (mJy)<br />
| style="text-align:center;"| 59.4<br />
| style="text-align:center;"| 0.16<br />
|-<br />
| style="text-align:center;"| Pol int (mJy)<br />
| style="text-align:center;"| 60.7<br />
| style="text-align:center;"| 0.159<br />
|-<br />
! scope="col"| P (%)<br />
! scope="col"| 16.4<br />
! scope="col"| 0.54<br />
|-<br />
! scope="col"| χ (deg)<br />
! scope="col"| 39.1<br />
! scope="col"| 0.029<br />
<br />
|}<br />
</center><br />
<br />
As you can see, the polarization intensity computed from the Q and U fluxes is the same as the one extracted from the polarization image (as it should be!).<br />
<br />
Note that '''the uncertainties quoted above are purely statistical.''' The systematic errors will be larger, and include (but are not limited to) any net bias in the position angle of the linear feeds in the antennas, the details of different observations (at what parallactic angles, etc.), and other data quality variations (including source structure). We conservatively estimate the position angle uncertainty to be '''± 2º'''.<br />
<br />
And now, a quick comparison with previous results from IRAM and CARMA.<br />
<br />
[http://adsabs.harvard.edu/abs/2012A%26A...541A.111A Agudo et al. (2012)] presented the results of single-dish millimeter measurements of 3C286 with the IRAM 30 m Telescope.<br />
Their observations at 1 mm (229 GHz) and 3 mm (86 GHz) produced the following results: <br />
{| class="wikitable"<br />
|-<br />
| <br />
|style="text-align:center;"| 1 mm<br />
|style="text-align:center;"| 3 mm<br />
|-<br />
|P (%)<br />
|style="text-align:center;"|14.4 ± 1.8 <br />
|style="text-align:center;"|13.5 ± 0.3<br />
|-<br />
|χ (deg)<br />
|style="text-align:center;"|'''33.1 ± 5.7'''<br />
|style="text-align:center;"|37.3 ± 0.8<br />
|}<br />
<br />
[http://arxiv.org/abs/1506.04771 Hull & Plambeck 2015 (JAI)] and [https://www.mmarray.org/memos/#64 CARMA Memo 64] report interferometric polarization data of 3C286 taken with CARMA at 1 mm (225 GHz). Their results are as follows:<br />
{| class="wikitable"<br />
|-<br />
| <br />
|style="text-align:center;"| 1 mm<br />
|-<br />
|χ (deg)<br />
|style="text-align:center;"|'''39.2 ± 1.0'''<br />
|}<br />
<br />
==Export images in fits format==<br />
<br />
To get all images in fits format you need to use the task {{exportfits}}.<br />
<br />
You can add the parameters imagename (the input image) and fitsimage (the name of the output image) one by one or you can <br />
run a python cycle over all the images:<br />
<br />
<source lang="python"><br />
# In CASA <br />
import glob<br />
<br />
imagenames=glob.glob("*image") <br />
<br />
for name in imagenames:<br />
exportfits(imagename=name, fitsimage=name+'.fits')<br />
<br />
</source> <br />
<br />
<br />
<br />
{{Checked 5.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_6.1&diff=312293C286 Band6Pol Calibration for CASA 6.12021-10-13T13:09:57Z<p>Jbertone: /* Applying the calibrations and splitting of the corrected data */</p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging]]'''.<br />
<br />
'''This guide is designed for CASA 6.1.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 6.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casalith<br />
version = '.'.join(map(str,casalith.version()))<br />
print ("You are using {}".format(version))<br />
if (version < '6.1.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
from casarecipes.almahelpers import fixsyscaltimes<br />
<br />
#path=os.getenv("CASAPATH").split(' ')[0]<br />
#execfile(path+'/lib/py/lib/python3.6/almahelpers.py')<br />
#execfile(path+'/lib/py/lib/python3.6/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:X10a_plotants_6.1.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X85c183_X10a ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotms}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_6.1.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_6.1.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotms(vis=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',iteraxis='antenna',coloraxis='scan',showgui=True)<br />
<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', showgui=True,<br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_6.1.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_6.1.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_6.1.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in {{plotms}}.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-calibration] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.4.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration Antennae Band 7 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
We will use refantmode='strict' in all {{gaincal}} commands during calibration. This forces the reference antenna to stay constant, which will be important in the polarization calibration. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refantmode='strict',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa_6.1.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Bscan',xaxis='frequency',yaxis='amp',showgui=True,<br />
coloraxis='antenna1',iteraxis='spw',gridrows=2,gridcols=2)<br />
</source><br />
<br />
Figure 7 shows the bandpass solutions table colored by antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# make sure usescratch = True<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012',<br />
usescratch = True)<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 8.57485 +/- 0.0223143 (SNR = 384.276, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 8.48545 +/- 0.0223276 (SNR = 380.043, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.14844 +/- 0.0241318 (SNR = 337.664, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.08103 +/- 0.0244413 (SNR = 330.63, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 0.925954 +/- 0.0075512 (SNR = 122.623, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.91457 +/- 0.0079036 (SNR = 115.716, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.876758 +/- 0.00810444 (SNR = 108.182, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.874631 +/- 0.00834362 (SNR = 104.826, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 8.31972 +/- 0.0099499 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.732687 +/- 0.0345553 covariance matrix for the fit: covar(0,0)=1.97575e-06 covar(0,1)=1.76551e-05 covar(1,0)=1.76551e-05 covar(1,1)=0.00874527<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.897732 +/- 0.00160112 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.728998 +/- 0.0513932 covariance matrix for the fit: covar(0,0)=1.98331e-05 covar(0,1)=0.000153745 covar(1,0)=0.000153745 covar(1,1)=0.0873129<br />
<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', showgui=True,<br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_6.1.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:amp_parangle_J1310+3220_6.1.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equations above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
refantmode = 'strict',<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_6.1.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='scan',yaxis='GainAmp',coloraxis='antenna1',correlation='/',showgui=True)<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_6.1.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use {{Polfromgain}} . <br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=polfromgain(vis='3c286_Band6.ms',tablein='3c286_Band6.ms.G1',caltable='3c286_Band6.ms.G1a')<br />
print(qu)<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. For each spectral window, the dictionary contains a list of the results in the order [I,Q,U,V]. The source polarization reported for all spws should be reasonably consistent. This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates. <br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'J1337-1257': {<br />
'Spw0': [1.0, 0.013214756975602053, 0.039836986736821615, 0.0], <br />
'Spw1': [1.0, 0.013857538011564944, 0.03946794978872145, 0.0], <br />
'Spw2': [1.0, 0.014536642899391663, 0.040715752347746374, 0.0], <br />
'Spw3': [1.0, 0.014842848559351099, 0.04151034475082878, 0.0], <br />
'SpwAve': [1.0, 0.01411294661147744, 0.04038275840602956, 0.0]}}<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_6.1.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
showgui=True,<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_6.1.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running {{plotms}} again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:Imaginary_plane_6.1.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
showgui=True,<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in {{gaincal}}, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.4.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:AfterXY0_imagreal_5.4.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.XY0amb',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.XY0',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
refantmode='strict',<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.4.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
plotms(vis='3c286_Band6.ms.G2.polcal',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.4.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_5.4.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_5.4.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
niter=1000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
weighting='briggs',<br />
robust=0.5,<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.4.png | Stokes I.<br />
File:ComparisonQ_5.4.png | Stokes Q.<br />
File:ComparisonU_5.4.png | Stokes U.<br />
File:ComparisonV_5.4.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_5.4.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
showgui=True,<br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_5.4.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:AfterAll_allants_phase_5.4.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs phase for all antennas after the whole calibration]]<br />
| [[File:AfterAll_1baseline_phase_5.4.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-0.015,0.015,-0.015,0.015], <br />
coloraxis='corr',showgui=True)<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_5.4.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
| [[Image:Field4_afterall_5.4.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_6.1 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_6.1&diff=312273C286 Band6Pol Calibration for CASA 6.12021-10-13T13:03:35Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging]]'''.<br />
<br />
'''This guide is designed for CASA 6.1.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 6.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casalith<br />
version = '.'.join(map(str,casalith.version()))<br />
print ("You are using {}".format(version))<br />
if (version < '6.1.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
from casarecipes.almahelpers import fixsyscaltimes<br />
<br />
#path=os.getenv("CASAPATH").split(' ')[0]<br />
#execfile(path+'/lib/py/lib/python3.6/almahelpers.py')<br />
#execfile(path+'/lib/py/lib/python3.6/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:X10a_plotants_6.1.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X85c183_X10a ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotms}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_6.1.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_6.1.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotms(vis=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',iteraxis='antenna',coloraxis='scan',showgui=True)<br />
<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', showgui=True,<br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_6.1.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_6.1.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_6.1.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in {{plotms}}.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-calibration] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.4.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration Antennae Band 7 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
We will use refantmode='strict' in all {{gaincal}} commands during calibration. This forces the reference antenna to stay constant, which will be important in the polarization calibration. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refantmode='strict',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa_6.1.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Bscan',xaxis='frequency',yaxis='amp',showgui=True,<br />
coloraxis='antenna1',iteraxis='spw',gridrows=2,gridcols=2)<br />
</source><br />
<br />
Figure 7 shows the bandpass solutions table colored by antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# make sure usescratch = True<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012',<br />
usescratch = True)<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 8.57485 +/- 0.0223143 (SNR = 384.276, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 8.48545 +/- 0.0223276 (SNR = 380.043, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.14844 +/- 0.0241318 (SNR = 337.664, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.08103 +/- 0.0244413 (SNR = 330.63, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 0.925954 +/- 0.0075512 (SNR = 122.623, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.91457 +/- 0.0079036 (SNR = 115.716, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.876758 +/- 0.00810444 (SNR = 108.182, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.874631 +/- 0.00834362 (SNR = 104.826, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 8.31972 +/- 0.0099499 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.732687 +/- 0.0345553 covariance matrix for the fit: covar(0,0)=1.97575e-06 covar(0,1)=1.76551e-05 covar(1,0)=1.76551e-05 covar(1,1)=0.00874527<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.897732 +/- 0.00160112 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.728998 +/- 0.0513932 covariance matrix for the fit: covar(0,0)=1.98331e-05 covar(0,1)=0.000153745 covar(1,0)=0.000153745 covar(1,1)=0.0873129<br />
<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', showgui=True,<br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_6.1.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:amp_parangle_J1310+3220_6.1.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equations above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
refantmode = 'strict',<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_6.1.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='scan',yaxis='GainAmp',coloraxis='antenna1',correlation='/',showgui=True)<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_6.1.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use {{Polfromgain}} . <br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=polfromgain(vis='3c286_Band6.ms',tablein='3c286_Band6.ms.G1',caltable='3c286_Band6.ms.G1a')<br />
print(qu)<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. For each spectral window, the dictionary contains a list of the results in the order [I,Q,U,V]. The source polarization reported for all spws should be reasonably consistent. This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates. <br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'J1337-1257': {<br />
'Spw0': [1.0, 0.013214756975602053, 0.039836986736821615, 0.0], <br />
'Spw1': [1.0, 0.013857538011564944, 0.03946794978872145, 0.0], <br />
'Spw2': [1.0, 0.014536642899391663, 0.040715752347746374, 0.0], <br />
'Spw3': [1.0, 0.014842848559351099, 0.04151034475082878, 0.0], <br />
'SpwAve': [1.0, 0.01411294661147744, 0.04038275840602956, 0.0]}}<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_6.1.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
showgui=True,<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_6.1.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running {{plotms}} again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:Imaginary_plane_6.1.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
showgui=True,<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in {{gaincal}}, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.4.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:AfterXY0_imagreal_5.4.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.XY0amb',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.XY0',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
refantmode='strict',<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.4.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
plotms(vis='3c286_Band6.ms.G2.polcal',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.4.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_5.4.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_5.4.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
niter=1000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
weighting='briggs',<br />
robust=0.5,<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.4.png | Stokes I.<br />
File:ComparisonQ_5.4.png | Stokes Q.<br />
File:ComparisonU_5.4.png | Stokes U.<br />
File:ComparisonV_5.4.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_5.4.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
showgui=True,<br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_5.4.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:AfterAll_allants_phase_5.4.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs phase for all antennas after the whole calibration]]<br />
| [[File:AfterAll_1baseline_phase_5.4.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-0.015,0.015,-0.015,0.015], <br />
coloraxis='corr',showgui=True)<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_5.4.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
| [[Image:Field4_afterall_5.4.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_5.4 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_6.1&diff=312203C286 Band6Pol Calibration for CASA 6.12021-10-11T21:27:53Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging]]'''.<br />
<br />
'''This guide is designed for CASA 6.1.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 6.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casalith<br />
version = '.'.join(map(str,casalith.version()))<br />
print ("You are using {}".format(version))<br />
if (version < '6.1.0'):<br />
print("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
from casarecipes.almahelpers import fixsyscaltimes<br />
<br />
#path=os.getenv("CASAPATH").split(' ')[0]<br />
#execfile(path+'/lib/py/lib/python3.6/almahelpers.py')<br />
#execfile(path+'/lib/py/lib/python3.6/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:X10a_plotants_6.1.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X85c183_X10a ]]<br />
</figure><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotms}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_6.1.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
</figure><br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_6.1.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
</figure><br />
<source lang="python"><br />
<br />
# In CASA<br />
plotms(vis=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',iteraxis='antenna',coloraxis='scan',showgui=True)<br />
<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', showgui=True,<br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_6.1.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
</figure><br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_6.1.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
</figure><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_6.1.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
</figure><br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in {{plotms}}.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-calibration] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.4.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration Antennae Band 7 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
We will use refantmode='strict' in all {{gaincal}} commands during calibration. This forces the reference antenna to stay constant, which will be important in the polarization calibration. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refantmode='strict',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa_6.1.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
</figure><br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Bscan',xaxis='frequency',yaxis='amp',showgui=True,<br />
coloraxis='antenna1',iteraxis='spw',gridrows=2,gridcols=2)<br />
</source><br />
<br />
Figure 7 shows the bandpass solutions table colored by antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# make sure usescratch = True<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012',<br />
usescratch = True)<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 8.57485 +/- 0.0223143 (SNR = 384.276, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 8.48545 +/- 0.0223276 (SNR = 380.043, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.14844 +/- 0.0241318 (SNR = 337.664, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.08103 +/- 0.0244413 (SNR = 330.63, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 0.925954 +/- 0.0075512 (SNR = 122.623, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.91457 +/- 0.0079036 (SNR = 115.716, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.876758 +/- 0.00810444 (SNR = 108.182, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.874631 +/- 0.00834362 (SNR = 104.826, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 8.31972 +/- 0.0099499 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.732687 +/- 0.0345553 covariance matrix for the fit: covar(0,0)=1.97575e-06 covar(0,1)=1.76551e-05 covar(1,0)=1.76551e-05 covar(1,1)=0.00874527<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.897732 +/- 0.00160112 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.728998 +/- 0.0513932 covariance matrix for the fit: covar(0,0)=1.98331e-05 covar(0,1)=0.000153745 covar(1,0)=0.000153745 covar(1,1)=0.0873129<br />
<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', showgui=True,<br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_6.1.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
</figure><br />
<br />
[[Image:amp_parangle_J1310+3220_6.1.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
</figure><br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equations above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
refantmode = 'strict',<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_6.1.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
</figure> <br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='scan',yaxis='GainAmp',coloraxis='antenna1',correlation='/',showgui=True)<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_6.1.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
</figure><br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use {{Polfromgain}} . <br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=polfromgain(vis='3c286_Band6.ms',tablein='3c286_Band6.ms.G1',caltable='3c286_Band6.ms.G1a')<br />
print(qu)<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. For each spectral window, the dictionary contains a list of the results in the order [I,Q,U,V]. The source polarization reported for all spws should be reasonably consistent. This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates. <br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'J1337-1257': {<br />
'Spw0': [1.0, 0.013214756975602053, 0.039836986736821615, 0.0], <br />
'Spw1': [1.0, 0.013857538011564944, 0.03946794978872145, 0.0], <br />
'Spw2': [1.0, 0.014536642899391663, 0.040715752347746374, 0.0], <br />
'Spw3': [1.0, 0.014842848559351099, 0.04151034475082878, 0.0], <br />
'SpwAve': [1.0, 0.01411294661147744, 0.04038275840602956, 0.0]}}<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_6.1.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
</figure><br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
showgui=True,<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_6.1.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
</figure><br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running {{plotms}} again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:Imaginary_plane_6.1.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
</figure><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
showgui=True,<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in {{gaincal}}, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
</figure><br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.4.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
</figure><br />
<br />
<br />
[[File:AfterXY0_imagreal_5.4.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
</figure><br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.XY0amb',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.XY0',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
refantmode='strict',<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.4.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
</figure><br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
plotms(vis='3c286_Band6.ms.G2.polcal',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.4.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]</figure><br />
| [[Image:Df0_real_5.4.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]</figure><br />
| [[Image:Df0_imag_5.4.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]</figure> <br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
niter=1000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
weighting='briggs',<br />
robust=0.5,<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.4.png | Stokes I.<br />
File:ComparisonQ_5.4.png | Stokes Q.<br />
File:ComparisonU_5.4.png | Stokes U.<br />
File:ComparisonV_5.4.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
</figure><br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_5.4.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
</figure><br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
showgui=True,<br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_5.4.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]</figure><br />
| [[Image:AfterAll_allants_phase_5.4.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs phase for all antennas after the whole calibration]]</figure><br />
| [[File:AfterAll_1baseline_phase_5.4.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]</figure><br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-0.015,0.015,-0.015,0.015], <br />
coloraxis='corr',showgui=True)<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_5.4.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]</figure><br />
| [[Image:Field4_afterall_5.4.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]</figure><br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_5.4 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_5.7&diff=312193C286 Band6Pol Calibration for CASA 5.72021-10-11T20:44:44Z<p>Jbertone: /* Confirm your version of CASA */</p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging]]'''.<br />
<br />
'''This guide is designed for CASA 5.7.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.7. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '5.7.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import fixsyscaltimes <br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:X1a_plotants_5.7.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X85c183_X10a ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotms}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_5.7.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_5.7.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotms(vis=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',iteraxis='antenna',coloraxis='scan',showgui=True)<br />
<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', showgui=True,<br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_5.7.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_5.7.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_5.7.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in {{plotms}}.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-calibration] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.4.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration Antennae Band 7 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
We will use refantmode='strict' in all {{gaincal}} commands during calibration. This forces the reference antenna to stay constant, which will be important in the polarization calibration. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refantmode='strict',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa_5.7.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Bscan',xaxis='frequency',yaxis='amp',showgui=True,<br />
coloraxis='antenna1',iteraxis='spw',gridrows=2,gridcols=2)<br />
</source><br />
<br />
Figure 7 shows the bandpass solutions table colored by antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# make sure usescratch=True<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012',<br />
usescratch = True)<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 8.57485 +/- 0.0223144 (SNR = 384.275, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 8.48545 +/- 0.0223275 (SNR = 380.044, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.14846 +/- 0.0241319 (SNR = 337.664, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.08105 +/- 0.0244413 (SNR = 330.631, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 0.925953 +/- 0.00755122 (SNR = 122.623, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.914569 +/- 0.00790359 (SNR = 115.716, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.87676 +/- 0.00810445 (SNR = 108.183, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.874633 +/- 0.00834364 (SNR = 104.826, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 8.31973 +/- 0.00995183 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.73265 +/- 0.034562 covariance matrix for the fit: covar(0,0)=1.97575e-06 covar(0,1)=1.76548e-05 covar(1,0)=1.76548e-05 covar(1,1)=0.00874527<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.897733 +/- 0.00160117 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.728967 +/- 0.0513949 covariance matrix for the fit: covar(0,0)=1.98331e-05 covar(0,1)=0.000153743 covar(1,0)=0.000153743 covar(1,1)=0.0873131<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', showgui=True,<br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_5.7.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:amp_parangle_J1310+3220_5.7.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
<br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equation above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
refantmode = 'strict',<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_5.7.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='scan',yaxis='GainAmp',coloraxis='antenna1',correlation='/',showgui=True)<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_5.7.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use {{polfromgain}} .<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=polfromgain(vis='3c286_Band6.ms',tablein='3c286_Band6.ms.G1',caltable='3c286_Band6.ms.G1a')<br />
print(qu)<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. For each spectral window, the dictionary contains a list of the results in the order [I,Q,U,V]. The source polarization reported for all spws should be reasonably consistent. <br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'J1337-1257': {<br />
'Spw0': [1.0, 0.013214757041403838, 0.039836986697640311, 0.0], <br />
'Spw1': [1.0, 0.013857537985592696, 0.039467949703625056, 0.0], <br />
'Spw2': [1.0, 0.014536642945002339, 0.040715752233138704, 0.0], <br />
'Spw3': [1.0, 0.014842848610760032, 0.041510344688499477, 0.0], '<br />
SpwAve': [1.0, 0.014112946645689725, 0.040382758330725889, 0.0]}}<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_5.7.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
showgui=True,<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_5.7.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running {{plotms}} again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
[[File:Imaginary_plane_5.7.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
showgui=True,<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in {{gaincal}}, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.4.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:AfterXY0_imagreal_5.4.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.XY0amb',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.XY0',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
refantmode='strict',<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.4.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
plotms(vis='3c286_Band6.ms.G2.polcal',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.4.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_5.4.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_5.4.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
niter=1000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
weighting='briggs',<br />
robust=0.5,<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.4.png | Stokes I.<br />
File:ComparisonQ_5.4.png | Stokes Q.<br />
File:ComparisonU_5.4.png | Stokes U.<br />
File:ComparisonV_5.4.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_5.4.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
showgui=True,<br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_5.4.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:AfterAll_allants_phase_5.4.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs phase for all antennas after the whole calibration]]<br />
| [[File:AfterAll_1baseline_phase_5.4.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-0.015,0.015,-0.015,0.015], <br />
coloraxis='corr',showgui=True)<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_5.4.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
| [[Image:Field4_afterall_5.4.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_5.4 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.7.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_5.7&diff=312183C286 Band6Pol Calibration for CASA 5.72021-10-11T20:43:08Z<p>Jbertone: /* Polarization Calibration */</p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging]]'''.<br />
<br />
'''This guide is designed for CASA 5.7.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.4. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import fixsyscaltimes <br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:X1a_plotants_5.7.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X85c183_X10a ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotms}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_5.7.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_5.7.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotms(vis=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',iteraxis='antenna',coloraxis='scan',showgui=True)<br />
<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', showgui=True,<br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_5.7.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_5.7.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_5.7.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in {{plotms}}.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-calibration] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.4.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration Antennae Band 7 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
We will use refantmode='strict' in all {{gaincal}} commands during calibration. This forces the reference antenna to stay constant, which will be important in the polarization calibration. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refantmode='strict',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa_5.7.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Bscan',xaxis='frequency',yaxis='amp',showgui=True,<br />
coloraxis='antenna1',iteraxis='spw',gridrows=2,gridcols=2)<br />
</source><br />
<br />
Figure 7 shows the bandpass solutions table colored by antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# make sure usescratch=True<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012',<br />
usescratch = True)<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 8.57485 +/- 0.0223144 (SNR = 384.275, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 8.48545 +/- 0.0223275 (SNR = 380.044, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.14846 +/- 0.0241319 (SNR = 337.664, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.08105 +/- 0.0244413 (SNR = 330.631, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 0.925953 +/- 0.00755122 (SNR = 122.623, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.914569 +/- 0.00790359 (SNR = 115.716, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.87676 +/- 0.00810445 (SNR = 108.183, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.874633 +/- 0.00834364 (SNR = 104.826, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 8.31973 +/- 0.00995183 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.73265 +/- 0.034562 covariance matrix for the fit: covar(0,0)=1.97575e-06 covar(0,1)=1.76548e-05 covar(1,0)=1.76548e-05 covar(1,1)=0.00874527<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.897733 +/- 0.00160117 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.728967 +/- 0.0513949 covariance matrix for the fit: covar(0,0)=1.98331e-05 covar(0,1)=0.000153743 covar(1,0)=0.000153743 covar(1,1)=0.0873131<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', showgui=True,<br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_5.7.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:amp_parangle_J1310+3220_5.7.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
<br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equation above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
refantmode = 'strict',<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_5.7.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='scan',yaxis='GainAmp',coloraxis='antenna1',correlation='/',showgui=True)<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_5.7.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use {{polfromgain}} .<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=polfromgain(vis='3c286_Band6.ms',tablein='3c286_Band6.ms.G1',caltable='3c286_Band6.ms.G1a')<br />
print(qu)<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. For each spectral window, the dictionary contains a list of the results in the order [I,Q,U,V]. The source polarization reported for all spws should be reasonably consistent. <br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'J1337-1257': {<br />
'Spw0': [1.0, 0.013214757041403838, 0.039836986697640311, 0.0], <br />
'Spw1': [1.0, 0.013857537985592696, 0.039467949703625056, 0.0], <br />
'Spw2': [1.0, 0.014536642945002339, 0.040715752233138704, 0.0], <br />
'Spw3': [1.0, 0.014842848610760032, 0.041510344688499477, 0.0], '<br />
SpwAve': [1.0, 0.014112946645689725, 0.040382758330725889, 0.0]}}<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_5.7.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
showgui=True,<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_5.7.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running {{plotms}} again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
[[File:Imaginary_plane_5.7.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
showgui=True,<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in {{gaincal}}, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.4.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:AfterXY0_imagreal_5.4.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.XY0amb',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.XY0',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
refantmode='strict',<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.4.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
plotms(vis='3c286_Band6.ms.G2.polcal',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.4.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_5.4.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_5.4.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
niter=1000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
weighting='briggs',<br />
robust=0.5,<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.4.png | Stokes I.<br />
File:ComparisonQ_5.4.png | Stokes Q.<br />
File:ComparisonU_5.4.png | Stokes U.<br />
File:ComparisonV_5.4.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_5.4.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
showgui=True,<br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_5.4.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:AfterAll_allants_phase_5.4.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs phase for all antennas after the whole calibration]]<br />
| [[File:AfterAll_1baseline_phase_5.4.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-0.015,0.015,-0.015,0.015], <br />
coloraxis='corr',showgui=True)<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_5.4.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
| [[Image:Field4_afterall_5.4.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_5.4 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.7.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_5.7&diff=312173C286 Band6Pol Calibration for CASA 5.72021-10-11T20:38:30Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging]]'''.<br />
<br />
'''This guide is designed for CASA 5.7.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.4. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import fixsyscaltimes <br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:X1a_plotants_5.7.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X85c183_X10a ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotms}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_5.7.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_5.7.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotms(vis=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',iteraxis='antenna',coloraxis='scan',showgui=True)<br />
<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', showgui=True,<br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_5.7.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_5.7.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_5.7.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in {{plotms}}.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',showgui=True,<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.4.0/synthesis-calibration] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.4.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration Antennae Band 7 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
We will use refantmode='strict' in all {{gaincal}} commands during calibration. This forces the reference antenna to stay constant, which will be important in the polarization calibration. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refantmode='strict',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa_5.7.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Bscan',xaxis='frequency',yaxis='amp',showgui=True,<br />
coloraxis='antenna1',iteraxis='spw',gridrows=2,gridcols=2)<br />
</source><br />
<br />
Figure 7 shows the bandpass solutions table colored by antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# make sure usescratch=True<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012',<br />
usescratch = True)<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
refantmode='strict',<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 8.57485 +/- 0.0223144 (SNR = 384.275, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 8.48545 +/- 0.0223275 (SNR = 380.044, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.14846 +/- 0.0241319 (SNR = 337.664, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.08105 +/- 0.0244413 (SNR = 330.631, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 0.925953 +/- 0.00755122 (SNR = 122.623, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.914569 +/- 0.00790359 (SNR = 115.716, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.87676 +/- 0.00810445 (SNR = 108.183, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.874633 +/- 0.00834364 (SNR = 104.826, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 8.31973 +/- 0.00995183 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.73265 +/- 0.034562 covariance matrix for the fit: covar(0,0)=1.97575e-06 covar(0,1)=1.76548e-05 covar(1,0)=1.76548e-05 covar(1,1)=0.00874527<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.897733 +/- 0.00160117 (freq=232.86 GHz) spidx: a_1 (spectral index) =-0.728967 +/- 0.0513949 covariance matrix for the fit: covar(0,0)=1.98331e-05 covar(0,1)=0.000153743 covar(1,0)=0.000153743 covar(1,1)=0.0873131<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', showgui=True,<br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_5.7.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:amp_parangle_J1310+3220_5.7.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<equation id="Visibilities"> <br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
</equation><br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equation above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
refantmode = 'strict',<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_5.7.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='scan',yaxis='GainAmp',coloraxis='antenna1',correlation='/',showgui=True)<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_5.7.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use {{polfromgain}} .<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=polfromgain(vis='3c286_Band6.ms',tablein='3c286_Band6.ms.G1',caltable='3c286_Band6.ms.G1a')<br />
print(qu)<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. For each spectral window, the dictionary contains a list of the results in the order [I,Q,U,V]. The source polarization reported for all spws should be reasonably consistent. <br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
{'J1337-1257': {<br />
'Spw0': [1.0, 0.013214757041403838, 0.039836986697640311, 0.0], <br />
'Spw1': [1.0, 0.013857537985592696, 0.039467949703625056, 0.0], <br />
'Spw2': [1.0, 0.014536642945002339, 0.040715752233138704, 0.0], <br />
'Spw3': [1.0, 0.014842848610760032, 0.041510344688499477, 0.0], '<br />
SpwAve': [1.0, 0.014112946645689725, 0.040382758330725889, 0.0]}}<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_5.7.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
showgui=True,<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_5.7.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running {{plotms}} again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
[[File:Imaginary_plane_5.7.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
showgui=True,<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in {{gaincal}}, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
refantmode='strict',<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.4.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:AfterXY0_imagreal_5.4.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.XY0amb',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.XY0',xaxis='frequency',yaxis='phase',iteraxis='antenna',coloraxis='spw',correlation='X',showgui=True)<br />
<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
refantmode='strict',<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.4.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.G1',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
plotms(vis='3c286_Band6.ms.G2.polcal',xaxis='time',yaxis='amp',field='J1337-1257',correlation='/',coloraxis='antenna1',showgui=True)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='amp',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='real',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='0,1',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
plotms(vis='3c286_Band6.ms.Df0',xaxis='frequency',yaxis='imag',spw='2,3',iteraxis='antenna',coloraxis='corr',showgui=True)<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.4.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_5.4.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_5.4.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, <br />
weighting='briggs',<br />
robust=0.5,<br />
niter=1000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
weighting='briggs',<br />
robust=0.5,<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.4.png | Stokes I.<br />
File:ComparisonQ_5.4.png | Stokes Q.<br />
File:ComparisonU_5.4.png | Stokes U.<br />
File:ComparisonV_5.4.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_5.4.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
showgui=True,<br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_5.4.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:AfterAll_allants_phase_5.4.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs phase for all antennas after the whole calibration]]<br />
| [[File:AfterAll_1baseline_phase_5.4.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-0.015,0.015,-0.015,0.015], <br />
coloraxis='corr',showgui=True)<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_5.4.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
| [[Image:Field4_afterall_5.4.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_5.4 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.7.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_5.1&diff=312163C286 Band6Pol Calibration for CASA 5.12021-10-11T18:48:59Z<p>Jbertone: /* Polarization Calibration */</p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging_for_CASA_5.1]]'''.<br />
<br />
'''This guide is designed for CASA 5.1.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:Uid_A002_X85c183_X10a_plotants5.1.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X84049a_X1e7 ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotcal}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_5.1.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_5.1.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotcal(caltable=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',subplot=111, <br />
iteration='antenna')<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', <br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_5.1.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_5.1.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_5.1.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in plotms.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.1.1/synthesis-calibration calibration chapter] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.0.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3 N3256_Band3 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa5.1.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotbandpass(caltable = '3c286_Band6.ms.Bscan', xaxis='freq', yaxis='amp', <br />
solutionTimeThresholdSeconds= 32.0, <br />
overlay='antenna', figfile='3c286_Band6.ms.Bscan.png')<br />
</source><br />
<br />
Figure 7 shows a colored curve for each antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. <br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012')<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 9.34578 +/- 0.0232958 (SNR = 401.179, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 9.24803 +/- 0.0233091 (SNR = 396.757, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.88068 +/- 0.0251928 (SNR = 352.509, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.80745 +/- 0.0255161 (SNR = 345.172, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 1.0092 +/- 0.00788337 (SNR = 128.017, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.996761 +/- 0.00825107 (SNR = 120.804, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.955552 +/- 0.00846077 (SNR = 112.939, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.953254 +/- 0.00871058 (SNR = 109.436, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 9.06751 +/- 0.0108899 (freq=232.86 GHz) spidx=-0.732841 +/- 0.0347008<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.978424 +/- 0.00175502 (freq=232.86 GHz) spidx=-0.729125 +/- 0.0516872<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', <br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_5.1.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:amp_parangle_J1310+3220_5.1.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration_for_CASA_5.1#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equation above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_5.1.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','scan','amp',field='',poln='/',subplot=111, figfile='G1_polnratio.png')<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_5.1.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use a function included in the almapolhelpers.py: qufromgain().<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=qufromgain('3c286_Band6.ms.G1')<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. The source polarization reported for all spws should be reasonably consistent.<br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Latitude = -23.0294371117<br />
Found as many as 5 fields.<br />
Found as many as 4 spws.<br />
Fld= 0 Spw= 0 (B=06, PA offset=-135.0deg) Gx/Gy= 0.99240530493 Q= 0.0120289687444 U= 0.0357750717038 P= 0.0377432357445 X= 35.7076779326<br />
Fld= 0 Spw= 1 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00460119738 Q= 0.0121162165327 U= 0.0342483624595 P= 0.0363284053355 X= 35.2587452748<br />
Fld= 0 Spw= 2 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00477771901 Q= 0.0132549977543 U= 0.0365819340249 P= 0.038909290182 X= 35.041330551<br />
Fld= 0 Spw= 3 (B=06, PA offset=-135.0deg) Gx/Gy= 0.997298298614 Q= 0.0135497265581 U= 0.0372695444178 P= 0.0396561978877 X= 35.0103714895<br />
For field id = 0 there are 4 good spws.<br />
Spw mean: Fld= 0 Q= 0.0127374773974 U= 0.0359687281515 (rms= 0.000673707205015 0.00112531166082 ) P= 0.0381574728629 X= 35.2497999082<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_5.1.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
# In CASA<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_5.1.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running plotms again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:Imaginary_plane_5.1.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in gaincal, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.1.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:AfterXY0_imagreal_5.1.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.XY0amb','freq','phase',antenna='0',poln='X',subplot=121)<br />
plotcal('3c286_Band6.ms.XY0','freq','phase',antenna='0',poln='X',subplot=122)<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.1.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','time','amp',field='J1337-1257',poln='/',subplot=121)<br />
plotcal('3c286_Band6.ms.G2.polcal','time','amp',field='J1337-1257',poln='/',subplot=122)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,0,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,0,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.1.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_5.1.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_5.1.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, niter=10000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.1.png | Stokes I.<br />
File:ComparisonQ_5.1.png | Stokes Q.<br />
File:ComparisonU_5.1.png | Stokes U.<br />
File:ComparisonV_5.1.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_casa5.1.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_casa5.1.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:AfterAll_allants_phase_casa5.1.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration]]<br />
| [[File:AfterAll_1baseline_phase_casa5.1.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-1,1,-1,1], <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_casa5.1.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
| [[Image:Field4_afterall_casa5.1.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_5.1 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.1.1}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_5.1&diff=312153C286 Band6Pol Calibration for CASA 5.12021-10-11T18:46:15Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging_for_CASA_5.1]]'''.<br />
<br />
'''This guide is designed for CASA 5.1.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casa<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
10 DA54 A063 12.0 m -067.45.16.1 -22.53.31.9 46.5810 -823.6376 21.9794 2225071.684905 -5440011.975810 -2481797.189115<br />
11 DA55 A080 12.0 m -067.45.14.7 -22.53.20.2 87.4828 -461.2364 21.1332 2225162.612020 -5440126.242512 -2481462.996898<br />
12 DA56 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0287 21.5198 2225109.988756 -5440002.411268 -2481782.630797<br />
13 DA57 A076 12.0 m -067.45.20.5 -22.53.33.8 -77.9911 -882.7200 24.5713 2224948.593535 -5440040.069448 -2481852.626444<br />
14 DA59 A021 12.0 m -067.45.17.2 -22.53.27.0 14.3185 -672.8117 21.8450 2225063.988913 -5440078.377183 -2481658.189073<br />
15 DA62 A016 12.0 m -067.45.16.4 -22.53.25.1 37.4652 -614.5612 21.7854 2225093.968954 -5440090.535537 -2481604.502426<br />
16 DV01 A072 12.0 m -067.45.12.6 -22.53.24.0 147.1726 -580.5878 18.1840 2225199.253543 -5440058.163697 -2481571.803410<br />
17 DV02 A087 12.0 m -067.45.08.3 -22.53.33.2 269.0953 -864.0700 16.2406 2225269.668853 -5439908.286857 -2481832.204625<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:Uid_A002_X85c183_X10a_plotants5.1.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X84049a_X1e7 ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [https://casaguides.nrao.edu/index.php/AntennaeBand7_Calibration here]) for details on {{flagmanager}}.<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run fixsyscaltimes task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotcal}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49_5.1.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21_5.1.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotcal(caltable=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',subplot=111, <br />
iteration='antenna')<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', <br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:Amp_time_all_field_5.1.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:Amp_uvdist_Ceres_5.1.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:Amp_time_field0_5.1.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in plotms.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [https://casa.nrao.edu/casadocs/casa-5.1.1/synthesis-calibration calibration chapter] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [https://casa.nrao.edu/casadocs/casa-5.0.0/reference-material/the-measurement-equation-calibration here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3 N3256_Band3 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details).<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:Bandpass_casa5.1.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotbandpass(caltable = '3c286_Band6.ms.Bscan', xaxis='freq', yaxis='amp', <br />
solutionTimeThresholdSeconds= 32.0, <br />
overlay='antenna', figfile='3c286_Band6.ms.Bscan.png')<br />
</source><br />
<br />
Figure 7 shows a colored curve for each antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. <br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012')<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24e+11 Hz) is: 9.34578 +/- 0.0232958 (SNR = 401.179, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26e+11 Hz) is: 9.24803 +/- 0.0233091 (SNR = 396.757, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.4e+11 Hz) is: 8.88068 +/- 0.0251928 (SNR = 352.509, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.42e+11 Hz) is: 8.80745 +/- 0.0255161 (SNR = 345.172, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24e+11 Hz) is: 1.0092 +/- 0.00788337 (SNR = 128.017, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26e+11 Hz) is: 0.996761 +/- 0.00825107 (SNR = 120.804, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.4e+11 Hz) is: 0.955552 +/- 0.00846077 (SNR = 112.939, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.42e+11 Hz) is: 0.953254 +/- 0.00871058 (SNR = 109.436, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 9.06751 +/- 0.0108899 (freq=232.86 GHz) spidx=-0.732841 +/- 0.0347008<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.978424 +/- 0.00175502 (freq=232.86 GHz) spidx=-0.729125 +/- 0.0516872<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', <br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:amp_parangle_3C279_5.1.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:amp_parangle_J1310+3220_5.1.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration_for_CASA_5.1#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<equation id="Visibilities"> <br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
</equation><br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equation above we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<br />
[[Image:Amp_parangle_J1337-1257_5.1.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','scan','amp',field='',poln='/',subplot=111, figfile='G1_polnratio.png')<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_5.1.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use a function included in the almapolhelpers.py: qufromgain().<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=qufromgain('3c286_Band6.ms.G1')<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. The source polarization reported for all spws should be reasonably consistent.<br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Latitude = -23.0294371117<br />
Found as many as 5 fields.<br />
Found as many as 4 spws.<br />
Fld= 0 Spw= 0 (B=06, PA offset=-135.0deg) Gx/Gy= 0.99240530493 Q= 0.0120289687444 U= 0.0357750717038 P= 0.0377432357445 X= 35.7076779326<br />
Fld= 0 Spw= 1 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00460119738 Q= 0.0121162165327 U= 0.0342483624595 P= 0.0363284053355 X= 35.2587452748<br />
Fld= 0 Spw= 2 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00477771901 Q= 0.0132549977543 U= 0.0365819340249 P= 0.038909290182 X= 35.041330551<br />
Fld= 0 Spw= 3 (B=06, PA offset=-135.0deg) Gx/Gy= 0.997298298614 Q= 0.0135497265581 U= 0.0372695444178 P= 0.0396561978877 X= 35.0103714895<br />
For field id = 0 there are 4 good spws.<br />
Spw mean: Fld= 0 Q= 0.0127374773974 U= 0.0359687281515 (rms= 0.000673707205015 0.00112531166082 ) P= 0.0381574728629 X= 35.2497999082<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:BG2_ph_freq_DA41DA48_5.1.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
# In CASA<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:AfterKcr_DA41DA48_5.1.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running plotms again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:Imaginary_plane_5.1.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in gaincal, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_5.1.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737477397375133, 0.035968728151496245)<br />
Spw = 0: Found QU = [ 0.01265498 0.04108791]<br />
...KEEPING X-Y phase 84.388794888 deg<br />
Spw = 1: Found QU = [-0.01267773 -0.04118295]<br />
...CONVERTING X-Y phase from -73.078910285 to 106.921089715 deg<br />
Spw = 2: Found QU = [ 0.01363056 0.0419152 ]<br />
...KEEPING X-Y phase 7.90123387877 deg<br />
Spw = 3: Found QU = [ 0.01352668 0.04204915]<br />
...KEEPING X-Y phase -21.4802993724 deg<br />
Ambiguity resolved (spw mean): Q= 0.0131224868819 U= 0.0415588011965 (rms= 0.000457682044256 0.000427337911239 ) P= 0.0435813448491 X= 36.2380589476<br />
Returning the following Stokes vector: [1.0, 0.013122486881911755, 0.04155880119651556, 0.0]<br />
<br />
</pre><br />
<br />
<br />
[[File:AfterXY0_DA41DA48_5.1.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:AfterXY0_imagreal_5.1.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.XY0amb','freq','phase',antenna='0',poln='X',subplot=121)<br />
plotcal('3c286_Band6.ms.XY0','freq','phase',antenna='0',poln='X',subplot=122)<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:GainsG1_G2polcal_5.1.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','time','amp',field='J1337-1257',poln='/',subplot=121)<br />
plotcal('3c286_Band6.ms.G2.polcal','time','amp',field='J1337-1257',poln='/',subplot=122)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000700173772305 U= -0.000493932905019 (rms= 0.000600375442618 0.00070264279 ) P= 0.000856862314543 X= -17.6004000037<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,0,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,0,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
</source><br />
{|<br />
| [[Image:Df0_amp_5.1.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_5.1.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_5.1.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, niter=10000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
tclean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
deconvolver='clarkstokes',<br />
interactive=True, niter=10000)<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:ComparisonI_5.1.png | Stokes I.<br />
File:ComparisonQ_5.1.png | Stokes Q.<br />
File:ComparisonU_5.1.png | Stokes U.<br />
File:ComparisonV_5.1.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:AfterAll_imagreal_casa5.1.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[True,True,False,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:AfterAll_allants_amp_casa5.1.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:AfterAll_allants_phase_casa5.1.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration]]<br />
| [[File:AfterAll_1baseline_phase_casa5.1.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[True,True,False,False,False,False],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-1,1,-1,1], <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:Field4_beforeall_casa5.1.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
| [[Image:Field4_afterall_casa5.1.png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_5.1 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 5.1.1}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_3.4&diff=31214AntennaeBand7 Imaging 3.42021-10-11T17:59:55Z<p>Jbertone: </p>
<hr />
<div>'''This guide is designed for CASA 3.4 (&ge;r19874). If you are using an older version of CASA please see [[Antennae_Band7_-_Imaging_for_CASA_3.3]].<br />
<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration_3.4]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
*'''For the streamlined Synthesis Imaging School version, see [[AntennaeBand7_North_Imaging_3.4]]'''<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 3.4.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casalog.version()<br />
print "You are using " + version<br />
if (int(version.split()[4][1:-1]) < 19874):<br />
print "\033[91m YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "\033[91m PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{clean}}. To invoke mosaic mode, you simply set the parameter '''imagermode='mosaic''''. The default subparameter '''ftmachine='mosaic'''' is then automatically set. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.flux.pbcoverage'' and ''<imagename>.flux'' (where the latter differs from the former only if the sensitivity of each field in the mosaic varies).<br />
<br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''. <br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{clean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/jott-mosaicking-school-04.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Figure 1), and from 70 to 100 in the northern mosaic (Figure 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_North-AMPvsCH.png')<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_South-AMPvsCH.png')<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.flux # the effective primary beam response (where '''pbcor=True''')<br />
** <imagename>.flux.pbcoverage # the primary beam coverage ('''ftmachine=’mosaic’ ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''mode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of clean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=F,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.27" x 0.71" P.A. 86.142 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre><br />
<br />
<br />
[[Image:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes there is definitely a detection in the vicinity of the Northern nucleus (see Figure 5 and Figure 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.4 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=T''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Clean*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=T, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only 100 iterations (see Figure 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{clean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{clean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{clean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. During {{clean}}, we repeat the process of making a clean mask. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.Cont.Clean*')<br />
clean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter='15',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=T, usescratch=False,<br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.11"x 0.71", and P.A.=65 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after 100 iterations (Figure 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.80 mJy/beam and the rms is 0.41 mJy/beam. For the Southern continuum we find a peak of 5.23 mJy/beam and an rms of 0.42 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Figure 5), then double click inside the polygon to have the statistic printed to the screen. We find rms=0.47 mJy/beam, 15% larger.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
We find rms=0.50 mJy/beam, 19% larger than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (<br />
raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00048,0.00380]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview (<br />
raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00050,0.00523]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub2}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub2(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub2(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[Image:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
To see what velocity parameters you want to set in {{clean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png') <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel.<br />
<br />
Notable parameters included in the {{clean}} call are:<br />
*'''mode='velocity' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'velocity' mode to set velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=10000''': Maximum number of clean iterations. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
<br />
<br />
[[Image:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about 900 iterations (Red X), when the artifacts start to look as bright as the residual. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Clean about 1200 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The continuum is far too weak so we will use the CO(3-2) line emission. The process of self-calibration tries to adjust the data to better match the model image that you give it. That is why it is important to make the model as good as you can by making clean masks. The clean parameter '''calready=T''' (true by default in the commands above) ensured that a uv-model of the resulting image was placed in the model data column. This model can then be used to self-calibrate the data using {{gaincal}}. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Another wrinkle is the self-calibration of spectral line data. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='data',plotfile='North_selfchan_data.png')<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='model',plotfile='North_selfchan_model.png')<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan')<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
<br />
[[Image:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png')<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{clean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 1200 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}],<br />
zoom={'channel': 43})<br />
</source><br />
<br />
Then in the viewer, select the blink toggle in the "tapedeck", now the tapedeck buttons cycle between the two images at the selected channel. To change the channel toggle back to "normal", change the channel number, and then toggle back to "blink". <br />
To make the figure shown in Figure 16, you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you need to toggle blink off, then use the tapedeck to move to the new channel of interest, and then turn blink back on to see both cubes again.<br />
<br />
Note: In "blink" mode, statistics will be reported for the image currently displayed. In "normal" mode, all open raster images will have their statistics reported.<br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png')<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=T,coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
<br />
[[Image:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 2200 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}],<br />
zoom={'channel': 31})<br />
</source><br />
<br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 22, 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=T''' and/or '''dropstokes=T'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 3.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_3.4&diff=31213AntennaeBand7 Calibration 3.42021-10-11T17:43:32Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
<br />
*'''This guide requires CASA 3.4 (&ge;r19874) and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7 Imaging 3.4]]'''.<br />
<br />
*'''For the streamlined Synthesis Imaging School version, see [[AntennaeBand7_North_Calibration_3.4]]'''<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step you through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection, for example data where one telescope blocks the line of sight of another. Then we will apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Then we will use observations of the calibrators Titan and 3c279 to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these to the data and then extract the calibrated source data into a file appropriate for imaging.<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
Once you have downloaded the Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
This guide requires Python module [[analysis_Utilities|analysisUtils]]. If you have not already installed [[analysis_Utilities|analysisUtils]] please follow the link to do so. <br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casapy<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 3.4.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casalog.version()<br />
print "You are using " + version<br />
if (int(version.split()[4][1:-1]) < 19874):<br />
print "\033[91m YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "\033[91m PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Install Analysis Utilities==<br />
<br />
Analysis Utilities (or analysisUtils for short) is a small set of Python scripts that provide a number of analysis and plotting utilities for ALMA data reduction. This guide uses a few of these utilities. They are very easy to install (just download and untar). See <br />
<br />
http://casaguides.nrao.edu/index.php?title=Analysis_Utilities<br />
<br />
for a full description and download instructions. If you do not wish to do this, see the CASA 3.3 version of the guide linked at the top of this page for alternative (but slow) plotting options. Analysis Utilities are updated frequently so if its been a while since you installed it, its probably worth doing it again. If you are at an ALMA site or ARC, the analysis utilities are probably already installed and up to date.<br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /export/lustre/ahale/CASA3.4TestingAntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /export/lustre/aleroy/Antennae_Band7_UnCalibratedMSandTablesForReduction/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Figure 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task {{plotants}}</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms',mode='manualflag', unflag=T, flagbackup=F)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', diameter=12.0, flagbackup = F)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow',diameter=12.0: flag shadowed data, taking into account that antennas are 12m diameter<br />
* '''flagbackup''' = F: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manualflag', intent='*POINTING*', flagbackup = F)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=F)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=T so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=T.<br />
<br />
For example try <br />
<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=F. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
</pre><br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
We will also want to look at Tsys as a function of frequency. This will use the analysisutils package mentioned at the beginning of this guide (called by the au. command)<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','Antennae']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
au.plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,buildpdf=False,<br />
interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. gencal will now be used put antenna position data into each observation. Again, gencal will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running gencal to create the information CASA can refer to for antenna positions.<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=F, calwt=T)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=F, calwt=T)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
In CASA 3.4: the MODEL column is deprecated (though it is still possible to use them by explicitly setting usescratch=T).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=F tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=F)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Figure 10 and Figure 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field')<br />
dummy_string = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Figure 15) and that Titan also shows evidence of a strong line (also Figure 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='manualflag', unflag= T, flagbackup = F)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Figure 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',spw = '0:0~7,0:3831~3839', flagbackup = F)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',spw = '0:3260~3320', flagbackup = F)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Figure 4 and Figure 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', antenna='DV04', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV04',flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV12',correlation='YY', flagbackup=F)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Figure 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', field='Titan', uvrange='0~40', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', field='Titan', uvrange='0~30', flagbackup = F)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Figure 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Figure 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms',timerange='00:53:47~01:08:00',flagbackup = F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=F)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=F)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms',timerange='21:18:00~21:22:15', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms',antenna='DV01&DV04', flagbackup=F)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Figure 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=T, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Figure 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=T, <br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=T,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=T,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
au.plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and then calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=F, <br />
field=['Titan'],<br />
spw=['0:1100~1700'])<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2010')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
2012-05-10 15:46:24 INFO setjy Titan (fld ind 1) spw 0 [I=2.8614, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)<br />
</pre><br />
<br />
Now we are going to take a look at the titan model for one dataset<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
</source><br />
<br />
Next we'll run a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.45 Jy, by averaging the fluxes obtained from the ten available datasets. This value agrees within 10% with the most recent 0.850 millimeter measurements from the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html] : (01 Jul 2011, SMA 9.75 ± 0.49).<br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 28.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 29.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 28 and Figure 29 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=T, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=T,avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=T)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=T)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_3.4 imaging guide].<br />
{{Checked 3.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.5&diff=31212AntennaeBand7 Calibration 4.52021-10-11T16:29:07Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 4.5.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7 Imaging 4.5]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step you through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection, for example data where one telescope blocks the line of sight of another. Then we will apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Then we will use observations of the calibrators Titan and 3c279 to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these to the data and then extract the calibrated source data into a file appropriate for imaging.<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
Once you have downloaded the Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casapy<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.5.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.5.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Figure 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task {{plotants}}</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=False)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = False)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = F: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = False)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=False)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=True so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=True.<br />
<br />
For example try <br />
<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=False. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
</pre><br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0, showgui = True) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0, showgui = True) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. gencal will now be used put antenna position data into each observation. Again, gencal will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running gencal to create the information CASA can refer to for antenna positions.<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=False, calwt=True)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=False, calwt=True)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
The MODEL column is now deprecated (though it is still possible to use it by explicitly setting usescratch=True).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=False tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=False)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Figure 10 and Figure 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Figure 15) and that Titan also shows evidence of a strong line (also Figure 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = False)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Figure 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = False)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = False)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Figure 4 and Figure 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Figure 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Figure 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Figure 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=False)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=False)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=False)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Figure 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=True, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Figure 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=True, <br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=True,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=True,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=False, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2010')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan (fld ind 1) spw 0 [I=2.8614, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)<br />
</pre><br />
<br />
Now we are going to take a look at the titan model for one dataset<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
</source><br />
<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model prodcued by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
[[File:2012_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model prodcued by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2010' '''. A more updated standard is also available: '''standard='Butler-JPL-Horizons 2012' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. The motivated reader may wish to rerun {{setjy}} using the updated model. We will highlight later in this tutorial, and in the imaging tutorial that follows, the effects of using the updated model.<br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.41 Jy, by averaging the fluxes obtained from the ten available datasets. This value agrees within 10% with the most recent 0.850 millimeter measurements from the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html] : (03 Feb 2014, SMA 10.93 ± 0.49).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the updated, Butler-JPL-Horizons 2012, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly higher. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.80 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=True, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=True,avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=True)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=True)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.5 imaging guide].<br />
{{Checked 4.5.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_4.3&diff=312103C286 Band6Pol Calibration for CASA 4.32021-10-06T16:45:58Z<p>Jbertone: /* Polarization Calibration */</p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging_for_CASA_4.3]]'''.<br />
<br />
'''This guide is designed for CASA 4.3.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casapy<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
<br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:uid___A002_X85c183_X10a_plotants_casa4.3.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X84049a_X1e7 ]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.2 here]) for details on {{flagmanager}}.<br />
<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run {{fixsyscaltimes}} task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotcal}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotcal(caltable=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',subplot=111, <br />
iteration='antenna')<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', <br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:all_field_casa4.3.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:ceres_uv_casa4.3.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:field0_casa3.4.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in plotms.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [http://casa.nrao.edu/docs/userman/casa_cookbook005.html calibration chapter] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [http://casa.nrao.edu/docs/UserMan/casa_cookbook016.html here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3 N3256_Band3 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details). '''IMPROVE THIS'''<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:bscan_casa3.4.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotbandpass(caltable = '3c286_Band6.ms.Bscan', xaxis='freq', yaxis='amp', <br />
solutionTimeThresholdSeconds= 32.0, <br />
overlay='antenna', figfile='3c286_Band6.ms.Bscan.png')<br />
</source><br />
<br />
Figure 7 shows a colored curve for each antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. <br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012')<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24984e+11 Hz) is: 9.34567 +/- 0.0232932 (SNR = 401.218, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26984e+11 Hz) is: 9.24788 +/- 0.0233105 (SNR = 396.726, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.39016e+11 Hz) is: 8.88454 +/- 0.0252485 (SNR = 351.884, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.41016e+11 Hz) is: 8.8074 +/- 0.0255162 (SNR = 345.169, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24984e+11 Hz) is: 1.0092 +/- 0.00788338 (SNR = 128.016, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26984e+11 Hz) is: 0.996775 +/- 0.00825086 (SNR = 120.809, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.39016e+11 Hz) is: 0.954589 +/- 0.00847336 (SNR = 112.658, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.41016e+11 Hz) is: 0.953251 +/- 0.00871098 (SNR = 109.431, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 9.06839 +/- 0.0086788 <br />
(freq=232.892 GHz) spidx=-0.831341 +/- 0.0314601<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.978163 +/- 0.00182649 <br />
(freq=232.892 GHz) spidx=-0.837124 +/- 0.0611923<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', <br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:3c279_afterBandG2parang_lastcheck.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:J1310_afterBandG2parang_lastcheck.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration_for_CASA_4.3#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equations we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
[[Image:J1337_afterBandG1_parang_casa4.3.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','scan','amp',field='',poln='/',subplot=111, figfile='G1_polnratio.png')<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_casa4.3.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use a function included in the almapolhelpers.py: qufromgain().<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=qufromgain('3c286_Band6.ms.G1')<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. The source polarization reported for all spws should be reasonably consistent.<br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Latitude = -23.0294371117<br />
Found as many as 5 fields.<br />
Found as many as 4 spws.<br />
Fld= 0 Spw= 0 (B=06, PA offset=-135.0deg) Gx/Gy= 0.992405226704 Q= 0.0120290146946 <br />
U= 0.0357751032462 P= 0.0377432802867 X= 35.7076525043<br />
Fld= 0 Spw= 1 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00460130437 Q= 0.0121161477985 <br />
U= 0.0342484704653 P= 0.0363284842332 X= 35.2588247799<br />
Fld= 0 Spw= 2 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00477793757 Q= 0.0132551417327<br />
U= 0.0365817946839 P= 0.0389092082244 X= 35.0411959343<br />
Fld= 0 Spw= 3 (B=06, PA offset=-135.0deg) Gx/Gy= 0.99729825908 Q= 0.013549612772 <br />
U= 0.0372694902549 P= 0.0396561081062 X= 35.0104353733<br />
For field id = 0 there are 4 good spws.<br />
Spw mean: Fld= 0 Q= 0.0127374792495 U= 0.0359687146626 (rms= 0.000673704330994 0.00112523439036 ) <br />
P= 0.038157460766 X= 35.2497952168<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:bG2_ph_freq_DA48DV23.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
# In CASA<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:afterKcr_DA48DV23.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running plotms again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:imaginary_plane.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in gaincal, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_casa4.3.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737479249452035, 0.035968714662568159)<br />
Spw = 0: Found QU = [ 0.01259385 0.04101928]<br />
...KEEPING X-Y phase 84.4169495884 deg<br />
Spw = 1: Found QU = [-0.01261223 -0.04112571]<br />
...CONVERTING X-Y phase from -73.064852046 to 106.935147954 deg<br />
Spw = 2: Found QU = [ 0.01355364 0.0418143 ]<br />
...KEEPING X-Y phase 8.60411636622 deg<br />
Spw = 3: Found QU = [ 0.01344682 0.04199271]<br />
...KEEPING X-Y phase -21.7400889849 deg<br />
Ambiguity resolved (spw mean): Q= 0.0130516346544 U= 0.0414879983291 <br />
(rms= 0.000450227601812 0.000421946318322 ) P= 0.0434925185809 X= 36.2685199328<br />
Returning the following Stokes vector: [1.0, 0.013051634654402733, 0.041487998329102993, 0.0]<br />
</pre><br />
<br />
[[File:afterXY0_DA48DV23.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:afterXY0_imagreal.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.XY0amb','freq','phase',antenna='0',poln='X',subplot=121)<br />
plotcal('3c286_Band6.ms.XY0','freq','phase',antenna='0',poln='X',subplot=122)<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:gainsG1_G2polcal_casa4.3.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','time','amp',field='J1337-1257',poln='/',subplot=121)<br />
plotcal('3c286_Band6.ms.G2.polcal','time','amp',field='J1337-1257',poln='/',subplot=122)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000838681858308 U= -0.000234770753915 (rms= 0.000600891446491 0.000705059408831 ) <br />
P= 0.000870921676357 X= -7.81921742693<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,0,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,0,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
</source><br />
{|<br />
| [[Image:Df0_amp_casa4.3.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_casa4.3.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_casa4.3.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True, niter=10000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True,niter=10000)<br />
<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:comparisonI.png | Stokes I.<br />
File:comparisonQ.png | Stokes Q.<br />
File:comparisonU.png | Stokes U.<br />
File:comparisonV.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:afterall_imagreal.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:afterAll_allantennas_amp_casa4.3.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:afterAll_allantennas_phase_casas4.3.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration]]<br />
| [[File:afterAll_1baseline_phase.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[T,T,F,F,F,F],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-1,1,-1,1], <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
|[[Image:field4_beforeaal_casa4.3.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
|[[Image:field4_afterall_casa43..png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_4.3 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_4.3&diff=312093C286 Band6Pol Calibration for CASA 4.32021-10-06T16:43:40Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging_for_CASA_4.3]]'''.<br />
<br />
'''This guide is designed for CASA 4.3.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casapy<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
<br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:uid___A002_X85c183_X10a_plotants_casa4.3.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X84049a_X1e7 ]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.2 here]) for details on {{flagmanager}}.<br />
<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run {{fixsyscaltimes}} task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotcal}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotcal(caltable=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',subplot=111, <br />
iteration='antenna')<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', <br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:all_field_casa4.3.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:ceres_uv_casa4.3.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:field0_casa3.4.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in plotms.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [http://casa.nrao.edu/docs/userman/casa_cookbook005.html calibration chapter] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [http://casa.nrao.edu/docs/UserMan/casa_cookbook016.html here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3 N3256_Band3 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details). '''IMPROVE THIS'''<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:bscan_casa3.4.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotbandpass(caltable = '3c286_Band6.ms.Bscan', xaxis='freq', yaxis='amp', <br />
solutionTimeThresholdSeconds= 32.0, <br />
overlay='antenna', figfile='3c286_Band6.ms.Bscan.png')<br />
</source><br />
<br />
Figure 7 shows a colored curve for each antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. <br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012')<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24984e+11 Hz) is: 9.34567 +/- 0.0232932 (SNR = 401.218, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26984e+11 Hz) is: 9.24788 +/- 0.0233105 (SNR = 396.726, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.39016e+11 Hz) is: 8.88454 +/- 0.0252485 (SNR = 351.884, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.41016e+11 Hz) is: 8.8074 +/- 0.0255162 (SNR = 345.169, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24984e+11 Hz) is: 1.0092 +/- 0.00788338 (SNR = 128.016, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26984e+11 Hz) is: 0.996775 +/- 0.00825086 (SNR = 120.809, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.39016e+11 Hz) is: 0.954589 +/- 0.00847336 (SNR = 112.658, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.41016e+11 Hz) is: 0.953251 +/- 0.00871098 (SNR = 109.431, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 9.06839 +/- 0.0086788 <br />
(freq=232.892 GHz) spidx=-0.831341 +/- 0.0314601<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.978163 +/- 0.00182649 <br />
(freq=232.892 GHz) spidx=-0.837124 +/- 0.0611923<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', <br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:3c279_afterBandG2parang_lastcheck.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:J1310_afterBandG2parang_lastcheck.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration_for_CASA_4.3#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<equation id="Visibilities"> <br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
</equation><br />
Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In the equations we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
[[Image:J1337_afterBandG1_parang_casa4.3.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','scan','amp',field='',poln='/',subplot=111, figfile='G1_polnratio.png')<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_casa4.3.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use a function included in the almapolhelpers.py: qufromgain().<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=qufromgain('3c286_Band6.ms.G1')<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. The source polarization reported for all spws should be reasonably consistent.<br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Latitude = -23.0294371117<br />
Found as many as 5 fields.<br />
Found as many as 4 spws.<br />
Fld= 0 Spw= 0 (B=06, PA offset=-135.0deg) Gx/Gy= 0.992405226704 Q= 0.0120290146946 <br />
U= 0.0357751032462 P= 0.0377432802867 X= 35.7076525043<br />
Fld= 0 Spw= 1 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00460130437 Q= 0.0121161477985 <br />
U= 0.0342484704653 P= 0.0363284842332 X= 35.2588247799<br />
Fld= 0 Spw= 2 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00477793757 Q= 0.0132551417327<br />
U= 0.0365817946839 P= 0.0389092082244 X= 35.0411959343<br />
Fld= 0 Spw= 3 (B=06, PA offset=-135.0deg) Gx/Gy= 0.99729825908 Q= 0.013549612772 <br />
U= 0.0372694902549 P= 0.0396561081062 X= 35.0104353733<br />
For field id = 0 there are 4 good spws.<br />
Spw mean: Fld= 0 Q= 0.0127374792495 U= 0.0359687146626 (rms= 0.000673704330994 0.00112523439036 ) <br />
P= 0.038157460766 X= 35.2497952168<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:bG2_ph_freq_DA48DV23.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
# In CASA<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:afterKcr_DA48DV23.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running plotms again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. Figure 14 shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:imaginary_plane.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in gaincal, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_casa4.3.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737479249452035, 0.035968714662568159)<br />
Spw = 0: Found QU = [ 0.01259385 0.04101928]<br />
...KEEPING X-Y phase 84.4169495884 deg<br />
Spw = 1: Found QU = [-0.01261223 -0.04112571]<br />
...CONVERTING X-Y phase from -73.064852046 to 106.935147954 deg<br />
Spw = 2: Found QU = [ 0.01355364 0.0418143 ]<br />
...KEEPING X-Y phase 8.60411636622 deg<br />
Spw = 3: Found QU = [ 0.01344682 0.04199271]<br />
...KEEPING X-Y phase -21.7400889849 deg<br />
Ambiguity resolved (spw mean): Q= 0.0130516346544 U= 0.0414879983291 <br />
(rms= 0.000450227601812 0.000421946318322 ) P= 0.0434925185809 X= 36.2685199328<br />
Returning the following Stokes vector: [1.0, 0.013051634654402733, 0.041487998329102993, 0.0]<br />
</pre><br />
<br />
[[File:afterXY0_DA48DV23.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:afterXY0_imagreal.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.XY0amb','freq','phase',antenna='0',poln='X',subplot=121)<br />
plotcal('3c286_Band6.ms.XY0','freq','phase',antenna='0',poln='X',subplot=122)<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in Figure 14. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:gainsG1_G2polcal_casa4.3.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','time','amp',field='J1337-1257',poln='/',subplot=121)<br />
plotcal('3c286_Band6.ms.G2.polcal','time','amp',field='J1337-1257',poln='/',subplot=122)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000838681858308 U= -0.000234770753915 (rms= 0.000600891446491 0.000705059408831 ) <br />
P= 0.000870921676357 X= -7.81921742693<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,0,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,0,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
</source><br />
{|<br />
| [[Image:Df0_amp_casa4.3.png|200px|thumb|left|'''Fig. 19.''' Df0 amp vs freq]]<br />
| [[Image:Df0_real_casa4.3.png|200px|thumb|left|'''Fig. 20.''' Df0 real vs freq]]<br />
| [[Image:Df0_imag_casa4.3.png|200px|thumb|center|'''Fig. 21.''' Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True, niter=10000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True,niter=10000)<br />
<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:comparisonI.png | Stokes I.<br />
File:comparisonQ.png | Stokes Q.<br />
File:comparisonU.png | Stokes U.<br />
File:comparisonV.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:afterall_imagreal.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:afterAll_allantennas_amp_casa4.3.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:afterAll_allantennas_phase_casas4.3.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration]]<br />
| [[File:afterAll_1baseline_phase.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[T,T,F,F,F,F],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-1,1,-1,1], <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
|[[Image:field4_beforeaal_casa4.3.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
|[[Image:field4_afterall_casa43..png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_4.3 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_4.3&diff=312083C286 Band6Pol Calibration for CASA 4.32021-10-06T15:59:23Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging_for_CASA_4.3]]'''.<br />
<br />
'''This guide is designed for CASA 4.3.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casapy<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
<br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<br />
[[File:uid___A002_X85c183_X10a_plotants_casa4.3.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X84049a_X1e7 ]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.2 here]) for details on {{flagmanager}}.<br />
<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run {{fixsyscaltimes}} task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotcal}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<br />
[[File:Tsys_plotcal_antDA49.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<br />
<br />
[[File:Tsys_plotcal_antDV21.png|200px|thumb|right|'''Fig. 3.''' Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
plotcal(caltable=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',subplot=111, <br />
iteration='antenna')<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', <br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<br />
[[File:all_field_casa4.3.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
<br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<br />
[[File:ceres_uv_casa4.3.png|200px|thumb|right|'''Fig. 5.''' Amplitude vs uv-distance for Ceres]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<br />
[[File:field0_casa3.4.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
<br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in plotms.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [http://casa.nrao.edu/docs/userman/casa_cookbook005.html calibration chapter] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [http://casa.nrao.edu/docs/UserMan/casa_cookbook016.html here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3 N3256_Band3 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details). '''IMPROVE THIS'''<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<br />
[[File:bscan_casa3.4.png|200px|thumb|right|'''Fig. 7.''' Bandpass amp solutions]]<br />
<br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotbandpass(caltable = '3c286_Band6.ms.Bscan', xaxis='freq', yaxis='amp', <br />
solutionTimeThresholdSeconds= 32.0, <br />
overlay='antenna', figfile='3c286_Band6.ms.Bscan.png')<br />
</source><br />
<br />
Figure 7 shows a colored curve for each antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. <br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012')<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24984e+11 Hz) is: 9.34567 +/- 0.0232932 (SNR = 401.218, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26984e+11 Hz) is: 9.24788 +/- 0.0233105 (SNR = 396.726, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.39016e+11 Hz) is: 8.88454 +/- 0.0252485 (SNR = 351.884, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.41016e+11 Hz) is: 8.8074 +/- 0.0255162 (SNR = 345.169, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24984e+11 Hz) is: 1.0092 +/- 0.00788338 (SNR = 128.016, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26984e+11 Hz) is: 0.996775 +/- 0.00825086 (SNR = 120.809, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.39016e+11 Hz) is: 0.954589 +/- 0.00847336 (SNR = 112.658, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.41016e+11 Hz) is: 0.953251 +/- 0.00871098 (SNR = 109.431, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 9.06839 +/- 0.0086788 <br />
(freq=232.892 GHz) spidx=-0.831341 +/- 0.0314601<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.978163 +/- 0.00182649 <br />
(freq=232.892 GHz) spidx=-0.837124 +/- 0.0611923<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', <br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<br />
[[Image:3c279_afterBandG2parang_lastcheck.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
[[Image:J1310_afterBandG2parang_lastcheck.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
<br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration_for_CASA_4.3#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<equation id="Visibilities"> <br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
</equation><br />
<xr id="Visibilities" nolink/>. Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In <xr id="Visibilities" /> we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
[[Image:J1337_afterBandG1_parang_casa4.3.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
<br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','scan','amp',field='',poln='/',subplot=111, figfile='G1_polnratio.png')<br />
<br />
</source><br />
<br />
<br />
[[Image:G1_polnratio_scan_casa4.3.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
<br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use a function included in the almapolhelpers.py: qufromgain().<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=qufromgain('3c286_Band6.ms.G1')<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. The source polarization reported for all spws should be reasonably consistent.<br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Latitude = -23.0294371117<br />
Found as many as 5 fields.<br />
Found as many as 4 spws.<br />
Fld= 0 Spw= 0 (B=06, PA offset=-135.0deg) Gx/Gy= 0.992405226704 Q= 0.0120290146946 <br />
U= 0.0357751032462 P= 0.0377432802867 X= 35.7076525043<br />
Fld= 0 Spw= 1 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00460130437 Q= 0.0121161477985 <br />
U= 0.0342484704653 P= 0.0363284842332 X= 35.2588247799<br />
Fld= 0 Spw= 2 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00477793757 Q= 0.0132551417327<br />
U= 0.0365817946839 P= 0.0389092082244 X= 35.0411959343<br />
Fld= 0 Spw= 3 (B=06, PA offset=-135.0deg) Gx/Gy= 0.99729825908 Q= 0.013549612772 <br />
U= 0.0372694902549 P= 0.0396561081062 X= 35.0104353733<br />
For field id = 0 there are 4 good spws.<br />
Spw mean: Fld= 0 Q= 0.0127374792495 U= 0.0359687146626 (rms= 0.000673704330994 0.00112523439036 ) <br />
P= 0.038157460766 X= 35.2497952168<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<br />
[[Image:bG2_ph_freq_DA48DV23.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
<br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
# In CASA<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:afterKcr_DA48DV23.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
<br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running plotms again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. <xr id="imagreal_bg1kcr"/> shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<br />
[[File:imaginary_plane.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in gaincal, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<br />
[[File:XY0_comparison_casa4.3.png|200px|thumb|right|'''Fig. 15.''' Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
<br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737479249452035, 0.035968714662568159)<br />
Spw = 0: Found QU = [ 0.01259385 0.04101928]<br />
...KEEPING X-Y phase 84.4169495884 deg<br />
Spw = 1: Found QU = [-0.01261223 -0.04112571]<br />
...CONVERTING X-Y phase from -73.064852046 to 106.935147954 deg<br />
Spw = 2: Found QU = [ 0.01355364 0.0418143 ]<br />
...KEEPING X-Y phase 8.60411636622 deg<br />
Spw = 3: Found QU = [ 0.01344682 0.04199271]<br />
...KEEPING X-Y phase -21.7400889849 deg<br />
Ambiguity resolved (spw mean): Q= 0.0130516346544 U= 0.0414879983291 <br />
(rms= 0.000450227601812 0.000421946318322 ) P= 0.0434925185809 X= 36.2685199328<br />
Returning the following Stokes vector: [1.0, 0.013051634654402733, 0.041487998329102993, 0.0]<br />
</pre><br />
<br />
[[File:afterXY0_DA48DV23.png|200px|thumb|right|'''Fig. 16.''' Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
<br />
<br />
<br />
[[File:afterXY0_imagreal.png|200px|thumb|right|'''Fig. 17.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
<br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see Figure 15). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.XY0amb','freq','phase',antenna='0',poln='X',subplot=121)<br />
plotcal('3c286_Band6.ms.XY0','freq','phase',antenna='0',poln='X',subplot=122)<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (Figure 16) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in <xr id="imagreal_bg1kcr" />. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<br />
[[Image:gainsG1_G2polcal_casa4.3.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
<br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','time','amp',field='J1337-1257',poln='/',subplot=121)<br />
plotcal('3c286_Band6.ms.G2.polcal','time','amp',field='J1337-1257',poln='/',subplot=122)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000838681858308 U= -0.000234770753915 (rms= 0.000600891446491 0.000705059408831 ) <br />
P= 0.000870921676357 X= -7.81921742693<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,0,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,0,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
</source><br />
{|<br />
| [[Image:Df0_amp_casa4.3.png|200px|thumb|left|'''Fig. 19.''' <xr id="D_amp" nolink/>. Df0 amp vs freq]]<br />
| [[Image:Df0_real_casa4.3.png|200px|thumb|left|'''Fig. 20.''' <xr id="D_real" nolink/>. Df0 real vs freq]]<br />
| [[Image:Df0_imag_casa4.3.png|200px|thumb|center|'''Fig. 21.''' <xr id="D_imag" nolink/>. Df0 imag vs freq]]<br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True, niter=10000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True,niter=10000)<br />
<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:comparisonI.png | Stokes I.<br />
File:comparisonQ.png | Stokes Q.<br />
File:comparisonU.png | Stokes U.<br />
File:comparisonV.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
<br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<br />
[[File:afterall_imagreal.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
<br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| [[Image:afterAll_allantennas_amp_casa4.3.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]<br />
| [[Image:afterAll_allantennas_phase_casas4.3.png|200px|thumb|center|'''Fig. 25.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration]]<br />
| [[File:afterAll_1baseline_phase.png|200px|thumb|right|'''Fig. 26.''' Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]<br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (Figure 25). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (Figure 26). Comparing this result with Figure 16 we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[T,T,F,F,F,F],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-1,1,-1,1], <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
|[[Image:field4_beforeaal_casa4.3.png|200px|thumb|center|'''Fig. 27.''' 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]<br />
|[[Image:field4_afterall_casa43..png|200px|thumb|right|'''Fig. 28.''' 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]<br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_4.3 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=3C286_Band6Pol_Calibration_for_CASA_4.3&diff=312073C286 Band6Pol Calibration for CASA 4.32021-10-06T15:27:54Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Polarization]]<br />
<br />
==Overview==<br />
<br />
This portion of the '''[[3C286_Polarization]]''' CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file 3C286_Band6_UncalibratedData.tgz from [[3C286_Polarization#Obtaining the Data]]'''.<br />
<br />
Details of the ALMA observations are provided at '''[[3C286_Polarization]]'''.<br />
<br />
To skip to the imaging portion of the guide, see: '''[[3C286_Band6Pol_Imaging_for_CASA_4.3]]'''.<br />
<br />
'''This guide is designed for CASA 4.3.'''<br />
<br />
''' Note that the calibration procedure described here is intended to provide an initial example of the reduction of ALMA polarization data. It may be updated in future, and is not necessarily the exact procedure by which Early Science data will be calibrated. A future Science Verification (SV) dataset on 3C 138 will demonstrate an updated calibration strategy. Details of the 3C 138 SV data can be found on the [https://almascience.nrao.edu/alma-data/science-verification/science-verification-wiki-info Science Verification webpage]'''<br />
<br />
==Unpack the Data==<br />
<br />
Once the file 3C286_Band6_UncalibratedData.tgz has been download, unpack it as follows:<br />
<source lang="bash"><br />
# In a terminal outside CASA<br />
tar -xvzf 3C286_Band6_UncalibratedData.tgz<br />
<br />
cd 3C286_Band6_UncalibratedData<br />
<br />
<br />
# Start CASA<br />
casapy<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
We need to import some scripts we will use during the calibration<br />
<br />
<source lang="python"><br />
# In CASA<br />
path=os.getenv("CASAPATH").split(' ')[0]<br />
execfile(path+'/lib/python2.7/recipes/almahelpers.py')<br />
execfile(path+'/lib/python2.7/recipes/almapolhelpers.py')<br />
</source><br />
<br />
<br />
<br />
==Initial Inspection, ''A priori '' calibration==<br />
We will eventually concatenate the three datasets into one dataset. However, we will keep them separate for now, as some of the steps to follow require individual datasets (specifically, the application of the Tsys and WVR tables). We therefore start by defining an array "basename" that includes the names of the three files in chronological order. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue this command. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Define a python list holding the names of all of our data sets<br />
basename=["uid___A002_X85c183_X10a","uid___A002_X85c183_X51a", "uid___A002_X85c183_X822"]<br />
</source><br />
<br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the observatory. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task importasdm.<br />
<br />
<source lang="python"><br />
#In CASA<br />
for name in basename:<br />
importasdm(asdm = name, asis='*')<br />
</source><br />
<br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Loop over each element in the list and create summary file using listobs<br />
<br />
for name in basename:<br />
os.system('rm '+name+'.listobs.txt')<br />
listobs(vis=name+'.ms', listfile=name+'.listobs.txt', verbose=True)<br />
<br />
</source><br />
<br />
Note that after cutting and pasting a for-loop you often have to press return several times to execute.<br />
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu42.html#UserMansu42.html logger], or saved in a text file. Here is a snippet extracted from the full '''verbose''' {{listobs}} output for the first file in the list:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 3400886<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 636058<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 546034<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 840937<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4701150<br />
Spectral Windows: (25 unique spectral windows and 3 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
1 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 215242.188 -15625.000 2000000.0 214250.0000 1 XX YY<br />
2 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 214234.375 1796875.000 1796875.0 214234.3750 1 XX YY<br />
3 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 217242.188 -15625.000 2000000.0 216250.0000 2 XX YY<br />
4 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 216234.375 1796875.000 1796875.0 216234.3750 2 XX YY<br />
5 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 229257.813 15625.000 2000000.0 230250.0000 3 XX YY<br />
6 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 230234.375 1796875.000 1796875.0 230234.3750 3 XX YY<br />
7 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 231257.813 15625.000 2000000.0 232250.0000 4 XX YY<br />
8 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 232234.375 1796875.000 1796875.0 232234.3750 4 XX YY<br />
9 ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 224992.188 -15625.000 2000000.0 224000.0000 1 XX YY<br />
10 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223976.562 1781250.000 1781250.0 223976.5625 1 XX YY<br />
11 ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 226992.188 -15625.000 2000000.0 226000.0000 2 XX YY<br />
12 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225976.562 1781250.000 1781250.0 225976.5625 2 XX YY<br />
13 ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 239007.813 15625.000 2000000.0 240000.0000 3 XX YY<br />
14 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239976.563 1781250.000 1781250.0 239976.5625 3 XX YY<br />
15 ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 241007.813 15625.000 2000000.0 242000.0000 4 XX YY<br />
16 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241976.563 1781250.000 1781250.0 241976.5625 4 XX YY<br />
17 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 224000.0000 1 XX XY YX YY<br />
18 ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 223968.750 1781250.000 1781250.0 223968.7500 1 XX XY YX YY<br />
19 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 226000.0000 2 XX XY YX YY<br />
20 ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 225968.750 1781250.000 1781250.0 225968.7500 2 XX XY YX YY<br />
21 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 240000.0000 3 XX XY YX YY<br />
22 ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 239968.750 1781250.000 1781250.0 239968.7500 3 XX XY YX YY<br />
23 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 242000.0000 4 XX XY YX YY<br />
24 ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 241968.750 1781250.000 1781250.0 241968.7500 4 XX XY YX YY<br />
[...]<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 DA41 A079 12.0 m -067.45.13.6 -22.53.35.0 116.8369 -920.2897 22.6287 2225122.700426 -5439951.133460 -2481886.481390<br />
1 DA42 A081 12.0 m -067.45.23.9 -22.53.32.5 -174.5620 -842.8378 21.0900 2224863.872997 -5440088.015712 -2481814.531125<br />
2 DA43 A091 12.0 m -067.45.28.7 -22.53.24.2 -312.9125 -584.7726 23.7306 2224774.741865 -5440235.548469 -2481577.816103<br />
3 DA44 A068 12.0 m -067.45.20.6 -22.53.25.7 -82.4244 -631.7829 23.5830 2224981.097391 -5440131.252519 -2481621.067210<br />
4 DA45 A070 12.0 m -067.45.11.9 -22.53.29.3 166.1822 -743.4936 19.8824 2225193.449573 -5439993.765650 -2481722.541223<br />
5 DA46 A058 12.0 m -067.45.17.3 -22.53.32.0 12.7400 -827.0339 21.9689 2225039.860423 -5440023.554679 -2481800.313874<br />
6 DA47 A074 12.0 m -067.45.12.1 -22.53.32.0 161.8144 -828.6212 19.2711 2225176.657096 -5439964.248716 -2481800.726738<br />
7 DA48 A046 12.0 m -067.45.17.0 -22.53.29.3 21.4254 -742.7993 21.6766 2225060.201673 -5440050.345525 -2481722.599573<br />
8 DA49 A029 12.0 m -067.45.18.2 -22.53.25.8 -12.9141 -636.4555 22.1366 2225044.239474 -5440102.024044 -2481624.809292<br />
9 DA51 A082 12.0 m -067.45.08.3 -22.53.29.2 269.0424 -740.9548 16.2831 2225287.766879 -5439952.669219 -2481718.802316<br />
[...]<br />
</pre><br />
<br />
This output shows that five fields were observed: J1337-1257, J1256-0547=3C279, Ceres, J1310+3220, and 3c286. <br />
The final column of the listobs output gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. These intents are also used for pipeline processing.<br />
Field 0 (J1337-1257) will serve as the polarization calibrator; field 1 (3c279) as bandpass calibrator; field 2 (Ceres) will serve as the flux calibrator; field 3 (J1310+3220) as phase calibrator; and 3c286 (field 4) is, of course, the science target. <br />
<br />
Spectral windows are marked with ID numbers from 0 to 24. <br />
If you look at one of the listobs.txt files, you may notice that there are additional SpwIDs listed in the "Sources" section that are not listed in the "Spectral Windows" section<br />
(this is not shown in the snippet above).<br />
These spws (15-62) are related to WVR measurements for each antenna, so you will not need them for the calibration either. <br />
<br />
Spectral window 0 contains the WVR data; spws 1 to 16 also contain calibration data.<br />
The Tsys information is stored in spws 9, 11, 13, 15. <br />
In the listobs output shown above, you notice that field 3 (J1310+3220) has no Tsys measurements, since it is missing spws 9 to 16 in the spwIds. <br />
To increase observation efficiency, Tsys measurements are usually done either on the phase calibrator or the target. In the application of Tsys tables, we will apply to sources missing direct Tsys measurements the closest measurements available (see below). <br />
<br />
The full polarization data are in spectral windows 17,19,21,23, which have 64 channels each. The channel width is 31.250 MHz and the total bandwidth is 2 GHz.<br />
There are two spws (17 and 19) in the lower sideband (LSB), centered at 224.984 and 226.984 GHz, respectively;<br />
and two spectral windows (spw 21 and 23) in the upper sideband (USB), centered at 239.015 and 241.015, respectively. <br />
Spectral windows 18,20,22,24 contain channel averages of the data in spectral windows 17,19,21,23, respectively. These are not useful for the offline data reduction. <br />
<br />
Thirty-eight antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-37. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}}. <br />
<br />
<figure id="uid___A002_X85c183_X10a_plotants_casa4.3.png"><br />
[[File:uid___A002_X85c183_X10a_plotants_casa4.3.png|200px|thumb|right|'''Fig. 1.''' Plotants output for the dataset uid___A002_X84049a_X1e7 ]]<br />
</figure><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')<br />
</source> <br />
<br />
This will plot the antenna configuration on your screen (see Figure 1) as well as save it under the specified filename for future reference. We need to choose a reference antenna that is close to the center of the array (and is also stable and present for the entire observation). We will use antenna DV23 as reference antenna, and we define a variable to be used in the following tasks: <br />
<source lang="python"><br />
# In CASA<br />
refant='DV23'<br />
</source><br />
<br />
The other two datasets ("uid___A002_X85c183_X51a", "uid___A002_X85c183_X822") have been observed after "uid___A002_X85c183_X10a".<br />
From their listobs output we can see that, in both datasets, Ceres was not observed, and the field J1310+3220 does not have Tsys measurements.<br />
<br />
<br />
==== Flagging ====<br />
The first editing we will do is some ''a priori'' flagging with {{flagdata}}. ALMA data contain both the cross correlation and autocorrelation data, but here we are only interested in the cross-correlation data. <br />
<br />
Now we will loop over the datasets, running these flagging commands:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', autocorr=True, flagbackup=False)<br />
<br />
</source><br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'. Similarly, we can flag the scans corresponding to atmospheric calibration since we no longer need them:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='manual', <br />
intent='*POINTING*,*SIDEBAND_RATIO*,*ATMOSPHERE*', flagbackup=False)<br />
</source><br />
<br />
We also flag "shadowed" data where one antenna blocks the line of sight of another.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis=name+'.ms',mode='shadow',flagbackup=False)<br />
</source><br />
<br />
We will then store the current flagging state for each dataset using the {{flagmanager}}: <br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagmanager(vis=name+'.ms', mode='save', versionname='Apriori')<br />
</source><br />
<br />
See the Antennae casaguide (click [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.2 here]) for details on {{flagmanager}}.<br />
<br />
<br />
==== Tsys and WVR calibration table generation ====<br />
<br />
It turns out that two execution blocks (uid://A002/X85c183/X10a and X51a) out of the three taken on 2014-07-01 were affected by an ASDM SYSCal table issue. As a result, the Tsys values were incorrect and an entire scan is unintentionally flagged after applying the Tsys table. <br />
We need to run {{fixsyscaltimes}} task on these datasets before making the Tsys table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixsyscaltimes("uid___A002_X85c183_X10a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X51a.ms")<br />
fixsyscaltimes("uid___A002_X85c183_X822.ms")<br />
</source><br />
<br />
We will loop over the datasets, generating the System Temperature (Tsys) and Water Vapour Radiometer (WVR) calibration tables for each of them.<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging.<br />
Each ms file contains Tsys measurements; the task {{gencal}} is used to generate calibration tables from these measurements.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
gencal(vis = name+'.ms',<br />
caltable = name+'.ms.tsys',<br />
caltype = 'tsys')<br />
</source><br />
<br />
There are different ways to inspect Tsys calibration tables, in order to identify possibly wrong antenna solutions,<br />
see the TWhydraBand7 casaguide (click [http://casaguides.nrao.edu/index.php?title=TWHydraBand7 here]) for details.<br />
<br />
Here we use {{plotcal}} to interactively plot Tsys vs frequency for all antennas.<br />
<br />
<figure id="Tsys_DA49"><br />
[[File:Tsys_plotcal_antDA49.png|200px|thumb|right|'''Fig. 2.''' Example Tsys vs freq plot for antenna DA49 in the dataset uid___A002_X85c183_X10a]]<br />
</figure><br />
<br />
<figure id="Tsys_DV21"><br />
[[File:Tsys_plotcal_antDV21.png|200px|thumb|right|'''Fig. 3.''' <xr id="Tsys_DV21" nolink/>. Example Tsys vs freq plot for antenna DV21 in the dataset uid___A002_X85c183_X10a]]<br />
</figure><br />
<source lang="python"><br />
<br />
# In CASA<br />
plotcal(caltable=basename[0]+'.ms.tsys', xaxis='freq', yaxis='tsys', spw='',subplot=111, <br />
iteration='antenna')<br />
</source><br />
<br />
By using the next button it is possible to see the plots for each antenna.<br />
You will notice that all antennas have more or less the same behavior as antenna DA49 shown in Figure 2.<br />
Antenna DV21 clearly shows higher value of Tsys (see Figure 3), we will flag that antenna later in the combined dataset.<br />
<br />
The plots look acceptable, aside from the large amplitudes in the edge channels.<br />
We flag the first four channels of each spw from the calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
flagdata(vis = name+'.ms.tsys',<br />
spw='9:0~3, 11:0~3, 13:0~3, 15:0~3')<br />
<br />
</source><br />
<br />
<br />
To create the WVR calibration tables <br />
we run the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for name in basename:<br />
mylogfile = casalog.logfile()<br />
casalog.setlogfile(name+'.ms.wvrgcal')<br />
wvrgcal(vis = name+'.ms',<br />
caltable = name+'.ms.wvr',<br />
toffset = 0,<br />
tie = ['3c286,J1310+3220'],<br />
statsource = 'J1256-0547',<br />
wvrflag=['DA54'])<br />
casalog.setlogfile(mylogfile) <br />
</source><br />
<br />
* casalog.setlogfile(mylogfile): we used this method to send the log output back to the original CASA log file.<br />
<br />
==== WVR Correction and Tsys Calibration ====<br />
<br />
We will now apply the Tsys and the WVR calibration tables to the data with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. <br />
<br />
It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected.<br />
<br />
In addition to looping over data sets, we can define the list of unique source names and loop over these. We set '''field''' and '''gainfield''' to ensure that the appropriate Tsys and WVR calibrations applied to each field. For J1337-1257, J1256-0547, and 3c286, this means using calibration obtained specifically on each of these fields; for J1310+3220, the Tsys from 3c286 is used. We will only correct '''spw''' 17,19,21,23, our science windows, because we will drop the other data in a moment.<br />
<br />
We also need to tell applycal to use the Tsys information in spectral windows 9,11,13,15 for spectral window 17,19,21,23, respectively.<br />
We do so by specifying a list of spws. Normally we would use the almahelper function tsysspwmap (see [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3#Tsys here] for details); however, in this case it gives a wrong result so we just define this list by hand:<br />
<br />
<source lang="python"><br />
# In CASA<br />
tsysmap = [0,1,2,3,4,5,6,7,8,9,9,11,11,13,13,15,15,9,9,11,11,13,13,15,15]<br />
</source><br />
<br />
Here we cycle over all datasets and fields (except Ceres):<br />
<br />
<source lang="python"><br />
# In CASA<br />
field_names = ['J1337-1257','J1256-0547','3c286']<br />
for name in basename:<br />
# General calibrators that have their own Tsys<br />
for field_name in field_names:<br />
applycal(vis = name+'.ms',<br />
field = field_name,<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = [field_name,field_name],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
<br />
# J1310+3220 uses 3c286's Tsys<br />
applycal(vis = name+'.ms',<br />
field = 'J1310+3220',<br />
spw = '17,19,21,23',<br />
gaintable = [name+'.ms.tsys',name+'.ms.wvr'],<br />
gainfield = ['3c286','J1310+3220'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
As anticipated in the listobs description, Ceres has been observed only in the first dataset. We correct it separately here:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis = basename[0]+'.ms',<br />
field = 'Ceres',<br />
spw = '17,19,21,23',<br />
gaintable = [basename[0]+'.ms.tsys',basename[0]+'.ms.wvr'],<br />
gainfield = ['Ceres','Ceres'],<br />
interp = 'linear',<br />
spwmap = [tsysmap,[]],<br />
calwt = True,<br />
flagbackup = False)<br />
</source><br />
<br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
Now we can concatenate the three individual data sets of each day into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
comvis=[]<br />
for name in basename:<br />
comvis.append(name+'.ms')<br />
<br />
os.system('rm -rf 3c286_Band6_concat.ms*')<br />
concat(vis=comvis, concatvis='3c286_Band6_concat.ms')<br />
<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 17,19,21,23. This will get rid of the extraneous spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms')<br />
split(vis='3c286_Band6_concat.ms', outputvis='3c286_Band6.ms',<br />
datacolumn='corrected', spw='17,19,21,23')<br />
<br />
</source><br />
<br />
The WVR and Tsys tables are now applied in the DATA column of the new measurement sets.<br />
While running this task you will see a list of warnings. They are not relevant, you can ignore them.<br />
<br />
==Additional Data Inspection==<br />
<br />
We run a {{listobs}} on the resulting concatenated data sets to check that they contain the fields, spectral windows and observing times that we expected:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='3c286_Band6.ms', listfile='3c286_Band6.ms.listobs') <br />
<br />
</source><br />
<br />
Below is an extract from the dataset listobs: <br />
<br />
<pre style="background-color: #fffacd;"><br />
Observation: ALMA<br />
<br />
Telescope Observation Date Observer Project <br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91097e+09]knakanishi uid://A002/X845868/X11<br />
ALMA [ 4.91097e+09, 4.91098e+09]knakanishi uid://A002/X845868/X11<br />
Data records: 7499520 Total elapsed time = 11154.7 seconds<br />
Observed from 01-Jul-2014/21:21:27.3 to 02-Jul-2014/00:27:21.9 (UTC)<br />
[...]<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none J1337-1257 13:37:39.782784 -12.57.24.69312 J2000 0 1666560<br />
1 none J1256-0547 12:56:11.166560 -05.47.21.52458 J2000 1 535680<br />
2 none Ceres 13:29:06.769601 -01.11.10.65438 J2000 2 148800<br />
3 none J1310+3220 13:10:28.663200 +32.20.43.80000 J2000 3 773760<br />
4 none 3c286 13:31:08.288100 +30.30.32.96000 J2000 4 4374720<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) BBC Num Corrs <br />
0 ALMA_RB_06#BB_1#SW-01#FULL_RES 64 TOPO 224984.375 -31250.000 2000000.0 1 XX XY YX YY<br />
1 ALMA_RB_06#BB_2#SW-01#FULL_RES 64 TOPO 226984.375 -31250.000 2000000.0 2 XX XY YX YY<br />
2 ALMA_RB_06#BB_3#SW-01#FULL_RES 64 TOPO 239015.625 31250.000 2000000.0 3 XX XY YX YY<br />
3 ALMA_RB_06#BB_4#SW-01#FULL_RES 64 TOPO 241015.625 31250.000 2000000.0 4 XX XY YX YY<br />
[...]<br />
</pre><br />
<br />
The scans have been combined and their original ID numbers have been offset to make scan numbers unique. The information about the different datasets is kept in the ObservationID number, this means that in the next steps when combining all the scans we will need to add also combine='obs' (see below).<br />
<br />
Only the four scientific spws with full polarization are included.<br />
<br />
Now that the data are concatenated into one dataset, we will do some additional inspection with {{plotms}}. First we will plot amplitude versus channel, averaging over time and baselines in order to speed up the plotting process.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='channel', yaxis='amp', field='1', correlation='XX', <br />
averagedata=True, avgbaseline=True, avgtime='1e8', avgscan=True, coloraxis='spw')<br />
</source><br />
<br />
From these plots we see that the edge channels have high or low amplitudes in the different spws. We will use {{Flagdata}} to remove the edge channels from both sides of the bandpass:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', spw='*:0~5,*:58~63')<br />
</source><br />
<br />
Next, we flag the antenna with high Tsys, as identified during the Tsys calibration inspection:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna='DV21')<br />
</source><br />
<br />
<figure id="allfields"><br />
[[File:all_field_casa4.3.png|200px|thumb|right||'''Fig. 4.''' Amplitude vs time for the dataset 3c286_Band6.ms, spw 0 all channels averaged ]]<br />
</figure><br />
<br />
To identify antennas or time ranges that differ from the average values, we carefully inspect the data with {{plotms}}, plotting different axes and colorizing by different parameters. Don't forget to average the data if possible to speed the plotting process. <br />
<br />
We will look first at amplitude versus time, averaging over all channels and colorizing by field (see Figure 4). <br />
<br />
<figure id="ceres"><br />
[[File:ceres_uv_casa4.3.png|200px|thumb|right|'''Fig. 5.''' <xr id="ceres" nolink/>. Amplitude vs uv-distance for Ceres]]<br />
</figure><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='time', yaxis='amp', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', coloraxis='field', spw='0')<br />
<br />
</source><br />
<br />
<figure id="field0"><br />
[[File:field0_casa3.4.png|200px|thumb|right|'''Fig. 6.''' Amp vs time for field 0.]]<br />
</figure><br />
<br />
Figure 6 shows some time ranges which need to be flagged. It is possible to zoom in, or identify the time ranges by using the Tool Hover in plotms.<br />
<br />
Here is how to plot the amplitude of the flux calibrator as a function of the uv distance.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='uvdist', yaxis='amp', field='Ceres', correlation='XX,YY',<br />
averagedata=True, avgchannel='128', avgtime='1e8', coloraxis='antenna1')<br />
</source><br />
<br />
Continue to inspect the data with {{plotms}}, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. You will find the following flags are needed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA49') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', field='0,3', spw='1') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DA45', scan='48', field='0', spw='2')<br />
flagdata(vis='3c286_Band6.ms', antenna = 'DV14') #low gain<br />
flagdata(vis='3c286_Band6.ms', antenna='DV04', scan='93',field='0') <br />
flagdata(vis='3c286_Band6.ms', timerange='2014/07/02/00:25:00~2014/07/02/00:25:10')<br />
<br />
</source><br />
<br />
Now that we have completed the above inspection and flagging, we can start the data calibration.<br />
<br />
== Calibrating the data ==<br />
We can now begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. <br />
For <em>much</em> more discussion on the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see the [http://casa.nrao.edu/docs/userman/casa_cookbook005.html calibration chapter] in the CASA Reference Manual. The CASA software solves the Hamaker-Bregman-Sault Measurement Equation (ME), described in detail [http://casa.nrao.edu/docs/UserMan/casa_cookbook016.html here].<br />
<br />
For full polarization calibration there are additional calibration steps, as described in previous casaguides.<br />
<br />
We start with the determination of the bandpass and gains. The method is similar to what shown in other casaguides (e.g. the [http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3 N3256_Band3 casaguide]).<br />
<br />
We do not actually measure absolute phase values; instead, we fix the phase of a reference antenna to zero for both polarizations, which yields relative phases for all antennas. Difference among antennas in each polarization are preserved, so there is no effect on parallel-hand calibration.<br />
<br />
After that we will proceed with the polarization calibration which will correct for the cross-hand influenced terms:<br />
<math>D^{r}</math>, <math>X^{r}</math> and <math>P</math> (see below for details). '''IMPROVE THIS'''<br />
<br />
<br />
=== Bandpass ===<br />
<br />
We are now ready to calibrate the bandpass.<br />
<br />
See the<br />
[http://casaguides.nrao.edu/index.php?title=NGC3256_Band3_Calibration_for_CASA_4.3#Bandpass_Calibration N3256_Band3 casaguide] for a more detailed description about the bandpass calibration, including inspection of data before applying it.<br />
<br />
We first run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time. This short integration time is possible because the bandpass calibrator is a very bright point source, so we have very high signal-to-noise (SNR) and a perfect model. This will correct for any phase variations in the bandpass calibrator as a function of time, a step which will prevent decorrelation of the vector-averaged bandpass solutions. We will then apply these solutions on-the-fly when we run {{bandpass}}. <br />
<br />
Note that we use the average of channels 20 to 45 to increase our signal-to-noise in the determination of the antenna-based phase solutions. Averaging over a subset of channels near the center of the bandpass is acceptable when the phase variation as a function of channel is small, which it is here. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G0ph')<br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.G0ph', <br />
field='1', #3c279<br />
gaintype='G',solint='int',calmode='p',<br />
spw='*:20~45',<br />
refant=refant,smodel=[1,0,0,0])<br />
</source><br />
<br />
* <tt>caltable='*.G0ph'</tt> : the gain solutions will be stored in an external table.<br />
* <tt>field=1</tt> : we select the bandpass calibrator 3c279<br />
* <tt>gaintype='G'</tt>: we solve for gains for each polarization<br />
* <tt>solint='int'</tt>: we determine one solution for each integration<br />
* <tt>calmode='p'</tt>: we solve for phase only<br />
* <tt>smodel=[1,0,0,0]</tt>: we assume an unpolarized model. The values correspond to [I,Q,U,V].<br />
<br />
Now that we have a first measurement of the phase variations as a function of time, we can determine the bandpass solutions with {{bandpass}}. We will apply the phase calibration table on-the-fly with the parameter "gaintable". <br />
Do not worry about the message "Insufficient unflagged antennas", which relates to the flagged edge channels.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Bscan')<br />
bandpass(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.Bscan',<br />
field='1', #3c279<br />
solint='inf',combine='scan, obs',<br />
refant=refant,solnorm=True, <br />
gaintable=['3c286_Band6.ms.G0ph'],interp=['nearest'])<br />
<br />
</source><br />
<br />
* The default type of bandpass solution is bandtype='B' , which does a channel by channel solution for each specified spw<br />
*caltable = '*.Bscan': Output bandpass calibration table<br />
*gaintable = '.G0ph': gain calibration table to be applied on the fly<br />
*minblperant=3: Minimum number of baselines required per antenna for each solve<br />
*minsnr=2: Minimum signal-to-noise (SNR) for solutions<br />
*solint='inf',combine='scan, obs': this setting, sets the solution interval to the entire observation. The G0ph table corrects the phase variations as a function of time so that the per-channel bandpass solution is coherent and high SNR. To the default combine='scan', we need to add also 'obs' since the dataset we are working on is a combination of different observations.<br />
*solnorm=True: Normalize the bandpass amplitudes and phases of the corrections to unity<br />
*interp=['nearest']: apply the nearest solution from the calibration table. <br />
<br />
<figure id="Bscan"><br />
[[File:bscan_casa3.4.png|200px|thumb|right|'''Fig. 7.''' <xr id="Bscan" nolink/>. Bandpass amp solutions]]<br />
</figure><br />
<br />
We now plot the bandpass solutions:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotbandpass(caltable = '3c286_Band6.ms.Bscan', xaxis='freq', yaxis='amp', <br />
solutionTimeThresholdSeconds= 32.0, <br />
overlay='antenna', figfile='3c286_Band6.ms.Bscan.png')<br />
</source><br />
<br />
Figure 7 shows a colored curve for each antenna, in each of the four spws. In spw 1 and 3 the presence of atmospheric lines is clearly visible, <br />
The solutions seem reasonable, so we will apply them on-the-fly during gain calibration in the next section.<br />
<br />
===Gain Calibration===<br />
<br />
We first set the flux density for the flux calibrators using {{setjy}}. Ceres has been used as flux calibrator.<br />
The standard catalog Butler-JPL-Horizons 2012 model fits Ceres's resolved visibilities to an emission model appropriate for the distance to Ceres on the observation date. <br />
<br />
<source lang="python"><br />
# In CASA<br />
setjy(vis = '3c286_Band6.ms',<br />
field = '2', # Ceres<br />
spw = '0,1,2,3',<br />
standard = 'Butler-JPL-Horizons 2012')<br />
<br />
<br />
</source><br />
<br />
Now we determine a gain calibration for the bandpass, flux, and phase calibrators. We will calibrate the polarization calibrator separately.<br />
We solve first for phase only, applying the bandpass calibration on-the-fly.<br />
<br />
<source lang="python"><br />
# In CASA<br />
#Gaincal for 3C279 and Ceres and J1310+3220<br />
os.system('rm -rf 3c286_Band6.ms.G2ph')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2ph', <br />
gaintype= 'G',<br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
calmode = 'p', <br />
gaintable=['3c286_Band6.ms.Bscan'],<br />
interp=['nearest'])<br />
<br />
</source><br />
*os.system('rm -rf *.G2ph'): removes the table if already existing. <br />
*caltable = '*.G2ph': the output gain calibration table<br />
*gaintype = 'G': is the default type of gain solution , which determine gains for each polarization and spectral window<br />
*field='1,2,3': to specify the bandpass, flux, and phase calibrators.<br />
*calmode = 'p': to solve for phase only.<br />
*solint='int': this setting will solve for one solution per integration<br />
*gaintable = ['*.Bscan']: we apply the bandpass calibration on-the-fly<br />
*interp=['nearest']: as before we use the nearest solution in the table<br />
<br />
We solve for the amplitude, applying on-the-fly the bandpass and the phase corrections just computed.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2amp')<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2amp', <br />
field='1,2,3',<br />
solint='int',refant=refant,<br />
gaintype = 'T',<br />
calmode = 'a', #solve amplitude using G2ph<br />
gaintable=['3c286_Band6.ms.Bscan', '3c286_Band6.ms.G2ph'],<br />
interp=['nearest', 'nearest'])<br />
<br />
</source><br />
<br />
*gaintype='T': this time we obtain one solution for both polarizations. <br />
<br />
We now bootstrap the flux density of the secondary calibrators from that of Ceres using the task {{fluxscale}}. <br />
The new flux table *.flux, containing the correctly-scaled gains, will replace the previous *.G2amp table when we apply the calibration to the data later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fluxscale(vis='3c286_Band6.ms', <br />
caltable='3c286_Band6.ms.G2amp',<br />
fluxtable = '3c286_Band6.ms.flux',<br />
reference='2', #Ceres<br />
transfer='1,3') #'J1256-0547', 'J1310+3220'<br />
<br />
</source><br />
<br />
*caltable='*.G2amp': this is the input table we want to scale<br />
*fluxscale='*.flux': output table, containing scaled gains.<br />
*reference='Ceres': here we specify the source with the known flux density (Ceres or Titan)<br />
*transfer='1,3': these are the sources whose amplitude gains are to be rescaled: 'J1256-0547', and 'J1310+3220'<br />
<br />
The logger produces the following output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Ceres<br />
Found transfer field(s): J1256-0547 J1310+3220<br />
Flux density for J1256-0547 in SpW=0 (freq=2.24984e+11 Hz) is: 9.34567 +/- 0.0232932 (SNR = 401.218, N = 28)<br />
Flux density for J1256-0547 in SpW=1 (freq=2.26984e+11 Hz) is: 9.24788 +/- 0.0233105 (SNR = 396.726, N = 28)<br />
Flux density for J1256-0547 in SpW=2 (freq=2.39016e+11 Hz) is: 8.88454 +/- 0.0252485 (SNR = 351.884, N = 28)<br />
Flux density for J1256-0547 in SpW=3 (freq=2.41016e+11 Hz) is: 8.8074 +/- 0.0255162 (SNR = 345.169, N = 28)<br />
Flux density for J1310+3220 in SpW=0 (freq=2.24984e+11 Hz) is: 1.0092 +/- 0.00788338 (SNR = 128.016, N = 27)<br />
Flux density for J1310+3220 in SpW=1 (freq=2.26984e+11 Hz) is: 0.996775 +/- 0.00825086 (SNR = 120.809, N = 26)<br />
Flux density for J1310+3220 in SpW=2 (freq=2.39016e+11 Hz) is: 0.954589 +/- 0.00847336 (SNR = 112.658, N = 27)<br />
Flux density for J1310+3220 in SpW=3 (freq=2.41016e+11 Hz) is: 0.953251 +/- 0.00871098 (SNR = 109.431, N = 27)<br />
Fitted spectrum for J1256-0547 with fitorder=1: Flux density = 9.06839 +/- 0.0086788 <br />
(freq=232.892 GHz) spidx=-0.831341 +/- 0.0314601<br />
Fitted spectrum for J1310+3220 with fitorder=1: Flux density = 0.978163 +/- 0.00182649 <br />
(freq=232.892 GHz) spidx=-0.837124 +/- 0.0611923<br />
</pre><br />
<br />
{{fluxscale}} prints to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument. You should examine the output to ensure that the fluxes are reasonable. <br />
The ALMA Calibrator Source Catalogue [http://almascience.eso.org/sc/] can be used to check whether the derived flux densities are reasonable. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.<br />
<br />
We apply the solutions so far, for inspection purposes only. We will apply all the calibrations at the end:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='1,2,3',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux'],<br />
interp=['nearest','linear','linear'], parang=False)<br />
<br />
</source><br />
<br />
*calwt=True: the weight will be calibrated.<br />
*interp=['nearest','linear','linear']: this list indicates the interpolation method to be used when applying the corresponding table. For bandpass table the nearest solution to the source is taken, while for the gain tables a linear interpolation in time is used.<br />
<br />
To inspect the results we plot the corrected data for the bandpass and phase calibrators:<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='1', spw='0', xaxis='parang', yaxis='amp', <br />
correlation='XX,YY', ydatacolumn='corrected', <br />
averagedata=True, avgchannel='128', avgbaseline=True, coloraxis='corr')<br />
</source><br />
<br />
<figure id="3c279_BG2"><br />
[[Image:3c279_afterBandG2parang_lastcheck.png|200px|thumb|left|'''Fig. 8.''' 3C279 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
</figure><br />
<figure id="J1310_BG2"><br />
[[Image:J1310_afterBandG2parang_lastcheck.png|200px|thumb|center|'''Fig. 9.''' J1310+3220 amp vs time after bandpass and gain calibration. Spw 0: all baselines have been averaged.]]<br />
</figure><br />
<br />
<br />
Figure 8 and Figure 9 show the XX and YY data for the bandpass and phase calibrators after the parallel-hand calibration. <br />
The effect of source polarization is clearly evident.<br />
<br />
===Polarization Calibration===<br />
<br />
A list of useful references about mathematical and theoretical details of polarization calibration is given in the [[3C286_Band6Pol_Calibration_for_CASA_4.3#References | last section of this guide]]. Here we introduce some basic concepts needed to guide you through the calibration steps.<br />
<br />
Correlations between circularly polarized feeds are sensitive to total intensity and the source circular polarization, while <br />
correlations between linearly polarized feeds are sensitive predominantly to total intensity plus a contribution from the linear polarization. Since the large majority of extragalactic radio sources are quite weakly circularly polarized, the separation of the calibration of the two parallel hand gains from each other and from the instrumental polarization is much easier in interferometers using circularly polarized feeds. For interferometers with linearly polarized feeds the situation is more complex: a contribution from Stokes ''Q'', ''U'', and parallactic angle ''ψ'' appears in the real part of ''all'' correlations. This complicates the analysis somewhat, but also provides better constraints for estimating and correcting the instrumental polarization.<br />
<br />
The polarization response can be described assuming that each feed is perfectly coupled to the polarization state to which it is sensitive, with the addition of a complex factor times the orthogonal polarization; this is called the "leakage" or "D-term" model.<br />
The instrumental contribution to the cross-polarized interferometer response (i.e., the effect due to leakage) is independent of parallactic angle, whereas the contribution from the source rotates with parallactic angle for alt-az mount antennas. This makes it possible to uniquely separate the source and instrumental contributions to the polarized response.<br />
<br />
In the limit of nearly perfect feeds (any higher order terms involving instrumental polarization can be ignored) and zero circular polarization, the linearized approximation for crossed linearly polarized feeds on one baseline is: <br />
<br />
<center><br />
<equation id="Visibilities"> <br />
<br />
<math>\ V_{XX} = (I + Q_{\psi}) + U_{\psi}(d_{Xj}^{*} + d_{Xi})</math><br />
<br />
<math>\ V_{XY} = U_{\psi} + I(d_{Yj}^{*} + d_{Xi}) + Q_{\psi}</math><br />
<math>\ (d_{Yj}^{*} - d_{Xi})</math><br />
<br />
<math>\ V_{YX} = U_{\psi} + I(d_{Yi} + d_{Xj}^{*}) + Q_{\psi}</math><br />
<math>\ (d_{Yi} - d_{Xj}^{*}) </math><br />
<br />
<math>\ V_{YY} = (I - Q_{\psi}) + U_{\psi}(d_{Yi} + d_{Yj}^{*})</math><br />
<br />
<br />
</equation><br />
<xr id="Visibilities" nolink/>. Visibilities for one baseline. Linearized approximation for linear feed.<br />
</center><br />
<br />
<br />
where the functions: <br />
<math> <br />
Q_{\psi}= Q cos2\psi + U sin2\psi <br />
</math> <br />
and <br />
<math> <br />
U_{\psi} = -Q sin2\psi + U cos2\psi <br />
</math> <br />
show the dependence of the source polarization on the parallactic angle (see Figure 8).<br />
<br />
In <xr id="Visibilities" /> we can recognize a constant complex offset proportional to I in cross-hands visibilities, and a time-dependent contribution of linear polarization from the source, scaled by the D-terms, in all correlations.<br />
In order to separate the source and instrumental contributions to the correlations, observations with an array using linear feeds need to include frequent measurements of an unresolved calibrator over a wide range of parallactic angle.<br />
<br />
The following steps will allow us to determine all these factors to properly correct the data. <br />
<br />
<br />
====Gains for the polarization calibrator====<br />
<br />
We start examining the polarization calibrator. Since we are concerned only about its fractional polarization, we do not worry about its absolute total flux density, and just use I=1.0.<br />
The first gains we determine will absorb all the polarization contributions, and we will revise them once the source polarization is correctly estimated.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G1')<br />
gaincal(vis = '3c286_Band6.ms',<br />
caltable = '3c286_Band6.ms.G1',<br />
field = '0',<br />
solint = 'int',<br />
refant = refant,<br />
smodel=[1,0,0,0],<br />
gaintable = '3c286_Band6.ms.Bscan',<br />
interp='nearest')<br />
<br />
</source><br />
<br />
We explicitly show some of the default parameters, but they could also be left out -- when a parameter is not specified, its default value is used.<br />
<br />
<figure id="J1337_BG1"><br />
[[Image:J1337_afterBandG1_parang_casa4.3.png|200px|thumb|right|'''Fig. 10.''' J1337-1257 amp vs parang after the application of the bandpass and G1 tables.]]<br />
</figure> <br />
<br />
* caltable = '*.G1': the output table. This table will be revised later on, when the source polarization will be estimated.<br />
* calmode = 'ap': this is the default value. We are solving for both amplitude and phase.<br />
* gaintype='G': this is the default value. We are determining gains for each polarization.<br />
* solint = 'int': one solution per integration.<br />
* smodel=[1,0,0,0]: we are using an unpolarized model with a Stokes I flux equal to 1. <br />
<br />
To inspect the results we generate a partially calibrated dataset, applying this gain table, as well as the bandpass, to the polarization calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'], parang=False)<br />
<br />
<br />
</source><br />
<br />
<br />
<br />
Figure 10 shows the polarization calibrator's amplitude versus time, after the application of the bandpass and the gain table G1, we just computed.<br />
<br />
The source's linear polarization, which rotates with parallactic angle as a function of time, has been absorbed by the gains, since we assumed an unpolarized model for the source.<br />
<br />
The presence of source polarization in the gains can be seen by plotting the amp vs time for this table using poln='/', which forms the complex polarization ratio.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','scan','amp',field='',poln='/',subplot=111, figfile='G1_polnratio.png')<br />
<br />
</source><br />
<br />
<figure id="G1_polnratio"><br />
[[Image:G1_polnratio_scan_casa4.3.png|thumb|'''Fig. 11.''' Polarization ratio from the table 3c286_Band6.ms.G1 versus scan number]]<br />
</figure><br />
<br />
The antenna-based complex polarization ratio reveals the sinusoidal (in parallactic angle) variation, which clearly shows the presence of polarization in the source.<br />
<br />
In order to extract the source polarization information hidden in the gains, we use a function included in the almapolhelpers.py: qufromgain().<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
qu=qufromgain('3c286_Band6.ms.G1')<br />
<br />
</source><br />
<br />
This script estimates the Q and U amplitudes from the gains, for each spectral window. The results are stored in the python dictionary qu, which will be used later on. The source polarization reported for all spws should be reasonably consistent.<br />
This estimate is not as good as can be obtained from the cross-hands, since it relies on the gain amplitude polarization ratio being stable, which may not be precisely true. However, it will be useful later on in removing an ambiguity that occurs in the cross-hand estimates.<br />
<br />
The output will be something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Latitude = -23.0294371117<br />
Found as many as 5 fields.<br />
Found as many as 4 spws.<br />
Fld= 0 Spw= 0 (B=06, PA offset=-135.0deg) Gx/Gy= 0.992405226704 Q= 0.0120290146946 <br />
U= 0.0357751032462 P= 0.0377432802867 X= 35.7076525043<br />
Fld= 0 Spw= 1 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00460130437 Q= 0.0121161477985 <br />
U= 0.0342484704653 P= 0.0363284842332 X= 35.2588247799<br />
Fld= 0 Spw= 2 (B=06, PA offset=-135.0deg) Gx/Gy= 1.00477793757 Q= 0.0132551417327<br />
U= 0.0365817946839 P= 0.0389092082244 X= 35.0411959343<br />
Fld= 0 Spw= 3 (B=06, PA offset=-135.0deg) Gx/Gy= 0.99729825908 Q= 0.013549612772 <br />
U= 0.0372694902549 P= 0.0396561081062 X= 35.0104353733<br />
For field id = 0 there are 4 good spws.<br />
Spw mean: Fld= 0 Q= 0.0127374792495 U= 0.0359687146626 (rms= 0.000673704330994 0.00112523439036 ) <br />
P= 0.038157460766 X= 35.2497952168<br />
</pre><br />
<br />
From this output we see that the Q and U values reported for all spws are consistent at ~0.1% level.<br />
<br />
====Cross-hand delay==== <br />
<br />
When the data have more than one channel per spectral window, it is important to be sure there is no large cross-hand (XY, YX) delay still present in the data. <br />
The gain and bandpass calibration will only correct for parallel-hand (XX, YY) delay residuals, since the two polarizations are referenced independently. <br />
Plots of cross-hand phases as a function of frequency for a strongly polarized source (i.e., a source whose polarization dominates the instrumental polarization) will show the cross-hand delay as a phase slope with frequency (see Figure 12). This slope will be the same magnitude on all baselines, but with different sign in the two cross-hand correlations (XY, YX). <br />
<br />
Note that in this dataset you can produce even higher-SNR plots of the cross-hand delay by using 3C279. However, in this guide we proceed by using the polarization calibrator for all of the polarization calibration steps (cross-hand delay, XY-phase, and leakages).<br />
<br />
<figure id="kcross_1bl"><br />
[[Image:bG2_ph_freq_DA48DV23.png|200px|thumb|right|'''Fig. 12.''' Baseline DA41&DA48: XY and YX phase vs freq before the cross-hand delay calibration.]]<br />
</figure><br />
<br />
<br />
We can plot the corrected column of the dataset (we applied so far the bandpass and the G1 tables) to show this effect on one baseline:<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xaxis='freq', yaxis='phase',<br />
field='0', <br />
avgtime='1e9',<br />
correlation='XY,YX',<br />
spw='', antenna='DA48',<br />
iteraxis='baseline',coloraxis='corr',<br />
plotrange=[0,0,-180,180])<br />
</source><br />
<br />
This cross-hand delay can be estimated using the gaintype=’KCROSS’ mode of {{gaincal}}. <br />
<br />
For the cross-hand delay calculation we need to choose a scan where the source's cross-hand contribution is maximum (in absolute value), since this will minimize the mean effect of instrumental polarization. This scan would be the one where the gain ratio is near the mean value in Figure 11: scan 48.<br />
<br />
<source lang="python"><br />
<br />
os.system('rm -rf 3c286_Band6.ms.Kcrs')<br />
# In CASA<br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Kcrs',<br />
selectdata=True,<br />
scan='48',<br />
gaintype='KCROSS',<br />
solint='inf',refant=refant, <br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1'],<br />
interp=['nearest','linear'])<br />
<br />
<br />
</source><br />
<br />
<figure id="afterkcross_1bl"><br />
[[Image:afterKcr_DA48DV23.png|200px|thumb|right|'''Fig. 13.''' Baseline DA41&DA48: XY and YX phase vs freq after the cross-hand delay calibration.]]<br />
</figure><br />
<br />
*caltable='*.Kcrs': the output table.<br />
*selectdata=True: this parameter allows further selection data parameters. Selection based on antennas, timerange, uvrange, scan, etc.<br />
*scan='48': we select the scan with strong polarization, and we calculate the gain solution using only this scan.<br />
*gaintype='KCROSS': solves for a global cross-hand delay.<br />
*smodel=[1,0,1,0]: here we are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity. <br />
*gaintable=['*.Bscan','*.G1']: we apply on-the-fly the bandpass and the first gain tables<br />
<br />
We apply this new cross-hand delay calibration to the dataset, along with the bandpass and gain tables:<br />
<br />
<source lang="python"><br />
applycal(vis='3c286_Band6.ms', <br />
field='0',<br />
calwt=True,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1', '3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear', 'nearest'])<br />
<br />
</source><br />
<br />
Running plotms again we can verify the effect of this calibration (Figure 13). These plots are a nice example of the effect of Kcrs correction on the cross-hand phases:<br />
by applying the Kcrs table to the data, the phase slopes vs frequency are corrected. The remaining frequency structure in the cross-hand phase is due to the remaining cross-hand bandpass phase, as well as the instrumental polarization.<br />
<br />
====XY-phase and QU====<br />
<br />
We have so far computed the bandpass and gain tables adequate for the parallel hands, and the Kcross table which corrects the cross-hands for a systematic linear phase slope across each spw. In computing bandpass and gains we don't measure absolute phase values; instead, we set the phase of the reference antenna to zero in both polarizations, yielding relative phases for all other antennas. As a result, a single residual phase bandpass relating the phase of the two hands of polarization in the reference antenna remains in the cross-hands of all baselines. The Kcross solution has already accounted for any linear phase slope in this phase bandpass, but we need to solve for any non-linear phase bandpass shape in the XY-phase so that both cross- and parallel-hands can be combined to extract correct Stokes parameters.<br />
<br />
To visualize the effect of this XY-phase offset on the data, we can plot the real and imaginary part of the data corrected for bandpass, gains, and cross-hands delay. <xr id="imagreal_bg1kcr"/> shows the cross-hands visibilities of the polarization calibrator, averaged per scan and over all baselines, in one spw (averaged in channel). The XY-phase is clearly visible as the slope of cross-hand visibilities in the complex plane. The XY and YX correlations have the same slope with ''opposite'' signs because they are complex conjugates of each other. For SNR purposes, this plot is averaged in channel; each channel has its own slope, in fact. The Kcross solution ensures that the channel-average is reasonably coherent.<br />
<br />
<figure id="imagreal_bg1kcr"><br />
[[File:imaginary_plane.png|200px|thumb|right|'''Fig. 14.''' Imag vs real. Calibration tables applied to the data are B, G1, Kcr.]]<br />
</figure><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms',<br />
ydatacolumn='corrected',<br />
xdatacolumn='corrected',<br />
xaxis='real', yaxis='imag',<br />
field='0',<br />
avgtime='1e9',<br />
avgchannel='128',<br />
avgbaseline=True,<br />
correlation='XY,YX',<br />
spw='3', <br />
coloraxis='corr',<br />
plotrange=[-0.06,0.06,-0.06,0.06])<br />
</source><br />
<br />
By using the XYf+QU solve in gaincal, we can estimate both the XY-phase offset and the source polarization from the cross-hands.<br />
The XYf+QU solve averages all baselines together and first solves for the XY-phase as a function of channel. <br />
It then solves for a channel-averaged source polarization (with the channel-dependent XY-phase corrected). <br />
<br />
Note that we only require a sufficiently strongly polarized source for this calibration, so that there is an adequate reference signal against which to measure the instrumental effects. It is not necessary to know the precise polarization properties of the calibrator, as long as there is sufficient time for the source polarization contribution to change via parallactic angle rotation.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.XY0amb') <br />
gaincal(vis='3c286_Band6.ms',caltable='3c286_Band6.ms.XY0amb', <br />
field='0',<br />
gaintype='XYf+QU',<br />
solint='inf',<br />
combine='scan,obs',<br />
preavg=300, <br />
refant=refant,<br />
smodel=[1,0,1,0],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs'],<br />
interp=['nearest','linear','nearest'])<br />
</source><br />
<br />
*caltable='*.XY0amb': the output table. In the name we just want to remember that there could still be a 180 deg ambiguity in the solutions obtained in this table.<br />
*gaintype='XYf+QU': estimate both the XY-phase offset and source polarization from the cross-hands<br />
*solint='inf'; combine='scan,obs': we get one solution for the all dataset. The option "obs" takes into account the different Obs ID which are usually present in different scheduling blocks.<br />
*preavg=300: pre-averaging interval (sec). This limits averaging within the solution interval to within scan boundaries, since the parallactic angle changes from scan to scan are important for the solution.<br />
*smodel=[1,0,1,0]: we again are using a polarized model (a non-zero value for U is assumed). This is just to enforce the assumption of non-zero source polarization signature in the cross-hands in the ratio between data and model. It is not important to specify the polarization Stokes parameters correctly, since here we are only solving for a phase-like quantity.<br />
*gaintable=['*.Bscan','*.G1','*.Kcrs']: we apply on-the-fly the bandpass, the gain and the cross-hand delay tables.<br />
<br />
<figure id="XYcomparison"><br />
[[File:XY0_comparison_casa4.3.png|200px|thumb|right|'''Fig. 15.''' <xr id="XYcomparison" nolink/>. Comparison between the XY-phase tables before and after resolving the ambiguity]]<br />
</figure><br />
<br />
The output table contains the values of Q, U and the XY-phase. A <br />
degeneracy may still be present<br />
(XY-phase, Q, U) ---> (XY-phase + π, -Q, -U). <br />
This ambiguity can be resolved by using the Q and U values derived from the <br />
gains in G1 table (qufromgain() results obtained above). We use the <br />
script xyamb() utility contained in the almapolhelpers.py.<br />
<br />
<source lang="python"><br />
# In CASA<br />
S=xyamb(xytab='3c286_Band6.ms.XY0amb',qu=qu[0],xyout='3c286_Band6.ms.XY0')<br />
</source><br />
<br />
This is an extract of the output of the xyamb() utility:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Expected QU = (0.012737479249452035, 0.035968714662568159)<br />
Spw = 0: Found QU = [ 0.01259385 0.04101928]<br />
...KEEPING X-Y phase 84.4169495884 deg<br />
Spw = 1: Found QU = [-0.01261223 -0.04112571]<br />
...CONVERTING X-Y phase from -73.064852046 to 106.935147954 deg<br />
Spw = 2: Found QU = [ 0.01355364 0.0418143 ]<br />
...KEEPING X-Y phase 8.60411636622 deg<br />
Spw = 3: Found QU = [ 0.01344682 0.04199271]<br />
...KEEPING X-Y phase -21.7400889849 deg<br />
Ambiguity resolved (spw mean): Q= 0.0130516346544 U= 0.0414879983291 <br />
(rms= 0.000450227601812 0.000421946318322 ) P= 0.0434925185809 X= 36.2685199328<br />
Returning the following Stokes vector: [1.0, 0.013051634654402733, 0.041487998329102993, 0.0]<br />
</pre><br />
<br />
<figure id="afterXY0"><br />
[[File:afterXY0_DA48DV23.png|200px|thumb|right|'''Fig. 16.''' <xr id="afterXY0" nolink/>. Baseline DA41&DA48: XY and YX phase vs channels after the XY0 calibration.]]<br />
</figure><br />
<br />
<figure id="afterXY0_imagreal"><br />
[[File:afterXY0_imagreal.png|200px|thumb|right|'''Fig. 17.''' <xr id="afterXY0_imagreal" nolink/>. Imag vs real. Calibration tables applied to the data are B, G1, Kcr, XY0.]]<br />
</figure><br />
<br />
The utility produces a new table with the unambiguous values of the XY-phases as well as a python variable S, reported in the last line,<br />
containing the mean source model (Stokes I=1; Q/I; U/I; V=0). This <br />
can be used in a revision of the gain calibration and instrumental <br />
polarization calibration. <br />
<br />
The XY-phase value in spw 1 has been converted from -73 to 107 degrees, as reported in the output. This can be seen when we compare the phases from both tables (see <xr id="XYcomparison" />). <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.XY0amb','freq','phase',antenna='0',poln='X',subplot=121)<br />
plotcal('3c286_Band6.ms.XY0','freq','phase',antenna='0',poln='X',subplot=122)<br />
</source><br />
<br />
To clarify the effect produced by this calibration on the cross-hand phases, we can apply it to the data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='3c286_Band6.ms',<br />
field='0',<br />
calwt=[True,True,False,False],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G1','3c286_Band6.ms.Kcrs',<br />
'3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear','nearest','nearest'])<br />
</source><br />
<br />
Plotting the corrected phase versus frequency (<xr id="afterXY0" />) and comparing the result with Figure 13, we see that the application of this table does indeed remove spw- and channel-dependent residual phases from the data. The remaining phase offsets (per baseline and correlation) are due to the instrumental polarization effects which will be calibrated below.<br />
<br />
In terms of the real and imaginary parts, the application of the XY-phase calibraiton removes the slope seen in <xr id="imagreal_bg1kcr" />. What we can see now in the complex plane (Figure 17) is the effect of parallactic angle variation of the source polarization occurring entirely in the real part (over a range between -0.06 and 0.06), with a small complex offset due to the mean instrumental polarization that will be solved for below.<br />
<br />
====Revise gain with good source pol estimate====<br />
<br />
Now we revise the gain calibration using the full polarization source model we derived above.<br />
In this way we will remove from the gains any signature of the source polarization, which is now correctly accounted for in the model "S".<br />
Cross-hands are not used here, so cross-hand calibration tables are not needed.<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.G2.polcal') <br />
gaincal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.G2.polcal', <br />
field='0',<br />
solint='int',<br />
refant=refant,<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan'],interp=['nearest'],<br />
parang=True)<br />
<br />
<br />
</source><br />
<br />
<figure id="G1_G2_ratio"><br />
[[Image:gainsG1_G2polcal_casa4.3.png|200px|thumb|right|'''Fig. 18.''' Complex polarization ratio comparison between table G1 and the revised G2.polcal]]<br />
</figure><br />
<br />
*caltable='*.G2.polcal': the output table. The name reminds us that these are gains for the polarization calibrator.<br />
*smodel=S: here we use as source model the full Stokes model we obtained above.<br />
*parang=True: this time the parallactic angle dependence is corrected for in computing the gains.<br />
<br />
With parang=True the supplied source linear polarization is properly rotated in the parallel-hand visibility model. This new gain solution can be plotted with poln=’/’ as above to show that the source polarization is no longer distorting it. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal('3c286_Band6.ms.G1','time','amp',field='J1337-1257',poln='/',subplot=121)<br />
plotcal('3c286_Band6.ms.G2.polcal','time','amp',field='J1337-1257',poln='/',subplot=122)<br />
</source><br />
<br />
In Figure 18 a small distortion can be noticed, in the central scans of table G2.polcal. It is due to antenna DV02 in spw 1, which shows a variation in the ratio slightly different than the average.<br />
<br />
Just to check: if qufromgain() is run on this new gain table, the reported source polarization should be statistically indistinguishable from zero. <br />
<br />
<source lang="python"><br />
# In CASA<br />
qufromgain('3c286_Band6.ms.G2.polcal')<br />
</source><br />
<br />
We show here just the average values obtained, which can be compared with the last line of qufromgain() results, reported above.<br />
We can see that now the Q and U values are comparable to the rms.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Spw mean: Fld= 0 Q= 0.000838681858308 U= -0.000234770753915 (rms= 0.000600891446491 0.000705059408831 ) <br />
P= 0.000870921676357 X= -7.81921742693<br />
</pre><br />
<br />
====Solving for the Leakage Terms====<br />
<br />
Finally, we can now solve for the instrumental polarization. <br />
This solve will produce an absolute instrumental polarization solution that is registered to the assumed source polarization and prior calibrations.<br />
To do that we apply all the previous calibrations in order to extract the D-terms from the corrected I, Q and U.<br />
<br />
The task used to estimate the D-terms is {{polcal}}:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.ms.Df0*') <br />
polcal(vis='3c286_Band6.ms',<br />
caltable='3c286_Band6.ms.Df0', <br />
field='0', #J1337-1257<br />
solint='inf',combine='obs,scan',<br />
preavg=300,<br />
poltype='Dflls',<br />
refant='', #solve absolute D-term<br />
smodel=S,<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
gainfield=['', '', '', ''],<br />
interp=['nearest','linear','nearest','nearest'])<br />
<br />
</source><br />
<br />
*caltable='*.Df0': the output table.<br />
*poltype='Dflls': frequency-dependent LLS solver for instrumental polarization<br />
*refant='': no reference antenna is needed here<br />
*smodel=S[0]: we use as model the "S" we calculated above<br />
*gaintable=['*.Bscan','*.G2.polcal','*.Kcrs', '*.XY0']: we apply the bandpass, gain, and cross-hand calibration tables (Kcrs, and XY0) <br />
<br />
We do not use a reference antenna here because the polarized source provides sufficient constraints to solve for all instrumental polarization parameters on all antennas, relative to the specified source polarization. This is in contrast to the case of an unpolarized calibrator (or the general ciricular basis treatment, even if the calibrator is polarized), where only relative instrumental polarization factors among the antennas may be determined, with one feed on one antenna set to zero instrumental polarization. <br />
<br />
Here we plot the amplitude, the real part, and the imaginary part of leakage (D-term) solutions, computed in frame of the telescope.<br />
<br />
<source lang="python"><br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,0,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','amp', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,0,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','real', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='0,1', iteration='antenna',subplot=231, plotrange=[223000,227000,-0.06,0.06])<br />
plotcal('3c286_Band6.ms.Df0','freq','imag', spw='2,3', iteration='antenna',subplot=231, plotrange=[239000,243000,-0.06,0.06])<br />
<br />
</source><br />
{|<br />
| <figure id="D_amp"> [[Image:Df0_amp_casa4.3.png|200px|thumb|left|'''Fig. 19.''' <xr id="D_amp" nolink/>. Df0 amp vs freq]]</figure><br />
| <figure id="D_real"> [[Image:Df0_real_casa4.3.png|200px|thumb|left|'''Fig. 20.''' <xr id="D_real" nolink/>. Df0 real vs freq]]</figure><br />
| <figure id="D_imag"> [[Image:Df0_imag_casa4.3.png|200px|thumb|center|'''Fig. 21.''' <xr id="D_imag" nolink/>. Df0 imag vs freq]]</figure> <br />
|}<br />
<br />
The values obtained--a few percent for most of the antennas--are reasonable, and we will apply them to the data.<br />
<br />
Before applying them we need to modify the D-term table in order for it to be applied to the parallel-hands as<br />
well as the cross-hands.<br />
We do that using the module Dgen() of casapolhelpers.py:<br />
<br />
<source lang="python"><br />
<br />
# In CASA<br />
Dgen(dtab='3c286_Band6.ms.Df0',dout='3c286_Band6.ms.Df0gen')<br />
<br />
</source><br />
<br />
Before applying all the calibration tables, it is worth verifying the images resulting from a calibration not including the D-terms correction. <br />
This test will be done in the following section.<br />
<br />
==Imaging of the polarization calibrator with and without applying D-terms corrections==<br />
<br />
Before proceeding with the application of all the calibration tables to the data, it makes sense to verify the actual effect of D-terms correction on the images. <br />
For verification purposes only we now apply to the polarization calibrator all calibration tables computed in the calibration guide ''except'' the D-terms corrections.<br />
In the dir 3C286_Band6_pol_UnCalibrated you should now have all the calibration tables, and you can proceed as follows:<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs', '3c286_Band6.ms.XY0'],<br />
interp=['nearest','linear', 'nearest','nearest'], <br />
gainfield=['', '', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
Now we make a full-polarization image of the polarization calibrator, just for inspection purposes. <br />
We use the task {{clean}} interactively, adding a mask around the central source. <br />
In the following section you will find a more clear explanation of how to define the cell and imsize parameters.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.noDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.noDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True, niter=10000)<br />
<br />
</source><br />
<br />
After the first 500 iterations the residual image is noise-like and we stop the cleaning. The clean flux is of 1.06 Jy.<br />
<br />
Now we apply all the calibration tables, including Df0gen:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal','3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
<br />
</source><br />
<br />
And we clean again:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf Field0.withDterm.Stokes.clean*')<br />
clean(vis='3c286_Band6.ms',<br />
field='0',<br />
imagename='Field0.withDterm.Stokes.clean',<br />
cell=['0.1arcsec'],<br />
imsize=[250,250],<br />
stokes='IQUV',<br />
psfmode='clarkstokes',<br />
interactive=True,niter=10000)<br />
<br />
</source><br />
<br />
To reach the same clean flux (~ 1.06 Jy) more iterations are needed and the residuals level decreases.<br />
<br />
As you can see in Figure 22, the application of D-term calibration clearly improves the images of Stokes Q, U, and V.<br />
<br />
<figure id="comparison"><br />
<gallery widths=220px heights=150px perrow=2 mode="nolines" ><br />
File:comparisonI.png | Stokes I.<br />
File:comparisonQ.png | Stokes Q.<br />
File:comparisonU.png | Stokes U.<br />
File:comparisonV.png | Stokes V.<br />
</gallery><br />
'''Fig. 22.''' Comparison between images of the polarization calibrator without (left panels) and with (right panels) the D-term corrections applied.<br />
</figure><br />
<br />
==Applying the calibrations and splitting of the corrected data==<br />
<br />
<figure id="afterall_imagreal"><br />
[[File:afterall_imagreal.png|200px|thumb|right|'''Fig. 23.''' Imag vs real. Calibration tables applied to the data are B, G2polcal, Kcr, XY0, and D.]]<br />
</figure><br />
<br />
Now we will use {{applycal}} to apply the calibration tables that we generated in the previous sections. <br />
First, we apply the calibration tables to the polarization calibrator. <br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 0<br />
applycal(vis='3c286_Band6.ms', <br />
field='0', <br />
calwt=[T,T,F,F,F],<br />
gaintable=['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2.polcal',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear', 'linear','nearest', 'nearest'], <br />
gainfield=['', '','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
'''NEW: We first plot the complex plane again, and compare the results with Figure 17. Figure 23 shows clearly that the imaginary part is now zero, while the real part is corrected for the variation shown before; its corrected value is ~0.04.''' <br />
<br />
Let's now examine the corrected amplitude and phase of the polarization calibrator.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', xaxis='chan', yaxis='amp',<br />
ydatacolumn='corrected', selectdata=True, field='0',<br />
averagedata=True, avgchannel='', <br />
avgtime='1e9', avgscan=True, <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
| <figure id="afterall_amp">[[Image:afterAll_allantennas_amp_casa4.3.png|200px|thumb|left|'''Fig. 24.''' XX,YY, XY and YX amp vs channels for all antennas after the whole calibration.]]</figure><br />
| <figure id="afterall_phase">[[Image:afterAll_allantennas_phase_casas4.3.png|200px|thumb|center|'''Fig. 25.''' <xr id="afterall_phase" nolink/>. XX,YY, XY and YX amp vs channels for all antennas after the whole calibration]]</figure><br />
| <figure id="afterall_phase_1bl">[[File:afterAll_1baseline_phase.png|200px|thumb|right|'''Fig. 26.''' <xr id="afterall_phase_1bl" nolink/>. Baseline DA41&DA48: XY and YX phase vs frequency after the whole calibration. All scans are averaged.]]</figure><br />
|}<br />
<br />
In Figure 24 we see that the XX,YY correlations have unitary amplitude, since we didn't scaled the flux. <br />
The XY,YX amplitude value is of the order of 0.04, which means the Stokes U of this source is ~0.04. This value is consistent with what the previous steps revealed.<br />
After the calibration all the phases are zero (<xr id="afterall_phase" />). <br />
To more clearly show the effect of the D-terms application on the polarization calibrator we also plot the corrected phases of the baseline DA48&DV23 (<xr id="afterall_phase_1bl" />). Comparing this result with <xr id="afterXY0" /> we see that the XY and YX phases are now corrected and around zero.<br />
<br />
We now apply the solutions from the phase calibrator (field='3') to the science target (field='4'). <br />
Its flux scale has also been bootstrapped to that of the flux calibrator, so it can serve as an amplitude and phase calibrator.<br />
We apply to these sources the polarization calibrations (Kcrs, XY0, and Df0gen) that we obtained for the polarization calibrator (field='0'). <br />
<br />
<source lang="python"><br />
# In CASA<br />
# Apply caltables to fields = 3 & 4<br />
applycal(vis='3c286_Band6.ms', <br />
field='3,4', <br />
calwt=[T,T,F,F,F,F],<br />
gaintable = ['3c286_Band6.ms.Bscan','3c286_Band6.ms.G2ph','3c286_Band6.ms.flux',<br />
'3c286_Band6.ms.Kcrs','3c286_Band6.ms.XY0','3c286_Band6.ms.Df0gen'],<br />
interp=['nearest','linear','linear','nearest','nearest','nearest'], <br />
gainfield=['', '3', '3','', '', ''],<br />
parang=True)<br />
</source><br />
<br />
* gaintable=['*.Bscan', '*.G2ph', '*.flux', '*.Kcrs', '*.XY0', '*.Df0gen']: these are all the tables to be applied to the data.<br />
* interp=['nearest', 'linear', 'linear', 'nearest', 'nearest', 'nearest']: the corresponding interpolation method <br />
* gainfield=['', '3', '3','', '', '']: select the field from the corresponding gaintable. Here the phase calibrator gains are applied to both the calibrator itself and the target.<br />
* parang=True: apply the parallactic angle correction<br />
<br />
To check the effects of calibration on the scientific target we can compare the plot of imaginary vs real part before the calibration (xdatacolumn & ydatacolumn='data') and after (xdatacolumn & ydatacolumn='corrected'):<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='3c286_Band6.ms', field='4', <br />
xaxis='real', yaxis='imag', spw='0', <br />
ydatacolumn='data', xdatacolumn='data', selectdata=True, <br />
averagedata=True, avgchannel='1000', <br />
avgtime='1e9', avgscan=True, plotrange=[-1,1,-1,1], <br />
coloraxis='corr')<br />
</source><br />
<br />
{|<br />
|<figure id="field4_before"> [[Image:field4_beforeaal_casa4.3.png|200px|thumb|center|'''Fig. 27.''' <xr id="field4_before" nolink/>. 3C286: zoom in of the plot of data columns. Imag vs real of parallel- and cross-hand correlations before the calibration. All scans have been averaged.]]</figure><br />
|<figure id="field4_aftall"> [[Image:field4_afterall_casa43..png|200px|thumb|right|'''Fig. 28.''' <xr id="field4_aftall" nolink/>. 3C286: Imag vs real of parallel- and cross-hand correlations after the whole calibration. The corrected amplitude has been flux scaled, so it is in units of Jy.]]</figure><br />
|}<br />
<br />
You can see in these plots how the calibration has corrected the real and imaginary parts of the target visibilities. The imaginary parts of both the parallel and the cross-hands are close to zero, while the real parts are ~340 mJy and ~60 mJy, respectively.<br />
<br />
We now split the scientific target (field 4) corrected data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3c286_Band6.pol.cal.ms') <br />
split(vis='3c286_Band6.ms',<br />
outputvis='3c286_Band6.pol.cal.ms',<br />
field='4',<br />
datacolumn='corrected',<br />
width='8')<br />
</source><br />
<br />
Using the parameter '''width''' we average the channels, in order to reduce the size of the output file (3c286_Band6.pol.cal.ms will have 16 channels).<br />
In Band 3 TDM observations, averaging more than <br />
about 8 channels in full Stokes would produce significant bandwidth smearing.<br />
<br />
Now you can continue on to the '''[[3C286_Band6Pol_Imaging_for_CASA_4.3 |imaging guide]]'''.<br />
<br />
==References==<br />
<br />
* Synthesis Imaging in Radio Astronomy II, 1999, ASP Conference Series, Vol. 180, Editors: Taylor, G. B.; Carilli, C. L.; Perley, R. A. <br />
* Interferometry and Synthesis in Radio Astronomy, 2nd Edition, 2001, Thompson, A.R., Moran, J.M., Swenson, G. W., Ed. Wiley<br />
* AT polarization calibration, 1991, Sault, R. J., Killeen,N.E.B., & Kesteven, M.J., ATNF memo <br />
* Understanding radio polarimetry. I. Mathematical foundations., 1996, Hamaker, J. P.; Bregman, J. D.; Sault, R. J., Astronomy and Astrophysics Supplement, v.117<br />
<br />
<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.0&diff=31204AntennaeBand7 Imaging 4.02021-10-05T19:05:03Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 4.0.0. If you are using an older version of CASA please see [[AntennaeBand7_Imaging_3.4]].<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration_4.0]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.0.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.0.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{clean}}. To invoke mosaic mode, you simply set the parameter '''imagermode='mosaic''''. The default subparameter '''ftmachine='mosaic'''' is then automatically set. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.flux.pbcoverage'' and ''<imagename>.flux'' (where the latter differs from the former only if the sensitivity of each field in the mosaic varies).<br />
<br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''. <br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{clean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/jott-mosaicking-school-04.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Fig. 1), and from 70 to 100 in the northern mosaic (Fig. 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_North-AMPvsCH.png')<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_South-AMPvsCH.png')<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.flux # the effective primary beam response (where '''pbcor=True''')<br />
** <imagename>.flux.pbcoverage # the primary beam coverage ('''ftmachine=’mosaic’ ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''mode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of clean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=F,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.27" x 0.71" P.A. 86.142 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
<br />
[[Image:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes there is definitely a detection in the vicinity of the Northern nucleus (see Fig. 5 and Fig. 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.4 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=T''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Clean*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=T, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only 100 iterations (see Fig. 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{clean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{clean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{clean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. During {{clean}}, we repeat the process of making a clean mask. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.Cont.Clean*')<br />
clean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter='15',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=T, usescratch=False,<br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.11"x 0.71", and P.A.=65 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after 100 iterations (Fig. 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.80 mJy/beam and the rms is 0.41 mJy/beam. For the Southern continuum we find a peak of 5.23 mJy/beam and an rms of 0.42 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Fig. 5), then double click inside the polygon to have the statistic printed to the screen. We find rms=0.47 mJy/beam, 15% larger.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
We find rms=0.50 mJy/beam, 19% larger than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (<br />
raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00048,0.00380]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview (<br />
raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00050,0.00523]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub2}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[Image:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
To see what velocity parameters you want to set in {{clean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png') <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel.<br />
<br />
Notable parameters included in the {{clean}} call are:<br />
*'''mode='velocity' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'velocity' mode to set velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=10000''': Maximum number of clean iterations. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
<br />
<br />
[[Image:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about 900 iterations (Red X), when the artifacts start to look as bright as the residual. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Clean about 1200 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{clean}} is run, it automatically saves a uv-model of the resulting image in the header of the measurement set. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='data',plotfile='North_selfchan_data.png')<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='model',plotfile='North_selfchan_model.png')<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan')<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
<br />
[[Image:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png')<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{clean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 1200 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}],<br />
zoom={'channel': 43})<br />
</source><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you need to toggle blink off, then use the tapedeck to move to the new channel of interest, and then turn blink back on to see both cubes again.<br />
<br />
Note: When the Images sub-panel is active (the check box next to Images is checked), statistics will be reported for the image currently displayed. When the Images sub-panel is not active, all open raster images will have their statistics reported.<br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png')<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=T,coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
<br />
[[Image:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 2200 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}],<br />
zoom={'channel': 31})<br />
</source><br />
<br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 22, 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=T''' and/or '''dropstokes=T'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 4.0.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.0&diff=31203AntennaeBand7 Calibration 4.02021-10-05T18:34:41Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 4.0.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7 Imaging 4.0]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step you through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection, for example data where one telescope blocks the line of sight of another. Then we will apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Then we will use observations of the calibrators Titan and 3c279 to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these to the data and then extract the calibrated source data into a file appropriate for imaging.<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
Once you have downloaded the Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
This guide requires Python module [[analysis_Utilities|analysisUtils]]. If you have not already installed [[analysis_Utilities|analysisUtils]] please follow the link to do so. <br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casapy<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.0.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.0.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Install Analysis Utilities==<br />
<br />
Analysis Utilities (or analysisUtils for short) is a small set of Python scripts that provide a number of analysis and plotting utilities for ALMA data reduction. This guide uses a few of these utilities. They are very easy to install (just download and untar). See <br />
<br />
http://casaguides.nrao.edu/index.php?title=Analysis_Utilities<br />
<br />
for a full description and download instructions. Analysis Utilities are updated frequently so if its been a while since you installed it, its probably worth doing it again. If you are at an ALMA site or ARC, the analysis utilities are probably already installed and up to date.<br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Fig. 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task {{plotants}}</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=F)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = F)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = F: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = F)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=F)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=T so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=T.<br />
<br />
For example try <br />
<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=F. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
</pre><br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
We will also want to look at Tsys as a function of frequency. This will use the analysisutils package mentioned at the beginning of this guide (called by the au. command)<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','Antennae']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
au.plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,buildpdf=False,<br />
interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. gencal will now be used put antenna position data into each observation. Again, gencal will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running gencal to create the information CASA can refer to for antenna positions.<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=F, calwt=T)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=F, calwt=T)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
In CASA 4.0: the MODEL column is deprecated (though it is still possible to use them by explicitly setting usescratch=T).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=F tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=F)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Figure 10 and Figure 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field')<br />
dummy_string = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Fig. 15) and that Titan also shows evidence of a strong line (also Fig. 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = F)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Fig. 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = F)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = F)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Fig. 4 and Fig. 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Fig. 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Fig. 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Fig. 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=F)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=F)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=F)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Fig. 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=T, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Fig. 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=T, <br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=T,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=T,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
au.plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=F, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2010')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan (fld ind 1) spw 0 [I=2.8614, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)<br />
</pre><br />
<br />
Now we are going to take a look at the titan model for one dataset<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
</source><br />
<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model prodcued by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
[[File:2012_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model prodcued by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2010' '''. A more updated standard is also available: '''standard='Butler-JPL-Horizons 2012' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. The motivated reader may wish to rerun {{setjy}} using the updated model. We will highlight later in this tutorial, and in the imaging tutorial that follows, the effects of using the updated model.<br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.45 Jy, by averaging the fluxes obtained from the ten available datasets. This value agrees within 10% with the most recent 0.850 millimeter measurements from the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html] : (01 Jul 2011, SMA 9.75 ± 0.49).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the updated, Butler-JPL-Horizons 2012, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly higher. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.80 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=T, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=T,avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=T)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=T)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.0 imaging guide].<br />
{{Checked 4.0.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.1&diff=31202AntennaeBand7 Imaging 4.12021-10-05T14:48:22Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 4.1.0. If you are using an older version of CASA please see [[AntennaeBand7_Imaging_3.4]].<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration_4.1]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.1.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.1.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{clean}}. To invoke mosaic mode, you simply set the parameter '''imagermode='mosaic''''. The default subparameter '''ftmachine='mosaic'''' is then automatically set. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.flux.pbcoverage'' and ''<imagename>.flux'' (where the latter differs from the former only if the sensitivity of each field in the mosaic varies).<br />
<br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''. <br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{clean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/jott-mosaicking-school-04.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.'''Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Fig. 1), and from 70 to 100 in the northern mosaic (Fig. 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_North-AMPvsCH.png')<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_South-AMPvsCH.png')<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.flux # the effective primary beam response (where '''pbcor=True''')<br />
** <imagename>.flux.pbcoverage # the primary beam coverage ('''ftmachine=’mosaic’ ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''mode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of clean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=F,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.27" x 0.71" P.A. 86.142 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
<br />
[[Image:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes there is definitely a detection in the vicinity of the Northern nucleus (see Fig. 5 and Fig. 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.4 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=T''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Clean*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=T, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only 100 iterations (see Fig. 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{clean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{clean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{clean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. During {{clean}}, we repeat the process of making a clean mask. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.Cont.Clean*')<br />
clean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter='15',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=T, usescratch=False,<br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.11"x 0.71", and P.A.=65 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after 100 iterations (Fig. 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.66 mJy/beam and the rms is 0.47 mJy/beam. For the Southern continuum we find a peak of 4.49 mJy/beam and an rms of 0.45 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Fig. 5), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (<br />
raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00048,0.00380]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview (<br />
raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00050,0.00523]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[Image:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
To see what velocity parameters you want to set in {{clean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png') <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel.<br />
<br />
Notable parameters included in the {{clean}} call are:<br />
*'''mode='velocity' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'velocity' mode to set velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=10000''': Maximum number of clean iterations. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
<br />
<br />
[[Image:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about 900 iterations (Red X), when the artifacts start to look as bright as the residual. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Clean about 1200 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{clean}} is run, it automatically saves a uv-model of the resulting image in the header of the measurement set. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='data',plotfile='North_selfchan_data.png')<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='model',plotfile='North_selfchan_model.png')<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
<br />
[[Image:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png')<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{clean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 1200 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}],<br />
zoom={'channel': 43})<br />
</source><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you need to toggle blink off, then use the tapedeck to move to the new channel of interest, and then turn blink back on to see both cubes again.<br />
<br />
Note: When the Images sub-panel is active (the check box next to Images is checked), statistics will be reported for the image currently displayed. When the Images sub-panel is not active, all open raster images will have their statistics reported.<br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
<br />
[[Image:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png')<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=T,coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
<br />
[[Image:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 2200 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}],<br />
zoom={'channel': 31})<br />
</source><br />
<br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 22, 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=T''' and/or '''dropstokes=T'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 4.1.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.1&diff=31201AntennaeBand7 Calibration 4.12021-10-05T14:44:14Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 4.1.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7 Imaging 4.1]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step you through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection, for example data where one telescope blocks the line of sight of another. Then we will apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Then we will use observations of the calibrators Titan and 3c279 to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these to the data and then extract the calibrated source data into a file appropriate for imaging.<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
Once you have downloaded the Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casapy<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.1.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.1.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Fig. 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>'''Fig. 1''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task {{plotants}}</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=F)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = F)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = F: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = F)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=F)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=T so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=T.<br />
<br />
For example try <br />
<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=F. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
</pre><br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Fig. 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. gencal will now be used put antenna position data into each observation. Again, gencal will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running gencal to create the information CASA can refer to for antenna positions.<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=F, calwt=T)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=F, calwt=T)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
In CASA 4.1: the MODEL column is deprecated (though it is still possible to use them by explicitly setting usescratch=T).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=F tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=F)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Fig. 10 and Fig. 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field')<br />
dummy_string = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Fig. 15) and that Titan also shows evidence of a strong line (also Fig. 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = F)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Fig. 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = F)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = F)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Fig. 4 and Fig. 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Fig. 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Fig. 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Fig. 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=F)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=F)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=F)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Fig. 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=T, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Fig. 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=T, <br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=T,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=T,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=F, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2010')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan (fld ind 1) spw 0 [I=2.8614, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)<br />
</pre><br />
<br />
Now we are going to take a look at the titan model for one dataset<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
</source><br />
<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model prodcued by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
[[File:2012_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model prodcued by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2010' '''. A more updated standard is also available: '''standard='Butler-JPL-Horizons 2012' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. The motivated reader may wish to rerun {{setjy}} using the updated model. We will highlight later in this tutorial, and in the imaging tutorial that follows, the effects of using the updated model.<br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.41 Jy, by averaging the fluxes obtained from the ten available datasets. This value agrees within 10% with the most recent 0.850 millimeter measurements from the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html] : (03 Feb 2014, SMA 10.93 ± 0.49).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the updated, Butler-JPL-Horizons 2012, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly higher. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.80 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=T, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=T,avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=T)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=T)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.1 imaging guide].<br />
{{Checked 4.1.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.2&diff=31200AntennaeBand7 Imaging 4.22021-10-05T14:34:01Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 4.2.0. If you are using an older version of CASA please see [[AntennaeBand7_Imaging_3.4]].<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration_4.2]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.2.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.2.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{clean}}. To invoke mosaic mode, you simply set the parameter '''imagermode='mosaic''''. The default subparameter '''ftmachine='mosaic'''' is then automatically set. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.flux.pbcoverage'' and ''<imagename>.flux'' (where the latter differs from the former only if the sensitivity of each field in the mosaic varies).<br />
<br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''. <br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{clean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/jott-mosaicking-school-04.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.'''Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Fig. 1), and from 70 to 100 in the northern mosaic (Fig. 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_North-AMPvsCH.png')<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_South-AMPvsCH.png')<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.flux # the effective primary beam response (where '''pbcor=True''')<br />
** <imagename>.flux.pbcoverage # the primary beam coverage ('''ftmachine=’mosaic’ ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''mode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of clean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=F,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.27" x 0.71" P.A. 86.142 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
<br />
[[Image:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes there is definitely a detection in the vicinity of the Northern nucleus (see Fig. 5 and Fig. 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.4 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=T''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Clean*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=T, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only 100 iterations (see Fig. 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{clean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{clean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{clean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. During {{clean}}, we repeat the process of making a clean mask. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.Cont.Clean*')<br />
clean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter='15',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=T, usescratch=False,<br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.11"x 0.71", and P.A.=65 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after 100 iterations (Fig. 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.66 mJy/beam and the rms is 0.47 mJy/beam. For the Southern continuum we find a peak of 4.49 mJy/beam and an rms of 0.45 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Fig. 5), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (<br />
raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00048,0.00380]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview (<br />
raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00050,0.00523]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[Image:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
To see what velocity parameters you want to set in {{clean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png') <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel.<br />
<br />
Notable parameters included in the {{clean}} call are:<br />
*'''mode='velocity' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'velocity' mode to set velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=10000''': Maximum number of clean iterations. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
<br />
<br />
[[Image:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about 900 iterations (Red X), when the artifacts start to look as bright as the residual. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Clean about 1200 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{clean}} is run, it automatically saves a uv-model of the resulting image in the header of the measurement set. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='data',plotfile='North_selfchan_data.png')<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='model',plotfile='North_selfchan_model.png')<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
<br />
[[Image:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png')<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{clean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 1200 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}],<br />
zoom={'channel': 43})<br />
</source><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you need to toggle blink off, then use the tapedeck to move to the new channel of interest, and then turn blink back on to see both cubes again.<br />
<br />
Note: When the Images sub-panel is active (the check box next to Images is checked), statistics will be reported for the image currently displayed. When the Images sub-panel is not active, all open raster images will have their statistics reported.<br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
<br />
[[Image:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png')<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=T,coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
<br />
[[Image:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 2200 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}],<br />
zoom={'channel': 31})<br />
</source><br />
<br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 22, 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=T''' and/or '''dropstokes=T'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 4.2.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.2&diff=31199AntennaeBand7 Calibration 4.22021-10-05T14:06:24Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 4.2.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7 Imaging 4.2]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step you through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection, for example data where one telescope blocks the line of sight of another. Then we will apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Then we will use observations of the calibrators Titan and 3c279 to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these to the data and then extract the calibrated source data into a file appropriate for imaging.<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
Once you have downloaded the Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casapy<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.2.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.2.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Fig. 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>'''Fig. 1''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task {{plotants}}</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=F)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = F)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = F: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = F)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=F)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=T so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=T.<br />
<br />
For example try <br />
<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=F. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
</pre><br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Fig. 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. gencal will now be used put antenna position data into each observation. Again, gencal will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running gencal to create the information CASA can refer to for antenna positions.<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=F, calwt=T)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=F, calwt=T)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
In CASA 4.2: the MODEL column is deprecated (though it is still possible to use them by explicitly setting usescratch=T).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=F tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=F)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Fig. 10 and Fig. 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field')<br />
dummy_string = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Fig. 15) and that Titan also shows evidence of a strong line (also Fig. 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = F)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Fig. 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = F)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = F)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Fig. 4 and Fig. 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Fig. 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Fig. 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Fig. 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=F)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=F)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=F)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Fig. 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=T, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Fig. 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=T, <br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=T,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=T,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=F, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2010')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan (fld ind 1) spw 0 [I=2.8614, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)<br />
</pre><br />
<br />
Now we are going to take a look at the titan model for one dataset<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
</source><br />
<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model prodcued by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
[[File:2012_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model prodcued by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2010' '''. A more updated standard is also available: '''standard='Butler-JPL-Horizons 2012' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. The motivated reader may wish to rerun {{setjy}} using the updated model. We will highlight later in this tutorial, and in the imaging tutorial that follows, the effects of using the updated model.<br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.41 Jy, by averaging the fluxes obtained from the ten available datasets. This value agrees within 10% with the most recent 0.850 millimeter measurements from the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html] : (03 Feb 2014, SMA 10.93 ± 0.49).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the updated, Butler-JPL-Horizons 2012, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly higher. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.80 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=T, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=T,avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=T)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=T)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.2 imaging guide].<br />
{{Checked 4.2.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.3&diff=31198AntennaeBand7 Imaging 4.32021-10-05T13:40:47Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 4.3.0. If you are using an older version of CASA please see [[AntennaeBand7_Imaging_4.2]].<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration_4.3]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{clean}}. To invoke mosaic mode, you simply set the parameter '''imagermode='mosaic''''. The default subparameter '''ftmachine='mosaic'''' is then automatically set. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.flux.pbcoverage'' and ''<imagename>.flux'' (where the latter differs from the former only if the sensitivity of each field in the mosaic varies).<br />
<br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''. <br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{clean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/jott-mosaicking-school-04.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Fig. 1), and from 70 to 100 in the northern mosaic (Fig. 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_North-AMPvsCH.png', showgui=True)<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_South-AMPvsCH.png', showgui=True)<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.flux # the effective primary beam response (where '''pbcor=True''')<br />
** <imagename>.flux.pbcoverage # the primary beam coverage ('''ftmachine=’mosaic’ ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''mode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of clean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=False,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.27" x 0.71" P.A. 86.142 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
<br />
[[Image:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes there is definitely a detection in the vicinity of the Northern nucleus (see Fig. 5 and Fig. 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.4 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=True''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Clean*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only 100 iterations (see Fig. 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{clean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{clean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{clean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. During {{clean}}, we repeat the process of making a clean mask. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.Cont.Clean*')<br />
clean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter='15',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=True, usescratch=False,<br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.11"x 0.71", and P.A.=65 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after 100 iterations (Fig. 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.66 mJy/beam and the rms is 0.47 mJy/beam. For the Southern continuum we find a peak of 4.49 mJy/beam and an rms of 0.45 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Fig. 5), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (<br />
raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00048,0.00380]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview (<br />
raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00050,0.00523]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[Image:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
To see what velocity parameters you want to set in {{clean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png', showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png', showgui=True) <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel.<br />
<br />
Notable parameters included in the {{clean}} call are:<br />
*'''mode='velocity' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'velocity' mode to set velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=10000''': Maximum number of clean iterations. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
<br />
<br />
[[Image:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about 900 iterations (Red X), when the artifacts start to look as bright as the residual. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Clean about 1200 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{clean}} is run, it automatically saves a uv-model of the resulting image in the header of the measurement set. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='data',plotfile='North_selfchan_data.png', showgui=True)<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='model',plotfile='North_selfchan_model.png', showgui=True)<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data', showgui=True)<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
[[Image:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png', showgui=True)<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{clean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 1200 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}],<br />
zoom={'channel': 43})<br />
</source><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you need to toggle blink off, then use the tapedeck to move to the new channel of interest, and then turn blink back on to see both cubes again.<br />
<br />
Note: When the Images sub-panel is active (the check box next to Images is checked), statistics will be reported for the image currently displayed. When the Images sub-panel is not active, all open raster images will have their statistics reported.<br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
<br />
[[Image:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png', showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png', showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=True,coloraxis='scan',ydatacolumn='data', showgui=True)<br />
</source><br />
<br />
<br />
[[Image:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png', showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 2200 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}],<br />
zoom={'channel': 31})<br />
</source><br />
<br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 22, 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=True''' and/or '''dropstokes=True'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.3&diff=31197AntennaeBand7 Calibration 4.32021-10-05T13:24:22Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 4.3.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7 Imaging 4.3]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step you through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection, for example data where one telescope blocks the line of sight of another. Then we will apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Then we will use observations of the calibrators Titan and 3c279 to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these to the data and then extract the calibrated source data into a file appropriate for imaging.<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
Once you have downloaded the Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casapy<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Fig. 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task {{plotants}}</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=False)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup=False)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' =False: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup=False)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=False)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=True so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=True.<br />
<br />
For example try <br />
<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=False. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
</pre><br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0, showgui=True) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0, showgui=True) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. gencal will now be used put antenna position data into each observation. Again, gencal will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running gencal to create the information CASA can refer to for antenna positions.<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=False, calwt=True)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=False, calwt=True)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
The MODEL column is now deprecated (though it is still possible to use it by explicitly setting usescratch=True).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', showgui=True)<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', showgui=True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', showgui=True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', showgui=True)<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="data", showgui=True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="corrected", showgui=True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data", showgui=True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected", showgui=True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=False tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=False)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Fig. 10 and Fig. 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field')<br />
dummy_string = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Fig. 15) and that Titan also shows evidence of a strong line (also Fig. 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup=False)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Fig. 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup=False)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup=False)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Fig. 4 and Fig. 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Fig. 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup=False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup=False)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Fig. 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Fig. 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=False)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=False)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=False)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Fig. 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=True, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Fig. 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=True, <br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=True,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=True,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=False, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2010')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan (fld ind 1) spw 0 [I=2.8614, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)<br />
</pre><br />
<br />
Now we are going to take a look at the titan model for one dataset<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr')<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr')<br />
</source><br />
<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model prodcued by 'Butler-JPL-Horizons 2010'.</caption>]][[File:2012_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model prodcued by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2010' '''. A more updated standard is also available: '''standard='Butler-JPL-Horizons 2012' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. The motivated reader may wish to rerun {{setjy}} using the updated model. We will highlight later in this tutorial, and in the imaging tutorial that follows, the effects of using the updated model.<br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.41 Jy, by averaging the fluxes obtained from the ten available datasets. This value agrees within 10% with the most recent 0.850 millimeter measurements from the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html] : (03 Feb 2014, SMA 10.93 ± 0.49).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the updated, Butler-JPL-Horizons 2012, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly higher. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.80 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=True, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=True,avgbaseline=False, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=True)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=True)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.3 imaging guide].<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.4&diff=31196AntennaeBand7 Imaging 4.42021-10-05T12:10:40Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 4.4.0. If you are using an older version of CASA please see [[AntennaeBand7_Imaging_4.3]].<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration_4.4]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.4.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.4.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{clean}}. To invoke mosaic mode, you simply set the parameter '''imagermode='mosaic''''. The default subparameter '''ftmachine='mosaic'''' is then automatically set. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.flux.pbcoverage'' and ''<imagename>.flux'' (where the latter differs from the former only if the sensitivity of each field in the mosaic varies).<br />
<br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''. <br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{clean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/jott-mosaicking-school-04.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Fig. 1), and from 70 to 100 in the northern mosaic (Fig. 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_North-AMPvsCH.png')<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=T,plotfile='Antennae_South-AMPvsCH.png')<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.flux # the effective primary beam response (where '''pbcor=True''')<br />
** <imagename>.flux.pbcoverage # the primary beam coverage ('''ftmachine=’mosaic’ ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''mode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of clean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=F,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.27" x 0.71" P.A. 86.142 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
<br />
[[Image:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes there is definitely a detection in the vicinity of the Northern nucleus (see Fig. 5 and Fig. 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.4 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=T''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Clean*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=T, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only 100 iterations (see Fig. 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{clean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{clean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{clean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. During {{clean}}, we repeat the process of making a clean mask. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.Cont.Clean*')<br />
clean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter='15',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=T, usescratch=False,<br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.11"x 0.71", and P.A.=65 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after 100 iterations (Fig. 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.66 mJy/beam and the rms is 0.47 mJy/beam. For the Southern continuum we find a peak of 4.49 mJy/beam and an rms of 0.45 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Fig. 5), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (<br />
raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00048,0.00380]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview (<br />
raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00050,0.00523]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
== CO(3-2) Imaging ==<br />
<br />
[[Image:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
To see what velocity parameters you want to set in {{clean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=T,transform=T,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png') <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel.<br />
<br />
Notable parameters included in the {{clean}} call are:<br />
*'''mode='velocity' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'velocity' mode to set velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=10000''': Maximum number of clean iterations. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
<br />
[[Image:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about 900 iterations (Red X), when the artifacts start to look as bright as the residual. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Clean about 1200 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{clean}} is run, it automatically saves a uv-model of the resulting image in the header of the measurement set. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='data',plotfile='North_selfchan_data.png')<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=T,ydatacolumn='model',plotfile='North_selfchan_model.png')<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
[[Image:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png')<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{clean}} always uses the corrected data column if it exists:<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 1200 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}],<br />
zoom={'channel': 43})<br />
</source><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you need to toggle blink off, then use the tapedeck to move to the new channel of interest, and then turn blink back on to see both cubes again.<br />
<br />
Note: When the Images sub-panel is active (the check box next to Images is checked), statistics will be reported for the image currently displayed. When the Images sub-panel is not active, all open raster images will have their statistics reported.<br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png')<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=T,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=T,coloraxis='scan',ydatacolumn='data')<br />
</source><br />
<br />
[[Image:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=F)<br />
</source><br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=T,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 2200 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview (raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}],<br />
zoom={'channel': 31})<br />
</source><br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figure 22, 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview (raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=T''' and/or '''dropstokes=T'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 4.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_4.4&diff=31195AntennaeBand7 Calibration 4.42021-10-05T11:51:14Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 4.4.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7 Imaging 4.4]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step you through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection, for example data where one telescope blocks the line of sight of another. Then we will apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Then we will use observations of the calibrators Titan and 3c279 to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these to the data and then extract the calibrated source data into a file appropriate for imaging.<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
Once you have downloaded the Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casapy<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.4.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.4.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Fig. 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task {{plotants}}</caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=F)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = F)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = F: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = F)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=F)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=T so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=T.<br />
<br />
For example try <br />
<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=F. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
</pre><br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. gencal will now be used put antenna position data into each observation. Again, gencal will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running gencal to create the information CASA can refer to for antenna positions.<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=F, calwt=T)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=F, calwt=T)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
The MODEL column is now deprecated (though it is still possible to use it by explicitly setting usescratch=T).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=T, spw='1', <br />
avgtime='1e8',avgscan=T,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=T, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=F,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data')<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=T, coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data")<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected")<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=F tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=F)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Fig. 10 and Fig. 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field')<br />
dummy_string = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=T, coloraxis='corr')<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Fig. 15) and that Titan also shows evidence of a strong line (also Fig. 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = F)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Fig. 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = F)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = F)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Figure 4 and Figure 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=F)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Fig. 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = F)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Fig. 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Fig. 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = F)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=F)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = F)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=F)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=F)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=F)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=F)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Fig. 22).<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=T, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Fig. 23).<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=T, <br />
coloraxis='baseline', iteraxis='antenna')<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=T,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=T,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=F, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2010')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan (fld ind 1) spw 0 [I=2.8614, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)<br />
</pre><br />
<br />
Now we are going to take a look at the titan model for one dataset<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
</source><br />
<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model prodcued by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
[[File:2012_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model prodcued by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2010' '''. A more updated standard is also available: '''standard='Butler-JPL-Horizons 2012' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. The motivated reader may wish to rerun {{setjy}} using the updated model. We will highlight later in this tutorial, and in the imaging tutorial that follows, the effects of using the updated model.<br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.41 Jy, by averaging the fluxes obtained from the ten available datasets. This value agrees within 10% with the most recent 0.850 millimeter measurements from the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html] : (03 Feb 2014, SMA 10.93 ± 0.49).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the updated, Butler-JPL-Horizons 2012, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly higher. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.80 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=F,calwt=F)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=F,calwt=F)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=T, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=F, avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=T,avgbaseline=F, coloraxis='corr')<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=T)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=T)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.4 imaging guide].<br />
{{Checked 4.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_4.5&diff=31194AntennaeBand7 Imaging 4.52021-10-05T11:26:42Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 4.5.0. If you are using an older version of CASA please see [[AntennaeBand7_Imaging_4.3]].<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration_4.5]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.5.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.5.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{clean}}. To invoke mosaic mode, you simply set the parameter '''imagermode='mosaic''''. The default subparameter '''ftmachine='mosaic'''' is then automatically set. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.flux.pbcoverage'' and ''<imagename>.flux'' (where the latter differs from the former only if the sensitivity of each field in the mosaic varies).<br />
<br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''. <br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{clean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review for example http://www.aoc.nrao.edu/events/synthesis/2010/lectures/jott-mosaicking-school-04.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Fig. 1), and from 70 to 100 in the northern mosaic (Fig. 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_North-AMPvsCH.png', showgui = True)<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_South-AMPvsCH.png', showgui = True)<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.flux # the effective primary beam response (where '''pbcor=True''')<br />
** <imagename>.flux.pbcoverage # the primary beam coverage ('''ftmachine=’mosaic’ ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''mode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of clean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=False,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.27" x 0.71" P.A. 86.142 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
<br />
[[Image:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes there is definitely a detection in the vicinity of the Northern nucleus (see Fig. 4 and Fig. 5 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.4 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=True''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Clean*')<br />
clean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter='12',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only 100 iterations (see Fig. 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{clean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{clean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{clean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[Image:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. During {{clean}}, we repeat the process of making a clean mask. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.Cont.Clean*')<br />
clean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter='15',<br />
mode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=True, usescratch=False,<br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.11"x 0.71", and P.A.=65 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after 100 iterations (Fig. 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 4.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.66 mJy/beam and the rms is 0.47 mJy/beam. For the Southern continuum we find a peak of 4.49 mJy/beam and an rms of 0.45 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Fig. 4), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 5.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00048,0.00380]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview(raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':T,'range':[-0.00050,0.00523]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[Image:North_CO3_2_vel.png|thumb|<caption>'''Fig. 7.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
[[Image:South_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
To see what velocity parameters you want to set in {{clean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png', showgui = True) <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel.<br />
<br />
Notable parameters included in the {{clean}} call are:<br />
*'''mode='velocity' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'velocity' mode to set velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=10000''': Maximum number of clean iterations. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
<br />
<br />
[[Image:North_interact.png|200px|thumb|right|<caption>'''Fig. 9.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Figure 9 shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about 900 iterations (Red X), when the artifacts start to look as bright as the residual. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[Image:South_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,usescratch=False,<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
Clean about 1200 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{clean}} is run, it automatically saves a uv-model of the resulting image in the header of the measurement set. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[Image:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 11.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='data',plotfile='North_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
<br />
[[Image:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='model',plotfile='North_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
<br />
[[Image:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 13.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png', showgui = True)<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{clean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[Image:North_interact2.png|200px|thumb|right|<caption>'''Fig. 14.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='12',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=500,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 1200 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}],<br />
zoom={'channel': 43})<br />
</source><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 15, you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you need to toggle blink off, then use the tapedeck to move to the new channel of interest, and then turn blink back on to see both cubes again.<br />
<br />
Note: When the Images sub-panel is active (the check box next to Images is checked), statistics will be reported for the image currently displayed. When the Images sub-panel is not active, all open raster images will have their statistics reported.<br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[Image:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 15.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
<br />
[[Image:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 16.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[Image:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=True,coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
<br />
[[Image:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
<br />
[[Image:South_interact2.png|200px|thumb|right|<caption>'''Fig. 19.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
clean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter='15',<br />
mode='velocity',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
imagermode='mosaic',usescratch=False,<br />
imsize=750,cell='0.13arcsec',minpb=0.2,<br />
interactive=True,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=10000, threshold='5.0mJy')<br />
</source><br />
<br />
After 600 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 2200 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images:<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}],<br />
zoom={'channel': 31})<br />
</source><br />
<br />
[[Image:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 20.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 4% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 21.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[Image:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 21, 22, 23, 24, 25, and 26)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':T,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':T},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=True''' and/or '''dropstokes=True'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 27 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[Image: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 27.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 4.5.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_5.1&diff=30895AntennaeBand7 Imaging 5.12021-08-20T20:35:45Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 5.1.1. If you are using an older version of CASA please see [[AntennaeBand7_Imaging_4.5]].<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.1.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '5.1.1'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review, for example, https://science.nrao.edu/science/meetings/2016/15th-synthesis-imaging-workshop/documents/wilner_vla16.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{tclean}}. To invoke mosaic mode, you simply set the parameter '''gridder='mosaic''''. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.pb''.<br />
<br />
<!-- <br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''.<br />
--><br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{tclean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review, for example, https://science.nrao.edu/science/meetings/2016/15th-synthesis-imaging-workshop/documents/bsmMosaicking2016.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Figure 1), and from 70 to 100 in the northern mosaic (Figure 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_North-AMPvsCH.png', showgui = True)<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_South-AMPvsCH.png', showgui = True)<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{clean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{clean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.pb # the primary beam response<br />
** <imagename>.image.pbcor # the primary beam-corrected image (if '''pbcor=True''')<br />
** <imagename>.weight # the primary beam coverage ('''gridder=’mosaic’ or 'awproject' ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''specmode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.net/ splatalogue].<br />
*'''niter=0''': Maximum number of tclean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
tclean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter=12,<br />
specmode='mfs',<br />
deconvolver='hogbom',<br />
restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
gridder='mosaic',<br />
imsize=300,cell='0.2arcsec',<br />
interactive=False,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.23" x 0.72", with a position angle (P.A.) of 85.4 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
[[File:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes, there is definitely a detection in the vicinity of the Northern nucleus (see Figure 5 and Figure 6 below). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.5 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). We have included an example mask to use in Antennae_Band7_CalibratedData.tgz. If you would like to use your own mask, make sure to removed the mask file before running the command below. Otherwise, you can edit the mask within the clean GUI. See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=True''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_North.Cont.Clean'+ext)<br />
<br />
tclean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter=12,<br />
deconvolver='hogbom',<br />
specmode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
gridder='mosaic',<br />
imsize=500,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only ~30 iterations (see Figure 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{tclean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{tclean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{tclean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[File:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. Again, you can use the provided mask or make your own.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_South.Cont.Clean'+ext)<br />
<br />
tclean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter=15,<br />
specmode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
gridder='mosaic',deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.13"x 0.67", and P.A.= 61 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after ~50 iterations (Figure 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.77 mJy/beam and the rms is 0.49 mJy/beam. For the Southern continuum we find a peak of 4.74 mJy/beam and an rms of 0.41 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (Figure 5), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python">exit<br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic made with {{imview}}.</caption>]]<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic made with {{imview}}.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':True,'range':[-0.00049,0.00377]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview(raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':True,'range':[-0.00041,0.00474]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
In CASA 5.1, you will see a warning that uvcontsub is using the original Visibility Iterator, which is the underlying iteration and retrieval mechanism used when processing visibility data. <br />
A new version of the Visibility Iterator has been developed and deployed in several other calibration tasks in CASA 5.0 and later (see [https://science.nrao.edu/enews/casa_005/ CASA eNews] for more information). <br />
We can ignore this warning. <br />
<br />
<pre style="background-color: #fffacd;"><br />
WARN calibrater Forcing use of OLD VisibilityIterator.<br />
</pre><br />
<br />
== CO(3-2) Imaging ==<br />
<br />
[[File:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
[[File:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''specmode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. <br />
When making spectral images you have three choices for the '''specmode''' parameter: '''mfs''', '''cube''', and '''cubedata'''. <br />
As noted above, setting '''specmode='mfs'''' gives an output image with only one channel, and is used for continuum imaging. The other '''specmode''' options create output data cubes with one or more channels.<br />
<br />
Data are taken using constant frequency channels. <br />
For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. <br />
For '''specmode='cube'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. <br />
The Doppler Shift is taken out during the regridding to the desired outframe in {{tclean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{tclean}}. <br />
<br />
To see what velocity parameters you want to set in {{tclean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png', showgui = True) <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we provide a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel. You can choose to use and edit the provided mask or create your own mask.<br />
<br />
Notable parameters included in the {{tclean}} call are:<br />
*'''specmode='cube' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'cube' mode to create an output data cube. We set the velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=20000''': Maximum number of clean iterations. With complex emission structure like that found in the Antennae CO data, we will need a large number of iterations to clean the emission. Note that you can increase the number of iterations allowed on-the-fly in the interactive {{tclean}} window. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
*'''savemodel='modelcolumn'''': This saves the clean model to the ms file. We will need this for self-calibration later. <br />
<br />
<br />
[[File:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_North.CO3_2Line.Clean'+ext)<br />
<br />
tclean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter=12,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',<br />
deconvolver='hogbom',<br />
imsize=500,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Cycle through the channels until you reach ~1650 km/s, where you can more easily see where to draw the clean box around the CO emission. Be sure the check the circle next to 'All Channels' when making your clean mask. Now clean using the green circle arrow. You can watch the progress in the logger. When the first ~650 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about ~7000 iterations (Red X), when the artifacts start to look as bright as the residual. Note again that we are not cleaning to the theoretical noise level. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[File:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('ntennae_South.CO3_2Line.Clean'+ext)<br />
<br />
tclean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter=15,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
Clean about 8,000 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{clean}} is run, it automatically saves a uv-model of the resulting image in the header of the measurement set. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
[[File:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='data',plotfile='North_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
You will notice that while the model looks reasonable across the channels where we imaged the data, the model amplitudes are set to 1 elsewhere. When {{tclean}} initializes a model, it defaults to an amplitude of 1 and phase 0 (i.e., a point source with an amplitude of 1 in the center of the image). When we set '''savemodel = 'modelcolumn'''' in the {{tclean}} call, the default model was replaced 'only where {{tclean}} had a model to replace it with', namely, over the channels we imaged. In future, CASA will set a flag to let users know that these model points are not valid for the updated model. In our case, we will perform self-calibration only on those channels with significant signal in the line emission, omitting the rest of the channels. <br />
<br />
<br />
[[File:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='model',plotfile='North_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
[[File:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='north_pcal1_phase.png', showgui = True)<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{tclean}} always uses the corrected data column if it exists:<br />
<br />
<br />
[[File:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
tclean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter=12,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',deconvolver='hogbom',<br />
imsize=500,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
After 3000 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 9,000 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}])<br />
<br />
</source><br />
<br />
<!--<br />
zoom={'channel': 43}) doesn't work with two images<br />
--><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, cycle to channel 43. Then you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you can use the tapedeck to move to the new channel of interest and see the changes in both cubes.<br />
<br />
To compare the statistics for each image, click the "Next" button in the "Statistics" tab of the "Region" sub-panel. <br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[File:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 25% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[File:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
<br />
[[File:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=True,coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
[[File:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteration='antenna',subplot=531,plotrange=[0,0,-80,80],<br />
figfile='south_pcal1_phase.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
[[File:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "All Channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
tclean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter=15,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=40000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
After 4000 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Stop after 25,000 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images (look, for example, at channel 31):<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}])<br />
</source><br />
<br />
<!--<br />
zoom={'channel': 31})<br />
doesn't work for multiple images<br />
--><br />
<br />
<br />
[[File:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 24% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
[[File:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
[[File:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
[[File:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 22, 23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':True,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':True,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=True''' and/or '''dropstokes=True'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[File: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 5.1.1}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_5.1&diff=30894AntennaeBand7 Calibration 5.12021-08-20T20:19:42Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 5.1.1 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7_Imaging]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection; for example, data where one telescope blocks the line of sight of another. We will then apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Using observations of the calibrators Titan and 3c279, it will be shown how to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these solutions to the data and then extract the calibrated source data into a file appropriate for imaging in the [[AntennaeBand7_Imaging_5.1 | Antennae Band 7 imaging guide]].<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
This guide requires CASA 5.1.1 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]. <br />
<br />
Once you have downloaded the data, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casa<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
<!--<br />
==Confirm your version of CASA==<br />
<br />
Doesn't work in 5.1:<br />
<br />
This guide has been written for CASA release 4.5.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.5.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
--><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Figure 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT.png|200px|thumb|right|<caption>'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task plotants. </caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
</pre><br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=False)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = False)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = F: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = False)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=False)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=True so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=True.<br />
<br />
For example try <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
</source><br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=False. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window spw=5 with the task {{plotcal}}. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. Here and throughout we focus on spw 1, which contains CO(3-2):<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
antenna='0~7',<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page1.png',<br />
fontsize=7.0, showgui = True) <br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotcal(caltable=asdm+'.tdm.tsys', <br />
xaxis="time",yaxis="tsys",<br />
antenna='8~15',<br />
spw='5:50~50',plotsymbol=".", subplot=421,<br />
iteration='antenna', figfile=asdm+'.tsys_vs_time.page2.png',<br />
fontsize=7.0, showgui = True) <br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotcal}}:<br />
<br />
* '''subplot'''=421 parameter sets up a 4 x 2 panel grid. <br />
* '''iteration''' tells {{plotcal}} to make a separate plot for each antenna.<br />
* '''spw''' '5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations.<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two plotcal calls: one for the first 8 antennas ('''antenna'''=0~7), and then for the remaining antennas ('''antenna'''=8~15). The '''fontsize''' needs to be set to a small value or the text overlaps.<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''figfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer.<br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figure 4, Figure 5, and Figure 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
<br />
In general, the most appropriate antenna positions for the dataset will be determined and updated if needed during QA2. If the antenna positions need to be updated after QA2, we inform the PIs of the affected datasets and provide instructions on how to update the antenna positions. For the data in this guide, we do need to update the antenna positions. <br />
<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. {{gencal}} will now be used put antenna position data into each observation. Again, {{gencal}} will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running {{gencal}} to create the information CASA can refer to for antenna positions.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=False, calwt=True)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=False, calwt=True)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
The MODEL column is now deprecated (though it is still possible to use it by explicitly setting usescratch=True).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
<!--<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
--><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=False tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=False)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Figure 10 and Figure 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field', showgui = True)<br />
user_input = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.[type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
user_input = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+". ")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Figure 15) and that Titan also shows evidence of a strong line (see Figure 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = False)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Figure 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = False)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = False)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Figure 4 and Figure 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Figure 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Figure 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Figure 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=False)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=False)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=False)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Figure 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=True, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Figure 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=True, <br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=True,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=True,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. '''For what systems is this true?''' As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
<!--<br />
Again this warning:<br />
No ASDM_CALWVR, ASDM_CALATMOSPHERE, or CalWVR.xml table found. Using PWV 1.0mm.<br />
--><br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27). We run the following additional flagging step:<br />
<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=False, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set.<br />
<br />
<source lang="python"><br />
# in CASA<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=False,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2012')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 3.3 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan: spw0 Flux:[I=3.2637,Q=0.0,U=0.0,V=0.0] +/- [I=0.0,Q=0.0,U=0.0,V=0.0] Jy @ 344.85GHz<br />
</pre><br />
<br />
Now we are going to take a look at the Titan model for one dataset:<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
</source><br />
<br />
[[File:2012_titan_model_5.1.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model produced by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model produced by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2012' '''. Previous versions of this guide used an older flux density standard '''standard='Butler-JPL-Horizons 2010' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. <br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.79 Jy, by averaging the fluxes obtained from the ten available datasets. If we check this value against reported flux measurements in the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html], we find that a flux of 10.79 Jy agrees within a few per cent of a linear interpolation between the two 0.85 mm measurements closest in time to the ALMA observations: (23 May 2011, SMA 11.644 ± 0.583; 01 July 2011, 9.748 ± 0.488).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the older, Butler-JPL-Horizons 2010, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly lower. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.41 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page1.png',<br />
antenna='0~7', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase',<br />
iteration = 'antenna', plotrange=[0,0,-180,180],<br />
showgui=False, subplot=421, figfile=asdm+'.scanphase.page2.png',<br />
antenna='8~15', fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='0~7', iteration='antenna',<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page1.png',<br />
fontsize=6.0)<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotcal(caltable = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp',<br />
plotrange = [0,0,0,0],<br />
antenna='8~15', iteration='antenna', fontsize=6.0,<br />
showgui=False, subplot=421, figfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=True, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=True,avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
Note that you will see a warning that the last output channel in the new spw 0 will be dropped, as it has less than 23 channels over which to average:<br />
<br />
<pre style="background-color: #fffacd;"><br />
INFO MSTransformDataHandler The last output channel of spw 0 has only 22 input channels.<br />
WARN MSTransformManager Not enough input channels to populate output channel n# 166 from SPW 0.<br />
WARN MSTransformManager The channel will be dropped in order to have an uniform grid.<br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=True)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=True)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging imaging guide].<br />
{{Checked 5.1.1}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_5.4&diff=30889AntennaeBand7 Imaging 5.42021-08-20T19:24:44Z<p>Jbertone: </p>
<hr />
<div>* '''This guide is designed for CASA 5.4.0. If you are using an older version of CASA please see [[AntennaeBand7]] for earlier versions of this CASAguide.<br />
* '''This tutorial picks up where [[AntennaeBand7_Calibration]] leaves off: with fully calibrated, split science target measurement sets. If you wish to skip the Calibration guide: obtain the calibrated data from [[AntennaeBand7#Obtaining_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
* '''Details of the ALMA data are provided at [[AntennaeBand7]]'''.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.4.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '5.4.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Imaging Mosaics ==<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean, pause here and <br />
review, for example, https://science.nrao.edu/science/meetings/2016/15th-synthesis-imaging-workshop/documents/wilner_vla16.pdf<br />
</pre><br />
<br />
Mosaics like other kinds of images are created in the CASA task {{tclean}}. To invoke mosaic mode, you simply set the parameter '''gridder='mosaic''''. This is a joint deconvolution algorithm that works in the uv-plane. A convolution of the primary beam patterns for each pointing in the mosaic is created: the primary beam response function. The corresponding image of the mosaic response function will be called ''<imagename>.pb''. Note that the mosaic gridder should also be used even for single pointings if 7m and 12m data are being imaged simultaneously.<br />
<br />
<!-- <br />
Well-sampled mosaics like the patterns observed for the Northern and Southern Antennae mosaics shown on the [[AntennaeBand7#ALMA_Data_Overview | Overview]] page are well-suited to joint deconvolution. However, if you have a poorly-sampled, or very irregular mosaic you may need to use the slower '''ftmachine='ft'''' which combines data in the image plane, similar to what is done in other packages like ''MIRIAD''.<br />
--><br />
<br />
Additionally, for mosaics it is essential to pick the center of the region to be imaged explicitly using the '''phasecenter''' parameter. Otherwise it will default to the first pointing included in the '''field''' parameter -- since this is often at one corner of the mosaic, the image will not be centered. For the Northern mosaic, the center pointing corresponds to field id 12. Note that during the final '''split''' in the calibration section that selected only the Antennae fields, the field ids were renumbered, so that the original centers (shown in the [[AntennaeBand7#ALMA_Data_Overview | Overview]]) have changed: field id 14 becomes 12 for the Northern mosaic and field 18 becomes 15 for the Southern mosaic. You can also set an explicit coordinate (see the {{tclean}} help for syntax).<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you want to learn more about mosaicing, pause here and <br />
review, for example, https://science.nrao.edu/science/meetings/2016/15th-synthesis-imaging-workshop/documents/bsmMosaicking2016.pdf<br />
</pre><br />
<br />
== Continuum Imaging ==<br />
<br />
[[File:Antennae_South-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 1.''' Southern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 50 to 100</caption>]]<br />
<br />
<br />
[[File:Antennae_North-AMPvsCH.png|200px|thumb|right|<caption>'''Fig. 2.''' Northern mosaic: Amplitude vs. channel. The CO(3-2) line is seen from 70 to 100</caption>]]<br />
<br />
<br />
We will make 345 GHz continuum images for the two regions covered by the mosaics. We use the task {{clean}} over the channels that are free of the line emission; we avoid the edge channels which tend to be noisier due to bandpass rolloff effects. The line-free channels are found by plotting the average spectrum (all fields). We find the CO(3-2) line from channels 50 to 100 in the southern mosaic (Figure 1), and from 70 to 100 in the northern mosaic (Figure 2). <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North-AMPvsCH.png')<br />
plotms(vis='Antennae_North.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_North-AMPvsCH.png', showgui = True)<br />
</source><br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South-AMPvsCH.png')<br />
plotms(vis='Antennae_South.cal.ms',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,plotfile='Antennae_South-AMPvsCH.png', showgui = True)<br />
</source><br />
<br />
The '''avgtime''' is set to a large value so that it averages over all the integrations, and '''avgscan''' is set to allow averaging of the different scans.<br />
<br />
Next we create continuum images from the line-free channels. <br />
<br />
==== Northern Continuum Mosaic ====<br />
<br />
For illustrative purposes we first make a dirty image to see if there is emission and what the exact beam size is. It should be on the order of 1" but this will vary a bit according to the uv-coverage in the actual data. We will start with a '''cell''' size of 0.2" to oversample the beam by a factor of 5. The '''imsize''' needs to be large enough given the cell size to comfortably encompass the mosaic. From the mosaic footprints shown in the [[AntennaeBand7#ALMA_Data_Overview | overview]], we can see that the Northern mosaic '''imsize''' needs to be about 1 arcmin. With 0.2" pixels, this requires '''imsize=300'''. <br />
<br />
Other essential {{tclean}} parameters for this case include:<br />
*'''vis='Antennae_North.cal.ms' ''': The calibrated dataset on the science target. {{tclean}} will always use the CORRECTED DATA column (if it exists).<br />
*'''imagename='Antennae_North.Cont.Dirty' ''': The base name of the output images:<br />
** <imagename>.image # the final restored image<br />
** <imagename>.pb # the primary beam response<br />
** <imagename>.image.pbcor # the primary beam-corrected image (if '''pbcor=True''')<br />
** <imagename>.weight # the primary beam coverage ('''gridder=’mosaic’ or 'awproject' ''' only)<br />
** <imagename>.model # the model image<br />
** <imagename>.residual # the residual image<br />
** <imagename>.psf # the synthesized (dirty) beam<br />
*'''spw='0:1~50;120~164' ''': To specify only the line-free channels of spectral window 0.<br />
*'''specmode='mfs' ''': Multi-Frequency Synthesis: The default mode, which produces one image from all the specified data combined. This will grid each channel independently before imaging. For wide bandwidths this will give a far superior result compared to averaging the channels and then imaging. <br />
*'''restfreq='345.79599GHz' ''': The rest frequency of CO(3-2) can be found with [http://www.splatalogue.online/ splatalogue].<br />
*'''niter=0''': Maximum number of tclean iterations. ('''niter=0 will do no cleaning''')<br />
<br />
In CASA 5.4 and later, {{tclean}} calls with '''gridder''' = ''''mosaic'''' have an additional parameter '''mosweight''' with a default of '''True'''. When '''mosweight''' = '''True''', the gridder weights each field in the mosaic independently. The '''mosweight''' parameter is particularly important for mosaics with non-uniform sensitivity, with rectangular shapes, or when using more uniform values of robust Briggs weighting. For more information on '''mosweight''', please see the {{tclean}} documentation. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.Cont.Dirty*')<br />
tclean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Dirty',<br />
field='',phasecenter=12,<br />
specmode='mfs',<br />
deconvolver='hogbom',<br />
restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
gridder='mosaic', mosweight=True,<br />
imsize=300,cell='0.2arcsec',<br />
interactive=False,niter=0)<br />
</source><br />
<br />
The reported beam size is about 1.23" x 0.72", with a position angle (P.A.) of 85.4 degrees. <br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Dirty.image')<br />
</source><br />
<br />
<!-- COMMENT OUT VIEWER VIDEO. UNCOMMENT WHEN A MORE UP-TO-DATE VIDEO IS AVAILABLE.<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the CASA viewer it is a good idea to pause here and watch the demo at <br />
http://casa.nrao.edu/CasaViewerDemo/casaViewerDemo.html<br />
</pre> --><br />
<br />
[[File:North_cont_clean.png|200px|thumb|right|<caption>'''Fig. 3.''' Residual for Northern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
Yes, there is definitely a detection in the vicinity of the Northern nucleus (see Figure 5 and Figure 6). Using the square region icon, and drawing a box near but not including the emission, we find the rms noise is about 0.5 mJy/beam in the dirty image.<br />
<br />
Next we switch to refined values of '''cell='0.13arcsec' ''' and '''imsize=500''' based on the observed beam size. We also switch to interactive mode so that you can create a clean mask using the polygon tool (note you need to double click inside the polygon region to activate the mask). We have included an example mask to use in Antennae_Band7_CalibratedData.tgz. If you would like to use your own mask, make sure to removed the mask file before running the command below. Otherwise, you can edit the mask within the clean GUI. See [[TWHydraBand7_Imaging_3.4#Image_and_Self-Calibrate_the_Continuum | TWHya casaguide]] for a more complete description of interactive clean and mask creation. <br />
<br />
*'''niter=1000''': Maximum number of clean iterations -- '''we will stop interactively'''<br />
*'''threshold='0.4mJy' ''': Stop cleaning if the maximum residual is below this value (the dirty rms noise)<br />
*'''interactive=True''': Clean will be periodically interrupted to show the residual clean image. Interactive clean mask can be made. If no mask is created, no cleaning is done.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_North.Cont.Clean'+ext)<br />
<br />
tclean(vis='Antennae_North.cal.ms',imagename='Antennae_North.Cont.Clean',<br />
field='',phasecenter=12,<br />
deconvolver='hogbom',<br />
specmode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~50;120~164',<br />
gridder='mosaic', mosweight=True,<br />
imsize=500,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The residuals are "noise-like" after only ~30 iterations (see Figure 3), so hit the red X symbol in the interactive window to stop cleaning here. <br />
<br />
Note that if you run the {{tclean}} task again with the same '''imagename''', without deleting the existing <imagename>.* files, {{tclean}} assumes that you want to continue cleaning the existing images. We put an <tt>rm</tt> command before the {{tclean}} command to guard against this, but sometimes this is what you want.<br />
<br />
==== Southern Continuum Mosaic ====<br />
<br />
<br />
[[File:South_cont_clean.png|200px|thumb|right|<caption>'''Fig. 4.''' Residual for Southern continuum mosaic after 100 iterations; the clean mask is shown by the white contour.</caption>]]<br />
<br />
<br />
For the southern mosaic we modify the '''phasecenter''', mosaic '''imsize''', and the line-free channels ('''spw''') to be consistent for this mosaic. We also bypass the dirty image step we did above for the Northern mosaic. As above, the mosaic size can be judged from the [[AntennaeBand7#ALMA_Data_Overview | Overview]]. We expect the beam to be about the same and use the same '''cell''' size. Again, you can use the provided mask or make your own.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_South.Cont.Clean'+ext)<br />
<br />
tclean(vis='Antennae_South.cal.ms',imagename='Antennae_South.Cont.Clean',<br />
field='',phasecenter=15,<br />
specmode='mfs',restfreq='345.79599GHz',<br />
spw='0:1~30;120~164',<br />
gridder='mosaic', mosweight=True, deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',<br />
interactive=True, <br />
niter=1000, threshold='0.4mJy')<br />
</source><br />
<br />
The beam size reported in the logger for the Southern mosaic: 1.13"x 0.67", and P.A.= 61 deg; which is a little better than the beam for the North owing to better uv-coverage.<br />
<br />
Stop after ~50 iterations (Figure 4).<br />
<br />
==== Image Statistics ====<br />
<br />
<br />
[[File:North_contstat.png|200px|thumb|right|<caption>'''Fig. 5.''' Example polygon for rms determination using the viewer.</caption>]]<br />
<br />
<br />
<br />
You can determine statistics for the images using the task {{imstat}}:<br />
<source lang="python"><br />
# In CASA<br />
imstat('Antennae_North.Cont.Clean.image')<br />
imstat('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
From this we find, that for the Northern continuum image the peak is 3.86 mJy/beam and the rms is 0.49 mJy/beam. For the Southern continuum we find a peak of 4.70 mJy/beam and an rms of 0.47 mJy/beam. <br />
<br />
However, the calculation of the rms comes with a couple of caveats. First, the mosaic primary beam response rolls off toward the edges of the mosaic, as do correspondingly the flux density and rms. Thus if you don't restrict the measurement to areas of full sensitivity, the apparent rms is skewed downward. Second, since there is real emission in the image, the rms will be skewed upward (with the error increasing with brighter emission). Both can be solved by picking boxes that exclude the edges of the mosaic and the real emission. <br />
<br />
It is often easier to use either the {{viewer}} directly or {{imview}} (a task wrapper for the {{viewer}}; see '''help imview''' for more info) to display the image and then interactively use the region tools to get the statistics.<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.Cont.Clean.image')<br />
</source><br />
<br />
Adjust colorscale using the middle mouse button (or reassign to a different mouse button by clicking on "plus" symbol icon). Next, select the polygon tool by clicking on its symbol with a mouse button. Then draw a region that avoids edges and emission (see Figure 5), then double click inside the polygon to have the statistic printed to the screen. You may measure an rms that differs by a few percent from the value reported by imstat.<br />
<br />
<source lang="python">exit<br />
# In CASA<br />
viewer('Antennae_South.Cont.Clean.image')<br />
</source><br />
<br />
<br />
[[File:Antennae_North.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 6.''' 345 GHz continuum image of the northern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.Cont.Clean.image.png|200px|thumb|right|<caption>'''Fig. 7.''' 345 GHz continuum image of the southern mosaic.</caption>]]<br />
<br />
<br />
Again, you may measure an rms slightly different than what we got from using {{imstat}} with no constraints. <br />
<br />
How does this compare to theory? You can find out using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator]. The continuum bandwidth of the Northern mosaic is about 1 GHz and about 0.85 GHz for the Southern mosaic. The number of antennas is about 12 and the time on a single pointing is about 300s. This yields an expected rms of about 1 mJy/beam. However, this needs to be decreased by about a factor of 2.5 near the center of the mosaic due to the hexagonal Nyquist sampling of the mosaic (radial spacing~0.37*HPBW) for a theoretical rms of about 0.4 mJy/beam, in good agreement with observation. <br />
<br />
Next make hardcopy plots of the continuum images using {{imview}}. To make the contrast a bit better, set the data range from -1 x sigma to the peak determined above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.Cont.Clean.image',<br />
'colorwedge':True,'range':[-0.00049,0.00386]},<br />
zoom=1, out='Antennae_North.Cont.Clean.image.png')<br />
imview(raster={'file': 'Antennae_South.Cont.Clean.image',<br />
'colorwedge':True,'range':[-0.00047,0.00470]},<br />
zoom=1, out='Antennae_South.Cont.Clean.image.png')<br />
</source><br />
<br />
==Continuum subtraction ==<br />
<br />
In these data, the continuum emission is too weak to contaminate the line emission (i.e. the peak continuum emission is less than the rms noise in the spectral line channels). Nevertheless, for illustrative purposes we demonstrate how to subtract the continuum emission in the uv-domain using the task {{uvcontsub}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='Antennae_North.cal.ms',fitspw='0:1~50;120~164',fitorder = 1)<br />
# In CASA<br />
uvcontsub(vis='Antennae_South.cal.ms',fitspw='0:1~30;120~164',fitorder = 1)<br />
</source><br />
<br />
Here, '''fitspw''' gives the line-free channels for each mosaic and '''fitorder=1'''. Higher order fits are not recommended. If you do not have line-free channels on both sides of the line '''fitorder=0''' is recommended. The output MS will have ''.contsub'' appended to the name.<br />
<br />
In CASA 5.4, you will see a warning that uvcontsub is using the original Visibility Iterator, which is the underlying iteration and retrieval mechanism used when processing visibility data. <br />
A new version of the Visibility Iterator has been developed and deployed in several other calibration tasks in CASA 5.0 and later (see [https://science.nrao.edu/enews/casa_005/ CASA eNews] for more information). <br />
We can ignore this warning. <br />
<br />
<pre style="background-color: #fffacd;"><br />
WARN calibrater Forcing use of OLD VisibilityIterator.<br />
</pre><br />
<br />
== CO(3-2) Imaging ==<br />
<br />
<br />
[[File:North_CO3_2_vel.png|thumb|<caption>'''Fig. 8.''' Northern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
[[File:South_CO3_2_vel.png|thumb|<caption>'''Fig. 9.''' Southern CO(3-2) uv-spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''specmode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. <br />
When making spectral images you have three choices for the '''specmode''' parameter: '''mfs''', '''cube''', and '''cubedata'''. <br />
As noted above, setting '''specmode='mfs'''' gives an output image with only one channel, and is used for continuum imaging. The other '''specmode''' options create output data cubes with one or more channels.<br />
<br />
Data are taken using constant frequency channels. <br />
For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. <br />
For '''specmode='cube'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for scientific analysis. <br />
The Doppler Shift is taken out during the regridding to the desired outframe in {{tclean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{tclean}}. <br />
<br />
To see what velocity parameters you want to set in {{tclean}}, it is useful to make a plot in {{plotms}} in the desired output frame. Note these plots take a little longer because of the frame shift. In order to compare with recent SMA data, we chose LSRK, but it should be noted that many papers of this source are in the BARY frame. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf North_CO3_2_vel.png')<br />
plotms(vis='Antennae_North.cal.ms.contsub/',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='North_CO3_2_vel.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf South_CO3_2_vel.png')<br />
plotms(vis='Antennae_South.cal.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotfile='South_CO3_2_vel.png', showgui = True) <br />
</source><br />
<br />
==== Northern Mosaic ====<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we provide a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel. You can choose to use and edit the provided mask or create your own mask.<br />
<br />
Notable parameters included in the {{tclean}} call are:<br />
*'''specmode='cube' ''','''outframe='LSRK' ''','''restfreq='345.79599GHz' '''. We use 'cube' mode to create an output data cube. We set the velocity information in the Local Standard of Rest frame (kinematic definition), and use the rest frequency of the CO(3-2) line.<br />
*'''nchan=60''','''start='1300km/s' ''','''width='10km/s' ''': To produce a data cube with 60 channels ("nchan"=60), starting at 1300km/s and with velocity widths of 10 km/s. That will include all the CO(3-2) emission in both mosaics.<br />
*'''imsize=500''','''cell='0.13arcsec' ''': An image size of 65 arcsec, with pixels of 0.13 arcsec (about one-fifth the minor axis of the synthesized beam). We make the imsize larger than the mosaic to increase the dirty beam image size, as the emission is quite extended along the mosaic. <br />
*'''weighting='briggs' ''','''robust=0.5''': Weighting to apply to visibilities. We use Briggs' weighting and robustness parameter 0.5 (in between natural and uniform weighting).<br />
*'''niter=20000''': Maximum number of clean iterations. With complex emission structure like that found in the Antennae CO data, we will need a large number of iterations to clean the emission. Note that you can increase the number of iterations allowed on-the-fly in the interactive {{tclean}} window. <br />
*'''threshold='5.0mJy' ''': Stop cleaning if the maximum residual is below this value. <br />
The threshold is set to roughly the rms of a single line-free channel for each dataset. <br />
*'''savemodel='modelcolumn'''': This saves the clean model to the ms file. We will need this for self-calibration later. <br />
<br />
[[File:North_interact.png|200px|thumb|right|<caption>'''Fig. 10.''' Interactive cleaning of northern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_North.CO3_2Line.Clean'+ext)<br />
<br />
tclean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter=12,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic', mosweight=True,<br />
deconvolver='hogbom',<br />
imsize=500,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
Figure 10 shows what the final clean box should look like. Cycle through the channels until you reach ~1650 km/s, where you can more easily see where to draw the clean box around the CO emission. Be sure the check the circle next to 'All Channels' when making your clean mask. Now clean using the green circle arrow. You can watch the progress in the logger. When the first ~650 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each channel with significant signal. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. To make it go a bit faster, you can increase '''iteration''' interactively, to 200 or 300, but don't overdo it.<br />
<br />
These data are pretty severely ''dynamic range'' limited, in part due to the sparse uv coverage. In other words, the noise in bright channels is set by a maximum signal-to-noise. This effectively prevents us from stopping clean based on an rms based '''threshold''' because the effective rms changes as a function of the brightest signal in each channel. Thus, in this case we need to stop clean interactively. Below an approximate number of '''iterations''' is given to help you decide when to quit. NOTE: the '''threshold''' that is set in the CO(3-2) clean commands are about equal to the noise in a line-free channel and is only there to prevent clean running forever if you fail to stop it. You are not intended to clean to this '''threshold'''.<br />
<br />
Also, we are going to self-calibrate the data so it's best to be conservative here - it can be difficult in images like these to discern real features from artifacts. If in doubt, don't include it in the clean mask. Keep cleaning and see if the feature becomes weaker. You cannot lose real emission by not masking it, but you can create a brighter feature by masking an artifact. <br />
<br />
Stop cleaning after about ~7000 iterations (Red X), when the artifacts start to look as bright as the residual. Note again that we are not cleaning to the theoretical noise level. <br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_North.CO3_2Line.Clean.image')<br />
</source><br />
<br />
==== Southern Mosaic ====<br />
<br />
[[File:South_interact.png|200px|thumb|right|<caption>'''Fig. 11.''' Interactive cleaning of southern mosaic. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
Repeat process for Southern mosaic.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for ext in ['.image','.model','.image.pbcor','.psf','.residual','.pb','.sumwt','.weight']:<br />
rmtables('Antennae_South.CO3_2Line.Clean'+ext)<br />
<br />
tclean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean',<br />
spw='0',field='',phasecenter=15,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
Clean about 8,000 iterations before stopping.<br />
<br />
Inspect the resulting data cube:<br />
<source lang="python"><br />
# In CASA<br />
viewer('Antennae_South.CO3_2Line.Clean.image')<br />
</source><br />
<br />
For each mosaic use the '''viewer''' polygon tool as described above for the continuum, to find the image statistics in both a line-free channel and the channel with the strongest emission. <br />
<br />
The line-free channel rms for the northern and southern mosaics are about 4.0 mJy/beam and 3.5 mJy/beam, respectively.<br />
However in a bright channel this degrades to about 18 mJy/beam and 9 mJy/beam, respectively. This is the aforementioned dynamic range limit. <br />
<br />
Using the [https://almascience.nrao.edu/call-for-proposals/sensitivity-calculator ALMA sensitivity calculator], a bandwidth of 10 km/s, dual polarization, 12 antennas, and the time on a single pointing of about 300s yields an rms of about 9.0 mJy/beam. As described above the Nyquist sampled hexagonal mosaic improves this by a factor of about 2.5 for an expected rms noise of about 3.6 mJy. So in line-free channels we are doing close to theoretical.<br />
<br />
== Self Calibration == <br />
<br />
Next we attempt to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model image that you give it. When {{tclean}} is run, it saves a uv-model of the resulting image in the MODEL column of the measurement set if ''savemodel'' = ''modelcolumn''. This model can be used to self-calibrate the data using {{gaincal}}. That is why it is important to make the model as good as you can by making clean masks. <br />
<br />
Mosaics can be tricky to self-calibrate. The reason is that typically only a few of the pointings may have strong emission. The stronger the signal, the stronger the signal-to-noise will be for a given self-calibration solution interval ('''solint'''). This means that typically only a few fields are strong enough for self-calibration, though it can still bring about overall improvement to apply these solutions to all fields. The fine tuning of each field in the mosaic over small solutions intervals is often impossible. Indeed fine-tuning only a few fields in the mosaic on small time scales can result in a mosaic with dramatically different noise levels as a function of position. Below, we will simply attempt to self-calibrate on the strongest field in the mosaic, obtaining one average solution per original input dataset. <br />
<br />
Here, the continuum is far too weak so we will use the CO(3-2) line emission. To increase signal-to-noise it is often helpful to limit solutions to the range of channels containing the strongest line emission. <br />
<br />
We will also use '''gaintype='T'''' to tell CASA to average the polarizations before deriving solutions which gains us a sqrt(2) in sensitivity.<br />
<br />
==== Northern Mosaic ====<br />
<br />
<br />
[[File:north_selfchan.png|200px|thumb|right|<caption>'''Fig. 12.''' Plot of the CO(3-2) emission from the data column of field 12 of the Northern mosaic.</caption>]]<br />
<br />
<br />
The strongest emission in the Northern mosaic comes from the center pointing, already identified above as '''field='12' '''. Below we make a uv-plot of the spectral line emission from this field to chose only the strongest channels to include in the self-calibration. ''It is essential that the self-calibration channel range be chosen based on the UV data and not on the image cube.'' This is because the channels in the image cube were modified by the velocity/channel specifications in the first invocation of clean. This plot shows the channel range of the uv data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='data',plotfile='North_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
We also want to check the channel amplitudes in the model column of the measurement set. The depth of the initial clean will determine what model channels have emission. (A very shallow clean could potentially produce a model with emission over a small range of channels.) Self calibration will attempt to make the data look like the model. Therefore, it is important that we select a channel range where the model is realistic. <br />
<br />
You will notice that while the model looks reasonable across the channels where we imaged the data, the model amplitudes are set to 1 elsewhere. When {{tclean}} initializes a model, it defaults to an amplitude of 1 and phase 0 (i.e., a point source with an amplitude of 1 in the center of the image). When we set '''savemodel = 'modelcolumn'''' in the {{tclean}} call, the default model was replaced ''only where {{tclean}} had a model to replace it with'', namely, over the channels we imaged. In future, CASA will set a flag to let users know that these model points are not valid for the updated model. In our case, we will perform self-calibration only on those channels with significant signal in the line emission, omitting the rest of the channels. <br />
<br />
<br />
[[File:North_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 13.''' Plot of the CO(3-2) emission from the model column of field 12 of the northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',field='12',<br />
avgtime='1e8',avgscan=True,ydatacolumn='model',plotfile='North_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
Next remind ourselves of the timing of the various observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_North.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='12',<br />
avgchannel='167',coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
Zooming in on each dataset we see that they are about 1 hour long (you can also check the {{listobs}} output), and that each dataset has 2 or 3 observations of field 12. Each time, it's observed for about 25 seconds. We set '''solint='3600s'''' to get one solution per dataset.<br />
<br />
[[File:north_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 14.''' Phase-only self-calibration solutions for the Northern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_North.cal.ms.contsub',caltable='north_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='12',<br />
refant='DV09',spw='0:82~89',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Plot the calibration solutions. Note that when '''iteraxis''' is set, {{plotms}} appends the axes being iterated over to the output filename specified in '''plotfile'''; this can make for rather unwieldy filenames but is useful for ALMA pipeline purposes. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='north_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteraxis='antenna',gridrows=5,gridcols=3,plotrange=[0,0,-80,80],<br />
plotfile='north_pcal1_phase.png', showgui = True)<br />
<br />
</source><br />
<br />
Apply the solutions to all fields and channels with {{applycal}}, which will overwrite the corrected data column:<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_North.cal.ms.contsub',field='',<br />
gaintable=['north_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
Re-image the data with the selfcal applied. Remember that {{tclean}} always uses the corrected data column if it exists:<br />
<br />
[[File:North_interact2.png|200px|thumb|right|<caption>'''Fig. 15.''' Interactive cleaning of Northern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "all channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_North.CO3_2Line.Clean.pcal1*')<br />
tclean(vis='Antennae_North.cal.ms.contsub',<br />
imagename='Antennae_North.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter=12,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',mosweight=True,deconvolver='hogbom',<br />
imsize=500,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,mask='Antennae_North.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=20000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
After 3000 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Remember to select the "all channels" toggle. Stop after 9,000 or so iterations.<br />
<br />
Compare the un-selfcal'ed and selfcal'ed data to determine if there has been improvement.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_North.CO3_2Line.Clean.image',<br />
'range': [-0.07,0.4]},<br />
{'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.07,0.4]}])<br />
<br />
</source><br />
<br />
<!--<br />
zoom={'channel': 43}) doesn't work with two images<br />
--><br />
<br />
Then in the viewer, activate the Animator Images sub-panel by clicking the check box next to "Images". Then use the tape deck controls to cycle between the two images. To change the channel use the Channels sub-panel in the Animator. <br />
To make the figure shown in Figure 16, cycle to channel 43. Then you can select the "p wrench" icon and change the number of panels in x to 2. You will need to drag your viewer window rather wide to get both panels to fit well. If you want to compare other channels, you can use the tapedeck to move to the new channel of interest and see the changes in both cubes.<br />
<br />
To compare the statistics for each image, click the "Next" button in the "Statistics" tab of the "Region" sub-panel. <br />
<br />
What you should notice is that the peak flux density has increased substantially while the rms noise is about the same. This is reasonable for this self-calibration case because we have mostly adjusted relative position offsets between the datasets and not taken out any short-term phase variations which would reduce the rms. <br />
<br />
Note that attempts to push the selfcal to shorter solint (for example to each scan of field=12) did not yield additional improvement. <br />
<br />
<br />
[[File:north_ch43compare.png|600px|thumb|center|<caption>'''Fig. 16.''' Comparison of channel 43 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 25% in the peak flux density (in channel 43).<br />
</div><br />
<br />
==== Southern Mosaic ====<br />
<br />
<br />
[[File:south_selfchan.png|200px|thumb|right|<caption>'''Fig. 17.''' Plot of the CO(3-2) emission from the data column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[File:South_selfchan_model.png|200px|thumb|right|<caption>'''Fig. 18.''' Plot of the CO(3-2) emission from the model column for Field 7 of the Southern mosaic.</caption>]]<br />
<br />
<br />
Now repeat the self-calibration process for the Southern mosaic. It is a bit tougher in this case to pick the best field. The most compact bright emission is toward field id 11 in the [[AntennaeBand7#Science_Target_Overview | Overview]]. Adjusting for the renumbering of field ids after the final calibration split, this becomes field='7'. <br />
<br />
From the following plots we can assess the brightest channels to pick and see that the timing in the 6 Southern datasets is similar to the North, with each lasting about 1 hour. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='data',<br />
plotfile='South_selfchan_data.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='channel',yaxis='amp',<br />
field='7',avgtime='1e8',avgscan=True,ydatacolumn='model',<br />
plotfile='South_selfchan_model.png', showgui = True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Antennae_South.cal.ms.contsub',spw='',xaxis='time',yaxis='amp',field='7',<br />
avgchannel='167',avgscan=True,coloraxis='scan',ydatacolumn='data', showgui = True)<br />
</source><br />
<br />
<br />
[[File:south_pcal1_phase.png|200px|thumb|right|<caption>'''Fig. 19.''' Phase-only self-calibration solutions for the Southern mosaic.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='Antennae_South.cal.ms.contsub',caltable='south_self_1.pcal',<br />
solint='3600s',combine='scan',gaintype='T',field='7',<br />
refant='DV09',spw='0:70~85',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='south_self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',<br />
iteraxis='antenna',gridrows=5,gridcols=3,plotrange=[0,0,-80,80],<br />
plotfile='south_pcal1_phase.png', showgui = True)<br />
<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='Antennae_South.cal.ms.contsub',field='',<br />
gaintable=['south_self_1.pcal'],calwt=False)<br />
</source><br />
<br />
<br />
[[File:South_interact2.png|200px|thumb|right|<caption>'''Fig. 20.''' Interactive cleaning of Southern mosaic after self-cal. The clean mask is shown in white. In this example we used the same mask for all channels by selecting "All Channels" before drawing mask.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Antennae_South.CO3_2Line.Clean.pcal1*')<br />
tclean(vis='Antennae_South.cal.ms.contsub',<br />
imagename='Antennae_South.CO3_2Line.Clean.pcal1',<br />
spw='0',field='',phasecenter=15,<br />
specmode='cube',outframe='LSRK',restfreq='345.79599GHz',<br />
nchan=70,start='1200km/s',width='10km/s',<br />
gridder='mosaic',mosweight=True,deconvolver='hogbom',<br />
imsize=750,cell='0.13arcsec',pblimit=0.2,<br />
restoringbeam='common',<br />
interactive=True,mask='Antennae_South.CO3_2Line.Clean.mask',<br />
weighting='briggs',robust=0.5,<br />
niter=40000, threshold='5.0mJy',<br />
savemodel='modelcolumn')<br />
</source><br />
<br />
After 4000 iterations or so, inspect each channel to see if there is more emission that needs to be included in the mask. Stop after 25,000 or so iterations.<br />
<br />
As for the Northern mosaic, you can compare the before and after self-cal images (look, for example, at channel 31):<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file': 'Antennae_South.CO3_2Line.Clean.image',<br />
'range': [-0.05,0.2]},<br />
{'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.05,0.2]}])<br />
</source><br />
<br />
<!--<br />
zoom={'channel': 31})<br />
doesn't work for multiple images<br />
--><br />
<br />
<br />
[[File:South_ch31compare.png|600px|thumb|center|<caption>'''Fig. 21.''' Comparison of channel 31 before selfcal (left) and after selfcal (right).</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used '''standard='Butler-JPL-Horizons 2012' ''' when running {{setjy}} during the calibration process, you will see an increase of approximately 24% in the peak flux density (in channel 31).<br />
</div><br />
<br />
== Image Analysis ==<br />
<br />
==== Moment Maps ====<br />
<br />
<br />
[[File:Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 22.''' The CO(3-2) integrated intensity map (moment 0) of the Northern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 23.''' The CO(3-2) velocity field (moment 1) of the Northern mosaic.</caption>]]<br />
<br />
[[File:Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 24.''' The CO(3-2) velocity dispersion (moment 2) of the Northern mosaic.</caption>]]<br />
<br />
[[File:Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png|200px|thumb|right|<caption>'''Fig. 25.''' The CO(3-2) integrated intensity map (moment 0) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png|200px|thumb|right|<caption>'''Fig. 26.''' The CO(3-2) velocity field (moment 1) of the Southern mosaic.</caption>]]<br />
<br />
<br />
[[File:Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png|200px|thumb|right|<caption>'''Fig. 27.''' The CO(3-2) velocity dispersion (moment 2) of the Southern mosaic.</caption>]]<br />
<br />
<br />
Next we will make moment maps for the CO(3-2) emission: Moment 0 is the integrated intensity; Moment 1 is the intensity weighted velocity field; and Moment 2 is the intensity weighted velocity dispersion. <br />
<br />
Above we determined the rms noise levels for both the North and South mosaics in both a line-free and a line-bright channel. We want to limit the channel range of the moment calculations to those channels with significant emission. One good way to do this is to open the cube in the viewer overlaid with 3-sigma contours, with sigma corresponding to the line-free rms. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.004],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 33~63.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'range': [-0.04,0.4]},<br />
contour={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
'levels': [0.0035],'unit': 5})<br />
</source><br />
<br />
We find a channel range for significant emission of 12~64.<br />
<br />
For moment 0 (integrated intensity) maps you do not typically want to set a flux threshold because this will tend to noise bias your integrated intensity.<br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[0],chans='33~63',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
moments=[0], chans='12~64', <br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0')<br />
</source><br />
<br />
For higher order moments it is very important to set a conservative flux threshold. Typically something like 3sigma, using sigma from a bright line channel works well. We do this with the '''mask''' parameter in the commands below. When making multiple moments, {{immoments}} appends the appropriate file name suffix to the value of '''outfile'''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_North.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='33~63', <br />
mask='Antennae_North.CO3_2Line.Clean.pcal1.image>0.018*3',<br />
outfile='Antennae_North.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
immoments('Antennae_South.CO3_2Line.Clean.pcal1.image', <br />
moments=[1,2], chans='12~64',<br />
mask=' Antennae_South.CO3_2Line.Clean.pcal1.image>0.009*3',<br />
outfile='Antennae_South.CO3_2Line.Clean.pcal1.image.mom')<br />
</source><br />
<br />
Next we can create the six moment maps<br />
(Figures 22,<br />
23, 24, 25, 26, and 27)<br />
from these images using '''imview'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':True,'scaling': -0.5},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
'colorwedge':True,'scaling': -0.5},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.png')<br />
<br />
os.system('rm -f Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png')<br />
imview(raster={'file': 'Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
'colorwedge':True},<br />
zoom=1,out='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.png') <br />
</source><br />
<br />
==== Exporting Images to Fits ====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_North.Cont.Clean.image',<br />
fitsimage='Antennae_North.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_North.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
exportfits(imagename='Antennae_South.Cont.Clean.image',<br />
fitsimage='Antennae_South.Cont.Clean.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom0.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_coord.fits')<br />
exportfits(imagename='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord',<br />
fitsimage='Antennae_South.CO3_2Line.Clean.pcal1.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=True''' and/or '''dropstokes=True'''.<br />
<br />
==Comparison with previous SMA CO(3-2) data==<br />
<br />
We compare with SMA CO(3-2) data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65). Figure 28 shows a comparison plot between the moment 0 maps of ALMA and SMA data using the {{viewer}}. The fluxes, peak locations, and large scale structure are consistent. Both southern and northern components have been combined.<br />
<br />
<br />
[[File: alma_sma_compare.png|500px|thumb|center|<caption>'''Fig. 28.''' The CO(3-2) total intensity map (moment 0) comparison with SMA data. Colour image is ALMA data, combining southern and northern mosaics. Contours show SMA data (Ueda, Iono, Petitpas et al. 2012, ApJ, 745, 65).</caption>]]<br />
<br />
{{Checked 5.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_5.4&diff=30887AntennaeBand7 Calibration 5.42021-08-20T19:08:04Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 5.4.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7_Imaging]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection; for example, data where one telescope blocks the line of sight of another. We will then apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Using observations of the calibrators Titan and 3c279, it will be shown how to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these solutions to the data and then extract the calibrated source data into a file appropriate for imaging in the [[AntennaeBand7_Imaging | Antennae Band 7 imaging guide]].<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
This guide requires CASA 5.4.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]. <br />
<br />
Once you have downloaded the data, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casa<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.4.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '5.4.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Figure 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT_5.4.png|200px|thumb|right|<caption>'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task plotants. </caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
</pre><br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=False)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = False)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = False: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = False)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=False)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=True so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=True.<br />
<br />
For example try <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
</source><br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=False. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window '''spw''' = 5 with the task {{plotms}}. In previous CASA versions, we used {{plotcal}} for this task, but most of '''plotcal''''s functionality is now included in {{plotms}}, and '''plotcal''' will be deprecated in a future CASA release. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. <!--Here and throughout we focus on spw 1, which contains CO(3-2): --><br />
<br />
<pre style="background-color: #E0FFFF;"><br />
Note that right now, plotms can't select individual channels when plotting caltables (as was possible with plotcal). In addition, when iteraxis is set, plotms appends the axes being iterated over to the output filename specified in plotfile; this can make for rather unwieldy filenames (particularly when iteraxis='antenna'!) but is useful for the ALMA pipeline. <br />
</pre><br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotms(vis=asdm+'.tdm.tsys',<br />
xaxis='time',yaxis='tsys',coloraxis='corr',<br />
spw='5:50~50',<br />
gridrows=4,gridcols=2,<br />
antenna='0~7',iteraxis='antenna',<br />
plotfile=asdm+'.tsys_vs_time.page1.png',overwrite=True)<br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotms(vis=asdm+'.tdm.tsys',<br />
xaxis='time',yaxis='tsys',coloraxis='corr',<br />
spw='5:50~50',<br />
gridrows=4,gridcols=2,<br />
antenna='8~15',iteraxis='antenna',<br />
plotfile=asdm+'.tsys_vs_time.page2.png',overwrite=True)<br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotms}}:<br />
<br />
* '''gridrows'''=4 and '''gridcols'''=2 sets up a 4 x 2 panel grid. <br />
* '''iteraxis'''='antenna' tells {{plotms}} to make a separate plot for each antenna.<br />
* '''spw'''='5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations. '''Note that this is a functionality in plotcal but not yet in {{plotms}}, and plotms is actually showing all channels from spw 5 at this time.'''<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two {{plotms}} calls: one for the first 8 antennas ('''antenna'''='0~7'), and then for the remaining antennas ('''antenna'''='8~15').<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''plotfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer. As noted above, here plotms is adding the antenna information to the filenames because we set '''iteraxis''' = 'antenna', so check your directory for the impressively long filenames to view them. <br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
You will see the following statement in the logger, but this is fine: <br />
<br />
<pre style="background-color: #fffacd;"><br />
No ASDM_CALWVR, ASDM_CALATMOSPHERE, or CalWVR.xml table found. Using PWV 1.0mm.<br />
</pre><br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figures 4, 5, and 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
<br />
In general, the most appropriate antenna positions for the dataset will be determined and updated if needed during QA2. If the antenna positions need to be updated after QA2, we inform the PIs of the affected datasets and provide instructions on how to update the antenna positions. For the data in this guide, we do need to update the antenna positions. <br />
<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. {{gencal}} will now be used put antenna position data into each observation. Again, {{gencal}} will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running {{gencal}} to create the information CASA can refer to for antenna positions.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=False, calwt=True)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=False, calwt=True)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
The MODEL column is now deprecated (though it is still possible to use it by explicitly setting usescratch=True).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
<!--<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
--><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=False tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=False)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Figure 10 and Figure 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field', showgui = True)<br />
user_input = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.[type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
user_input = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+". Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Figure 15) and that Titan also shows evidence of a strong line (see Figure 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = False)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Figure 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = False)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = False)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Figure 4 and Figure 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Figure 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Figure 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Figure 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=False)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=False)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=False)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Figure 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=True, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Figure 23).<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=True, <br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=True,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=True,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. '''For what systems is this true?''' As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
<!--<br />
Again this warning:<br />
No ASDM_CALWVR, ASDM_CALATMOSPHERE, or CalWVR.xml table found. Using PWV 1.0mm.<br />
--><br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27. We run the following additional flagging step:<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=False, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator.<br />
<br />
<source lang="python"><br />
# in CASA<br />
# make sure usescratch=True<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=True,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2012')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 3.3 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan: spw0 Flux:[I=3.2637,Q=0.0,U=0.0,V=0.0] +/- [I=0.0,Q=0.0,U=0.0,V=0.0] Jy @ 344.85GHz<br />
</pre><br />
<br />
Now we are going to take a look at the Titan model for one dataset:<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
</source><br />
<br />
[[File:2012_titan_model_5.1.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model produced by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model produced by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2012' '''. Previous versions of this guide used an older flux density standard '''standard='Butler-JPL-Horizons 2010' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. <br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.79 Jy, by averaging the fluxes obtained from the ten available datasets. If we check this value against reported flux measurements in the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html], we find that a flux of 10.79 Jy agrees within a few per cent of a linear interpolation between the two 0.85 mm measurements closest in time to the ALMA observations: (23 May 2011, SMA 11.644 ± 0.583; 01 July 2011, 9.748 ± 0.488).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the older, Butler-JPL-Horizons 2010, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly lower. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.41 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotms(vis=asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase', coloraxis='corr',<br />
iteraxis='antenna',antenna='0~7',plotrange=[0,0,-180,180],<br />
gridrows=4,gridcols=1,showgui=False,<br />
plotfile=asdm+'.scanphase.page1.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotms(vis=asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase', coloraxis='corr',<br />
iteraxis = 'antenna', antenna='8~15', plotrange=[0,0,-180,180],<br />
showgui=False, gridrows=4, gridcols=1, <br />
plotfile=asdm+'.scanphase.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotms(vis = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp', coloraxis='corr',<br />
iteraxis='antenna', antenna='0~7', plotrange = [0,0,0,0],<br />
showgui=False, gridrows=4,gridcols=1, <br />
plotfile=asdm+'.flux.page1.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotms(vis = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp', coloraxis='corr',<br />
iteraxis='antenna', antenna='8~15', plotrange = [0,0,0,0],<br />
showgui=False, gridrows=4,gridcols=1, <br />
plotfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=True, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=True,avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
Note that you will see a warning that the last output channel in the new spw 0 will be dropped, as it has less than 23 channels over which to average:<br />
<br />
<pre style="background-color: #fffacd;"><br />
INFO MSTransformDataHandler The last output channel of spw 0 has only 22 input channels.<br />
WARN MSTransformManager Not enough input channels to populate output channel n# 166 from SPW 0.<br />
WARN MSTransformManager The channel will be dropped in order to have an uniform grid.<br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=True)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=True)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging imaging guide].<br />
{{Checked 5.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_5.4&diff=30886AntennaeBand7 Calibration 5.42021-08-20T19:05:54Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide requires CASA 5.4.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7_Imaging]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection; for example, data where one telescope blocks the line of sight of another. We will then apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Using observations of the calibrators Titan and 3c279, it will be shown how to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these solutions to the data and then extract the calibrated source data into a file appropriate for imaging in the [[AntennaeBand7_Imaging | Antennae Band 7 imaging guide]].<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
This guide requires CASA 5.4.0 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]. <br />
<br />
Once you have downloaded the data, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
<br />
casa<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 5.4.0. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casadef<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '5.4.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print asdm<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (Figure 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT_5.4.png|200px|thumb|right|<caption>'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task plotants. </caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print "Antenna configuration for : "+asdm<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = raw_input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "raw_input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print asdm<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print basename_all". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
</pre><br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Resetting flags for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=False)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging shadowed data for "+asdm<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = False)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = False: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging calibration scans for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = False)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Flagging autocorrelation data for "+asdm<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=False)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Backing up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print "Restoring up 'a priori' flags for "+asdm<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=True so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=True.<br />
<br />
For example try <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
</source><br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=False. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print "Creating TDM Tsys Table for "+asdm<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window '''spw''' = 5 with the task {{plotms}}. In previous CASA versions, we used {{plotcal}} for this task, but most of '''plotcal''''s functionality is now included in {{plotms}}, and '''plotcal''' will be deprecated in a future CASA release. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. <!--Here and throughout we focus on spw 1, which contains CO(3-2): --><br />
<br />
<pre style="background-color: #E0FFFF;"><br />
Note that right now, plotms can't select individual channels when plotting caltables (as was possible with plotcal). In addition, when iteraxis is set, plotms appends the axes being iterated over to the output filename specified in plotfile; this can make for rather unwieldy filenames (particularly when iteraxis='antenna'!) but is useful for the ALMA pipeline. <br />
</pre><br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.tsys_vs_time.page2.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotms(vis=asdm+'.tdm.tsys',<br />
xaxis='time',yaxis='tsys',coloraxis='corr',<br />
spw='5:50~50',<br />
gridrows=4,gridcols=2,<br />
antenna='0~7',iteraxis='antenna',<br />
plotfile=asdm+'.tsys_vs_time.page1.png',overwrite=True)<br />
#dummy_string = raw_input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotms(vis=asdm+'.tdm.tsys',<br />
xaxis='time',yaxis='tsys',coloraxis='corr',<br />
spw='5:50~50',<br />
gridrows=4,gridcols=2,<br />
antenna='8~15',iteraxis='antenna',<br />
plotfile=asdm+'.tsys_vs_time.page2.png',overwrite=True)<br />
#dummy_string = raw_input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotms}}:<br />
<br />
* '''gridrows'''=4 and '''gridcols'''=2 sets up a 4 x 2 panel grid. <br />
* '''iteraxis'''='antenna' tells {{plotms}} to make a separate plot for each antenna.<br />
* '''spw'''='5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations. '''Note that this is a functionality in plotcal but not yet in {{plotms}}, and plotms is actually showing all channels from spw 5 at this time.'''<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two {{plotms}} calls: one for the first 8 antennas ('''antenna'''='0~7'), and then for the remaining antennas ('''antenna'''='8~15').<br />
<br />
The 'raw_input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''plotfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer. As noted above, here plotms is adding the antenna information to the filenames because we set '''iteraxis''' = 'antenna', so check your directory for the impressively long filenames to view them. <br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV02.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.DV08.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_Xb.tdm.tsys.3c279.PM01.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X215db8_X392.tdm.tsys.3c279.DV11.spw5.CASA3_4.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
You will see the following statement in the logger, but this is fine: <br />
<br />
<pre style="background-color: #fffacd;"><br />
No ASDM_CALWVR, ASDM_CALATMOSPHERE, or CalWVR.xml table found. Using PWV 1.0mm.<br />
</pre><br />
<br />
Now have a look at the Tsys vs. frequency plots or see Figures 4, 5, and 6 for examples on the first data set. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7 for an example from a later data set). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
<br />
In general, the most appropriate antenna positions for the dataset will be determined and updated if needed during QA2. If the antenna positions need to be updated after QA2, we inform the PIs of the affected datasets and provide instructions on how to update the antenna positions. For the data in this guide, we do need to update the antenna positions. <br />
<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. {{gencal}} will now be used put antenna position data into each observation. Again, {{gencal}} will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running {{gencal}} to create the information CASA can refer to for antenna positions.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from recipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=False, calwt=True)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print "Apply Tsys, WVR, and Antenna Position calibrations to "+asdm<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=False, calwt=True)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
The MODEL column is now deprecated (though it is still possible to use it by explicitly setting usescratch=True).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the first Northern Mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-corr.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
[[File:plotms_amp_vs_time_south.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=True, spw='1:1200~1300', <br />
avgchannel='1000',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
<!--<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
--><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print "Showing baseline "+ant_str+" for data set "+asdm<br />
print "Use this to inspect effect of applying wvrcal and Tsys calibrations."<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print "... "+y_axis+" vs. frequency for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print "... "+y_axis+" vs. time for DATA:" <br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print "... "+y_axis+" vs. time for CORRECTED:"<br />
plotms(vis=asdm+'.ms', spw='1:1200~1300', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="1000", coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = raw_input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
[[File:plotms_amp_vs_freq_Titan.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=False tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print "Splitting out corrected data for "+asdm<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=False)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Figure 10 and Figure 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field', showgui = True)<br />
user_input = raw_input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.[type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12 shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
user_input = raw_input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+". Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Figure 15) and that Titan also shows evidence of a strong line (also Figure 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = False)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Figure 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = False)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = False)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Figure 4 and Figure 6).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Figure 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Figure 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Figure 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=False)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=False)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=False)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Figure 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=True, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Figure 23).<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1300', antenna='*&*',correlation='XX',<br />
avgchannel='1000', avgscan=True, <br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = raw_input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print "Running a short solution interval phase calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=True,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
for asdm in basename_all:<br />
print "Running a bandpass calibration for "+asdm<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=True,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. '''For what systems is this true?''' As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1.CASA3.4.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
<!--<br />
Again this warning:<br />
No ASDM_CALWVR, ASDM_CALATMOSPHERE, or CalWVR.xml table found. Using PWV 1.0mm.<br />
--><br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate (you can see the affected channel range in Figure 27. We run the following additional flagging step:<br />
<br />
[[File:Antennae-TitanLine.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Flagging CO(3-2) in Titan for "+asdm<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=False, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator.<br />
<br />
<source lang="python"><br />
# in CASA<br />
# make sure usescratch=True<br />
for asdm in basename_all:<br />
print "Reading model for Titan into "+asdm<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=True,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2012')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 3.3 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan: spw0 Flux:[I=3.2637,Q=0.0,U=0.0,V=0.0] +/- [I=0.0,Q=0.0,U=0.0,V=0.0] Jy @ 344.85GHz<br />
</pre><br />
<br />
Now we are going to take a look at the Titan model for one dataset:<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
</source><br />
<br />
[[File:2012_titan_model_5.1.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model produced by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model produced by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2012' '''. Previous versions of this guide used an older flux density standard '''standard='Butler-JPL-Horizons 2010' '''. Figure 28 and Figure 29 show the differences in the Titan models produced using these two standards. <br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out short timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Carrying out longer timescale phase solution for "+asdm<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Solving for longer (scan) interval amplitude solution for "+asdm<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Scaling amplitude calibration to match Titan for "+asdm<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.79 Jy, by averaging the fluxes obtained from the ten available datasets. If we check this value against reported flux measurements in the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html], we find that a flux of 10.79 Jy agrees within a few per cent of a linear interpolation between the two 0.85 mm measurements closest in time to the ALMA observations: (23 May 2011, SMA 11.644 ± 0.583; 01 July 2011, 9.748 ± 0.488).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the older, Butler-JPL-Horizons 2010, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly lower. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.41 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Plotting solutions for "+asdm<br />
plotms(vis=asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase', coloraxis='corr',<br />
iteraxis='antenna',antenna='0~7',plotrange=[0,0,-180,180],<br />
gridrows=4,gridcols=1,showgui=False,<br />
plotfile=asdm+'.scanphase.page1.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotms(vis=asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase', coloraxis='corr',<br />
iteraxis = 'antenna', antenna='8~15', plotrange=[0,0,-180,180],<br />
showgui=False, gridrows=4, gridcols=1, <br />
plotfile=asdm+'.scanphase.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotms(vis = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp', coloraxis='corr',<br />
iteraxis='antenna', antenna='0~7', plotrange = [0,0,0,0],<br />
showgui=False, gridrows=4,gridcols=1, <br />
plotfile=asdm+'.flux.page1.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
plotms(vis = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp', coloraxis='corr',<br />
iteraxis='antenna', antenna='8~15', plotrange = [0,0,0,0],<br />
showgui=False, gridrows=4,gridcols=1, <br />
plotfile=asdm+'.flux.page2.png')<br />
# dummy_string = raw_input("Hit <Enter> to see next plot.")<br />
<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print "Applying calibrations for "+asdm<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
<br />
[[File:xb-checkcal-pha-cha.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=True, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print "Examining calibrated data for "+asdm<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=True,avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = raw_input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
Note that you will see a warning that the last output channel in the new spw 0 will be dropped, as it has less than 23 channels over which to average:<br />
<br />
<pre style="background-color: #fffacd;"><br />
INFO MSTransformDataHandler The last output channel of spw 0 has only 22 input channels.<br />
WARN MSTransformManager Not enough input channels to populate output channel n# 166 from SPW 0.<br />
WARN MSTransformManager The channel will be dropped in order to have an uniform grid.<br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print "Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!"<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=True)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=True)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print cal_south_vis'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging imaging guide].<br />
{{Checked 5.4.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=AntennaeBand7_Calibration_6.1&diff=30884AntennaeBand7 Calibration 6.12021-08-20T18:43:10Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This guide is written for CASA 6.1 and uses Python 3. The same functionality is implemented in Python 2 using CASA 5.7 at [[AntennaeBand7 Calibration 5.7]]'''<br />
<br />
*'''This guide requires CASA 6.1 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]'''<br />
<br />
*'''Details of the ALMA observations are provided at [[AntennaeBand7]]<br />
<br />
*'''This portion of the guide covers calibration of the raw visibility data. To skip to the imaging portion of the guide, see: [[AntennaeBand7_Imaging 6.1]]'''.<br />
<br />
==Overview==<br />
<br />
This part of the Antennae Band 7 CASA guide will step through the calibration of the visibility data. We will begin by flagging (marking as bad) data known to be useless before any inspection; for example, data where one telescope blocks the line of sight of another. We will then apply telescope-generated calibration tables to partially correct for atmospheric effects. After inspecting the data, we will flag some additional data that exhibit pathologies. Using observations of the calibrators Titan and 3c279, it will be shown how to derive the phase and amplitude response of individual antennas as a function of time and frequency ("phase", "amplitude", and "bandpass" calibrations). We will apply these solutions to the data and then extract the calibrated source data into a file appropriate for imaging in the [[AntennaeBand7_Imaging | Antennae Band 7 imaging guide]].<br />
<br />
The general procedure in this guide follows the other ALMA CASA guides: [[NGC3256Band3]] and [[TWHydraBand7]].<br />
<br />
==Unpack the Data==<br />
<br />
This guide requires CASA 6.1 and assumes that you have downloaded Antennae_Band7_UnCalibratedMSAndTablesForReduction.tgz from [[AntennaeBand7#Obtaining_the_Data]]. <br />
<br />
Once you have downloaded the data, unpack the file in a terminal outside CASA using<br />
<br />
<source lang="bash"><br />
# in bash<br />
<br />
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz<br />
<br />
</source><br />
<br />
then change directory to the new directory <br />
<br />
<source lang="bash"><br />
# in bash<br />
<br />
cd Antennae_Band7_UnCalibratedMSandTablesForReduction<br />
<br />
</source><br />
<br />
You may wish to type<br />
<br />
<source lang="bash"><br />
# in bash<br />
<br />
ls<br />
<br />
</source><br />
<br />
to look at the files present. You should see a set of files with extension ".ms". These are CASA measurement set (MS) files. The data have already been converted to MS format from the native ALMA format using the CASA task {{importasdm}}. In addition to the data, we provide you with calibration tables containing system temperature (Tsys), water vapor radiometer (WVR), and antenna position information. For Early Science, these tables will either be pre-applied or supplied with the data.<br />
<br />
To begin, start CASA by typing<br />
<br />
<source lang="bash"><br />
# in bash<br />
<br />
casa<br />
<br />
</source><br />
<br />
Be sure that you are using the version indicated at the top of this page.<br />
<br />
==Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 6.1. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
import casalith<br />
version = casalith.version_string()<br />
print("You are using {}".format(version))<br />
if (version < '6.1'):<br />
print ("YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE.")<br />
print ("PLEASE UPDATE IT BEFORE PROCEEDING.")<br />
else:<br />
print ("Your version of CASA is appropriate for this guide.")<br />
</source><br />
<br />
==Initial Inspection==<br />
<br />
First we will take stock of what we have. If you have not already done so, begin by reviewing the description of the observations here: [[AntennaeBand7]]. The 10 data sets each target either the Northern or the Southern Mosaic, as follows:<br />
<br />
Northern Mosaic:<br />
*uid___A002_X1ff7b0_Xb.ms<br />
*uid___A002_X207fe4_X3a.ms<br />
*uid___A002_X207fe4_X3b9.ms<br />
*uid___A002_X2181fb_X49.ms<br />
<br />
Southern Mosaic:<br />
*uid___A002_X1ff7b0_X1c8.ms<br />
*uid___A002_X207fe4_X1f7.ms<br />
*uid___A002_X207fe4_X4d7.ms<br />
*uid___A002_X215db8_X18.ms<br />
*uid___A002_X215db8_X1d5.ms<br />
*uid___A002_X215db8_X392.ms<br />
<br />
The first step is to get basic information about the data: targets observed, time range, spectral setup, and so on. We do this using the task {{listobs}}, which will output a detailed summary of each dataset. Enter the following commands into CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# Define a python list holding the names of all of our data sets<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
# Initialize user-input string. (for Python testing)<br />
dummy_string = ''<br />
<br />
# Loop over each element in the list and create summary file using listobs<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.listobs.txt')<br />
listobs(vis=asdm+'.ms', listfile=asdm+'.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note that after cutting and pasting a 'for' loop like this you often have to press return twice to execute. You may also want to take care to paste a line at a time if you are having trouble copy and pasting. Even better, you can use "cpaste" to paste blocks of code. To do so type "cpaste" at the CASA prompt, paste your commands, and then type "--" and hit return on the final (otherwise empty) line. This should look something like this:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
CASA <8>: cpaste<br />
Pasting code; enter '--' alone on the line to stop.<br />
:basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
: "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
: "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
: "uid___A002_X215db8_X392"]<br />
:<br />
:for asdm in basename_all:<br />
: print (asdm)<br />
:--<br />
uid___A002_X1ff7b0_Xb<br />
uid___A002_X207fe4_X3a<br />
uid___A002_X207fe4_X3b9<br />
uid___A002_X2181fb_X49<br />
uid___A002_X1ff7b0_X1c8<br />
uid___A002_X207fe4_X1f7<br />
uid___A002_X207fe4_X4d7<br />
uid___A002_X215db8_X18<br />
uid___A002_X215db8_X1d5<br />
uid___A002_X215db8_X392<br />
<br />
CASA <9>: <br />
<br />
</pre><br />
<br />
cpaste should be ''much'' more robust than copying-and-pasting directly into the shell but if you have trouble, just carefully paste one line at a time directly into CASA and hit return until the desired command executes.<br />
<br />
These commands define a python list called "basename_all", which contains the name of all 10 MS files. The "for" loop executes for each element in basename_all, calling listobs and directing the output to a file called, e.g., "uid___A002_X1ff7b0_Xb.listobs.txt" for the first measurement set. You can browse through the listobs output as you would normally look at a text file (use emacs, vi, or another editor). You can also send the output to the terminal from inside of CASA. To do so type:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('cat uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
or<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('more uid___A002_X1ff7b0_Xb.listobs.txt')<br />
</source><br />
<br />
CASA knows a few basic shell commands like 'cat', 'ls', and 'rm' but for more complex commands you will need to run them inside 'os.system("command")'. For more information see http://casa.nrao.edu/ .<br />
<br />
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms, which targets the Northern Mosaic. You would see this if you had specified '''verbose''' to be False in the listobs call:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_Xb.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA(11 antennas)<br />
Data records: 181357 Total integration time = 4931.71 seconds<br />
Observed from 28-May-2011/01:25:27.6 to 28-May-2011/02:47:39.3 (UTC)<br />
<br />
Fields: 26<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 39116 <br />
1 none Titan 12:42:43.94813 -01.43.38.3190 J2000 1 10626 <br />
2 none NGC4038 - Antennae *12:01:53.17008 -18.52.37.9200 J2000 2 4818 <br />
3 none NGC4038 - Antennae *12:01:51.90301 -18.51.49.9437 J2000 2 6435 <br />
4 none NGC4038 - Antennae *12:01:52.43086 -18.51.49.9437 J2000 2 6424 <br />
5 none NGC4038 - Antennae *12:01:52.95871 -18.51.49.9437 J2000 2 5478 <br />
6 none NGC4038 - Antennae *12:01:53.48656 -18.51.49.9436 J2000 2 6435 <br />
7 none NGC4038 - Antennae *12:01:54.01441 -18.51.49.9436 J2000 2 6446 <br />
8 none NGC4038 - Antennae *12:01:52.16693 -18.51.56.4319 J2000 2 6435 <br />
9 none NGC4038 - Antennae *12:01:52.69478 -18.51.56.4318 J2000 2 6424 <br />
10 none NGC4038 - Antennae *12:01:53.22263 -18.51.56.4318 J2000 2 6446 <br />
11 none NGC4038 - Antennae *12:01:53.75049 -18.51.56.4318 J2000 2 6446 <br />
12 none NGC4038 - Antennae *12:01:51.90301 -18.52.02.9201 J2000 2 6435 <br />
13 none NGC4038 - Antennae *12:01:52.43086 -18.52.02.9200 J2000 2 6424 <br />
14 none NGC4038 - Antennae *12:01:52.95871 -18.52.02.9200 J2000 2 4818 <br />
15 none NGC4038 - Antennae *12:01:53.48656 -18.52.02.9200 J2000 2 4829 <br />
16 none NGC4038 - Antennae *12:01:54.01441 -18.52.02.9199 J2000 2 4829 <br />
17 none NGC4038 - Antennae *12:01:52.16694 -18.52.09.4082 J2000 2 4829 <br />
18 none NGC4038 - Antennae *12:01:52.69479 -18.52.09.4082 J2000 2 4829 <br />
19 none NGC4038 - Antennae *12:01:53.22264 -18.52.09.4082 J2000 2 3872 <br />
20 none NGC4038 - Antennae *12:01:53.75049 -18.52.09.4081 J2000 2 4829 <br />
21 none NGC4038 - Antennae *12:01:51.90301 -18.52.15.8964 J2000 2 4829 <br />
22 none NGC4038 - Antennae *12:01:52.43087 -18.52.15.8964 J2000 2 4829 <br />
23 none NGC4038 - Antennae *12:01:52.95872 -18.52.15.8963 J2000 2 4818 <br />
24 none NGC4038 - Antennae *12:01:53.48657 -18.52.15.8963 J2000 2 4818 <br />
25 none NGC4038 - Antennae *12:01:54.01442 -18.52.15.8963 J2000 2 4840 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 354971.074 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 354916.143 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11 'name'='station' <br />
ID= 0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', <br />
ID= 4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', <br />
ID= 8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'<br />
================================================================================<br />
<br />
</pre><br />
<br />
And here is an extract from the full '''verbose''' {{listobs}} for uid___A002_X1ff7b0_X1c8.ms, which targets the Southern Mosaic (note that we have snipped out the record of individual scans):<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
================================================================================<br />
MeasurementSet Name: /lustre/naasc/almauser/AntennaeBand7/uid___A002_X1ff7b0_X1c8.ms MS Version 2<br />
================================================================================<br />
Observer: Unknown Project: T.B.D. <br />
Observation: ALMA<br />
Data records: 175615 Total integration time = 4927.1 seconds<br />
Observed from 28-May-2011/02:50:18.2 to 28-May-2011/04:12:25.3 (UTC)<br />
<br />
<snip><br />
<br />
Fields: 33<br />
ID Code Name RA Decl Epoch SrcId nVis <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 12232 <br />
1 none Titan 12:42:44.82765 -01.43.41.4224 J2000 1 10615 <br />
2 none 3c279 12:56:11.16600 -05.47.21.5250 J2000 2 27764 <br />
3 none Antennae 12:01:53.17008 -18.52.37.9200 J2000 3 4829 <br />
4 none Antennae 12:01:52.18699 -18.53.30.3952 J2000 3 3883 <br />
5 none Antennae 12:01:52.64413 -18.53.26.6494 J2000 3 3883 <br />
6 none Antennae 12:01:53.10127 -18.53.22.9035 J2000 3 3872 <br />
7 none Antennae 12:01:53.55841 -18.53.19.1577 J2000 3 4818 <br />
8 none Antennae 12:01:54.01554 -18.53.15.4119 J2000 3 4829 <br />
9 none Antennae 12:01:54.47268 -18.53.11.6661 J2000 3 4829 <br />
10 none Antennae 12:01:54.92982 -18.53.07.9203 J2000 3 3872 <br />
11 none Antennae 12:01:55.38696 -18.53.04.1744 J2000 3 3883 <br />
12 none Antennae 12:01:55.84409 -18.53.00.4286 J2000 3 4840 <br />
13 none Antennae 12:01:56.30123 -18.52.56.6828 J2000 3 4818 <br />
14 none Antennae 12:01:52.18700 -18.53.22.9033 J2000 3 4829 <br />
15 none Antennae 12:01:52.64414 -18.53.19.1575 J2000 3 4818 <br />
16 none Antennae 12:01:53.10128 -18.53.15.4116 J2000 3 4818 <br />
17 none Antennae 12:01:53.55842 -18.53.11.6658 J2000 3 4840 <br />
18 none Antennae 12:01:54.01555 -18.53.07.9200 J2000 3 3872 <br />
19 none Antennae 12:01:54.47269 -18.53.04.1742 J2000 3 4829 <br />
20 none Antennae 12:01:54.92983 -18.53.00.4284 J2000 3 4829 <br />
21 none Antennae 12:01:55.38697 -18.52.56.6825 J2000 3 4829 <br />
22 none Antennae 12:01:55.84410 -18.52.52.9367 J2000 3 4829 <br />
23 none Antennae 12:01:51.72988 -18.53.19.1572 J2000 3 4818 <br />
24 none Antennae 12:01:52.18702 -18.53.15.4114 J2000 3 4829 <br />
25 none Antennae 12:01:52.64415 -18.53.11.6656 J2000 3 4829 <br />
26 none Antennae 12:01:53.10129 -18.53.07.9197 J2000 3 2266 <br />
27 none Antennae 12:01:53.55843 -18.53.04.1739 J2000 3 2266 <br />
28 none Antennae 12:01:54.01557 -18.53.00.4281 J2000 3 3212 <br />
29 none Antennae 12:01:54.47270 -18.52.56.6823 J2000 3 3234 <br />
30 none Antennae 12:01:54.92984 -18.52.52.9365 J2000 3 3212 <br />
31 none Antennae 12:01:55.38698 -18.52.49.1906 J2000 3 2266 <br />
32 none Antennae 12:01:55.84411 -18.52.45.4448 J2000 3 3223 <br />
(nVis = Total number of time/baseline visibilities per field) <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 3840 TOPO 344845.586 488.28125 1875000 XX YY <br />
2 1 TOPO 343908.086 1875000 1875000 XX YY <br />
3 3840 TOPO 356845.586 488.28125 1875000 XX YY <br />
4 1 TOPO 343908.086 1875000 1875000 XX YY <br />
5 128 TOPO 344900.518 15625 2000000 XX YY <br />
6 1 TOPO 343892.705 1796875 1796875 XX YY <br />
7 128 TOPO 356900.518 15625 2000000 XX YY <br />
8 1 TOPO 343892.705 1796875 1796875 XX YY <br />
Antennas: 11:<br />
ID Name Station Diam. Long. Lat. <br />
0 DV02 A015 12.0 m -067.45.15.3 -22.53.26.0 <br />
1 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8 <br />
2 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1 <br />
3 DV07 A004 12.0 m -067.45.15.9 -22.53.28.0 <br />
4 DV08 A072 12.0 m -067.45.12.6 -22.53.24.0 <br />
5 DV09 A008 12.0 m -067.45.15.4 -22.53.26.8 <br />
6 DV10 A009 12.0 m -067.45.16.1 -22.53.26.1 <br />
7 DV11 A016 12.0 m -067.45.16.4 -22.53.25.1 <br />
8 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1 <br />
9 PM02 A017 12.0 m -067.45.15.9 -22.53.26.8 <br />
10 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0 <br />
</pre><br />
<br />
This output shows that three sources were observed in each data set: 3c279, Titan, and the Antennae.<br />
<br />
* The '''Antennae''' are our science target. Note that the source name changes between the Northern Mosaic, where it is "NGC4038 - Antennae", and the Southern Mosaic, where it is just "Antennae". Also note that the source corresponds to a number of individual fields (see the Field ID column). These are the individual mosaic pointings. There are 23 for the Northern Mosaic and 29 for the Southern Mosaic.<br />
* '''Titan''' is observed once and will be used to set the absolute flux scale of the data.<br />
* '''3c279''' plays two roles: it will serve as our bandpass calibrator, to characterize the frequency response of the antennas, and because it is fairly close on the sky to the Antennae (18.6 degrees away) it will serve as our secondary calibrator (also referred to as the "phase calibrator" or "gain calibrator"), to track changes in the phase and amplitude response of the telescopes over time. Observations of 3c279 are interleaved with observations of the Antennae.<br />
<br />
The output also shows that the data contain many spectral windows. Using the labeling scheme in the listobs above these are:<br />
<br />
* '''spw 0''' targets ~185 GHz and holds water vapor radiometer data<br />
* '''spw 1''' and '''spw 3''' hold our science data. These are "Frequency Domain Mode" (FDM) data with small (0.49 MHz) channel width and wide (1.875 GHz) total bandwidth. As a result these have a lot of channels (3840). spw 1 holds the lower sideband (LSB) data and includes the CO(3-2) line. We will focus on these data. For the CO(3-2) line the channel width corresponds to 0.426 km/s and the bandwidth of spw 1 to 1634 km/s.<br />
* '''spw 2''' and '''spw 4''' hold frequency-averaged versions of spw 1 and 3 ("Channel 0" for those familiar with AIPS). These are used for some quick automated inspection. We will not use them here but we will carry out an equivalent inspection using spw 1.<br />
* '''spw 5''' and '''spw 7''' hold lower a resolution processing ("Time Domain Mode", TDM) of the data from the same part of the spectrum (baseband) as spws 1 and 3. These data have only 128 channels across 2 GHz bandwidth and so have a much coarser channel spacing than the FDM data. These were used to generate the calibration tables that we include in the tarball but will not otherwise appear in this guide.<br />
<br />
The final column of the listobs output in the logger (not shown above) gives the scan intent. Later we will use this information to flag the pointing scans and the hot and ambient load calibration scans.<br />
<br />
We'll now have a look at the configuration of the antennas used to take the data using the task {{plotants}} (see Figure 1).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb.ANT_6.1.png|200px|thumb|right|'''Fig. 1.''' Position of antennas in dataset uid_A002_X1ff7b0_Xb obtained using task plotants. ]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
<br />
for asdm in basename_all:<br />
print ("Antenna configuration for : "+asdm)<br />
plotants(vis=asdm+'.ms', figfile=asdm+'.plotants.png')<br />
clearstat()<br />
dummy_string = input("Hit <Enter> to see the antenna configuration for the next data set.")<br />
</source> <br />
<br />
This will loop through all 10 data sets, show you the antenna position for each, and save that as a file named, e.g., "uid___A002_X1ff7b0_Xb.plotants.png" for the first data set. The "input" command asks CASA to wait for your input before proceeding. If you would prefer to just browse the .png files after the fact you can remove this. Notice that the antenna setup changes, but only slightly, over the course of the 10 data sets.<br />
<br />
==How to Deal With 10 Measurement Sets==<br />
<br />
It should already be clear from the initial inspection that dealing with 10 data sets at the same time can be a bit problematic. This is especially tricky in our case because the Antennae data contain two distinct sets of observations: the Northern and Southern Mosaics. The source name changes between these two scripts and there are different numbers of fields in the mosaic.<br />
<br />
''As a general rule one would reduce each individual observation separately or at the very least only group data observed in a uniform way and very close in time.'' <br />
<br />
Unfortunately, a CASA Guide stepping through the reduction for each of 10 data sets would quickly become unwieldy. Therefore we will use a few tricks to reduce the Antennae data in a kind of batch mode. You have already seen the first trick: we can define a python list holding the names of each data set and then loop over this list to execute the same command on each data set. For example:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print (asdm)<br />
</source><br />
<br />
You only need to define your list of MS files once per CASA session. Then "basename_all" will be a variable in the casapy shell. You can check if it exists by typing "print (basename_all)". In the interests of allowing you to easily exit and restart CASA and pick this guide up at any point we will redefine "basename_all" in each section of the guide. Feel free to skip this step if you've already defined it in your session.<br />
<br />
This page will step you through the reduction of the whole Antennae Band 7 SV data set using these 'for' loops. We will not be able to show every diagnostic plot but we give an example of each and the syntax to generate the rest. Also please be aware that even on a very fast machine this whole process can take a while, we are simply dealing with a lot of data.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
One potential "gotcha" is that the source name changes between the two data sets. Therefore at several points we will break apart our loop to iterate separately over the Northern and Southern Mosaics, where the source can be referred unambiguously. Another subtle point is that 3c279 appears with two distinct field IDs in the Southern Mosaic, but only one in the Northern Mosaic. We will largely avoid this by referring to the source by its name but if you tried to use field ID numbers and mingled the two data sets this could cause confusion.<br />
</pre><br />
<br />
==''A Priori'' Flagging==<br />
<br />
Even before we look in detail, we know that there are some data that we wish to exclude. We will start by flagging "shadowed" data where one antenna blocks the line of sight of another. We will also flag scans that were used to carry out pointing and atmospheric calibration, identified by their scan intent. Finally, we'll flag the autocorrelation data (the correlation of the signal from an antenna with itself) as we are only interested in cross-correlation data to make an interferometric image.<br />
<br />
Start by defining our list of MS files:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
You may want to reset the flagging if you have tried this step before and are starting over though this is not necessary on your first time through. Do so using {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print ("Resetting flags for "+asdm)<br />
flagdata(vis=asdm+'.ms', mode='unflag', flagbackup=False)<br />
</source><br />
<br />
Then flag shadowed data using the command {{flagdata}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print ("Flagging shadowed data for "+asdm)<br />
flagdata(vis=asdm+'.ms',mode = 'shadow', flagbackup = False)<br />
</source><br />
<br />
In the {{flagdata}} task we choose:<br />
<br />
* '''vis''' = asdm+'.ms' : each measurement set<br />
* '''mode''' = 'shadow': flag shadowed data<br />
* '''flagbackup''' = False: Do not automatically back up the flag files. We will save all of the ''a priori'' flags together using {{flagmanager}} at the end of this subsection and save some space and time.<br />
<br />
The relevant calibration information has already been extracted from the pointing and atmospheric scans and we will not need them below. Now flag the pointing scans using {{flagdata}} in 'manualflag' mode and selecting on 'intent':<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print ("Flagging calibration scans for "+asdm)<br />
flagdata(vis=asdm+'.ms', mode='manual', intent='*POINTING*', flagbackup = False)<br />
</source><br />
<br />
Note that because the atmospheric calibration scans contain only TDM spectral windows, they will be removed automatically when we separate out the FDM data below.<br />
<br />
Now flag the autocorrelation data:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print ("Flagging autocorrelation data for "+asdm)<br />
flagdata(vis=asdm+'.ms',autocorr=True,flagbackup=False)<br />
</source><br />
<br />
Finally store the current flags information using {{flagmanager}}: <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print ("Backing up 'a priori' flags for "+asdm)<br />
flagmanager(vis = asdm+'.ms', mode = 'save', versionname = 'Apriori')<br />
</source><br />
<br />
Where '''versionname''' is just the name we assign to the current flagging state in the data. We can now roll back the flags to match the current version, called 'Apriori', whenever we want. There is no reason to do this now, but the syntax would be:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print ("Restoring up 'a priori' flags for "+asdm)<br />
flagmanager(vis = asdm+'.ms', mode = 'restore', versionname = 'Apriori')<br />
</source><br />
<br />
<br />
It would have been possible to set {{flagdata}} to '''flagbackup'''=True so that it stores the flags at each of the flagging step automatically, but this way it is a bit more efficient with regard to time and disk space. More important, lumping all the ''a priori'' flagging together gives us a natural point to roll back to if we get in trouble.<br />
<br />
<br />
The flagmanager task will also allow you to view the saved flagging versions,<br />
including those created by running flagdata with flagbackup=True.<br />
<br />
For example try <br />
<br />
<source lang="python"><br />
# In CASA<br />
flagmanager(vis='uid___A002_X2181fb_X49.ms', mode='list')<br />
</source><br />
<br />
The output in the logger should list the Apriori flagging that we have applied.<br />
It will also indicate versions associated with any flagdata command where you<br />
did not set flagbackup=False. Other tasks, for example applycal, will also create<br />
flag version tables.<br />
<br />
==Create and Apply Tsys, WVR, and Antenna Position Calibration Tables==<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
</source><br />
<br />
===Tsys===<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency and associates weights with each visibility that persists through imaging.<br />
<br />
Use gencal to create the Tsys calibration tables from the spectral windows with CALIBRATE_ATMOSPHERE intents in listobs (i.e. the TDM spectral windows with 128 channels), these are spectral windows 5 and 7.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#Removing existing Tsys information<br />
os.system('rm -rf *tdm.tsys')<br />
for asdm in basename_all:<br />
print ("Creating TDM Tsys Table for "+asdm)<br />
gencal(vis=asdm+'.ms',caltable=asdm+'.tdm.tsys',spw='5,7',caltype='tsys')<br />
</source><br />
<br />
Later in the applycal stage this TDM Tsys table will be interpolated to the FDM (3840 channels per spw) science spectral windows 1 and 3.<br />
<br />
Next we inspect the Tsys tables for the spectral window '''spw''' = 5 with the task {{plotms}}. In previous CASA versions, we used {{plotcal}} for this task, but most of '''plotcal''''s functionality is now included in {{plotms}}, and '''plotcal''' will be deprecated in a future CASA release. We want to check that Tsys data have reasonable values and identify any unexpected features as a function of either time or frequency. To get an idea of sensible Tsys under average atmospheric observations consult the ALMA sensitivity calculator, accessible from http://www.almascience.org (via the "Documents & Tools" link).<br />
<br />
We start by plotting the Tsys for all the antennas and polarizations (XX and YY) as a function of time for each. <!--Here and throughout we focus on spw 1, which contains CO(3-2): --><br />
<br />
<pre style="background-color: #E0FFFF;"><br />
Note that right now, plotms can't select individual channels when plotting caltables (as was possible with plotcal). In addition, when iteraxis is set, plotms appends the axes being iterated over to the output filename specified in plotfile; this can make for rather unwieldy filenames (particularly when iteraxis='antenna'!) but is useful for the ALMA pipeline. <br />
</pre><br />
<br />
<br />
[[File:Xb.Tsys_vs_time.page1.6.1.png|200px|thumb|right|<caption>'''Fig. 2.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 8 antennas. Note the high y-axis values for DV04. The two different colors indicate the two polarizations (XX and YY).</caption>]]<br />
<br />
<br />
[[File:Xb.Tsys_vs_time.page2.6.1.png|200px|thumb|right|<caption>'''Fig. 3.''' Tsys vs. time plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Plotting Tsys vs. time for "+asdm)<br />
plotms(vis=asdm+'.tdm.tsys',<br />
xaxis='time',yaxis='tsys',coloraxis='corr',<br />
spw='5:50~50',<br />
gridrows=4,gridcols=2,<br />
antenna='0~7',iteraxis='antenna',<br />
plotfile=asdm+'.tsys_vs_time.page1.png',overwrite=True)<br />
#dummy_string = input("First eight antennas for "+asdm+" . Hit <Enter> to continue.")<br />
plotms(vis=asdm+'.tdm.tsys',<br />
xaxis='time',yaxis='tsys',coloraxis='corr',<br />
spw='5:50~50',<br />
gridrows=4,gridcols=2,<br />
antenna='8~15',iteraxis='antenna',<br />
plotfile=asdm+'.tsys_vs_time.page2.png',overwrite=True)<br />
#dummy_string = input("Remaining antennas for "+asdm+" . Hit <Enter> to continue.")<br />
</source><br />
<br />
This sequence loops over all of our files and plots Tsys as a function of time for channel 50 in spectral window 5. In the call to {{plotms}}:<br />
<br />
* '''gridrows'''=4 and '''gridcols'''=2 sets up a 4 x 2 panel grid. <br />
* '''iteraxis'''='antenna' tells {{plotms}} to make a separate plot for each antenna.<br />
* '''spw'''='5:50~50' selects spw '5' channel 50-50. This allows us to cleanly separate time variations from frequency variations. '''Note that this is a functionality in plotcal but not yet in {{plotms}}, and plotms is actually showing all channels from spw 5 at this time.'''<br />
<br />
Because 8 panels (2 panels for each antenna - LSB and USB) is not enough to show all antennas on one page, there are two {{plotms}} calls: one for the first 8 antennas ('''antenna'''='0~7'), and then for the remaining antennas ('''antenna'''='8~15').<br />
<br />
The 'input' commands will wait for you to hit Enter before issuing the next plot command. In the example above these are commented out (the leading "#" means that CASA will ignore them). If you would like to interactively cycle through the plots, uncomment them by removing the "#". Otherwise, the '''plotfile''' parameter directs the output to .png files for later inspection. The easiest way to look at the 20 plots produced here is to simply inspect the .png files using your favorite viewer. As noted above, here plotms is adding the antenna information to the filenames because we set '''iteraxis''' = 'antenna', so check your directory for the impressively long filenames to view them. <br />
<br />
The Tsys values in Figure 2 and Figure 3 look reliable, with typical values ~150 K except for some large values of Tsys at ~400 and 500 K for DV04. We will flag the data for that antenna later. <br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.tdm.tsys.*Antennae*.DV02.spw05_6.1.png|200px|thumb|right|<caption>'''Fig. 4.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). First 4 antennas. Note the high y-axis values for DV04 and the mesospheric line near 343.2 GHz.</caption>]]<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.tdm.tsys.*Antennae*.DV08.spw05_6.1.png|200px|thumb|right|<caption>'''Fig. 5.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Next 4 antennas.</caption>]]<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.tdm.tsys.*Antennae*.PM01.spw05_6.1.png|200px|thumb|right|<caption>'''Fig. 6.''' Tsys vs. frequency plot for uid_A002_X1ff7b0_Xb (northern mosaic). Remaining antennas.</caption>]]<br />
<br />
[[File:uid___A002_X215db8_X392.tdm.tsys.3c279.PM01.spw05_6.1.png|200px|thumb|right|<caption>'''Fig. 7.''' Tsys vs. frequency plot for uid___A002_X215db8_X392. Note the pathological behavior for DV12.</caption>]]<br />
<br />
<br />
We will also want to look at Tsys as a function of frequency. <br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
os.system('rm -rf cal_plots/Tsys_plots/'+asdm+'.tdm.tsys*')<br />
tsysfields=['3c279','Titan','*Antennae*']<br />
caltable=asdm+'.tdm.tsys'<br />
for field in tsysfields:<br />
plotbandpass(caltable=caltable,yaxis='amp',field=field,xaxis='freq',<br />
showatm=True,overlay='time',<br />
figfile='cal_plots/Tsys_plots/'+caltable+'.'+field,<br />
buildpdf=False,interactive=False,chanrange='8~120',subplot=42)<br />
</source><br />
<br />
You will see the following statement in the logger, but this is fine: <br />
<br />
<pre style="background-color: #fffacd;"><br />
No ASDM_CALWVR, ASDM_CALATMOSPHERE, or CalWVR.xml table found. Using PWV 1.0mm.<br />
</pre><br />
<br />
Now have a look at the Tsys vs. frequency plots. You can see the effect of a close pair of atmospheric ozone absorption lines at about 343.2 GHz that makes Tsys larger near that frequency in all antennas. Applying the Tsys calibration tables will minimize the contribution of these atmospheric lines. Again DV04 stands out with its very high Tsys. Although not present in the first data sets, Antenna DV12 exhibits periodic spikes in Tsys vs. frequency for one polarization (see Figure 7). It may or may not be possible to calibrate that behavior out. We will make a note to look carefully at DV12 further on in the calibration process.<br />
<br />
===WVR===<br />
The WVR calibration uses observations of the wings of the 183 GHz atmospheric water line to correct for phase variations as a function of time. As noted in the tsys section, this run of gencal will append to existing WVR produced tables. To avoid this, preexisting wvrgcal data is removed. <br />
<source lang="python"><br />
#In CASA<br />
#Removing existing WVR information<br />
os.system('rm -rf *.wvrgcal')<br />
for asdm in basename_all:<br />
wvrgcal(vis=asdm+'.ms',caltable=asdm+'.wvrgcal',toffset=-1)<br />
</source><br />
<br />
===Antenna Positions===<br />
<br />
In general, the most appropriate antenna positions for the dataset will be determined and updated if needed during QA2. If the antenna positions need to be updated after QA2, we inform the PIs of the affected datasets and provide instructions on how to update the antenna positions. For the data in this guide, we do need to update the antenna positions. <br />
<br />
The antenna position table reflects refinements in the measured positions of the antennas from those stored in the data. {{gencal}} will now be used put antenna position data into each observation. Again, {{gencal}} will merely append to existing antenna position data, ruining any subsequent results. We start by removing any existing antenna position refinements, followed by defining the antenna names, then their refinements (both as arrays), finally running {{gencal}} to create the information CASA can refer to for antenna positions.<br />
<br />
<source lang="python"><br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_Xb.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:21]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3a.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X3b9.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X2181fb_X49.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV01,DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
-0.00004, 0.00026, 0.00019, #DV01-A137<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X1ff7b0_X1c8.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV06,DV07,DV08,DV09,DV10,DV11,PM01,PM02,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00034, 0.00092, 0.00027, #PM02-A017 Before July 1<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X1f7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X207fe4_X4d7.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV11,DV12,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X18.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:22]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X1d5.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
<br />
#In CASA<br />
#The measurement set we are refining the antenna positions for<br />
data='uid___A002_X215db8_X392.ms'<br />
#Removing existing antenna refinements<br />
os.system('rm -rf '+data[0:23]+'.antpos')<br />
antenna='DV02,DV04,DV05,DV06,DV07,DV08,DV09,DV10,DV11,DV12,DV13,PM01,PM03'<br />
parameter = [<br />
0.00000, 0.00000, 0.00000, #DV02-A015<br />
-0.00004, 0.00059, 0.00024, #DV04-J505<br />
-0.00019,-0.00003, 0.00035, #DV05-A067<br />
-0.00027, 0.00033, 0.00022, #DV06-T704<br />
0.00006, 0.00057, 0.00032, #DV07-A004<br />
-0.00027, 0.00025,-0.00003, #DV08-A072<br />
-0.00037, 0.00025, 0.00003, #DV09-A008<br />
-0.00056, 0.00028, 0.00026, #DV10-A009<br />
-0.00028, 0.00024, 0.00011, #DV11-A016<br />
-0.00018, 0.00010,-0.00014, #DV12-A011<br />
0.00009, 0.00033, 0.00023, #DV13-A075<br />
-0.00017, 0.00013, 0.00030, #PM01-T702<br />
-0.00007, 0.00034, -0.00004] #PM03-J504 <br />
gencal(vis=data,caltable='%s.antpos'%(data.split('.')[0]),caltype='antpos',<br />
antenna=antenna,parameter=parameter)<br />
</source><br />
<br />
===Applycal===<br />
<br />
We are now ready to apply the Tsys and the WVR calibration tables to the data with {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. Again, we loop through all the datasets. It is important to only apply Tsys and WVR corrections obtained close in time to the data being corrected, so in addition to looping over data sets we define the list of unique source names and loop over these. Then by setting '''gainfield''' and '''field''' to the same value we ensure that Tsys and WVR calibrations are only applied to the source for which they are measured. Because the source has a different name in the Northern Mosaic and the Southern Mosaic, we will carry out two loops. We will only correct '''spw''' 1 and 3, our science windows, because we will drop the other data in a moment.<br />
<br />
The applycal task now has much more flexibility for interpolating and applying calibrations derived in one spectral window to another, even if they do not share the same spectral shape (number of channels and channel width). This new functionality is used below to interpolate the TDM (128 channel) Tsys measurements to the FDM (3840 channel) spectral windows. This is controlled through the spectral window mapping parameter *spwmap*. Because this can be a bit confusing, we've written a "helper" function that will tell you what you should put for the Tsys calibration table part of spwmap. We only need to run it on one of the datasets because they are all the same in this regard<br />
<br />
<source lang="python"><br />
# In CASA<br />
from casarecipes.almahelpers import tsysspwmap<br />
tsysspwmap(vis='uid___A002_X207fe4_X3a.ms',tsystable='uid___A002_X207fe4_X3a.tdm.tsys')<br />
</source><br />
<br />
This will print to your screen: [0, 5, 5, 7, 5, 5, 5, 7]. Recall that the TDM spectral windows are spw=5 and 7 and the FDM spectral windows are 1 and 3. This spwmap tells applycal to use the Tsys information in spectral window 5 for spectral window 1; and to use the information in spectral window 7 for spectral window 3. Beyond that, it is only important that there are placeholders for the other spectral windows, its also useful to note that you need not give placeholders for spectral window ids larger than the largest spectral window you need to use, in this case 7.<br />
<br />
You will also need to put in placeholders for other tables that you give the gaintables. So if the Tsys table is 2nd in the list, spwmap=[[],[0,5,5,7,5,5,5,7],[]]<br />
<br />
Now run the applycal commands.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
# A new list of file names that contain only data from the Northern Mosaic<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names_north = ['Titan','3c279','NGC*']<br />
<br />
for asdm in basename_north:<br />
print ("Apply Tsys, WVR, and Antenna Position calibrations to "+asdm)<br />
for field in field_names_north:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=['',field,field],<br />
interp=['','linear,spline','nearest'], <br />
gaintable=[asdm+'.antpos',asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]],<br />
flagbackup=False, calwt=True)<br />
<br />
# A new list of file names that contain only data from the Southern Mosaic<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names_south = ['Titan','3c279','Ant*']<br />
<br />
for asdm in basename_south:<br />
print ("Apply Tsys, WVR, and Antenna Position calibrations to "+asdm)<br />
for field in field_names_south:<br />
applycal(vis=asdm+".ms", spw='1,3', <br />
field=field, gainfield=["",field,field],<br />
interp=['','linear,spline','nearest'],<br />
gaintable=[asdm+".antpos",asdm+'.tdm.tsys',asdm+'.wvrgcal'],<br />
spwmap=[[],[0, 5, 5, 7, 5, 5, 5, 7],[]], <br />
flagbackup=False, calwt=True)<br />
</source><br />
<br />
where:<br />
<br />
*'''field''': the field ''to'' which we will apply the calibration, <br />
*'''gainfield''': the field ''from'' which we wish to take the calibration table<br />
*'''interp''' = 'nearest' : apply the nearest solution from the calibration table rather than interpolating.<br />
<br />
As you browse through the whole data set, you will probably note some problems along the same lines as the DV04 issue we saw above. We'll apply these as additional data flagging in just a moment.<br />
<br />
==Inspect Data==<br />
We are not quite done with the original ".ms" data sets yet. Before going further it will be useful to use {{plotms}} to show the effects of applying the calibration. In the process we'll take a quick look at each antenna and search for pathologies in the data.<br />
<br />
For this basic inspection, we want to compare the phase and amplitude as a function of frequency and time in the DATA and CORRECTED columns of each measurement set. The CORRECTED column has had the Tsys and WVR calibrations applied and so we expect lower phase scatter and flatter amplitude response as a function of time and frequency. We are looking for antenna-based issues, so cycling through a set of baselines that includes each antenna once will be a good start. We'll focus these plots on the phase+bandpass calibrator, 3c279, and on baselines that include antenna DV11, which we will make our reference antenna in just a bit.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
Each CASA Measurement Set has up to three "columns" of data: DATA, CORRECTED, and MODEL (though it is possible<br />
for a MS to hold only a DATA column if it has not been processed at all). <br />
<br />
A column entry exists for each data point (baseline, frequency, time). <br />
<br />
The DATA column holds the current raw data, though using split as we just did we can change the definition of<br />
"raw" data. <br />
<br />
The CORRECTED column holds the result of applying one or more calibration tables (e.g., via applycal) to the<br />
DATA column and so represents a corrected version of the DATA column. <br />
<br />
The MODEL column is now deprecated (though it is still possible to use it by explicitly setting usescratch=True).<br />
Instead, the model is stored in the header of the ms. This is a great step forward in terms of saving space and time <br />
to create scratch columns <br />
<br />
To get an intuitive feel for the columns you may wish to explore using plotms (which can plot different data <br />
columns, as seen below) or the browsetable task.<br />
<br />
</pre><br />
<br />
<br />
[[File:plotms_amp_vs_freq_example-data_6.1.png|200px|thumb|right|<caption>'''Fig. 8.''' Example of Amplitude vs. Frequency before correction for the 3c 279.</caption>]]<br />
<br />
[[File:plotms_amp_vs_freq_example-corr_6.1.png|200px|thumb|right|<caption>'''Fig. 9.''' Same baseline as Figure 8 but now after correction using WVR and Tsys calibrations.</caption>]]<br />
<br />
<br />
First, we plot amplitude as a function of frequency for 3c279. We start by plotting the DATA column, set color to indicate the two correlations (i.e., the XX and YY polarizations), and ask plotms to iterate over baseline. By setting '''antenna''' to 'DV11&*' we select only baselines that include DV11. We ask plotms to average all data over a very long timescale, '''avgtime''' = 1e8 seconds ~ 3 years or much longer than the time spanned by the whole data set. By setting '''avgscan''' = True we allow {{plotms}} to average across scan boundaries. The result is a plot of average amplitude per channel vs. frequency.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
asdm=basename_all[0]<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='amp',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Notice the green arrows along the bottom of the {{plotms}} window. We asked {{plotms}} to iterate over baseline. As you click the arrows, the plot will rotate from baseline to baseline, always with DV11 so that each antenna shows up once. To see the effect of the calibration, go to the "Axes" tab along the left of the {{plotms}} window and pull down the Data Column menu under the Y Axis. Set this from DATA to CORRECTED and you should see the effects of the calibration. You may need to ensure that the "Force Reload" box is checked before clicking "Plot" (both buttons lie at the bottom of the panel). For the most part things get better (flatter), but as we noted before DV04 is problematic.<br />
<br />
<br />
[[File:plotms_amp_vs_time_north_6.1.png|200px|thumb|right|<caption>'''Fig. 10.''' Example of amplitude vs. time for a northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_time_south_6.1.png|200px|thumb|right|<caption>'''Fig. 11.''' Example of amplitude vs. time for a southern mosaic data set.</caption>]]<br />
<br />
<br />
You can now make analogous calls to examine the phase vs. frequency, amplitude vs. time, and phase vs. time.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='frequency', yaxis='phase',<br />
selectdata=True, spw='1', <br />
avgtime='1e8',avgscan=True,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='amp',<br />
selectdata=True, spw='1:1200~1299', <br />
avgchannel='100',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
<br />
plotms(vis=asdm+'.ms', <br />
field='3c279',<br />
xaxis='time', yaxis='phase',<br />
selectdata=True, spw='1:1200~1299', <br />
avgchannel='100',avgscan=False,<br />
coloraxis='corr',<br />
iteraxis='baseline',<br />
antenna='DV11&*',<br />
ydatacolumn='data', <br />
showgui = True)<br />
</source><br />
<br />
Where:<br />
<br />
* '''spw''' is set to cover only channels 1200~1300 for the time plots in order to isolate time dependent variations from frequency-dependent behavior. Those 101 channels represent only a small part of the total spw 1 bandpass.<br />
* '''avgchannel''' set to a large number causes the plots of phase and amplitude vs. time to average data at all frequencies into a single point for each measurement.<br />
* '''coloraxis''' corr sets the colors to correspond to the two polarizations of the data.<br />
<br />
In each case, you will want to examine each baseline, alternating between the DATA and CORRECTED columns.<br />
<br />
This is a lot of data inspection and that's only for one of 10 data sets! You can iterate across the data by hand, updating "asdm" to refer to each data set in order and cycling between baselines and DATA/CORRECTED. It is also possible to script CASA to show you the key plots in succession (see the next block down). However you approach the infrastructure, you are looking for:<br />
<br />
* Improved scatter and lower variability in phase and amplitude vs. frequency and time. This indicates that the WVR and Tsys calibrations helped. <br />
* Sudden jumps in phase or amplitude as a function of either time or frequency. These may indicate problems with the antenna during that observation.<br />
* Large gradients, especially full wraps, in phase as a function of frequency. This may indicate a problem in the delays, the signal path length to the telescopes.<br />
* Unusual magnitude, scatter, or patterns in any plot - though this may be better explored using plots that show all data together, which we'll make in a moment.<br />
* Missing data. For example, if the phase calibrator drops out for a period of time we will not be able to calibrate and will need to flag the data.<br />
<br />
As you look through, note individual potentially problematic antennas. If ''all'' antennas in a data set appear problematic it may be that your "reference" antenna, DV11 in the example above, is the source of the problem. In this case swap this reference antenna for another and see whether the problem is isolated to your original reference antenna.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
<br />
A brief aside on structure:<br />
<br />
This section (Data Inspection) and the next (Apply Flags) are closely linked. We will present them as two<br />
separate steps. The idea here is that you look through your data, note problems, and then write commands to<br />
flag problematic data. This mimics one approach to writing data reduction scripts for CASA, where you will<br />
group all flagging together into one convenient place. Other CASA guides take a different approach, interleaving <br />
flagging and plotting. There is no "right" answer here. Find an approach to data inspection that works for you.<br />
<br />
</pre><br />
<br />
If you do wish to semi-automate the plot generation, the following sequence will cycle between data and corrected plots for each data set in turn. Type "stop" at any input call to break out.<br />
<br />
<!--<br />
'''Note:''' In loops like this involving {{plotms}}, hitting enter before the last dataset completes loading may cause {{plotms}} to stop refreshing the plot window at each step, or cause the window to disappear altogether. This will be fixed in an upcoming CASA release. Right now, you may need to exit CASA and restart to bring up the {{plotms}} window again, but you should be able to continue in the script where you left off.<br />
--><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
user_input = ""<br />
for asdm in basename_all:<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
# Extract antenna list for this data set.<br />
tb.open(asdm+'.ms/ANTENNA', nomodify=True)<br />
ants = tb.getcol('NAME')<br />
tb.close<br />
# Define the reference antenna to make baselines with<br />
ref_ant = 'DV11'<br />
# Loop over antennas<br />
for ant in ants:<br />
# Check if the user wants to stop<br />
if user_input == "stop":<br />
break<br />
# Skip correlation of reference antenna with itself (autocorrelations are flagged anyhow) <br />
if ant == ref_ant:<br />
continue<br />
# Define the baseline with the reference antenna for current antenna<br />
ant_str = ref_ant+'&'+ant<br />
print ("Showing baseline "+ant_str+" for data set "+asdm)<br />
print ("Use this to inspect effect of applying wvrcal and Tsys calibrations.")<br />
# Loop over phase and amplitude as axes<br />
for y_axis in ["amp", "phase"]:<br />
# Make 'before' plot for frequency x-axis<br />
print ("... "+y_axis+" vs. frequency for DATA:") <br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for frequency x-axis<br />
print ("... "+y_axis+" vs. frequency for CORRECTED:")<br />
plotms(vis=asdm+'.ms', spw='1', field='3c279',<br />
antenna=ant_str, xaxis="frequency", yaxis=y_axis,<br />
avgtime="1e8", avgscan=True, coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'before' plot for time x-axis<br />
print ("... "+y_axis+" vs. time for DATA:") <br />
plotms(vis=asdm+'.ms', spw='1:1200~1299', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="100", coloraxis="corr",<br />
ydatacolumn="data", showgui = True)<br />
user_input = input("Hit <ENTER> to see CORRECTED data [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
# Make 'after' plot for time y-axis<br />
print ("... "+y_axis+" vs. time for CORRECTED:")<br />
plotms(vis=asdm+'.ms', spw='1:1200~1299', field='3c279',<br />
antenna=ant_str, xaxis="time", yaxis=y_axis,<br />
avgchannel="100", coloraxis="corr",<br />
ydatacolumn="corrected", showgui = True)<br />
user_input = input("Hit <ENTER> to proceed to next plot [type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
[[File:plotms_amp_vs_uvdist_north_6.1.png|200px|thumb|right|<caption>'''Fig. 12.''' Example of amplitude vs. uv-distance for 3c279 in the first northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_3c279_6.1.png|200px|thumb|right|<caption>'''Fig. 13.''' Example of amplitude vs. frequency for 3c279 in the second northern mosaic data set.</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Titan_6.1.png|200px|thumb|right|<caption>'''Fig. 14.''' Example of amplitude vs. frequency for Titan in the second northern mosaic data set. Note the strong line (this is CO 3-2)!</caption>]]<br />
<br />
<br />
[[File:plotms_amp_vs_freq_Antennae_6.1.png|200px|thumb|right|<caption>'''Fig. 15.''' Example of amplitude vs. frequency for the Antennae in the second northern mosaic data set. The CO(3-2) line is visible.</caption>]]<br />
<br />
<br />
A detailed explanation of the procedure is a bit outside the scope of this guide (for more on python see http://www.python.org/ and for more on the CASA toolkit see http://casa.nrao.edu/), but the basic process is to loop over each data set, baseline with the reference antenna (here DV11), and y-axis of interest (phase or amplitude) then plot the effect of the calibration vs. frequency and time for each combination. Running this to step through the data will give you about 200 "before and after" plots from which you could note a subset of problematic cases to be followed up by hand. Many other strategies to inspect the data are also viable.<br />
<br />
With the Tsys and WVR calibrations applied successfully and the ''a priori'' flagging taken care of we will now {{split}} out the corrected data. We will keep only the corrected data, specified via '''datacolumn''', and only spectral window 1, which contains the FDM (high spectral resolution) observations of the CO(3-2) line. Setting '''keepflags'''=False tells {{split}} not to carry over any fully flagged rows from the original data set to the new MS. We give the new MS files the extension ".wvrtsys.ms" to indicate that they have been corrected for WVR and Tsys effects. Because split will not overwrite existing files, we remove any previous versions of the new MS before beginning.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvrtsys.ms')<br />
print ("Splitting out corrected data for "+asdm)<br />
split(vis=asdm+'.ms', outputvis=asdm+'.wvrtsys.ms', <br />
datacolumn='corrected', spw='1', keepflags=False)<br />
</source><br />
<br />
Be patient, {{split}} may take awhile. The WVR and Tsys-corrected data now sit in the DATA column of the new measurement sets, which have only one spectral window (now labeled spectral window 0 though it was spectral window 1 in the original data). You may wish to run {{listobs}} to illustrate the changes:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
os.system('rm '+asdm+'.wvrtsys.listobs.txt')<br />
listobs(vis=asdm+'.wvrtsys.ms', listfile=asdm+'.wvrtsys.listobs.txt', verbose=True)<br />
</source><br />
<br />
Note the new spectral window information:<br />
<br />
<pre style="background-color: #fffacd;"><br />
<br />
2011-08-05 01:07:08 INFO listobs Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)<br />
2011-08-05 01:07:08 INFO listobs SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs <br />
2011-08-05 01:07:08 INFO listobs 0 3840 TOPO 344845.586 488.28125 1875000 344908.33 XX YY <br />
<br />
</pre><br />
<br />
<br />
Next we will do a bit more inspection using {{plotms}} to look at whole data sets. This will help us identify missing data or look for egregious outliers.<br />
<br />
First we plot amplitude versus time (see Fig. 10 and Fig. 11), averaging over all channels (by setting '''avgchannel''' to the very large value 10,000). We colorize by field so that scans on Titan are red, the bandpass and phase calibrator 3c279 is black (and orange in the Southern Mosaic where it has two field IDs), and the Antennae mosaic appears as a range of colors (one per pointing).<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='amp', <br />
avgchannel='3840',coloraxis='field', showgui = True)<br />
user_input = input("Examining amplitude vs. time for "+asdm+" . Hit <Enter> to proceed.[type 'stop'+<Enter> to break out].")<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
Here look for:<br />
<br />
* Missing data. The source needs to be flanked by phase calibrator scans, if those are missing for any reason we need to flag the appropriate time range.<br />
* Dramatic outliers. Does the source suddenly get very bright or the otherwise bright calibrator appear anomalously faint for a brief time? This likely indicates problematic data that should be identified and flagged. You can use the "select" (box with green plus along the bottom row in plotms) and "locate" (magnifying glass) buttons in {{plotms}} to isolate and identify problem data (it will print to the log).<br />
* Smooth variation with time. A sudden jump may indicate a problem and often the safest approach is to flag data near a discontinuity.<br />
<br />
Look through the amplitudes vs. time for each data set (remember that we've already examined the phases vs. time and amplitude vs. time for individual baselines above).<br />
<br />
There are two other very useful "averaging" plots worth making. First, we plot amplitude as a function of u-v distance (projected antenna separation). Discontinuities and spikes in this plot are often from non-astrophysical sources. In the phase analog to the plot, the effects of atmospheric decorrelation can be assessed from increased scatter at longer u-v distances. While using the moon Titan as our flux calibrator, we may want to watch for flaring amplitudes at short u-v distances. These may indicate that Saturn is contaminating our beam. For a perfect, bright point source, we expect flat amplitudes as a function of u-v distance at the source amplitudes. Figure 12shows an example of this plot, generated via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='3c279',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
dummy_string = input("Examining amplitude vs. UV Distance for 3c279 for "+asdm+" . Hit <Enter> to proceed.")<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field='Titan',<br />
xaxis='uvdist', yaxis='amp', <br />
avgchannel='3840',coloraxis='corr', showgui = True)<br />
user_input = input("Examining amplitude vs. UV Distance for Titan for "+asdm+" . Hit <Enter> to proceed [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if user_input == "stop":<br />
break<br />
</source><br />
<br />
<br />
<br />
It can also be useful to examine the average amplitude as a function of frequency for each target. This allows one to check for lingering atmospheric effects, unexpected line emission or absorption in the calibrators, or decreased sensitivity due to "roll-off" of the telescope sensitivity at the band edges. These plots are created by the sequence of commands below, where we have broken up the script to loop separately over the Northern and Southern Mosaics to reflect the different source names.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
field_names = ["3c279","Titan","NGC*"]<br />
<br />
for asdm in basename_north:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = input("Examining amplitude vs. frequency for "+field+" in "+asdm+". Hit <Enter> to proceed.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
field_names = ["3c279","Titan","Ant*"]<br />
<br />
for asdm in basename_south:<br />
for field in field_names:<br />
plotms(vis=asdm+'.wvrtsys.ms', <br />
field=field,<br />
xaxis='frequency', yaxis='amp', <br />
avgtime='1e8',avgscan=True, coloraxis='corr', showgui = True)<br />
dummy_string = input("Examining amplitude vs. frequency for "+field+" in "+asdm+" . Hit <Enter> to proceed.")<br />
<br />
</source><br />
<br />
<br />
For this command notice that we can see the CO(3-2) line in the Antennae even before calibration (see Figure 15) and that Titan also shows evidence of a strong line (see Figure 14)! This will need to be flagged before we can use Titan to calibrate the flux scale of our data.<br />
<br />
This suite of plots (along with the earlier inspection of the Tsys tables) gives us the tools we need to identify problematic data through the data sets. We use this to generate a set of inspection-driven {{flagdata}} commands for each data set. We apply these before the bandpass and gain calibration.<br />
<br />
==Apply Flagging==<br />
Based on this inspection and the other plots we have made, we now flag problematic portions of the data. We break up the flags by reason for flagging in order to illustrate the process. As you reduce your own data it may be more efficient to group flags by data set and make use of the flagcmd command. Except for the "post-calibration" flagging, the inspection plots that we just looked through have already revealed all of the problems that we flag. We structure the guide so that the flagging is grouped in one place.<br />
<br />
As before, we may wish to reset our flags before beginning (particularly if one iterates this process) via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms',mode='unflag', flagbackup = False)<br />
</source><br />
<br />
Remember that we dropped the flagged data when splitting out after the WVR and Tsys calibration, so this should not undo your "A Priori" flagging of shadowed data, autocorrelations, etc.. In any case this unflagging step is not necessary during the first iteration.<br />
<br />
* '''Edge Channels'''<br />
<br />
ALMA's sensitivity decreases near the edge of the baseband and it is often useful to check for a 'roll-off' in sensitivity near the edge of the band. This will appear as a downturn in amplitude as a function of channel near the edge of the window in the uncalibrated data, as a flaring due to increased noise at the spw edges in the calibrated data. It will also be visible in the amplitude part of the bandpass calibration table. Because our FDM spw does not cover the full baseband, we do not see a strong roll off in our data (see Figure 16), where there is only a mild hint of a roll-off at the high end) but we do flag a (very) few channels at the high and low-frequency edge of the data set to be safe.<br />
<br />
<br />
[[File:amp_vs_channel_example.png|200px|thumb|right|<caption>'''Fig. 16.''' Amplitude vs. channel for one uncalibrated antenna pair. This kind of plot can be inspected to get an idea of the presence or magnitude of any roll-off in sensitivity near the edges of the spectral window.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:0~7,0:3831~3839', flagbackup = False)<br />
<br />
for asdm in basename_all:<br />
flagdata(vis = asdm+'.wvrtsys.ms', mode='manual', spw = '0:3260~3320', flagbackup = False)<br />
</source><br />
<br />
* '''Problematic Tsys measurements'''<br />
<br />
Above we noted issues with the Tsys measurements for both DV04 and DV12. We flag the affected data. Each of these issues should be visible in the Tsys plots you made above (e.g., see Figures 4 and 5).<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV04',flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X1d5"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
<br />
asdm="uid___A002_X215db8_X392"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV12',correlation='YY', flagbackup=False)<br />
</source><br />
<br />
* '''Unreliable Short-Spacing Measurements on Titan'''<br />
<br />
<br />
[[File:amp_vs_uv_titan_obs.png|200px|thumb|right|<caption>'''Fig. 17.''' Observed amplitude vs. uv-distance for observations of Titan in the first data set. Note the scatter for low projected antenna separations. We will flag these short-spacing data, which may reflect contamination by Saturn, and use only the more extended baselines for flux calibration.</caption>]]<br />
<br />
<br />
Saturn may contaminate the short u-v spacings from Titan. In any case these often show significant scatter (Figure 17), so we flag them. There are still enough baselines to determine a good amplitude calibration for each antenna.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X1ff7b0_Xb"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~40', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X1ff7b0_X1c8"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~50', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X4d7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', field='Titan', uvrange='0~30', flagbackup = False)<br />
</source><br />
<br />
* '''Delay Issues''' <br />
<br />
<br />
[[File:phase_vs_freq_DV13.png|200px|thumb|right|<caption>'''Fig. 18.''' Phase vs. frequency for DV13 in a data set where this antenna shows evidence of imperfect delay calibration.</caption>]]<br />
<br />
<br />
DV13 and a few other antennas show signatures of an imperfect delay calibration. This is most easily identified via strong "wrapping" of phase as a function of frequency (see Figure 18). Such effects can be calibrated out with mild delay issues largely accounted for by the bandpass solution. The phase wrapping in DV13 seems weak enough that we will trust the calibrations to remove it. For a more extreme example see the CASA guide describing the calibration of [[NGC3256Band3]].<br />
<br />
* '''Missing Phase Calibrator Observations'''<br />
<br />
<br />
[[File:amp_vs_time_missing_scans.png|200px|thumb|right|<caption>'''Fig. 19.''' Amplitude vs. time colored by field for a data set where the final visit to the phase calibrator is missing. We will flag the last set of source data to ensure that each visit to the source is flanked in time by visits to the phase calibrator.</caption>]]<br />
<br />
<br />
As a general rule, we want to be sure that observations of the phase calibrator (3c279) bracket each source observation. Two of the data sets do not include a final phase calibrator observation (see Figure 19) and for those two cases we flag the affected source observations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3b9"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='00:53:47~01:08:00',flagbackup = False)<br />
<br />
asdm="uid___A002_X215db8_X18"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV10',timerange='19:46:20~20:34:40',flagbackup=False)<br />
</source><br />
<br />
* '''Unexpected Scatter or Discontinuity in the Data'''<br />
<br />
For several antennas we find sudden jumps in the phase of the phase calibrator as a function of time. These are visible in the plots of uncalibrated phase vs. time for single baselines above, and we show an example in Figure 20. It will not be possible to effectively interpolate the phase between measurements when we see these discontinuities. The safest approach is to flag the source data across these jumps. We do so here (though note that the last two flaggings are borderline cases).<br />
<br />
<br />
[[File:phase_vs_time_DV09.png|200px|thumb|right|<caption>'''Fig. 20.''' Phase vs. time for DV09 on a problematic day. It may prove problematic to calibrate the data near this discontinuity so we flag data near this time.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09', timerange='21:24:09~21:35:35', flagbackup = False)<br />
<br />
asdm="uid___A002_X207fe4_X1f7" <br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV09',timerange='23:30:52~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM01',timerange='23:16:50~24:10:00',flagbackup=False)<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='PM03',timerange='23:16:50~24:10:00',flagbackup=False)<br />
</source><br />
<br />
* '''Outliers Visible After Calibration'''<br />
<br />
<br />
[[File:amp_vs_uvdist_outliers.png|200px|thumb|right|<caption>'''Fig. 21.''' Amplitude vs. u-v distance ''after'' calibration. Note the outlying data. We track these down using the select (box with green "+") and locate (magnifying glass) features inside plotms and identify several problematic baselines across our data. ''Note that you cannot make this plot at this point in the guide, these pathologies become evident after calibration, requiring an iterative approach to reduction.''</caption>]]<br />
<br />
<br />
Often issues with the data may become evident after calibration (i.e., after the next few steps that we apply). These data can appear as outliers in diagnostic plots for the calibrated data or even show up in the imaging stages. Once these are identified, best practice is to apply this new flagging then redo the calibration (if the issue is very minor, then re-calibrating may not be necessary).<br />
<br />
<source lang="python"><br />
# In CASA<br />
asdm="uid___A002_X207fe4_X3a"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', timerange='21:18:00~21:22:15', flagbackup=False)<br />
<br />
asdm="uid___A002_X2181fb_X49"<br />
flagdata(vis=asdm+'.wvrtsys.ms', mode='manual', antenna='DV01&DV04', flagbackup=False)<br />
</source><br />
<br />
Now that we've applied our flagging, back up the flags as version "User" using {{flagmanager}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis=asdm+'.wvrtsys.ms',mode='save',versionname ='User')<br />
</source><br />
<br />
Applying this flagging will remove the most egregious pathologies from the data. We are now ready to calibrate the data.<br />
<br />
==Bandpass Calibration==<br />
<br />
We begin by calibrating the phase and amplitude response of each antenna as a function of frequency, called "bandpass calibration." We have already seen that the data contain smooth but systematic variations in both phase and amplitude as a function of frequency. We can see this again in a more compact form by plotting phase as a function of frequency for all baselines associated with each antenna (Figure 22).<br />
<br />
<br />
[[File:plotms_phase_vs_freq_example_6.1.png|200px|thumb|right|<caption>'''Fig. 22.''' Phase vs. frequency for the bandpass calibrator, 3c279 in the first data set. We plot all baselines with DV02, averaged over time, and show only the 'XX' correlation.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='freq', yaxis='phase', <br />
selectdata=True, field='3c279', correlation='XX',<br />
avgtime='1e8', avgscan=True, antenna='*&*',<br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = input("Plotting phase vs. frequency for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Each plot shows phase as a function of frequency for all baselines with one antenna for 3c279. We plot only the 'XX' correlation, colorizing by baseline. With '''iteraxis''' set to antenna the green arrows at the bottom of {{plotms}} will cycle through antennas. By using '''avgscan''' and a large '''avgtime''' we average all scans and integrations. <br />
<br />
The phase (and amplitude) also varies as a function of time, as we saw before. Here are the similar plots for phase vs. time (see Figure 23).<br />
<br />
<br />
[[File:Uid_A002_X1ff7b0_Xb-phasecont_6.1.png|200px|thumb|right|<caption>'''Fig. 23.''' Phase vs. time for the bandpass calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
plotms(vis= asdm+'.wvrtsys.ms', <br />
xaxis='time', yaxis='phase', <br />
selectdata=True, field='3c279', <br />
spw='0:1200~1299', antenna='*&*',correlation='XX',<br />
avgchannel='100', avgscan=True, <br />
coloraxis='baseline', iteraxis='antenna', showgui = True)<br />
dummy_string = input("Plotting phase vs. time for "+asdm+". Hit <Enter> for next data set or cycle through antennas [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
Figure 23 shows that the phase varies with time. We need to take this temporal variation into account when we solve for the frequency variations. Therefore we carry out the bandpass calibration in two steps. First, we use {{gaincal}} to solve for the variation of phase as a function of time for 3c279 on very short timescales. We set {{gaincal}} to derive a separate phase solution for each antenna every integration by setting '''solint''' to 'int'. We solve, averaging together only a small fraction of the total bandpass (channels 1100-1300) to avoid the effects of the phase vs. frequency behavior. We will then apply this solution to remove time-dependent behavior when we solve for the frequency response of the antennas with {{bandpass}}.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
An integration is the smallest time quantum of observation in the data, usually a few seconds. <br />
You can check the integration length for each scan in listobs. <br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
for asdm in basename_all:<br />
print ("Running a short solution interval phase calibration for "+asdm)<br />
os.system('rm -rf '+asdm+'.bpphase.gcal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
selectdata=True,field = '3c279',spw = '0:1100~1300',<br />
caltable = asdm+'.bpphase.gcal',<br />
solint = 'int',refant = 'DV11',calmode='p')<br />
</source><br />
<br />
Now we use {{bandpass}} to solve for the frequency response of each antenna. To do this, we average all data in time by setting '''solint''' to 'inf' (that is, 'infinite'). We allow combination across scans and the different field IDs found for 3c279 by setting '''combine''' to "scan,field". We apply the phase vs. time calibration that we just derived on-the-fly using the parameter "gaintable".<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
print ("Running a bandpass calibration for "+asdm)<br />
os.system('rm -rf '+asdm+'.bandpass.bcal')<br />
bandpass(vis = asdm+'.wvrtsys.ms',<br />
field = '3c279',<br />
gaintable = asdm+'.bpphase.gcal',<br />
caltable = asdm+'.bandpass.bcal',<br />
bandtype='B',<br />
solint = 'inf',combine = 'scan,field', solnorm=True,refant = 'DV11',<br />
minblperant=3,minsnr=2,fillgaps=62)<br />
</source><br />
<br />
in these calls:<br />
<br />
*'''caltable''' specifies the output calibration table<br />
*'''gaintable''' specifies any calibration tables to be applied "on the fly" to the data before solving.<br />
*'''solint''' sets the time interval for which solutions are derived for each antenna.<br />
*'''refant''' set the reference antenna, here DV11, which is defined to have zero correction by construction.<br />
*'''calmode''' 'p' sets {{gaincal}} to calibrate the phase only, other options are 'a'mplitude or 'ap' - amplitude and phase.<br />
*'''minblperant''' sets the minimum baselines required per antenna for a successful solution.<br />
*'''minsnr''' sets the minimum signal-to-noise ratio required for a successful solution.<br />
*'''bandtype''' here 'B' tells {{bandpass}} to solve for the phase and frequency response of each antenna in each channel, averaging data in time across the solution interval.<br />
*'''fillgaps''' tells {{bandpass}} to interpolate across channel gaps 1 channel wide.<br />
*'''solnorm''' tells {{bandpass}} to normalize the bandpass amplitude and phase corrections to have magnitude unity. The absolute scale of the calibration will come from our later {{gaincal}} solutions.<br />
<br />
Do not worry about the message "Insufficient unflagged antennas" when running the {{bandpass}} task. This indicates that {{bandpass}} is failing on the flagged edge channels, which is expected.<br />
<br />
It is now a good idea to plot both sets of solutions to look for irregularities, especially:<br />
<br />
* discontinuities in the phase vs. time solution<br />
* rapid wrapping of phase in either phase vs. time or bandpass solution<br />
* large roll-off in the amplitude response near the band edge in the bandpass solution<br />
* large scatter in any solution.<br />
<br />
We loop through and using analysis utils, again generating .png files of each calibration. '''For what systems is this true?''' As the bandpass plots take a while you may want to go have a cup of coffee and inspect them using your favorite image viewer. Otherwise, switch the '''interactive''' parameter to True to see them in real time instead.<br />
<br />
<br />
[[File:uid___A002_X1ff7b0_X1c8.bpphase.page1_6.1.png|200px|thumb|right|<caption>'''Fig. 24.''' Short interval phase vs. time solution carried out and applied during bandpass calibration, here for the first set of antennas for the first data set. This solution removes any systematic variations in phase vs. time.</caption>]]<br />
<br />
<br />
[[File:Uid___A002_X1ff7b0_X1c8.bandpass.DV02.spw0.t1_6.1.png|200px|thumb|right|<caption>'''Fig. 25.''' Bandpass amplitude solution for the first set of antennas and the first data set.</caption>]]<br />
<br />
[[File:uid___A002_X1ff7b0_Xb.bcal_phase.page1.png|200px|thumb|right|<caption>'''Fig. 26.''' Phase vs. frequency calibration from the bandpass calibration for the first set of antennas and the first data set.</caption>]]<br />
<br />
<br />
<!--<br />
Again this warning:<br />
No ASDM_CALWVR, ASDM_CALATMOSPHERE, or CalWVR.xml table found. Using PWV 1.0mm.<br />
--><br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
os.system('rm -rf cal_plots/*bandpass*.png')<br />
for asdm in basename_all:<br />
plotbandpass(caltable=asdm+'.bandpass.bcal',<br />
field='',xaxis='freq',yaxis='both',<br />
figfile='cal_plots/'+asdm+'.bandpass.png',<br />
showatm=True,interactive=False,subplot=42)<br />
</source><br />
<br />
==Absolute Flux Calibration==<br />
<br />
The bandpass calibration will account for the phase and amplitude response of our antennas as a function of frequency. We now solve for the absolute flux scale of the data by referencing to Titan and in the next section we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
Before using Titan to set the flux, there is an important systematic to account for. When we looked at the integrated spectra of our targets above, remember that Titan showed a whopping spectral line, in fact the same CO(3-2) line that we wish to observe in the Antennae. We will set the flux of Titan (and thus all of our data) by referencing to a model in casa that does not account for this line. Therefore we need to flag the part of the Titan observations contaminated by the line before we calibrate. We run the following additional flagging step:<br />
<br />
[[File:Antennae-TitanLine_6.1.png|200px|thumb|right|<caption>'''Fig. 27.''' Uncalibrated amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.</caption>]]<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Flagging CO(3-2) in Titan for "+asdm)<br />
flagdata(vis=asdm+'.wvrtsys.ms',flagbackup=False, <br />
field='Titan', spw='0:1100~1700')<br />
<br />
for asdm in basename_all:<br />
flagmanager(vis =asdm+'.wvrtsys.ms',mode = 'save',versionname = 'Calibration')<br />
</source><br />
<br />
Next, we will use the {{setjy}} task to read the predicted complex visibilities for Titan into the MODEL column of each data set. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator.<br />
<br />
<source lang="python"><br />
# in CASA<br />
# make sure usescratch=True<br />
for asdm in basename_all:<br />
print ("Reading model for Titan into "+asdm)<br />
setjy(vis = asdm+'.wvrtsys.ms',field = 'Titan',<br />
usescratch=True,scalebychan=True,<br />
standard = 'Butler-JPL-Horizons 2012')<br />
</source><br />
<br />
{{setjy}} will output the flux of Titan to the CASA logger and it is worth recording this information. The flux of Titan at these frequencies is about 3.3 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.wvrtsys.ms:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Titan: spw0 Flux:[I=3.2637,Q=0.0,U=0.0,V=0.0] +/- [I=0.0,Q=0.0,U=0.0,V=0.0] Jy @ 344.85GHz<br />
</pre><br />
<br />
Now we are going to take a look at the Titan model for one dataset:<br />
<br />
<source lang="python"><br />
# in CASA<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
<br />
plotms(vis ='uid___A002_X1ff7b0_X1c8.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='model', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
</source><br />
<br />
[[File:2012_titan_model_6.1.png|thumb|right|200px|<caption>'''Fig. 28.''' Titan model produced by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
[[File:2010_titan_model.png|thumb|right|200px|<caption>'''Fig. 29.''' Titan model produced by 'Butler-JPL-Horizons 2010'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation made in this section uses the flux density standard '''standard='Butler-JPL-Horizons 2012' '''. Previous versions of this guide used an older flux density standard '''standard='Butler-JPL-Horizons 2010' '''. Figures 28 and 29 show the differences in the Titan models produced using these two standards. <br />
</div><br />
<br />
==Gain (Phase and Amplitude) Calibration==<br />
<br />
With the flux now properly scaled, we will calibrate the phase and amplitude behavior of the antennas as a function of time.<br />
<br />
We begin by running a short-solution interval {{gaincal}} to solve for phase variation on short timescales during observations of our two calibrators, with '''solint''' set to "int". By applying this on-the-fly, we can remove any decorrelation in the data due to phase scatter when we solve for the amplitude calibration. However, there is no benefit to using this short-timescale solution to calibrate the source because we only have information on the gain during calibrator visits (though see the Imaging portion of this guide). Instead we will solve for the gains to apply to the source using a longer '''solint''' in just a moment.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Carrying out short timescale phase solution for "+asdm)<br />
os.system('rm -rf '+asdm+'.intphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.intphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='int',minsnr=2.0, minblperant=4)<br />
</source><br />
<br />
Now we derive the longer timescale phase calibration table using '''solint''' set to "inf", but not allowing scan combination. This calibration has higher signal to noise due to combining more data, and for the purposes of correcting the source, it is just as precise as the short timescale solution.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Carrying out longer timescale phase solution for "+asdm)<br />
os.system('rm -rf '+asdm+'.scanphase.gcal')<br />
gaincal(vis=asdm+'.wvrtsys.ms',<br />
gaintable=asdm+'.bandpass.bcal', <br />
caltable=asdm+'.scanphase.gcal',<br />
calmode='p',<br />
field='Titan,3c279',<br />
spw='0:40~3800',<br />
refant='DV11', solint='inf',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
Now we apply the short-timescale phase solution and carry out a scan length ('''solint''' set to "inf") calibration of the data using '''calmode''' of 'ap'.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Solving for longer (scan) interval amplitude solution for "+asdm)<br />
os.system('rm -rf '+asdm+'.amp.cal')<br />
gaincal(vis = asdm+'.wvrtsys.ms',<br />
gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],<br />
caltable = asdm+'.amp.cal',<br />
calmode='ap',<br />
field = 'Titan,3c279',<br />
spw='0:40~3800',<br />
refant = 'DV11',solint = 'inf', minblperant=4)<br />
</source><br />
<br />
This "amp.cal" solution gives us the amplitude variations as a function of time, but they are not yet pinned to a realistic scale except in the case of Titan, where we have solved using the model input by {{setjy}}. We will set the flux of our secondary calibrator 3c279 with reference to Titan using {{fluxscale}}.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Scaling amplitude calibration to match Titan for "+asdm)<br />
os.system('rm -rf '+asdm+'.flux.cal')<br />
fluxscale(vis = asdm+'.wvrtsys.ms',<br />
caltable = asdm+'.amp.cal',<br />
fluxtable = asdm+'.flux.cal',<br />
reference = 'Titan',<br />
transfer = '3c279')<br />
</source><br />
<br />
This new correctly-scaled flux table ".flux.cal" replaces the previous ".amp.cal" table as the correct amplitude calibration table to apply to the data, i.e., the ".flux.cal" contains both the time variability of the amplitude solved for in ".amp.cal" and the correct flux scaling set with {{fluxscale}}.<br />
<br />
Fluxscale will output the derived flux for 3c279 to the CASA logger. This information is worth noting. We find that the flux of 3c279 is 10.79 Jy, by averaging the fluxes obtained from the ten available datasets. If we check this value against reported flux measurements in the SMA calibrator list [http://sma1.sma.hawaii.edu/callist/callist.html], we find that a flux of 10.79 Jy agrees within a few per cent of a linear interpolation between the two 0.85 mm measurements closest in time to the ALMA observations: (23 May 2011, SMA 11.644 ± 0.583; 01 July 2011, 9.748 ± 0.488).<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the older, Butler-JPL-Horizons 2010, flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for 3c279 will be slightly lower. The flux of 3c279, when averaged over all 10 measurement sets, will be 10.41 Jy.<br />
</div><br />
<br />
Now we plot the final phase and amplitude calibration tables for each data set. A well-behaved calibration table will show smooth variations as a function of time. Sudden jumps or wild variations among the antenna amplitude gains should prompt further investigation and possibly additional flagging.<br />
<br />
<source lang="python"><br />
# in CASA<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Plotting solutions for "+asdm)<br />
plotms(vis=asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase', coloraxis='corr',<br />
iteraxis='antenna',antenna='0~7',plotrange=[0,0,-180,180],<br />
gridrows=4,gridcols=1,showgui=False,<br />
plotfile=asdm+'.scanphase.page1.png')<br />
# dummy_string = input("Hit <Enter> to see next plot.")<br />
plotms(vis=asdm+'.scanphase.gcal',<br />
xaxis = 'time', yaxis = 'phase', coloraxis='corr',<br />
iteraxis = 'antenna', antenna='8~15', plotrange=[0,0,-180,180],<br />
showgui=False, gridrows=4, gridcols=1, <br />
plotfile=asdm+'.scanphase.page2.png')<br />
# dummy_string = input("Hit <Enter> to see next plot.")<br />
plotms(vis = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp', coloraxis='corr',<br />
iteraxis='antenna', antenna='0~7', plotrange = [0,0,0,0],<br />
showgui=False, gridrows=4,gridcols=1, <br />
plotfile=asdm+'.flux.page1.png')<br />
# dummy_string = input("Hit <Enter> to see next plot.")<br />
plotms(vis = asdm+'.flux.cal', <br />
xaxis = 'time',yaxis = 'amp', coloraxis='corr',<br />
iteraxis='antenna', antenna='8~15', plotrange = [0,0,0,0],<br />
showgui=False, gridrows=4,gridcols=1, <br />
plotfile=asdm+'.flux.page2.png')<br />
# dummy_string = input("Hit <Enter> to see next plot.")<br />
<br />
</source><br />
<br />
==Apply the Calibrations and Inspect ==<br />
Now we will use {{applycal}} to apply the bandpass, phase, and amplitude calibration tables that we generated in the previous sections to the data. We apply the solutions separately to the bandpass and secondary ("phase") calibrator 3c279, the flux calibrator Titan, and the target source. In most data sets, the bandpass and secondary calibrator will not be the same and this step would include one additional {{applycal}}.<br />
<br />
<source lang="python"><br />
#In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
for asdm in basename_north: <br />
print ("Applying calibrations for "+asdm)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='NGC*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_south: <br />
print ("Applying calibrations for "+asdm)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='3c279',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Titan',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['3c279','Titan','Titan'],flagbackup=False,calwt=False)<br />
applycal(vis=asdm+'.wvrtsys.ms',field='Ant*',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['3c279','3c279','3c279'],flagbackup=False,calwt=False)<br />
</source><br />
<br />
Here:<br />
* '''vis''' and '''field''' specify the measurement set and field to calibrate.<br />
* '''gaintable''' specifies the tables to apply to the data. There may be several tables applied all at once, here we apply the bandpass, phase, and flux calibrations.<br />
* '''interp''' specifies the type of interpolation used to match calibration tables to the data. If there are several gaintables then interp may be either a single value (e.g., "linear" or "nearest") or a list of values with size equal to the list of gaintables. In the case where interp is a list, applycal will match each element to the corresponding gaintable (e.g., first gaintable with first interp parameter).<br />
* '''gainfield''' specifies field from which to draw the calibration. As with interp, this can be a list of size matched to gaintable, so that the first element of gainfield will be the field used for the first table. For example, in the above commands the bandpass calibration (the first table) will always be applied from '3c279' (the first gainfield).<br />
* '''calwt''' indicates that the weights, which were derived from the Tsys measurements, should also be calibrated. Note that '''calwt''' is True by default.<br />
<br />
Once calibrations are applied, it is important to go back and inspect the calibrated data. New problematic antennas or baselines may be visible after calibration. Repeat the steps above, focusing on the CORRECTED data column. Bear in mind that for any point source calibrators we now expect to find phase scattering around zero and to find flat amplitudes as a function of u-v distance. Look for outliers and other signatures of problematic data. As a general rule, you will want to incorporate these data into your overall flagging script then rerun the whole calibration process, so that reduction is iterative. If the data only represent a minor problem, however, it may not be terribly harmful to flag them after the fact so that they do not interfere with imaging but trust that the calibrations are mostly unaffected.<br />
<br />
As an example of this inspection, we cycle through the corrected amplitudes and phases of 3c279 and Titan as a function u-v distance, to check that the phases are close to zero and the amplitudes are constant. <br />
<br />
[[File:xb-checkcal-pha-cha_6.1.png|200px|thumb|right|<caption>'''Fig. 30.''' Calibrated phase vs. channel plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<br />
[[File:xb-checkcal-amp-time_6.1.png|200px|thumb|right|<caption>'''Fig. 31.''' Calibrated amplitude vs. time plot for 3c279 (uid___A002_X1ff7b0_Xb dataset). </caption>]]<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Examining calibrated data for "+asdm)<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='3c279',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='time', yaxis='phase',<br />
ydatacolumn='corrected', field='3c279',<br />
avgchannel='3840', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
<br />
<br />
for asdm in basename_all:<br />
print ("Examining calibrated data for "+asdm)<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='phase',<br />
ydatacolumn='corrected', field='Titan',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Titan',avgtime='1e8',<br />
avgchannel='', avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot [type 'stop'+<Enter> to break out].")<br />
# check if a stop has been requested<br />
if dummy_string == "stop":<br />
break<br />
</source><br />
<br />
In Figure 30 and Figure 31 we plot phase vs. channel and amp vs. time for 3c279 for the uid___A002_X1ff7b0_Xb dataset. <br />
<br />
Finally we can use {{plotms}} to examine the corrected amplitude and phase of Antennae galaxies as a function of time and uv-distance (plots not shown):<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
for asdm in basename_north:<br />
print ("Examining calibrated data for "+asdm)<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='NGC*',<br />
avgtime='1e8', avgscan=True, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
for asdm in basename_south:<br />
print ("Examining calibrated data for "+asdm)<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='uvdist', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
averagedata=True, avgchannel='3840', avgtime='',<br />
avgscan=False, avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
plotms(vis = asdm+'.wvrtsys.ms', xaxis='freq', yaxis='amp',<br />
ydatacolumn='corrected', field='Ant*',<br />
avgtime='1e8', avgscan=True,avgbaseline=False, coloraxis='corr', showgui = True)<br />
dummy_string = input("Hit <Enter> for next plot.")<br />
</source><br />
<br />
==Split and Concatenate Data for Northern and Southern Mosaics==<br />
The individual data sets are now calibrated. We can safely split out the calibrated data for our science target and drop the calibrators. As we do so, we will smooth the data in frequency, averaging together groups of 23 channels by setting '''width=23''' in {{split}}. The new data will have a channel width corresponding to about ~10 km/s, very similar to the SMA data being verified. The factor of >20 drop in data volume will also make the imaging steps ''much'' more tractable.<br />
<br />
Note that you will see a warning that the last output channel in the new spw 0 will be dropped, as it has less than 23 channels over which to average:<br />
<br />
<pre style="background-color: #fffacd;"><br />
INFO MSTransformDataHandler The last output channel of spw 0 has only 22 input channels.<br />
WARN MSTransformManager Not enough input channels to populate output channel n# 166 from SPW 0.<br />
WARN MSTransformManager The channel will be dropped in order to have an uniform grid.<br />
</pre><br />
<br />
<source lang="python"><br />
#In CASA<br />
<br />
basename_north=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]<br />
<br />
basename_south=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",<br />
"uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_north:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'NGC*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
<br />
for asdm in basename_south:<br />
os.system('rm -rf '+asdm+'.cal.ms')<br />
split(vis = asdm+'.wvrtsys.ms',outputvis = asdm+'.cal.ms',<br />
field = 'Ant*',spw='0',width=23, keepflags=False)<br />
os.system('rm -rf '+asdm+'.cal.listobs.txt')<br />
listobs(asdm+'.cal.ms',listfile=asdm+'.cal.listobs.txt')<br />
</source><br />
<br />
For convenience we concatenate all data for the Northern Mosaic into a single big MS and place all data for the Southern Mosaic into another file. To do this, we construct a list that holds the names of all the Southern Mosaic MS files and another that holds the name of all the Northern Mosaic MS files then feed these into the {{concat}} task. <br />
<br />
Before we do this, we'll take a brief aside to remove Pointing information from the calibrated data. Normally, the pointing information would apply important corrections to be used while imaging the data. However, some aspects of this are still under development and the pointing information in these data interacts poorly with CASA. We will remove the pointing information using the CASA toolkit Table tool (tb). ''This kind of operation is dangerous and should not be a regular feature of your CASA reduction.''<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",<br />
"uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",<br />
"uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",<br />
"uid___A002_X215db8_X392"]<br />
<br />
for asdm in basename_all:<br />
print ("Manually removing pointing table from "+asdm+".cal.ms ... Please be careful!")<br />
tb.open(asdm+'.cal.ms/POINTING',nomodify=False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
tb.close()<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
cal_south_vis = [vis+'.cal.ms' for vis in basename_south]<br />
cal_north_vis = [vis+'.cal.ms' for vis in basename_north]<br />
<br />
os.system('rm -rf Antennae_South.cal.ms')<br />
concat(vis=cal_south_vis, concatvis='Antennae_South.cal.ms', timesort=True)<br />
<br />
os.system('rm -rf Antennae_North.cal.ms')<br />
concat(vis=cal_north_vis, concatvis='Antennae_North.cal.ms', timesort=True)<br />
</source><br />
<br />
The syntax used to construct the 'cal_south_vis' variable loops over basename_south and makes a list after adding '.cal.ms' to each member. To see the list 'print (cal_south_vis)'.<br />
<br />
==Continue on to Imaging of the Science Target==<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=AntennaeBand7_Imaging_6.1 imaging guide].<br />
<div style="background-color: #CCFF99;"><br />
<p>Last checked on CASA Version 6.1.0.<br />
</p><br />
</div></div>Jbertonehttps://casaguides.nrao.edu/index.php?title=Guide_To_Processing_ALMA_Data_for_Cycle_0&diff=30883Guide To Processing ALMA Data for Cycle 02021-08-19T21:14:16Z<p>Jbertone: </p>
<hr />
<div>== About this Guide, and Cycle 0 ALMA Data ==<br />
<br />
This guide describes steps you can take to make science-ready images from Cycle 0 ALMA data available from the ALMA data archive. We begin with the process of locating and downloading the data from the public archive.<br />
<br />
This guide is explicitly written for Cycle 0 data. Data for Cycle 1 and beyond will be delivered in a different format and will be described in a separate guide.<br />
<br />
For Cycle 0, ALMA data is delivered as a "data package" that includes a set of calibrated data files and a sample of reference images. The data were calibrated and imaged by an ALMA scientist at one of the ALMA Regional Centers (ARCs) using CASA. The scientist developed CASA scripts to complete the calibration and imaging. Cycle 0 data packages also include fully calibrated data sets. Most users will start with these calibrated data and proceed directly to the imaging steps. In many cases where a target is observed with high signal-to-noise, the imaging can be improved dramatically with "self-calibration" on the science target. Self-calibration is the process of using the detected signal in the target source, itself, to tune the phase (and perhaps amplitude) calibrations, as a function of time.<br />
<br />
Typically, users interested in making science-ready images with Cycle 0 data from the ALMA archive will take the following steps:<br />
<br />
# Download the data from the Archive<br />
# Inspect the Quality Assurance plots and data files<br />
# Inspect the reference images supplied with the data package<br />
# Combine the calibrated data sets into a single calibrated measurement set<br />
# Self-calibrate and image the combined data set<br />
# Generate moment maps and other analysis products<br />
<br />
Most users will not need to modify the calibration supplied in the data package. But in some cases, users may wish to review the calibration steps in detail, make modifications to the calibration scripts, and generate new calibrated data sets.<br />
<br />
<br />
=== About the Sample Data: H2D+ and N2H+ in TW Hya ===<br />
<br />
The data for this example comes from ALMA Project 2011.0.00340.S, "Searching for H2D+ in the disk of TW Hya v1.5", for which the PI is Chunhua Qi. Part of the data for this project has been published in [http://adsabs.harvard.edu/abs/2013Sci...341..630Q Qi et al. 2013].<br />
<br />
This observation has three scientific objectives:<br />
# Image the submm continuum structure in TW Hya<br />
# Image the H2D+ line structure (rest frequency 372.42138 GHz)<br />
# Image the N2H+ line structure (rest frequency 372.67249 GHz)<br />
<br />
Eventually we will see that the data is distilled to 4 "spectral windows" used for science. The N2H+ line is in one spectral window, the H2D+ is in another window, and the other two windows contribute to the continuum observation. The best continuum map will be generated from the line-free channels of all 4 spectral windows.<br />
<br />
Each spectral window covers 234.375 MHz in bandwidth, and the raw data contain 3840 channels, spaced by 61 kHz. Information on the spectral configuration and targets observed must be gleaned from the data themselves. In CASA, you can explore the contents of a data set using the '''listobs''' command.<br />
<br />
This project was completed by executing a scheduling block three times, leading to three measurement sets. These measurement sets have identical configurations. Our strategy will be to concatenate the three data sets into a single measurement set, and than image that one, combined data set.<br />
<br />
=== What other data sets are available? ===<br />
<br />
A [https://almascience.nrao.edu/alma-data/archive/delivery-list Delivery List] of publicly available data sets is provided on the ALMA Science Portal, in the "Data" tab.<br />
<br />
== Prerequisites : Computing Requirements ==<br />
<br />
ALMA data sets can be very large and require significant computing resources for efficient processing. This example begins with a download of 176 GB of data. For a recommendation on computing resources needed to process ALMA data, check [http://casa.nrao.edu/casa_hardware-requirements.shtml here]. Those who do not have sufficient computing power may wish to arrange a visit to one of the ARCs to use the computing facilities at these sites. To arrange a visit to an ARC, submit a ticket to the [https://help.almascience.org/ ALMA Helpdesk]. Remote use of ARC computing facilities is not available at this time.<br />
<br />
== Getting the Data: The ALMA Data Archive ==<br />
<br />
The ALMA data archive is part of the [http://www.almascience.org/ ALMA Science Portal]. A copy of the archive is stored at each of the ARCs, and you can connect to the nearest archive through these links:<br />
* [https://almascience.nrao.edu/alma-data/archive North America]<br />
* [http://almascience.eso.org/alma-data/archive Europe]<br />
* [http://almascience.nao.ac.jp/alma-data/archive East Asia]<br />
<br />
<br />
[[Image:Archive_interface.png|thumb|<caption>'''Fig. 1.''' The interface to the ALMA data archive.</caption>]]<br />
<br />
<br />
Upon entry into the ALMA Archive Query page, set the "Results View" option to "project" (see the red highlight #1 in Figure 1) and specify the Project Code to 2011.0.00340.S (red highlight #2). Note, if you leave the "Results View" set to "raw data", you will be presented with three rows of data sets in the results page. These correspond to three executions of the observing script. In fact, for Cycle 0 data these rows contains copies of the same data set, so use care not to download the (large!) data set three times. By setting "Results View" to "project", you see just one entry, and that is the one you'd like to download.<br />
<br />
You can download the data through the Archive GUI. For more control over the download process, you can use the Unix shell script provided on the Request Handler page. This script has a name like "downloadRequest84998259script.sh". You need to put this file into a directory that contains ample disk space, and execute the script in your shell. For example, in bash:<br />
<br />
<source lang="bash"><br />
# In bash<br />
% chmod +x downloadRequest84998259script.sh<br />
% ./downloadRequest84998259script.sh<br />
</source><br />
<br />
=== Unpacking the data ===<br />
The data you have downloaded for this particular project includes 17 tar files. Unpack these using the following command:<br />
<br />
<source lang="bash"><br />
# In bash<br />
% for i in $(ls *.tar); do echo 'Untarring ' $i; tar xvf $i; done<br />
</source><br />
<br />
At this point you will have a directory called "2011.0.00340.S" with the full data distribution.<br />
<br />
== Overview of Delivered Data and Products ==<br />
<br />
To get to the directory that contains the data package, do:<br />
<br />
<source lang="bash"><br />
# In bash<br />
% cd 2011.0.00340.S/sg_ouss_id/group_ouss_id/member_ouss_2012-12-05_id<br />
</source><br />
<br />
Here you will find the following entries:<br />
<br />
<source lang="bash"><br />
# In bash<br />
% ls<br />
calibrated calibration log product qa raw README script<br />
</source><br />
<br />
These directories are standard for all cycle 0 data packages. The README file describes the files in the distribution and includes notes from the ALMA scientist who performed the initial calibration and imaging.<br />
<br />
In our example, the directories contain the following files:<br />
<br />
<br />
-- calibrated/<br />
|-- uid___A002_X554543_X207.ms.split.cal/<br />
|-- uid___A002_X554543_X3d0.ms.split.cal/<br />
|-- uid___A002_X554543_X667.ms.split.cal/<br />
-- calibration/<br />
|-- uid___A002_X554543_X207.calibration/<br />
|-- uid___A002_X554543_X207.calibration.plots/<br />
|-- uid___A002_X554543_X3d0.calibration/<br />
|-- uid___A002_X554543_X3d0.calibration.plots/<br />
|-- uid___A002_X554543_X667.calibration/<br />
|-- uid___A002_X554543_X667.calibration.plots/<br />
-- log/<br />
|-- uid___A002_X554543_X207.calibration.log<br />
|-- uid___A002_X554543_X3d0.calibration.log<br />
|-- uid___A002_X554543_X667.calibration.log<br />
|-- Imaging.log<br />
|-- 340.log<br />
-- product/<br />
|-- TWHya.continuum.fits<br />
|-- TWHya.N2H+.fits<br />
|-- TWHya.continuum.mask/<br />
|-- TWHya.H2D+.mask/<br />
|-- TWHya.N2H+.mask/<br />
-- qa/<br />
|-- uid___A002_X554543_X207__textfile.txt<br />
|-- uid___A002_X554543_X207__qa2_part1.png<br />
|-- uid___A002_X554543_X207__qa2_part2.png<br />
|-- uid___A002_X554543_X207__qa2_part3.png<br />
|-- uid___A002_X554543_X3d0__textfile.txt<br />
|-- uid___A002_X554543_X3d0__qa2_part1.png<br />
|-- uid___A002_X554543_X3d0__qa2_part2.png<br />
|-- uid___A002_X554543_X3d0__qa2_part3.png<br />
|-- uid___A002_X554543_X667__textfile.txt<br />
|-- uid___A002_X554543_X667__qa2_part1.png<br />
|-- uid___A002_X554543_X667__qa2_part2.png<br />
|-- uid___A002_X554543_X667__qa2_part3.png<br />
-- raw/<br />
|-- uid___A002_X554543_X207.ms.split/<br />
|-- uid___A002_X554543_X3d0.ms.split/<br />
|-- uid___A002_X554543_X667.ms.split/<br />
-- script/<br />
|-- uid___A002_X554543_X207.ms.scriptForCalibration.py<br />
|-- uid___A002_X554543_X3d0.ms.scriptForCalibration.py<br />
|-- uid___A002_X554543_X667.ms.scriptForCalibration.py<br />
|-- scriptForImaging.py<br />
|-- import_data.py<br />
|-- scriptForFluxCalibration.py<br />
<br />
<br />
* calibrated: This directory contains a calibrated measurement set for each of the execution blocks in the project. In our case, there are three. These could be imaged separately, or combined to make a single calibrated measurement set. In this guide we will combine the three executions prior to imaging.<br />
* calibration: Auxiliary data files generated in the calibration process.<br />
** The "calibration.plots" directories contain (a few hundred) plots generated during the calibration process. These can be useful for the expert user to assess the quality of the calibration at each step.<br />
* log: Log files from the CASA sessions used by the ALMA scientist to calibrate and image the data. These are provided for reference.<br />
* product: The final data products from the calibration and imaging process. The directory contains "reference" images that are used to determine the quality of the observation, but they are not necessarily science-ready. The data files here can used for initial inspections.<br />
* qa: The result of standardized Quality Assessment tests. The data from each scheduling block goes through such a quality assessment, and all data delivered to the public ALMA Archive have passed the quality assessment. It is worthwhile to review the plots and text files contained here. You will find plots of the antenna configuration, UV coverage, calibration results, Tsys, and so on.<br />
* raw: The "raw" data files. If you would like to tune or refine the calibration, these files would be your starting point. In fact, these data files do have certain a priori calibrations already applied, amounting to steps 0-6 of the calibration scripts provided in the data package. These steps include a priori flagging and application of WVR, Tsys, and antenna position corrections.<br />
* script: These are the scripts developed and applied by the ALMA scientist, to calibrate the data and generate reference images. These scripts cannot be applied directly to the raw data provided in this data distribution, but they do serve as a valuable reference to see the steps that need to be taken to re-calibrate the data, if you so choose.<br />
<br />
== What if I need to redo the Calibration? ==<br />
<br />
In most cases, users should not need to redo the [[Guide_To_Processing_ALMA_DataGlossary#Calibration|calibration]]. However, you should consult [https://help.almascience.org/kb/articles/will-re-reduction-improve-the-cycle-0-data-products-provided-by-the-archive this knowledgebase article] for suggestions on instances when recalibration may be beneficial. You might also want to redo calibration to include your own preferences (for flagging, gain solutions calculation, flux references), or to tailor it to your specific needs. If you would like to learn how to redo calibration, proceed to the [[Guide_To_Redo_Calibration_For_ALMA_Cycle_0|Guide To Redo Calibration For ALMA Cycle 0]].<br />
<br />
<span id="DataConcatenation"></span><br />
<br />
== Data Concatenation ==<br />
<br />
The data package includes a fully calibrated data set for each of the execution blocks. Each data set has been calibrated first according to the steps in the '''scriptForCalibration''' scripts, and subsequently by the steps listed in the '''scriptForFluxCalibration.py''' script. The data delivery package does not include a concatenated measurement set, so we will do the concatenation here, then proceed to imaging. Let's make a new directory for imaging, and do all subsequent work from there.<br />
<br />
<source lang="bash"><br />
# In bash<br />
mkdir Imaging<br />
cd Imaging<br />
casapy<br />
</source><br />
<br />
and then, in CASA:<br />
<br />
<source lang="python"><br />
# In CASA<br />
concat(vis = ['../calibrated/uid___A002_X554543_X207.ms.split.cal', '../calibrated/uid___A002_X554543_X3d0.ms.split.cal',<br />
'../calibrated/uid___A002_X554543_X667.ms.split.cal'], concatvis = 'calibrated.ms')<br />
</source><br />
<br />
This operation could take an hour or longer. When completed, we have created a calibrated measurement set '''calibrated.ms''', which is ready for imaging.<br />
<br />
== Imaging ==<br />
<br />
A first step is to list the contents of the '''calibarated.ms''' measurement set using the CASA task {{listobs}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='calibrated.ms',listfile='calibrated.listobs')<br />
</source><br />
<br />
The text file '''calibrated.listobs''' now contains a summary of the contents of this data file. The {{listobs}} output includes the following section:<br />
<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) BBC Num Corrs <br />
0 ALMA_RB_07#BB_1#SW-01#FULL_RES 3840 TOPO 372286.368 61.035 234375.0 1 XX YY<br />
1 ALMA_RB_07#BB_2#SW-01#FULL_RES 3840 TOPO 372532.812 61.035 234375.0 2 XX YY<br />
2 ALMA_RB_07#BB_3#SW-01#FULL_RES 3840 TOPO 358727.946 -61.035 234375.0 3 XX YY<br />
3 ALMA_RB_07#BB_4#SW-01#FULL_RES 3840 TOPO 358040.598 -61.035 234375.0 4 XX YY<br />
<br />
We see that the data file contains 4 spectral windows. The H2D+ data are contained in SpwID=0 (rest frequency of H2D+: 372.42138 GHz) and the N2H+ data are contained in SpwID=1 (rest frequency of N2H+: 372.67249 GHz). Note that the Ch0 column gives the sky frequency at channel 0. To get central frequencies for the spectral windows, add 117.1875 MHz for Spectral Window IDs 0 and 1. Subtract 117.1875 MHz for IDs 2 and 3. (Note the channel width for these IDs is negative, indicating the frequencies increase in the opposite direction as SpwID 0 and 1.<br />
<br />
Our strategy will be to image and self-calibrate the continuum data first. We will then apply the self-cal solutions to each of the two line data files, and image those second. In general terms, we will follow the steps outlined in the [[TWHydraBand7_Imaging_4.2|Science Verification CASAguide on TW Hya]]. You can also refer to the '''ScriptForImaging.py''' script included with the data package.<br />
<br />
Other data sets may require different strategies. This strategy works best in this case because the continuum is bright, and suitable for self-calibration. A data set with a bright line may benefit from self-calibration on the line, instead of the continuum.<br />
<br />
=== Continuum Imaging ===<br />
<br />
First we will split off the TW Hya data and average the data over 100 channels, eliminating a few channels at each edge prior to averaging. This will generate a data set suitable for continuum imaging. The final data set has 6.1 MHz channels, sufficient to allow efficient cleaning with the "Multi-Frequency Synthesis" algorithm used by the CASA task {{clean}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
split(vis='calibrated.ms', outputvis='TWHya_cont.ms', datacolumn='data', keepflags=True, field='TW Hya', spw='0~3:21~3820', width=100)<br />
listobs(vis='TWHya_cont.ms', listfile='TWHya_cont.listobs')<br />
</source><br />
<br />
Now plot amplitude vs. channel with {{plotms}} to see what needs to be flagged. See Figure 2.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHya_cont.ms',spw='0~3',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,coloraxis='spw',iteraxis='spw',xselfscale=True)<br />
</source><br />
<br />
<br />
[[Image:Plotms_amp.png|thumb|<caption>'''Fig. 2.''' In SpwID = 1 we notice elevated amplitudes in channel 24. This arises from the strong N2H+ line emission. We'd like to flag this channel prior to imaging the continuum.</caption>]]<br />
<br />
<br />
We flag the line emission using the {{flagdata}} task as follows:<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='TWHya_cont.ms', mode='manual', spw='1:24~24')<br />
</source><br />
<br />
Next we will generate a first-pass continuum image. We will use mode "mfs" to generate a single continuum image from all the data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHya.continuum*')<br />
clean(vis = 'TWHya_cont.ms',<br />
imagename = 'TWHya.continuum',<br />
field = '1', # TW Hya<br />
spw = '0~3',<br />
mode = 'mfs',<br />
imagermode = 'csclean',<br />
interactive=True,<br />
imsize = [300, 300],<br />
cell = '0.1arcsec',<br />
phasecenter = 1,<br />
weighting = 'briggs',<br />
robust = 0.5)<br />
</source><br />
<br />
<br />
[[Image:Cont_clean1.png|thumb|<caption>'''Fig. 3.''' After a few minutes, the '''clean''' procedure returns a plot of the dirty map as shown.</caption>]]<br />
<br />
<br />
Note that at ~370 GHz the ALMA primary beam has a width of ~ 20 arcseconds FWHM. We want to image an area at least as large as that. Also, for this data set the synthesized beam is about 0.49 x 0.46 arcseconds. We have selected a cell size of 0.1 arcsec to fully sample the synthesized beam.<br />
<br />
We'd like to do some cleaning to achieve our best image. In {{clean}}, the multifrequency synthesis mode grids each spectral channel independently. Since the uv-spacing is a function of frequency, this will actually achieve greater uv-coverage than if all the channels and spectral windows had been averaged in advance.<br />
<br />
Our initial cleaned model will be used as the image model in the next self-calibration step. It is essential that only "real" signal be included in the image model, so we will clean the image conservatively. We set '''interactive=True''' to make the clean mask interactively. This also allows us to stop cleaning when the signal from the source starts to approach the surrounding "noise" areas in the residual map.<br />
<br />
Once the Clean Viewer opens, click on the ellipse tool with your left mouse button, and draw a circle around the continuum emission with the left mouse button. Double click inside the region with the left mouse button when you are happy with it. It will turn from green to white when the clean mask is accepted. To begin the clean use the green arrow in the "Next action" section on the Viewer Display GUI. <br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The red X will stop clean, the blue arrow will clean non-interactively <br />
until reaching the number of iterations requested (niter) or the flux <br />
density threshold (whichever comes first), and the green circle arrow <br />
will clean until it reaches the "iterations" parameter on the left side <br />
of the green area. <br />
</pre><br />
<br />
You should choose the green circle arrow. It will clean 100 iterations (as shown in the GUI) and then show the residual image. If necessary you can adjust the color scale to examine the residuals more clearly. The middle mouse button is initially assigned to the "plus" icon, which adjusts the color scale. Also, note in the logger that {{clean}} reports the clean flux recovered so far, the min/max residuals, and the synthesized beam size.<br />
<br />
In this example we will clean 500 iterations by clicking the green arrow five times, without changing the clean mask. After this we have cleaned about 1.5 Jy and we have residuals of about 5-7 mJy. Click the red and white "X" icon to conclude the cleaning process.<br />
<br />
There are a few points to note:<br />
* Real emission will not get cleaned if you do not put a clean box on it.<br />
* Conversely, if you put a clean box on a region that contains no real emission and clean deeply, you can amplify any artifacts or features in that region. <br />
* Thus, it's best to be conservative. Stop cleaning when the residuals inside the clean box approach those outside it. Start with a small clean box surrounding what is clearly real emission, and add to it after some iterations, if necessary. You can generally see fainter "real" features in the residuals.<br />
* There is no default mask. So if you start an interactive clean and do not make a mask, the algorithm will clean nothing.<br />
<br />
{{clean}} will generate several files:<br />
* TWHya.continuum.flux # the effective primary beam response (e.g. for pbcor)<br />
* TWHya.continuum.flux.pbcoverage # the primary beam coverage (ftmachine=’mosaic’ only)<br />
* TWHya.continuum.image # the final restored image<br />
* TWHya.continuum.mask # the clean mask<br />
* TWHya.continuum.model # the model image<br />
* TWHya.continuum.psf # the synthesized (dirty) beam<br />
* TWHya.continuum.residual # the residual image<br />
<br />
<br />
To examine the cleaned image with the CASA viewer, use:<br />
<source lang="python"><br />
# In CASA<br />
viewer('TWHya.continuum.image')<br />
</source><br />
<br />
=== First cycle of self-calibration: phase only ===<br />
Next we can use the clean model to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model you give it. That is why it is important to make the model as good as you can. {{clean}} places a uv-model of the resulting image in the file <tt>TWHydra_cont.model</tt>. This model can then be used to self-calibrate the data using {{gaincal}}. We use '''gaintype='T'''' to tell it to average the polarizations before deriving phase solutions (to improve sensitivity by a factor of sqrt(2)). The "solution interval" ('''solint''') is critical and may require some experimentation to get optimal results. From {{listobs}} we see that the integration time of these data is 6.05 seconds, so we want to select a solution interval that is at least this long, and is matched to the expected coherence time of the atmosphere, while maintaining adequate S/N. We can use '''solint=30s''' as a starting point. It is often best to start with a larger '''solint''' and then work your way down to the integration time if the S/N of the data permits. For the reasons described in the [[TWHydraBand7_Calibration_4.2|TW Hydra Science Verification]] tutorial, we want to determine the phases before tackling amplitudes. Se we set '''calmode='p''''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='TWHya_cont.ms',caltable='self_1.pcal',<br />
solint='30s',combine='',gaintype='T',<br />
refant='DV22',spw='',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<br />
[[Image:C0Guide_self_1_phase.png|thumb|<caption>'''Fig. 4.''' Phase solutions from self_1.pcal with solint=30s.</caption>]]<br />
<br />
<br />
We are using '''combine=' '''' to get independent solutions for each spw. If the S/N is insufficient to get independent solutions, you can try combining spectral windows with '''combine='spw''''. We have plenty of S/N in these data so we keep them separate.<br />
<br />
Next we'll examine the solutions using {{plotcal}}.<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',iteration='antenna',<br />
subplot=421,plotrange=[0,0,-80,80],figfile='self_1_phase.png')<br />
</source><br />
<br />
The magnitude of the corrections here gives you a sense of the accuracy of the phase transfer from the calibrators. Check that the phases are being well tracked by the solutions, i.e. smoothly varying functions of time within a given scan. Also the solutions for each spectral window should be nearly identical, indicating that the phase offsets between spectral windows are well calibrated already. In this example, the solutions look fine, overall. If they didn't you could try increasing the solint and/or combining the spw into a single solution. <br />
<br />
When you are happy with the solutions, apply them with {{applycal}}<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='TWHya_cont.ms',field='',gaintable=['self_1.pcal'],calwt=False)<br />
</source><br />
<br />
<br />
[[Image:Viewer_pcal1.png|thumb|<caption>'''Fig. 5.''' Clean residuals after 500 iterations and the 1st phase only self-cal. Residuals inside mask are higher than the "noise" outside.</caption>]]<br />
<br />
<br />
Next clean the source again. To save time we can start with the clean mask from the last iteration. If you make interactive adjustments to the clean mask, they will be written to the new mask '''TWHydra.continuum.1pcal.mask''', and not the old one. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHya.continuum.1pcal.*')<br />
clean(vis='TWHya_cont.ms',imagename='TWHya.continuum.1pcal',<br />
mode='mfs',imagermode='csclean',<br />
imsize=300,cell=['0.1arcsec'],spw='',<br />
weighting='briggs',robust=0.5, <br />
mask='TWHya.continuum.mask',usescratch=False,<br />
interactive=True,threshold='1mJy',niter=10000)<br />
</source><br />
<br />
Now after the first 500 iterations you see that the residuals inside the clean mask are higher than last time. This is because flux that had been spread out in the map due to poorly correlated phases has been recovered in the source, thereby increasing its flux. Let's continue cleaning for 1200 iterations.<br />
<br />
Now let's compare the maps made before and after self-calibration using {{imview}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file':'TWHya.continuum.image','range':[-0.01,0.05]},<br />
{'file':'TWHya.continuum.1pcal.image','range':[-0.01,0.05]}])<br />
</source><br />
<br />
Once the Viewer Display panel opens, click on the box next in the "Images" panel and then use the tape deck buttons in the panel to alternate between the two images. We use the '''range''' parameter to put the two images on the same color scale. It is obvious that the self-calibrated image has much better residuals. To quantify the improvement in rms noise, use the rectangle tool to draw a region that excludes the central source and then double click inside. Repeat that process for the second image. Statistics for the region will print to the terminal:<br />
<br />
<pre style="background-color: #fffacd;"><br />
(TWHya.continuum.image)<br />
Stokes Velocity Frame Doppler Frequency <br />
I -10.5133km/s LSRK RADIO 3.65287e+11 <br />
BrightnessUnit BeamArea Npts Sum FluxDensity <br />
Jy/beam 25.4321 20100 -8.748308e-01 -3.439866e-02 <br />
Mean Rms Std dev Minimum Maximum <br />
-4.352392e-05 2.133304e-03 2.132913e-03 -7.120122e-03 5.782832e-03 <br />
region count <br />
1 <br />
<br />
<br />
(TWHya.continuum.1pcal.image)<br />
Stokes Velocity Frame Doppler Frequency <br />
I -10.5133km/s LSRK RADIO 3.65287e+11 <br />
BrightnessUnit BeamArea Npts Sum FluxDensity <br />
Jy/beam 25.4701 20100 -4.660212e-02 -1.829681e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
-2.318513e-06 4.291876e-04 4.291920e-04 -1.593901e-03 1.666186e-03 <br />
region count <br />
1 <br />
</pre><br />
<br />
The self-calibrated image has improved the rms noise by a factor of 5.<br />
<br />
=== Second cycle of self-calibration: phase only ===<br />
Now that we have a better model, we can try to determine phase solutions again using a second round of phase-only self calibration. We will write the gain solutions to a new table, but these solutions will replace the solutions we determined in the first round. It may be useful to experiment with the solution interval. Keep in mind that the integration time is 6.05 seconds for this observation. (The integration time we refer to here is the "dump time" of the correlator, not the cumulative on-source time.)<br />
<br />
In this example, we will stick with the 30 second solution interval.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf self_2*')<br />
gaincal(vis='TWHya_cont.ms',caltable='self_2.pcal',<br />
solint='30s',combine='',gaintype='T',<br />
refant='DV22',spw='',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Now plot the new solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='self_2.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',iteration='antenna',<br />
subplot=421,plotrange=[0,0,-80,80],figfile='self_2_phase.png')<br />
</source><br />
<br />
The solutions look much the same as those after the first round of phase calibration, as expected, since the corrections from the (slightly better) new model are minor in this case. We can apply the new phase solutions and make a new image to see if there is improvement. <br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='TWHya_cont.ms',field='',gaintable=['self_2.pcal'],calwt=False)<br />
</source><br />
<br />
Note, here, that {{applycal}} applies the gain solutions to the raw data and overwrites any results in the "corrected data" column of the measurement set, so this calibration replaces what we applied after the first-round solutions.<br />
<br />
Now clean again, updating '''mask''' with the most recent mask file.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHya.continuum.2pcal.*')<br />
clean(vis='TWHya_cont.ms',imagename='TWHya.continuum.2pcal',<br />
mode='mfs',imagermode='csclean',<br />
imsize=300,cell=['0.1arcsec'],spw='',<br />
weighting='briggs',robust=0.5,usescratch=False,<br />
mask='TWHya.continuum.1pcal.mask',<br />
interactive=True,threshold='1mJy',niter=10000)<br />
</source><br />
<br />
Clean again, interactively. In general, you should pay attention to the total flux being cleaned after each round, and the residuals inside the clean box as compared to those outside. When the cleaned flux in the latest round is small and the residuals inside the clean box match those outside it, the image has been sufficiently cleaned. Stop by hitting the red X on the GUI. In this case, let's again clean for 1200 iterations. The task reports a total cleaned flux and residuals similar to what we saw after the first round of cleaning:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Model 0: max, min residuals = 0.00105882, -0.00017499 clean flux 1.78282<br />
</pre><br />
<br />
Now repeat the display, blinking, and image statistics steps described above for the 1st and 2nd phase-only self-cal iterations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file':'TWHya.continuum.1pcal.image','range':[-0.01,0.05]},<br />
{'file':'TWHya.continuum.2pcal.image','range':[-0.01,0.05]}])<br />
</source><br />
<br />
The second round made an improvement, but the difference is small:<br />
<pre style="background-color: #fffacd;"><br />
<br />
(TWHya.continuum.1pcal.image)<br />
Stokes Velocity Frame Doppler Frequency <br />
I -10.5133km/s LSRK RADIO 3.65287e+11 <br />
BrightnessUnit BeamArea Npts Sum FluxDensity <br />
Jy/beam 25.4696 21576 -7.237051e-02 -2.841450e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
-3.354213e-06 4.405892e-04 4.405866e-04 -1.534719e-03 1.791427e-03 <br />
region count <br />
1 <br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
(TWHya.continuum.2pcal.image)<br />
Stokes Velocity Frame Doppler Frequency <br />
I -10.5133km/s LSRK RADIO 3.65287e+11 <br />
BrightnessUnit BeamArea Npts Sum FluxDensity <br />
Jy/beam 25.3159 21576 -6.708313e-02 -2.649847e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
-3.109155e-06 4.225414e-04 4.225398e-04 -1.471975e-03 1.751540e-03 <br />
region count <br />
1 <br />
</pre><br />
<br />
This small improvement suggests that we have reached an end point with phase-only self-cal.<br />
<br />
=== Third cycle of self-calibration: amplitude & phase ===<br />
<br />
Next we will add amplitude self-calibration to the process. Amplitude changes more slowly than phase, so we will determine one solution per scan. We can set '''solint = 'inf'''', which tells {{gaincal}} to use the "combine" parameter to determine how to set the solution interval. With "combine" set to the default ('''combine = ''''') we will get one solution per scan.<br />
<br />
Also, note that we set "gaintable" to 'self_2.pcal'. This tells {{gaincal}} to apply the former phase-only solutions prior to determining the new amplitude+phase solutions. So, the new solutions will be incremental. Later when we apply these solutions to the data sets, we must be sure to apply both the phase-only solutions and the amplitude+phase solutions.<br />
<br />
[[Image:C0Guide_self_ap_phase.png|thumb|<caption>'''Fig. 6.''' Incremental phase solutions after the third round of self-calibration.</caption>]]<br />
<br />
<br />
<br />
[[Image:C0Guide_self_ap_amp.png|thumb|<caption>'''Fig. 7.''' Amplitude solutions after the third round of self-calibration.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='TWHya_cont.ms',caltable='self_ap.cal',<br />
solint='inf',combine='',gaintype='T',<br />
refant='DV22',spw='',minblperant=4,<br />
gaintable=['self_2.pcal'],<br />
calmode='ap',minsnr=2)<br />
<br />
plotcal(caltable='self_ap.cal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='',iteration='antenna',<br />
subplot=421,plotrange=[0,0,-10,10],figfile='self_ap_phase.png')<br />
<br />
plotcal(caltable='self_ap.cal',xaxis='time',yaxis='amp',<br />
spw='',field='',antenna='',iteration='antenna',<br />
subplot=421,plotrange=[0,0,0.5,1.5],figfile='self_ap_amp.png')<br />
</source><br />
<br />
Use the "next" button on the {{plotcal}} GUI to advance the plot to see additional antennas. Since phase corrections have already been applied, we see that the new incremental phase solutions are fairly small. The amplitude corrections are at the level of ~10% in many cases, but there is considerable variance among different antennas. See Figures 6 and 7 for the phase and amplitude solutions.<br />
<br />
We can apply the calibration now with {{applycal}}. Note we apply two gain tables.<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='TWHya_cont.ms',field='',gaintable=['self_2.pcal','self_ap.cal'],calwt=False)<br />
</source><br />
<br />
<br />
[[Image:C0Guide_selfcal_time.png|thumb|<caption>'''Fig. 8.''' Amplitudes as a function of time. We see that one of the scans has anomolously high amplitudes and should be flagged.</caption>]]<br />
<br />
<br />
[[Image:C0Guide_selfcal_uvdist.png|thumb|<caption>'''Fig. 9.''' Amplitudes as a function of u,v distance. The many high amplitudes come from scan 72.</caption>]]<br />
<br />
<br />
Let's plot the u,v data against both time and u,v distance. It is a good idea to check these after an amplitude self-calbration to be sure that it worked properly. Occasionally, a few spurious bits of data will get wildly incorrect values by the amplitude self-cal. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHya_cont.ms',xaxis='time',yaxis='amp',<br />
avgchannel='38',ydatacolumn='corrected',coloraxis='spw',<br />
plotfile='selfcal_time.png')<br />
<br />
# In CASA<br />
plotms(vis='TWHya_cont.ms',xaxis='uvdist',yaxis='amp',<br />
avgchannel='38',ydatacolumn='corrected',coloraxis='spw',<br />
plotfile='selfcal_uvdist.png')<br />
</source><br />
<br />
Figures 8 and 9 show results from the previous two commands.<br />
<br />
Using the "locate" tool in the {{plotms}} GUI, select a few bad data points and confirm the scan number. We can see that bad data points exist for, at least, spw 0 and 1. To be conservative, we can flag all of scan 72. It's not much data compared to the full observation.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='TWHya_cont.ms', mode='manual', scan='72')<br />
</source><br />
<br />
Replot the u,v data to confirm the bad data have been removed. IMPORTANT: when re-plotting this data set in {{plotms}}, you must remember to select the "force reload" button on the GUI. Otherwise, the plot will not update with the new flags.<br />
<br />
If the plots look good, you can proceed to make the final continuum image.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHya.continuum.apcal.*')<br />
clean(vis='TWHya_cont.ms',imagename='TWHya.continuum.apcal',<br />
mode='mfs',imagermode='csclean',<br />
imsize=300,cell=['0.1arcsec'],spw='',<br />
weighting='briggs',robust=0.5,usescratch=False,<br />
interactive=True,threshold='0.1mJy',niter=5000,npercycle=1000)<br />
</source><br />
<br />
<br />
[[Image:C0Guide_cont_clean_residuals.png|thumb|<caption>'''Fig. 10.''' Here we show the residuals plotted in the viewer during the cleaning process. We decided to stop cleaning when the residuals reached the level shown.</caption>]]<br />
<br />
<br />
You will need to execute over a thousand iterations in total to get the image fully cleaned, so we start the clean process with 1000 iterations and press the green circle. In the GUI, you should then adjust the number of iterations per cycle down to 100 for additional cleaning. Update the clean box if necessary, by expanding the region with a larger ellipse. Note that the pattern of residuals inside the clean box may not come to match that outside it, but you want to clean until the magnitude of the residuals is similar, inside it and out. In this case, the cleaning takes about 1500 iterations.<br />
<br />
The {{clean}} task reports a total cleaned flux of 1.76 Jy.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file':'TWHya.continuum.2pcal.image','range':[-0.01,0.05]},<br />
{'file':'TWHya.continuum.apcal.image','range':[-0.01,0.05]}])<br />
</source><br />
<br />
<pre><br />
(TWHya.continuum.2pcal.image)<br />
Stokes Velocity Frame Doppler Frequency <br />
I -10.5133km/s LSRK RADIO 3.65287e+11 <br />
BrightnessUnit BeamArea Npts Sum FluxDensity <br />
Jy/beam 25.3159 20553 1.654666e-01 6.536086e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
8.050727e-06 4.115417e-04 4.114729e-04 -1.471975e-03 1.612787e-03 <br />
region count <br />
1 <br />
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----<br />
(TWHya.continuum.apcal.image)<br />
Stokes Velocity Frame Doppler Frequency <br />
I -10.5133km/s LSRK RADIO 3.65287e+11 <br />
BrightnessUnit BeamArea Npts Sum FluxDensity <br />
Jy/beam 25.3653 20553 3.826559e-02 1.508581e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
1.861801e-06 2.410208e-04 2.410195e-04 -1.023332e-03 8.470871e-04 <br />
region count <br />
1 <br />
</pre><br />
<br />
We see that the amplitude+phase self-cal made a significant improvement in the rms of the image, compared to the phase-only self-cal from 0.41 mJy to 0.24 mJy.<br />
<br />
=== Apply Calibration to the Full Data Set and Split the Line Data ===<br />
<br />
Now we apply the self-calibration derived from the continuum emission to the full-resolution spectral cube.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='calibrated.ms',gaintable=['self_2.pcal','self_ap.cal'],calwt=False)<br />
</source><br />
<br />
Remember to flag the bad scan, 72.<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='calibrated.ms',scan='72')<br />
</source><br />
<br />
Next we can split off the spectral line data for further processing.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_H2D+.ms*')<br />
split(vis='calibrated.ms',outputvis='TWHydra_H2D+.ms',datacolumn='corrected',spw='0',field='TW Hya')<br />
<br />
os.system('rm -rf TWHydra_N2H+.ms*')<br />
split(vis='calibrated.ms',outputvis='TWHydra_N2H+.ms',datacolumn='corrected',spw='1',field='TW Hya')<br />
</source><br />
<br />
=== Subtracting the Continuum ===<br />
<br />
The next step in the processing is to subtract the continuum from the full data set, in preparation for imaging the line emission. We'll do the subtraction in the u,v plane. We determine the continuum from line-free channels, so we begin by plotting the u,v amplitudes using '''plotms''', to identify channels with the N2H+ line.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_N2H+.ms',xaxis='channel',yaxis='amp',avgtime='1e8',avgscan=True)<br />
</source><br />
<br />
<br />
[[Image:C0Guide_N2H+_amp.png|thumb|<caption>'''Fig. 11.''' Here we show the u,v amplitudes from {{plotms}} for the spectral window containing the N2H+ line, which is detected near channel 2440.</caption>]]<br />
<br />
In this data set, the line emission is evident directly in the u,v amplitudes. Although the line is not really strong enough to have much impact on continuum subtraction, we will avoid the N2H+ line anyway (good practice) and fit the continuum to the spectrum using line-free channels. In fitting for the continuum we will also avoid a few channels at the band edges, where there can sometimes be erratic data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='TWHydra_N2H+.ms',fitorder=1,fitspw='0:20~2400,0:2500~3800')<br />
</source><br />
<br />
It is useful at this stage to determine the velocity range of the detected emission. We'll use this later when we image the line emission. Using the {{plotms}} GUI, set the X axis to "Velocity" and then select the vertical "Trans" tab. Here, set the Frame to "LSRK", the velocity defn to "RADIO", and the Rest Freq to 372672.49 MHz. Then re-plot the data by clicking the "Plot" button. Notice that the emission falls between ~ 2-4 km/s, which is consistent with the known recession velocity of TW Hydra, about 2.9 km/s.<br />
<br />
=== Imaging N2H+ ===<br />
<br />
The final step is to image the N2H+ line emission using the {clean}} task. <br />
<br />
Many of the imaging parameters are the same as we used for the continuum, but now we have a few new parameters to set for the spectral cube: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe'''. When making spectral images you have three choices for '''mode''': "channel", "velocity", or "frequency". The data are recorded into channels spaced by a constant frequency increment, but for spectral line analysis it's often more useful to have a constant velocity increment. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units. <br />
<br />
ALMA does not do on-line Doppler Tracking and the native frame of the data is topocentric (TOPO). If you do not specify '''outframe''', the output cube will also be in TOPO, which is not very useful for spectral line work. By specifying the '''outframe''' parameter, {{clean}} will make the doppler shift correction. Alternatively it can be done prior to imaging with the {{cvel}} task. <br />
<br />
In our example here, we will image the N2H+ line in the range 0-4 km/s using steps of 0.1 km/s. Our choice of velocity increment is determined by the channel spacing. From the {{listobs}} output, we know the channel spacing is 61 kHz, which corresponds to ~ 0.05 km/s at 372 GHz. The true velocity resolution is worse than the channel spacing by about a factor of 2. So, 0.1 km/s steps in velocity is a good choice.<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf line.N2H+.*')<br />
clean(vis = 'TWHydra_N2H+.ms.contsub', imagename = 'line.N2H+', mode = 'velocity', nchan = 40, start = '1.0km/s',<br />
width = '0.1km/s', outframe = 'LSRK', restfreq = '372.67249GHz', imagermode = 'csclean',<br />
interactive=True, niter=1000, imsize = [300, 300], cell = '0.1arcsec', weighting = 'briggs', robust = 0.0)<br />
</source><br />
<br />
Imaging spectral data cubes with complex line emission can be an arduous task. In this case, we use the clean viewer to step through each spectral channel and define a clean mask for each channel individually. This is usually the best approach when the structure is complex and changing from one spectral channel to the next, as it is for the N2H+ line in TW Hydra. For simpler structures, it can be sufficient to define a single clean mask and apply it for all channels.<br />
<br />
After the viewer GUI appears, you may wish to maximize the size of the "Display" panel by deselecting the "Cursors" panel, if necessary. Then step through channels with the tape deck controls, and create a clean mask for each channel with detected line emission, as you did for the continuum cleaning. In this case, make sure that the radio button at the top of the GUI has "This Channel" selected, to create a separate mask for each spectral channel.<br />
<br />
Click the green arrow to clean with 100 iterations, then click the "play" button on the tape deck to see the residuals, You will see significant residuals remaining in each channel. Continue cleaning until the residuals inside the clean boxes match those outside. If you reach 5000 iterations, the {{clean}} task will end itself (since we specified niter=5000). If you'd like to stop sooner, click the red "X" button.<br />
<br />
That's it! You've got a data cube with N2H+ ready for analysis.<br />
<br />
Apart from imaging this targeted channel range, it is also important to image the full data cube to explore the full spectral bandwidth observed. A full imaging could be accomplished as follows:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf line.N2H+.*')<br />
clean(vis = 'TWHydra_N2H+.ms.contsub', imagename = 'line.N2H+', mode = 'channel', start=0, width=10, imagermode = 'csclean',<br />
interactive=False, niter=0, imsize = [300, 300], cell = '0.1arcsec', weighting = 'briggs', robust = 0.0)<br />
</source><br />
<br />
=== What About the H2D+? ===<br />
<br />
Recall the first spectral window (spw='0') in the '''calibrated.ms''' data set contains the anticipated H2D+ line emission. Imaging of that line can proceed in much the same manner as we have done for the N2D+ line. In this case, though, there is no obvious detection of the H2D+ line. So we will not repeat the steps in this guide.<br />
<br />
== Image Analysis ==<br />
<br />
<br />
[[Image:C0Guide_N2H+_1chan.png|thumb|<caption>'''Fig. 12''' Here we show one spectral channel in the final N2H+ image. The faint purple line marks an elliptical region that we use to determine an integrated spectrum, shown in the next figure.</caption>]]<br />
<br />
<br />
The CASA viewer offers some powerful capabilities for exploring spectral data cubes. We are currently preparing CASA Guides that discuss '''Using the Viewer to Extract Spectra''', and '''Making Moment Maps'''. Check back later!<br />
<br />
Here, we will demonstrate how to make moment maps from the N2H+ data cube. First examine the final image in the viewer:<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer('line.N2H+.image')<br />
</source><br />
<br />
You should see a result as shown in Figure 12.<br />
<br />
<br />
[[Image:C0Guide_N2H+_profile.png|thumb|<caption>'''Fig. 13.''' This figure shows a spectrum of N2H+ determined by integrating the flux in each channel over the area marked by the purple ellipse in the previous figure.</caption>]]<br />
<br />
<br />
Use the ellipse drawing tool to define an elliptical region that encompasses all the line emission, from all channels, and then select "Spectral Profile" from the "Tools" menu to generate a spectrum, as shown in Figure 13 to the right. We are showing the X axis in units of velocity. If we select "channels" from the Spectral Profile GUI, we can see that the line emission extends from channels 11-26.<br />
<br />
Now make the Moment 0 map:<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf line.N2H+.image.mom0')<br />
immoments(imagename='line.N2H+.image',moments=[0], outfile='line.N2H+.image.mom0', chans='11~26')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf ')<br />
os.system('rm -rf ')<br />
immoments(imagename='line.N2H+.image',moments=[1,2],<br />
outfile='line.N2H+.image.mom',<br />
chans='11~26',includepix=[0.1,100])<br />
</source><br />
<br />
Display all three moment maps in the viewer, overlaid with continuum contours.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview( raster=[ {'file':'line.N2H+.image.mom0'},<br />
{'file':'line.N2H+.image.mom.weighted_coord'},<br />
{'file':'line.N2H+.image.mom.weighted_dispersion_coord'} ], <br />
contour={'file':'TWHya.continuum.apcal.image', 'base':0, 'unit':0.0025, 'levels':[3,100]} )<br />
</source><br />
<br />
<br />
[[Image:C0Guide_3moments.png|thumb|<caption>'''Fig. 14.''' The first three moment images from the N2H+ data cube.</caption>]]<br />
<br />
<br />
To see all three raster images simultaneously, open the viewer's Panel Display Options (the "P-wrench" icon in the viewer tool bar), click "Number of panels", and change "Number of panels in x" to 3.</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=Solar_System_Models_in_CASA_4.0&diff=30882Solar System Models in CASA 4.02021-08-19T21:09:14Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]]<br />
<br />
== Overview ==<br />
<br />
The detailed radio wavelength spectral energy distributions, including time variability, and contributions from molecular line emission of Solar System Objects (SS Objects) is an active field of astronomical research. In preparation for ALMA commissioning and Early Science an initial set of Solar System Object models were compiled called Butler-JPL-Horizons 2010. These models represented what was known at that time about the millimeter to submm behavior of the most useful SS Objects. All Science Verification and Cycle 0 data were calibrated with the Butler-Horizons-2010 models and CASA 3.3 or 3.4. With the wealth of new information now available from telescopes like Herschel, along with more detailed models of some of the more prevalent atmospheric spectral lines in SS Objects, the models were updated for the CASA 4.0 release. The new models are called Butler-JPL- Horizons 2012 and represent the current best understanding from published or publicly available work. A memo describing the details of the models and how they work in CASA can be found in the ALMA Memo Series at: https://science.nrao.edu/facilities/alma/aboutALMA/Technology/ALMA_Memo_Series/alma594/abs594. Experts from around the world were consulted in the compilation of these models and they represent the current state-of-the-art. It is anticipated that for future releases of CASA we will continue to incorporate improvements to the models as the field continues to progress. <br />
<br />
'''The table below shows the fractional difference ([2012/2010 -1]) in the "zero spacing" flux density between the 2012 and 2010 models derived for March 1, 2013 at four fiducial "continuum" frequencies''', i.e. ones that should not be contaminated by significant line emission. The purpose of the table is to give an overall sense of the magnitude of the differences in the two models. The actual values of the models on any give date will change. For most objects this is (so far) only due to the change in apparent size of the SS Object as it moves nearer or further from us over time. Additionally, the model for Mars attempts to account for seasonal changes in the surface conditions and hence albedo of Mars over time as well. The way Vesta is calculated in the 2012 model also makes its correction compared to the 2010 model time-variable.<br />
<br />
{| class="wikitable" <br />
! align="Center"| Object<br />
! 103 GHz<br />
! 241 GHz<br />
! 349 GHz<br />
! 681 GHz<br />
|-<br />
|Venus || -0.01 || -0.01 || +0.00 || +0.38<br />
|-<br />
|Mars || +0.00 || +0.06 || +0.08 || +0.12<br />
|-<br />
|Ceres || +0.11 || +0.12 || +0.13 || +0.13<br />
|-<br />
|Jupiter || -0.05 || -0.02 || -0.03 || -0.13<br />
|-<br />
|Ganymede || -0.22 || -0.14 || -0.10 || -0.02<br />
|-<br />
|Callisto || -0.21 || -0.15 || -0.12 || -0.07<br />
|-<br />
|Pallas || +0.16 || +0.16 || +0.16 || +0.17<br />
|-<br />
|Neptune || -0.05 || -0.04 || -0.11 || +0.13<br />
|-<br />
|Titan || +0.07 || -0.01 || -0.01 || -0.01<br />
|-<br />
|Uranus || +0.02 || +0.00 || +0.03 || -0.11<br />
|-<br />
|Vesta || +0.13 || +0.13 || +0.13 || +0.13<br />
|-<br />
|}<br />
<br />
From this table it is clear that the differences are typically a few to 20%, with a significant outlier for Venus at Band 9 (38%). Whether the change is important for a specific dataset is highly dependent on individual science goals and which SS Object was used. Below we describe how you can find the values for specific observing dates, and how you can correct your data if you so chose.<br />
<br />
== Determine SS Object Flux Density on Specific Date ==<br />
<br />
[[File:Callisto2012_103GHz_2013-03-01-12-00-00.png|thumb|<caption>Sample plot of the 2012 model for Callisto at 241 GHz. The zero spacing flux of 6.233 Jy is given in a box in the top right corner of the plot.</caption>]]<br />
<br />
<br />
To determine the zero spacing flux density on a given date, one can use the {{predictcomp}} CASA task. It is essential that a valid time is also given and that you use a frequency that is representative of the data you are interested in. The task {{listobs}} can be used to determine an exact date, time, and representative frequency. A valid ALMA configuration file must be used, but the zero spacing flux does not depend on this choice. For example, to get the value for Callisto on March 1, 2013 for a frequency of 241 GHz, you can use<br />
<br />
<source lang="python"><br />
# In CASA 4.0 or later<br />
predictcomp(objname='Callisto',<br />
standard='Butler-JPL-Horizons 2010' ,<br />
epoch='2013-03-01-00:00:00',<br />
minfreq='241GHz',<br />
antennalist='alma_cycle1_1.cfg',<br />
savefig='Callisto2010_241GHz_2013-03-01-00-00-00.png',<br />
showbl0flux=True,include0amp=True,include0bl=True)<br />
<br />
</source><br />
<br />
The plot produced is shown in Figure 1. The zero spacing flux bl0= 6.233 Jy is given in the text box in the top right corner of the plot. If you repeat with 'Butler-JPL-Horizons 2010', you find bl0=7.332 Jy, for a percent difference of -15%, as also given in the Table above. Because Callisto does not have a time dependent model beyond that induced by its apparent size, the percent difference will be constant in time. Only Mars and Vesta will show changes in the percent difference in time.<br />
<br />
'''Caveat 1:''' For SS Objects with atmospheres it is important to avoid picking frequencies for evaluation that are coincident with strong emission lines like CO (Mars, Neptune, Titan). Some of the new 2012 models contain estimates for these lines (see the Memo for details), while the 2010 models do not. Since strong lines were flagged for the SS Objects in the Cycle 0 data reduction, a comparison made on such a strong line will be skewed.<br />
<br />
== Correcting Your Images ==<br />
<br />
For this technique you need to have a Cycle 0 image that is either only composed of one execution's worth of data, or all executions used the same SS Object (or were bootstrapped from a single SS Object). The delivered data reduction scripts will be helpful in figuring out what was done, what if any SS Objects were used. An easy way to correct such an image is to simply multiply by the fractional difference between the two models +1. For example, if the SS Object is Callisto we see from the table that the 2012 model flux density is 15% '''smaller''' than the 2010 model would have predicted, so the correction factor is =-0.15+1. The CASA task {{immath}} can be used to do the multiplication:<br />
<br />
<source lang="python"><br />
# In CASA 4.0 or later<br />
immath(imagename='ScienceTarget.image',mode='evalexpr',<br />
expr='0.85*IM0',<br />
outfile='ScienceTarget.image.BJH12')<br />
<br />
</source><br />
<br />
Note that both the signal and rms noise will be correctly adjusted. Datasets comprised of multiple executions that used multiple different SS Objects for absolute flux calibration cannot be so easily corrected, and it will be necessary to redo the calibration using the 2012 models for each SS Object. Alternatively, you can estimate the effect by taking a weighted average of the fractional differences needed for each dataset and apply this average. For example, if you have 4 executions, half with Callisto and half with Titan (which has almost no difference between the two models away from line emission), you could apply a correction that is 1/2 the Callisto difference.<br />
<br />
== Correcting Your Data ==<br />
<br />
You can also correct the data themselves:<br />
<br />
* If the data reduction scripts were created for CASA 3.4, then the line "if re.search('^3.4', casadef.casa_version) == None:" will appear near the top of the script. If this is the case, then you simply need to make sure you have installed CASA 4.0.1, change the '''{{setjy}} command in Step 9''' from standard = 'Butler-JPL-Horizons 2010' to standard = 'Butler-JPL-Horizons 2012', and re-run the scripts starting with the "raw" data. <br />
<br />
* If the scripts were created in CASA 3.3, they will need to be made compatible with CASA 3.4 before you can run them in the required version: CASA 4.0.1. See sample TDM and FDM CASA 3.4 scripts at http://casaguides.nrao.edu/index.php?title=ALMAguides<br />
<br />
* If you also have included in your data delivery package a script called scriptForFluxCalibration.py you may want to re-evaluate whether you want to do this equalization based on the outcome of the initial re-reduction -- some of the previous variation may have come from inconsistencies between the 2010 SS models. If you do, you will want to adjust the calibrator flux density values in the {{setjy}} steps to reflect the new average values that come from having used the 2012 models in the calibration scripts before re-running this script. To do this, it is helpful to look at the files with names ending in *.fluxscale that are produced during the calibration process which records the output of the {{fluxscale}} bootstrapping. You can find the values from the original reduction in the file called qa/*.textfile.txt in your data delivery under the heading of "Flux Density Determinations".<br />
<br />
* Once all the recalibration is done you can follow the imaging script to produce new images. <br />
<br />
If you have difficulties understanding just what to do, feel free to contact your ARC through the ALMA helpdesk (https://almascience.org).</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=DataWeightsAndCombination&diff=30881DataWeightsAndCombination2021-08-19T21:07:22Z<p>Jbertone: </p>
<hr />
<div>The purpose of this guide is to describe the CASA visibility weights, and how they relate to data combination. If you are trying to combine ALMA data from Cycles 0, 1, and/or 2, you should be aware of the issues described here. Only data calibrated in >=CASA 4.2.2 pipeline, or manual reduction in >=CASA 4.3 are free from any issues. All of the data to be combined must be in a consistent weight state. <br />
<br />
== Principles of Data Weighting ==<br />
<br />
==== Visibility Weights ====<br />
<br />
For optimal imaging performance it is critical that in a ''relative'' sense each visibility in the data have the correct weight after calibration -- that is, data with better sensitivity have more weight than data with less sensitivity. Formally, the post-calibration visibility weights should be equal to 1/&sigma;<sub>ij</sub><sup>2</sup> where &sigma;<sub>ij</sub> is the rms noise of a given visibility.<br />
<br />
<math> <br />
\sigma_{ij} (Jy) =\frac{2k}{\eta_{q}\eta_{c}A_{eff}} <br />
</math><br />
<math> <br />
\sqrt{\frac{T_{sys,i} T_{sys,j}}{2\Delta\nu_{ch} t_{ij}}}<br />
</math><br />
<math> <br />
\times 10^{26},<br />
</math><br />
<br />
where:<br />
<br />
''k'' is Boltzmann's constant. <br />
<br />
''A<sub>eff</sub>'' is the effective antenna area which is equal to the aperture efficiency (''&eta;<sub>a</sub>'') x the geometric area of the antenna (&pi; r<sup>2</sup>). The aperture efficiency depends on the rms antenna surface accuracy and is about 0.75 for ALMA dishes. <br />
<br />
''&eta;<sub>q</sub>'' and ''&eta;<sub>c</sub>'' are the quantization and correlator efficiencies, respectively. These have values of about 0.88 and 0.96, respectively. See the ALMA Technical Handbook for more information.<br />
<br />
''T<sub>sys,i</sub>'' is the system temperature for antenna i, and ''T<sub>sys,j</sub>'' is the system temperature for antenna j<br />
<br />
''&Delta;&nu;<sub>ch</sub>'' is the effective channel frequency width.<br />
<br />
''t<sub>ij</sub>'' is the integration time per visibility. <br />
<br />
Note, this equation can be extended from a single visibility to the expected imaging rms noise for one channel in an entire dataset by replacing ''T<sub>sys,i</sub>T<sub>sys,j</sub>'' by the average ''T<sub>sys</sub><sup>2</sup>'', ''t<sub>ij</sub>'' by the total time on source, and the <math>\sqrt{2}</math> in the denominator by <math>\sqrt{N(N-1)}</math> , where ''N'' is the number of antennas. Additionally, ''<math>\sqrt{n_p}</math>'' is included in the denominator, where ''n<sub>p</sub>'' is the number of polarizations (1 for single pol and 2 for dual or full pol data):<br />
<br />
<math> <br />
\sigma_{image} (Jy) =\frac{2k}{\eta_{q}\eta_{c}A_{eff}} <br />
</math><br />
<math> <br />
\sqrt{\frac{T_{sys}^{2}}{N(N-1)\Delta\nu_{ch} t_{source}n_{p}}}<br />
</math><br />
<math> <br />
\times 10^{26},<br />
</math><br />
<br />
'''In order to correctly combine and image data that have different ''T<sub>sys</sub>'', ''&Delta;&nu;<sub>ch</sub>'', ''t<sub>ij</sub>'', or antenna size it is essential to use visibility weights proportional to 1/&sigma;<sub>ij</sub><sup>2</sup>. The remainder of this guide deals with ensuring that the individual visibility weights are correct in your ALMA data.'''<br />
<br />
==== Relative Overall Weights between Configurations ====<br />
<br />
Additionally, when combining data from different antenna configurations, one will get optimal overall sensitivity to all spatial scales by matching the surface brightness sensitivity at each uv-distance. This can only be achieved by having time-on-source per configuration in the right proportion. This topic is covered in [http://library.nrao.edu/public/memos/alma/memo598.pdf ALMA Memo 598]. This memo informs the relative amount of time that ALMA observes a project with the 7m-array versus the 12m-array, and compact versus extended 12m-array configurations. However, since telescope time is expensive, one typically does not actually observe in the optimal proportion, in that case one will not fully realize the expected "impact" of adding the less sensitive configuration data. In that case, one may chose to "up-weight" the less sensitive configuration explicitly by changing its data weights, above and beyond 1/&sigma;<sub>ij</sub><sup>2</sup>. However, it should be noted that nothing comes for free, and such a change will come at the expense of overall image sensitivity though it may very well be the optimal choice for your science case -- for example, if you are particularly interested in large scale structures you might apply extra up-weighting to the dataset(s) with the shortest baselines. Finding the optimal up-weighting is a matter of experimentation, but can easily be explored using the visweightscale parameter in {{concat}}. As a general rule of thumb extra up-weighting by more than a factor of 4 or so is not recommended. Before experimenting with this however, its always best to start with data that simply has the correct 1/&sigma;<sub>ij</sub><sup>2</sup> weights. Additionally, some degree of relative re-weighting can also be accomplished simply by using the Briggs Robust parameter, without explicitly changing the visibility weights, and generally it is also a good idea to try this first before more aggressive up-weighting is attempted.<br />
<br />
== Weights in CASA ==<br />
<br />
A memo describing weights in CASA, in particular the significant changes that were made with CASA 4.2.2, can be found at [http://casa.nrao.edu/Memos/CASA-data-weights.pdf CASA-data-weights.pdf]<br />
<br />
To summarize the situation specifically for ALMA data processed by the ALMA Project:<br />
<br />
:'''CASE 1) CASA 4.2.1 and earlier''': Weights were calculated on a per spw bases and scaled by ''N<sub>chan</sub>/[(Tsys(i) * Tsys(j)]'' using calwt=True at the applycal stage for Tsys table. Assuming that (i) there aren't any unflagged antennas with significantly low gain due to for example pointing errors, or hardware problems, and (ii) all the antennas have similar efficiencies -- usually good assumptions for ALMA data, then to good approximation the data weights are close to internally consistent and can produce good imaging results, but should not be combined with other data that have different &Delta;&nu;<sub>ch</sub>, t<sub>ij</sub>, or antenna size without further modification. <br />
:'''CASE 2) CASA 4.2.2 and later''': Upon import data weights are scaled by 2&Delta;&nu;<sub>ch</sub>&Delta;t<sub>ij</sub> and then also scaled by ''1/[(Tsys(i) * Tsys(j)]'' using calwt=True at the applycal stage for Tsys table (thus making them per channel weights). Additionally:<br />
::'''(A) For data calibrated by the 4.2.2 CASA Pipeline''' the weights are further modified by [gain(i)<sup>2</sup> * gain(j)<sup>2</sup>] when the amplitude gain table is applied using calwt=True. Since the amplitude gains are directly proportional to the effective Antenna area, scaling the weights by the amplitude gains will take into account antenna size differences, and also down-weight antennas with comparatively low gain. Thus, these weights are correct (but see [https://casaguides.nrao.edu/index.php/DataWeightsAndCombination#Absolute_Accuracy_of_the_Data_Weights Absolute Accuracy of the Data Weights]).<br />
::'''(B) For data manually calibrated in CASA 4.2.2''', unfortunately calwt=False was still used to apply the antenna gain table, thus, these data have weights that are not correct in a relative sense when compared to other data with different antenna size by the factor [gain(i)<sup>2</sup> * gain(j)<sup>2</sup>].<br />
:'''CASE 3) CASA 4.3 and later:''' Data calibrated in either the pipeline or manually will have correct weights (but see [https://casaguides.nrao.edu/index.php/DataWeightsAndCombination#Absolute_Accuracy_of_the_Data_Weights Absolute Accuracy of the Data Weights]). An example of this situation is demonstrated in https://casaguides.nrao.edu/index.php/M100_Band3_Combine_4.3<br />
<br />
== How Do I Know the Situation For My ALMA Data?==<br />
<br />
First, remember that good imaging performance requires that the '''relative weights''' be correct for all data included in the imaging. For all of the situations below, the data are sufficiently internally consistent that they can be imaged fine on their own or with other data that share the same situation and have the same data properties. Problems arise when data are combined that (a) do not share the same weight situation and/or differences in antenna size, visibility integration time, or channel bandwidth are not accounted for. <br />
<br />
'''All datasets to be combined should be independently assessed to determine their weight situation and thus whether corrective action is required'''<br />
<br />
* '''Cycle 0''' and '''Cycle 1''' data were reduced in 4.2.1 or earlier versions and correspond to Case 1 above. ACA 7m-array data were first offered in Cycle 1. If you want to combine 12m-array and 7m-array data taken during Cycle 1, it is very likely you need to correct the visibility weights before imaging.<br />
<br />
* '''Cycle 2''' data has a more confused range of possibilities (this includes actual Cycle 2 projects and carry-over projects or parts of projects from Cycle 1). Recall from Case 2 above, that the key factor that determines your situation for Cycle 2 is whether the data were pipeline or manually calibrated. <br />
** Key dates:<br />
*** Start Cycle 2: June 1, 2014<br />
*** CASA 4.2.2 release date: Sept. 4, 2014<br />
*** Pipeline release date: Oct. 20, 2014<br />
** The earlier in Cycle 2 your data was taken, the more likely it was manually calibrated, also if you have any very narrow spws, high frequency, etc it is more likely data were manually calibrated.<br />
** If the data were manually calibrated early in Cycle 2 it is likely in Case 1 (certainly the case for data earlier than Sept. 2014). After Oct. 1, 2014 it is likely to be in Case 2B. <br />
** How can I distinguish between pipeline and manual data reduction for Cycle 2 data?<br />
*** In your data delivery, the README file should say but there were some oversights on this front. A sure way to tell: look in the directory called '''script''' within your data delivery for a particular member_ouss. If you see a file with '''PPR*.xml''', the data was calibrated by the pipeline. Otherwise it was done manually.<br />
<br />
== What Are the Options for Adjusting the Weights for Older Reductions?==<br />
<br />
If the data weights are not correct in ALL the datasets you want to combine there are three options to correct the situation. These different methods carry different levels of pain/complexity depending on your situation. The situation can also be extra confusing if your data fall into multiple categories above. For example, it is not uncommon that in Cycle 2 the 12m-array data could be pipeline calibrated, but the 7m-array data done manually. Pay close attention to recommendations in <span style="color:#B22222">Red</span> and the Caveats for each option.<br />
<br />
====Option 1: Re-calibrate your data in CASA 4.2.2 or later==== <br />
<br />
:<span style="color:#B22222"> &rArr; Option 1 is easiest to implement for Case 2B (Data observed in Cycle 2 manually calibrated in 4.2.2 or 4.3.1 with calwt=False for amplitude/flux gain table). </span><br />
<br />
* All data imported in 4.2.2 (or later) will automatically adjust the weights by 2&Delta;&nu;<sub>ch</sub>&Delta;t<sub>ij</sub>. The Tsys application should also already be correct in your scripts. To correct for antenna size and weight by the gains you must change calwt=True for the amplitude table {{applycal}}.<br />
<br />
* Where do I make the calwt change? <br />
** As described in your README file, you can obtain a fully calibrated measurement set by executing the '''scriptForPI.py'''. For manually calibrated data, within the '''script''' directory you will also find scripts with the format uid*scriptForCalibration.py, one for each execution for that Scheduling Block -- the sciptForPI.py actually executes these scripts, thus this is where any change must be made. <br />
** Toward the bottom of the uid*scriptForCalibration.py scripts (typically Step 17 or 18) you will find a step called '''# Application of the bandpass and gain cal tables'''. Change the calwt=False in all of the {{applycal}} calls to calwt=True. <br />
<br />
:'''Caveat 1:''' You must have the raw ALMA ASDM to run the '''scriptForPI.py''' (see the README file for more information). <br />
<br />
:'''Caveat 2:''' Most (all except early Cycle 0) ALMA manual calibration scripts have within them the CASA version used to create the script. For example, if your manual calibration script says the following, using any other version than 4.2.2 will give an error (even though using for example 4.3.1 would be fine in this case):<br />
<br />
if re.search('^4.2.2', casadef.casa_version) == None:<br />
sys.exit('ERROR: PLEASE USE THE SAME VERSION OF CASA THAT YOU USED FOR GENERATING THE SCRIPT: 4.2.2')<br />
<br />
::You must change the version number in the script to match the version you want to use or the script will not run. <br />
<br />
:'''Caveat 3:''' Scripts from earlier than 4.2.1 are likely to have commands that make them incompatible to run directly in later versions of CASA. It may be difficult for a non-expert to update the uid*scriptForCalibration.py script(s) to current syntax.<br />
<br />
====Option 2: Make an approximate overall relative correction to your calibrated science target data:====<br />
<br />
:<span style="color:#B22222"> &rArr; Option 2 is easy to implement for Cases 1 and 2B, but is not as accurate as Options 1 or 3. However, it is typically adequate for most purposes, and is MUCH better than doing nothing. </span><br />
<br />
* All ALMA data reduced in CASA should already have 1/Tsys<sup>2</sup> weighting which accounts for many of the factors that make data from some baselines more sensitive than others within an individual dataset. If we assume that antennas with abnormally low gains (typically due to poor pointing, hardware failures) have already been flagged and all antennas meet the surface accuracy specifications, then the amplitude gains will be fairly constant from antenna to antenna, with the value dominated by the antenna size. Other parameters that can often be different between datasets include the &Delta;&nu;<sub>ch</sub>, and t<sub>ij</sub>. <br />
* Thus, a reasonably good approximation for the relative weight scaling for data_a compared to data_b is:<br />
** Case 1: (antenna_size_b/antenna_size_a)<sup>4</sup> x (&Delta;&nu;<sub>ch</sub>_b/&Delta;&nu;<sub>ch</sub>_a) x (t<sub>ij</sub>_b/t<sub>ij</sub>_a) <br />
** Case 2B: (antenna_size_b/antenna_size_a)<sup>4</sup> <br />
* such an overall scale factor can be applied during the {{concat}} step using the visweightscale parameter. Note that because its a ratio, it doesn't matter if you use the antenna radius or diameter as long as you are consistent. An example is given below.<br />
<br />
:'''Caveat 1:''' This option requires that the datasets to be combined have only one (unflagged) antenna size per dataset. If this is not the case, then Options 1 or 3 must be used.<br />
<br />
:'''Caveat 2:''' This option also requires that both datasets to be combined are in the same weight situation (i.e. mixed Cases cannot be handled using simple overall relative scaling).<br />
<br />
====Option 3: Run the task {{statwt}} on your calibrated science target data==== <br />
<br />
:<span style="color:#B22222"> &rArr; Option 3 is pretty easy to implement for both Cases 1 and 2B, and will account for (likely small) Gain variations from antenna to antenna within the individual datasets, whereas Option 2 does not. However, see Caveat 1, for very complex line emission it may be difficult to get a good result. </span> <br />
<br />
* This task attempts to assess the sensitivity per visibility and adjust the weights accordingly. It is very commonly used for JVLA data (including their pipeline). <br />
<br />
:'''Caveat 1:''' One must limit the calculation to line-free channels using the fitspw parameter. For complex line projects this can be painful, however, typically the line-free channels are already known from the continuum subtraction, and can be reused here. However, it is best to run statwt before continuum subtraction, and it should always be run on the fully channelized data before any spectral averaging is applied.<br />
:'''Caveat 2:''' {{statwt}} does not currently account for channel-dependent flagging. It is critical to explicitly avoid the edge channels for any TDM spws in the data (128 channels per spw) data using fitspw, (example: for a dataset with only TDM spws, fitspw='*,7~120').<br />
<br />
== Example of Correcting Weights Using Option 2 (Relative Scaling)==<br />
<br />
<br />
[[File:12m_WT_4.2.1.png|200px|thumb|right|<caption> M100 12m-array weights from 4.2.1 reduction.</caption>]]<br />
<br />
<br />
<br />
[[File:7m_WT_4.2.1.png|200px|thumb|right|<caption> M100 7m-array weights from 4.2.1 reduction.</caption>]]<br />
<br />
<br />
Below we show what could have been done to approximately correct the weights in the M100 SV 7m-array and 12m-array data if it was reduced in CASA 4.2.1 (or earlier), i.e. Case 1 (For a complete treatment of the data processed in CASA 4.3 see https://casaguides.nrao.edu/index.php?title=M100_Band3_Combine_4.3).<br />
<br />
In CASA 4.2.1 and earlier, the data weights are 1 upon import, later in the standard calibration procedure, applycal scales the weights by ''N<sub>chan</sub>/[(Tsys(i) * Tsys(j)]'' because calwt=True for the Tsys table applycal. As an example, we plot the weights of 7m and 12m data imported in CASA 4.2.1. No averaging can be turned on <br />
when plotting the weights. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 7m_WT.png 12m_WT.png')<br />
plotms(vis='m100_12m_CO.ms',yaxis='wt',xaxis='uvdist',spw='0~2:200',<br />
coloraxis='spw',plotfile='12m_WT.png')<br />
#<br />
plotms(vis='m100_7m_CO.ms',yaxis='wt',xaxis='uvdist',spw='0~2:200',<br />
coloraxis='spw',plotfile='7m_WT.png')<br />
</source><br />
<br />
The Tsys for the 12m data for the spw containing CO(1-0) ranges from about 100 to 150 K, and the number <br />
of channels per spw is 3840, yielding an expected range for the 4.2.1 weights of 0.38 to 0.17 in good <br />
agreement with Fig. 1. <br />
<br />
As you can see from Figures 1 and 2, the weights are quite similar at this <br />
stage because the data were taken under similar weather conditions and <br />
hence Tsys, and this is the only thing the weights have been scaled by so far besides the <br />
number of channels per spw which is also the same. <br />
<br />
Recall that the rms noise in a single channel for a single visibility is:<br />
<br />
<math> <br />
\sigma_{ij} (Jy) =\frac{2k}{\eta_{q}\eta_{c}A_{eff}} <br />
</math><br />
<math> <br />
\sqrt{\frac{T_{sys,i} T_{sys,j}}{2\Delta\nu_{ch} t_{ij}}}<br />
</math><br />
<math> <br />
\times 10^{26},<br />
</math><br />
<br />
Beyond the obvious difference in the antenna dish sizes (''A<sub>eff</sub>''), looking at the listobs output for these two datasets the same channel width was used but that the integration time per visibility is 10.1 sec for the 7m-array and 6.05 sec for the 12m-array. Since dish area is in the denominator of <br />
the radiometer equation and integration time per visibility is in the <br />
denominator, and assuming WT &prop; 1/&sigma;<sub>i</sub><sub>j</sub><sup>2</sup>, <br />
the '''7m weight should be scaled by: (7./12.)<sup>4</sup> x (10.1/6.05) = 0.193''' <br />
to account for the difference in telescope size and integration time<br />
per visibility.<br />
<br />
12m data:<br />
<pre style="background-color: #fffacd;"><br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
10-Aug-2011/19:38:05.8 - 19:50:22.8 11 1 M100 1500 [0,1,2,3] [6.05, 6.05, 6.05, 6.05] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
<br />
Spectral Windows: (4 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 3840 TOPO 113726.419 488.281 1875000.0 114663.6750 1 XX YY<br />
1 3840 TOPO 111851.419 488.281 1875000.0 112788.6750 2 XX YY<br />
2 3840 TOPO 103663.431 -488.281 1875000.0 102726.1750 3 XX YY<br />
3 3840 TOPO 101850.931 -488.281 1875000.0 100913.6750 4 XX YY<br />
</pre><br />
<br />
<br />
7m data:<br />
<pre style="background-color: #fffacd;"><br />
<br />
ObservationID = 0 ArrayID = 0<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
17-Mar-2013/04:44:04.3 - 04:50:43.7 11 1 M100 261 [0,1,2,3] [10.1, 10.1, 10.1, 10.1] [OBSERVE_TARGET#ON_SOURCE]<br />
<br />
Spectral Windows: (6 unique spectral windows and 1 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs<br />
0 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 101945.850 -488.281 1992187.5 100950.0000 1 XX YY<br />
1 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 103761.000 -488.281 1992187.5 102765.1500 2 XX YY<br />
2 ALMA_RB_03#BB_3#SW-01#FULL_RES 4080 TOPO 111811.300 488.281 1992187.5 112807.1500 3 XX YY<br />
3 ALMA_RB_03#BB_4#SW-01#FULL_RES 4080 TOPO 113686.300 488.281 1992187.5 114682.1500 4 XX YY<br />
4 ALMA_RB_03#BB_1#SW-01#FULL_RES 4080 TOPO 111798.250 488.281 1992187.5 112794.1000 1 XX YY<br />
5 ALMA_RB_03#BB_2#SW-01#FULL_RES 4080 TOPO 113673.250 488.281 1992187.5 114669.1000 2 XX YY<br />
</pre><br />
<br />
<br />
[[File:Intcombo_0.193_WT.png|200px|thumb|right|<caption> M100 7m-array (red) and 12m-array (black) weights after scaling by relative sensitivity. </caption>]]<br />
<br />
<br />
''NOTE: If these data had been manually calibrated in CASA 4.2.2 with calwt=False for the gain table applycal (i.e. Case 2B), then the overall scale factor to apply would be (7./12.)<sup>4</sup> = 0.116, because the difference in visibility integration time would already be accounted for.''<br />
<br />
An easy way to perform this overall scaling is using the visweightscale parameter in {{concat}}.<br />
<br />
<source lang="python"><br />
# In CASA<br />
# Concat and scale weights<br />
os.system('rm -rf M100_Intcombo_0.193.ms')<br />
concat(vis=['m100_12m_CO.ms','m100_7m_CO.ms'],<br />
concatvis='M100_Intcombo_0.193.ms',<br />
visweightscale=[1,0.193])<br />
</source><br />
<br />
Now plot the concatenated weights to verify they are as expected.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Intcombo_0.193_WT.png')<br />
plotms(vis='M100_Intcombo_0.193.ms',yaxis='wt',xaxis='uvdist',spw='0~2:200',<br />
coloraxis='spw',plotfile='Intcombo_0.193_WT.png')<br />
</source><br />
<br />
These combined data with correct relative weights are now ready for imaging. See https://casaguides.nrao.edu/index.php?title=M100_Band3_Combine_4.3 for more on the imaging in CASA 4.3.<br />
<br />
Note that the per spw 12m-array 4.2.1 derived weights are quite similar to those shown in [https://casaguides.nrao.edu/index.php?title=M100_Band3_Combine_4.3 M100_Band3_Combine_4.3] from CASA 4.3 (Figure 3) using the correct per channel weights. This is purely accidental... for this example 1/&sigma;<sub>ij</sub><sup>2</sup> just happens to be approximately equal to ''N<sub>chan</sub>/[(Tsys(i) * Tsys(j)]'' for an antenna with diamter=12m, &Delta;&nu;<sub>ch</sub>=488.281 kHz, t<sub>ij</sub>=6.048s, and 3840 channels.<br />
<br />
== Example of Correcting Weights Using Option 3 (statwt)==<br />
<br />
<br />
[[File:12m_calwtF_WT.png|200px|thumb|right|<caption> M100 12m-array WT vs uvdist after calwt=False for amplitude gains. </caption>]]<br />
<br />
<br />
<br />
[[File:7m_calwtF_WT.png|200px|thumb|right|<caption> M100 7m-array WT vs uvdist after calwt=False for amplitude gains.</caption>]]<br />
<br />
<br />
[[File:12m_amp_ch_Spw0.png|200px|thumb|right|<caption> M100 12m-array amplitude vs channel. </caption>]]<br />
<br />
<br />
<br />
[[File:7m_amp_ch.png|200px|thumb|right|<caption> M100 7m-array amplitude vs channel. </caption>]]<br />
<br />
<br />
This is an example of running {{statwt}} on the M100 SV data. The starting point was data calibrated as if it had been manually calibrated in CASA 4.2.2, i.e. CASE 2b calwt=False for the gains. Note for this starting point, the 7m data actually have ''higher'' weight than the 12m data. This is because the average Tsys were similar but the visibility integration time of the 7m data is 10.1 versus 6.048s for the 12m data (see {{listobs}} outputs in previous section). Imaging these data would work very poorly because as already described, the 7m should actually have weights &approx; 0.193 times lower than the 12m data.<br />
<br />
<source lang="python"><br />
os.system('rm -rf 12m_calwtF_WT.png')<br />
plotms(vis='M100_Band3_12m_CalibratedData.cf.ms.fresh', spw='0',yaxis='weight',xaxis='uvdist',<br />
plotfile='12m_calwtF_WT.png')<br />
<br />
<br />
os.system('rm -rf 7m_calwtF_WT.png')<br />
plotms(vis='M100_Band3_7m_CalibratedData.cf.ms.fresh', spw='3,5',yaxis='weight',xaxis='uvdist',<br />
plotfile='7m_calwtF_WT.png')<br />
</source><br />
<br />
As mentioned above in the Option 3 {{statwt}} caveats, it is essential to avoid strong line channels when calculating <br />
the statistics. What is meant by strong? At minimum, if you can see it in a uvplot it should not be included. The easiest way to check is to make a plot of the uv-data in channel space.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 12m_amp_ch_Spw0.png')<br />
plotms(vis='M100_Band3_12m_CalibratedData.cf.ms',<br />
yaxis='amp',xaxis='channel',spw='0',avgtime='1e8',<br />
avgscan=True,plotfile='12m_amp_ch_Spw0.png')<br />
<br />
<br />
os.system('rm -rf 7m_amp_ch_Spw3_5.png')<br />
plotms(vis='M100_Band3_7m_CalibratedData.cf.ms',<br />
yaxis='amp',xaxis='channel',spw='5',avgtime='1e8',<br />
avgscan=True,plotfile='7m_amp_ch_Spw5.png') <br />
</source><br />
<br />
In statwt we need to avoid the line emission, while sampling the full spectral range. This is especially important in this example because the CO(1-0) transition is close to an oxygen line in the atmosphere -- this is what causes the noise to increase at higher channel numbers. Since we are deriving a spectrally averaged weight it would be unrealistic to only include the lower, less noisy channels. <br />
<br />
<source lang="python"><br />
# In CASA<br />
statwt(vis='M100_Band3_12m_CalibratedData.cf.ms', spw='0', <br />
fitspw='0:0~1800;2500~3839', datacolumn='data')<br />
<br />
# In CASA<br />
statwt(vis='M100_Band3_7m_CalibratedData.cf.ms', spw='3,5', <br />
fitspw='3:0~1900;2500~3839,5:0~1900;2500~3839', datacolumn='data')<br />
</source><br />
<br />
Now we can concat these datasets and make a plot to verify that the weights have the values expect.<br />
<br />
<br />
[[File:M100_statwt.png|200px|thumb|right|<caption> M100 7m-array (purple) and 12m-array (black) weights after running statwt. </caption>]]<br />
<br />
<br />
<source lang="python"><br />
concat(vis=['M100_Band3_12m_CalibratedData.cf.ms','M100_Band3_7m_CalibratedData.cf.ms'], <br />
concatvis='M100_Band3_statwt.ms')<br />
<br />
os.system('rm -rf M100_statwt.png')<br />
plotms(vis='M100_Band3_statwt.ms', spw='0,7,9',yaxis='weight',xaxis='uvdist',<br />
coloraxis='spw',plotfile='M100_statwt.png')<br />
</source><br />
These weights are in good agreement with those derived using the correct weights throughout CASA, i.e. <br />
Figure 3 in [https://casaguides.nrao.edu/index.php/M100_Band3_Combine_4.3 M100_Band3_Combine_4.3]. However, see the next section for caveats.<br />
<br />
== Absolute Accuracy of the Data Weights ==<br />
<br />
As mentioned several times in this guide, good imaging performance is only subject to correct relative weights, which is achieved as of CASA >4.2.2 provided calwt=True for the amplitude gain table. Even for Case 1 (4.2.1 and lower), the relative weights within a given dataset are reasonably accurate in most cases due to the 1/Tsys<sup>2</sup> scaling. However, there are some instances, for example, uv-model fitting where one would like to use the visibility weights as an absolute measure of 1/&sigma;<sup>2</sup>. Here we discuss the absolute accuracy of the visibility weights for options 1 and 3:<br />
<br />
:'''Option 1:''' All PI data taken so far with ALMA are Hanning smoothed online, but it is up to the user at proposal creation time to decide if additional channel averaging is done in the correlator (typically to reduce the data rate). Data that are Hanning smoothed but not channel averaged by the ALMA online system have a '''theoretical effective channel width 2.667 times that reported by for example {{listobs}}''' from the point of view of calculating the sensitivity of a single channel (it is a factor of 2.0 for the FWHM, i.e. channel resolution). The CASA initialization of 2&Delta;&nu;<sub>ch</sub>&Delta;t<sub>ij</sub> does not currently take this into account -- CASA does not have a way of knowing presently what the online system did, only how the data appears on disk, though this will change with future versions of the online software. Thus, presently CASA weights for this case are in principle a factor of 2.667 times smaller than they should be from a sensitivity point of view. For online channel averaging of two or more channels, the difference between the channel width and the effective channel width diminishes rapidly (2x channel averaging has a factor 1.6, 4x a factor 1.2, 8x a factor of 1.1). Additionally, the current weights do not include the correlator and quantization efficiencies ''&eta;<sub>q</sub>'' and ''&eta;<sub>c</sub>''. These have values of about 0.88 and 0.96, respectively.<br />
<br />
:<span style="color:#B22222"> &rArr; Starting in Cycle 3, the effective channel width will be recorded in the raw data and used by CASA to initialize the weights for CASA versions >=4.2.2.</span><br />
<br />
:'''Option 3:''' For CASA <=4.4 {{statwt}} does not take channel-specific flagging into account, only flags that apply to all channels. Thus, the user must avoid flagged channels with the fitspw parameter. Additionally, strong line emission must be avoided. Provided these conditions are met, our tests suggest that the weights produced by {{statwt}} are too low by a factor of 2 compared to the expected values. Both of these {{statwt}} issues will be fixed as of CASA 4.5. <br />
<br />
As a consequence, for online Hanning smoothed data with no online channel averaging, the average value of the weights resulting from Options 1 and 3 will appear to approximately match (see M100 examples), but for erroneous reasons. <br />
<br />
Our investigation into these matters continues so if the absolute weights are important to you, check back here or send a helpdesk ticket with your inquiry.</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=TWHydraBand7_Imaging_4.3&diff=30880TWHydraBand7 Imaging 4.32021-08-19T21:00:35Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Imaging]][[Category:Spectral Line]]<br />
<br />
* '''This tutorial picks up where '''[[TWHydraBand7_Calibration_4.3]]''' leaves off, with fully calibrated, split science target MS. If you wish to skip the Calibration guide: obtain the calibrated data from [[TWHydraBand7#Getting_the_Data]]; extract it using <tt>tar -xzvf FILENAME</tt>; and <tt>cd</tt> into the extracted directory.'''<br />
<br />
*'''This guide is designed for CASA 4.3. If you are using an older version of CASA please see [[TWHydraBand7_Imaging_4.3]].<br />
<br />
* '''Details of the ALMA data are provided at [[TWHydraBand7]]'''<br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
== Prepare Continuum Data for Further Processing==<br />
<br />
In order to speed up imaging/self-calibration for the continuum data we will average channels together, and then flag the few spectral features. This method works well in the case of only a few spectral lines, while retaining enough channelization to aid in multi-frequency synthesis and being able to remove spectral features in the averaged data. When generalizing this process to other datasets, the amount of channel averaging will vary. '''Note for wide bandwidth data it is never a good idea to use the continuum estimate generated by uv-continuum subtraction as your continuum data'''. This worked OK in the past with the narrow band systems like the VLA and low dynamic range instruments like the SMA, but is not advised for ALMA wide bandwidth, high dynamic range data. <br />
<br />
<br />
[[Image:Cont_spw2.png|thumb|<caption>CO(3-2) in spw=2 from the "continuum" channel averaged data.</caption>]]<br />
<br />
<br />
[[Image:Cont_spw3.png|thumb|<caption>Notice upswing at high channel numbers in spw=3.</caption>]]<br />
<br />
<br />
One can also flag the spectral features first but in that case make a backup of the file that is analogous to TWHydra_corrected.ms as you will need it later (unflagged) for spectral line imaging. We begin by excluding ~20 channels on each edge, and averaging over 100 channel intervals:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_cont.ms*')<br />
split(vis='TWHydra_corrected.ms',outputvis='TWHydra_cont.ms',<br />
spw='0~3:21~3820',width=100,datacolumn='data')<br />
</source><br />
<br />
Now make a plot of amplitude vs. channel to see what needs to be flagged<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_cont.ms',spw='0~3',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,coloraxis='spw',iteraxis='spw',xselfscale=True,showgui=True)<br />
</source><br />
<br />
The CO(3-2) and HCO+(4-3) lines are obvious. The 4th spw; spw=3 also shows an upswing on the highest channels that is problematic, it was seen on the calibrators too and is likely due to a weak atmospheric feature, we will flag the worst of it.<br />
<br />
<br />
[[Image:Cont_uvplot.png|thumb|<caption>UV-plot of the continuum emission, colors show spws.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
flagdata(vis='TWHydra_cont.ms', mode='manual', <br />
spw='0:16~16, 2:21~21, 3:33~37')<br />
</source><br />
<br />
Have a look at the continuum as a function of uv-distance (in the plot to the right, the 4 spw are shown in different colors on top of each other). If the data are flat as a function of uv-distance, the source is completely unresolved. If instead you see structure (in this case decreasing amplitude with uv-distance), the source is resolved. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_cont.ms',spw='',xaxis='uvdist',yaxis='amp',field='',avgchannel='38',<br />
coloraxis='spw',plotfile='cont_uvplot.png',showgui=True)<br />
</source><br />
<br />
Notice that the spw do not have the same amplitude. Unfortunately, the red spw (highest frequency) should be higher than the green one (lowest frequency) because optically thin dust goes as nu^4, and optically thick as nu^2. This inconsistency is due to imperfect amplitude/absolute flux calibration. The self-calibration below will bring them into agreement with each other, but this is the sort of thing that results in considerable absolute flux uncertainty at submillimeter wavelengths. We are working on how to do this better.<br />
<br />
==A priori Noise Estimates==<br />
<br />
<br />
[[File:Corrected_time.png|thumb|<caption>Plot for estimating time.</caption>]]<br />
<br />
[[File:sensitivity_cont.png|thumb|<caption>Sensitivity calculator setup with observed continuum parameters for these observations.</caption>]]<br />
<br />
<br />
To estimate what the expected noise levels are, we first need to know how much time we had on source. Looking at <br />
a plot of amplitude vs time for the concatenated but pre-split data (see below) there are 9x8min scans + 6x6min scans + 3x1min + 2x2min scans for about 1.9 hours on source. There are 2 polarizations and about 1.75 GHz of total continuum bandwidth taking into account flagging.<br />
<br />
Using this information and https://almascience.nrao.edu/proposing/sensitivity-calculator, we find that the '''expected rms noise for the continuum is about 0.3 mJy/beam'''.<br />
<br />
For the spectral line data, which we will image below, with 0.12 km/s channels (needs to be entered in the field ''bandwidth per polarization''), the '''expected rms noise for the spectral lines is about 32 mJy/beam'''.<br />
<br />
==Image and Self-Calibrate the Continuum==<br />
<br />
====Create Initial Clean Image====<br />
<br />
First we will make an initial image using {{clean}}, this task will both deconvolve and clean. <br />
<br />
<pre style="background-color: #E0FFFF;"><br />
If you are unfamiliar with the basic concepts of deconvolution and clean,<br />
pause here and review for example<br />
http://www.aoc.nrao.edu/events/synthesis/2010/lectures/wilner_synthesis10.pdf<br />
</pre><br />
<br />
In {{clean}}, we use '''mode='mfs'''' to do multifrequency synthesis, in other words grid each uv-channel independently. Since the uv-spacing is a function of frequency, this will actually achieve greater uv-coverage than if all the channels and spws had been averaged in advance. The '''imsize=100''' and '''cellsize=['0.3arcsec']''' were chosen to be appropriate for the Band and antenna configuration. At ALMA Band 7, the Full Width Half Power primary beam is about 20", so we want to make our image at least this big. We've made it a bit larger out to about the 25% power point. For the configuration used to take these data, the resolution is expected to be about 1.5" and it is a good idea to oversample the beam by setting the cell size to be a factor of 4 or 5 smaller than the beam (we have used 5 here). <br />
<br />
For doing self-calibration it is essential that only "real" signal be included in the image model, thus it is important to make a clean mask, and clean the first iteration rather conservatively (shallowly) so that you are sure that you do not have clean components in your model that do not represent real source emission. In the example below we set '''interactive=True''' so you can interactively make the clean mask and stop clean when the signal from the source starts to approach the surrounding "noise" areas in the residual map.<br />
<br />
'''NOTE:''' Real emission will not go away if you do not put a clean box on it. Conversely if you put a clean box on something that is NOT real and clean deeply you can amplify that feature. Thus, its best to be conservative and build up your clean box as you go, you can generally see fainter "real" features in the residuals.<br />
<br />
'''NOTE:''' If you start an interactive clean, and then do not make a mask, clean will stop when you tell it to go on because it has nothing to clean. There is no default mask.<br />
<br />
<br />
[[Image:Viewer_clean1.png|thumb|<caption>Interactive viewer.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_contall.*')<br />
clean(vis='TWHydra_cont.ms',imagename='TWHydra_contall',<br />
mode='mfs', imagermode='csclean',<br />
imsize=100,cell=['0.3arcsec'],spw='',<br />
weighting='briggs',robust=0.5, <br />
mask='',usescratch=False,<br />
interactive=True,threshold='1mJy',niter=10000)<br />
</source><br />
<br />
'''NOTE:'''{{clean}} will generate several files automatically with the '''imagename''' prepended:<br />
* <imagename>.image # the final restored image<br />
* <imagename>.flux # the effective primary beam response (e.g. for pbcor)<br />
* <imagename>.flux.pbcoverage # the primary beam coverage (ftmachine=’mosaic’ only)<br />
* <imagename>.model # the model image<br />
* <imagename>.residual # the residual image<br />
* <imagename>.psf # the synthesized (dirty) beam<br />
<br />
Once the Clean Viewer opens, click on the polygon tool with your left mouse button, and draw a polygon around the continuum emission with the left mouse button. Double click inside the region with the left mouse button when you are happy with it. It will turn from green to white when the clean mask is accepted. To continue with clean use the "Next action" buttons in the green area on the Viewer Display GUI. <br />
<br />
<pre style="background-color: #E0FFFF;"><br />
The red X will stop clean where you are, the blue arrow will stop <br />
the interactive part of clean, but continue to clean non-interactively <br />
until reaching the number of iterations requested (niter) or the flux <br />
density threshold (whichever comes first), and the green circle arrow <br />
will clean until it reaches the "iterations" parameter on the left side <br />
of the green area. <br />
</pre><br />
<br />
'''You should chose the green circle arrow'''. It will clean 100 iterations (number under iteration in the green area on the Viewer GUI) and then show the '''residual image''' again. The middle mouse button is automatically assigned to the "plus" icon, which adjusts the colorscale. '''If you cannot easily access multiple mouse buttons, you can always assign your one mouse button the icon you want by clicking on it'''. Adjust the color scale so you can see the structure in the noise better. Also, note that the logger is telling you the amount of clean flux recovered so far, the min/max residuals, and the synthesized beam:<br />
<br />
<pre style="background-color: #fffacd;"><br />
Model 0: max, min residuals = 0.0203454, 0.00267232 clean flux 1.30124<br />
Threshhold not reached yet.<br />
Beam used in restoration: 1.6962 by 1.54549 (arcsec) at pa 28.9043 (deg) <br />
</pre> <br />
<br />
Already with only 100 iterations, the residuals outside the clean mask are larger than that inside. So we will stop here by hitting the Red X symbol.<br />
<br />
====1st Round of Continuum Data Phase Self-calibration====<br />
<br />
Next we can use the clean model to self-calibrate the uv-data. The process of self-calibration tries to adjust the data to better match the model you give it. That is why it is important to make the model as good as you can. {{clean}} places a uv-model of the resulting image in the file <tt>TWHydra_cont.model</tt>. This model can then be used to self-calibrate the data using {{gaincal}}. We use '''gaintype='T'''' to tell it to average the polarizations before deriving solutions (which gains us a sqrt(2) in sensitivity). A critical parameter to play with is '''solint'''; we use '''solint=30s'''. From {{listobs}} the integration time of these data is about 10 seconds, so using '''solint=30s''' then gains us a sqrt(3) in sensitivity. It is often best to start with a larger '''solint''' and then work your way down to the integration time if the S/N of the data permits. For the same reasons as described in the [[TWHydraBand7_Calibration_4.3]] tutorial we want to fix up the phases before tackling amplitude by setting '''calmode='p''''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='TWHydra_cont.ms',caltable='self_1.pcal',<br />
solint='30s',combine='',gaintype='T',<br />
refant='DV06',spw='',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
<br />
[[Image:Self_1_phase.png|thumb|<caption>Phase solutions from self_1.pcal with solint=30s.</caption>]]<br />
<br />
<br />
Because '''combine=' '''', we are getting independent solutions for each spw. If you are desperate for S/N, you can try combining them with combine='spw'. We have plenty of S/N in these data so we keep them separate. Look at the solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='self_1.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='1~8',iteration='antenna',<br />
subplot=421,plotrange=[0,0,-80,80],figfile='self_1_phase.png',showgui=True)<br />
</source><br />
<br />
The magnitude of corrections gives you a sense of how accurate the phase transfer from the calibrators was. You are checking to see that the phases are being well tracked by the solutions, i.e. smoothly varying functions of time within a given scan. Also the solutions for each spw are lying basically on top of each other which indicates that the spw to spw phase offsets are well calibrated already. Overall, these solutions look quite good. If they didn't you could try increasing the solint and/or combining the spw into a single solution. <br />
<br />
After checking that you are happy with the solutions, apply them with {{applycal}}<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='TWHydra_cont.ms',field='',gaintable=['self_1.pcal'],calwt=False)<br />
</source><br />
<br />
<br />
[[Image:Viewer_pcal1.png|thumb|<caption>Clean residuals after 100 iterations and the 1st phase only self-cal. Residuals inside mask are higher than the "noise" outside.</caption>]]<br />
<br />
Next clean the source again, to save time we can start with the clean mask from the last iteration. If you make interactive adjustments to the clean mask, they will be written to the '''new''' mask TWHydra_contall_1pcal.mask, and not the old one. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_contall_1pcal.*')<br />
clean(vis='TWHydra_cont.ms',imagename='TWHydra_contall_1pcal',<br />
mode='mfs',imagermode='csclean',<br />
imsize=100,cell=['0.3arcsec'],spw='',<br />
weighting='briggs',robust=0.5, <br />
mask='TWHydra_contall.mask',usescratch=False,<br />
interactive=True,threshold='1mJy',niter=10000)<br />
</source><br />
<br />
Now after the first 100 iterations you see that the residuals inside the clean mask are higher than those outside. This is because flux that had been spread out in the map due to poorly correlated phases has been placed where it belongs on the source, thereby increasing its flux. You may also notice that your original clean mask is a little too small. If so, you can make it a bit larger, just by drawing a new polygon around the area you want to include. When you double click, it will incorporate the new mask into the current mask (if your box goes awry you can use the erase toggle [radio button] to erase and redo the mask). The new mask area need not overlap the original mask (though for this source it probably will). You might also want to use the magnifying glass to zoom in. Again, you assign mouse buttons by clicking on the icon you want. Clean another 100 iterations (for a total of 200) and stop by clicking the red X.<br />
<br />
Now open initial and 1st self-cal images to compare them using {{imview}}. This is an alternative version of the {{viewer}} that is somewhat scriptable. <br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file':'TWHydra_contall.image','range':[-0.01,0.05]},<br />
{'file':'TWHydra_contall_1pcal.image','range':[-0.01,0.05]}])<br />
</source><br />
<br />
Once the Viewer Display panel opens, click on the box next in the "Images" panel and then use the tapedeck buttons in the panel to alternate between the two images. Because the range was explicitly set, they are on the same color scale. It is obvious that the self-calibrated image is better. To quantify, 1st toggle back to "normal" and then draw a region that excludes the central source with the polygon tool and then double click inside (if you stay in the "Images" view, only the statistics for the last image opened will appear on the terminal instead of both). The basic statistics of the polygon region will print to the '''terminal''':<br />
<br />
<pre style="background-color: #fffacd;"><br />
TWHydra_contall.image<br />
Stokes Velocity Frame Doppler Frequency <br />
I 0km/s TOPO RADIO 3.50845e+11 <br />
BrightnessUnit BeamArea Npts Sum Flux <br />
Jy/beam 33.0039 2357 -9.434800e-01 -2.858690e-02 <br />
Mean Rms Std dev Minimum Maximum <br />
-4.002885e-04 7.906752e-03 7.898289e-03 -2.541723e-02 2.172779e-02 <br />
<br />
TWHydra_contall_1pcal.image<br />
Stokes Velocity Frame Doppler Frequency <br />
I 0km/s TOPO RADIO 3.50845e+11 <br />
BrightnessUnit BeamArea Npts Sum Flux <br />
Jy/beam 33.0039 2357 6.326584e-01 1.916918e-02 <br />
Mean Rms Std dev Minimum Maximum <br />
2.684168e-04 3.655786e-03 3.646692e-03 -1.215155e-02 1.109403e-02 <br />
</pre><br />
<br />
The self-calibrated image has more than a factor of 2 lower rms noise!<br />
<br />
====2nd Round of Continuum Data Phase Self-calibration====<br />
<br />
Now that we have a better model, we solve for solutions again. This time we will try making the solint smaller -- the integration time is about 10 seconds, so we set '''solint='int'''' to have it use the integration time.<br />
<br />
<br />
[[Image:Self_2_phase.png|thumb|<caption>Phase solutions from self_2.pcal with solint='int'.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='TWHydra_cont.ms',caltable='self_2.pcal',<br />
solint='int',combine='',gaintype='T',<br />
refant='DV06',spw='',minblperant=4,<br />
calmode='p',minsnr=2)<br />
</source><br />
<br />
Now plot the new solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='self_2.pcal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='1~8',iteration='antenna',<br />
subplot=421,plotrange=[0,0,-80,80],figfile='self_2_phase.png',showgui=True)<br />
</source><br />
<br />
The phases on this shorter '''solint''' still seem to be tracking well so we will apply it and make a new image and check to see if there is improvement. <br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='TWHydra_cont.ms',field='',gaintable=['self_2.pcal'],calwt=False)<br />
</source><br />
<br />
[[Image:Viewer_pcal2.png|thumb|<caption>Clean residuals after 2nd phase only self-cal and 250 iterations. The residuals inside the clean mask are lower than those outside, so its time to stop.</caption>]]<br />
<br />
<br />
Now clean again, updating '''mask''' with the most recent mask file.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_contall_2pcal.*')<br />
clean(vis='TWHydra_cont.ms',imagename='TWHydra_contall_2pcal',<br />
mode='mfs',imagermode='csclean',<br />
imsize=100,cell=['0.3arcsec'],spw='',<br />
weighting='briggs',robust=0.5,usescratch=False,<br />
mask='TWHydra_contall_1pcal.mask',<br />
interactive=True,threshold='1mJy',niter=10000)<br />
</source><br />
<br />
After 200 iterations<br />
<br />
<pre style="background-color: #fffacd;"><br />
Model 0: max, min residuals = 0.00489776, -0.00464541 clean flux 1.44461<br />
</pre><br />
<br />
It looks like it could use just a little more cleaning, so set '''iterations=50 in the GUI'''.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Clean used 50 iterations to approach a threshhold of 0.0015067<br />
0.0107122 Jy <- cleaned in this cycle for model 0 (Total flux : 1.45532Jy)<br />
Final maximum residual = 0.00326382<br />
Model 0: max, min residuals = 0.00326382, -0.00321361 clean flux 1.45532<br />
</pre><br />
<br />
The small amount of flux cleaned this cycle indicates that clean is converging and it's about time to stop. Also the residuals inside the clean mask are now smaller than the residuals outside (i.e. the noise) also indicating it's time to stop by hitting the red X on the GUI.<br />
<br />
Now repeat the display, blinking, and image statistics steps described above for the 1st and 2nd phase-only self-cal iterations.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file':'TWHydra_contall_1pcal.image','range':[-0.01,0.05]},<br />
{'file':'TWHydra_contall_2pcal.image','range':[-0.01,0.05]}])<br />
</source><br />
<br />
The difference is small:<br />
<pre style="background-color: #fffacd;"><br />
Stokes Velocity Frame Doppler Frequency <br />
I 0km/s TOPO RADIO 3.50845e+11 <br />
BrightnessUnit BeamArea Npts Sum Flux <br />
Jy/beam 33.0039 2466 4.611597e-02 1.397287e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
1.870072e-05 3.831524e-03 3.832255e-03 -1.414417e-02 1.109403e-02 <br />
<br />
TWHydra_contall_2pcal.image<br />
Stokes Velocity Frame Doppler Frequency <br />
I 0km/s TOPO RADIO 3.50845e+11 <br />
BrightnessUnit BeamArea Npts Sum Flux <br />
Jy/beam 33.0039 2466 1.159416e-01 3.512963e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
4.701606e-05 3.811716e-03 3.812199e-03 -1.383034e-02 1.078456e-02 <br />
</pre><br />
<br />
This small improvement suggests that we have gone as far as we can with phase self-cal.<br />
<br />
===Continuum Data Amplitude Self-Calibration===<br />
<br />
Now we will apply the best phase solution table on-the-fly while solving for the amplitude self-cal solutions. Since amplitude changes more slowly and is less constrained than phase, we use a longer '''solint='inf'''', as long as '''combine=' '''' this means to do one solution per scan. <br />
<br />
<source lang="python"><br />
# In CASA<br />
gaincal(vis='TWHydra_cont.ms',caltable='self_ap.cal',<br />
solint='inf',combine='',gaintype='T',<br />
refant='DV06',spw='',minblperant=4,<br />
gaintable=['self_2.pcal'],<br />
calmode='ap',minsnr=2)<br />
</source><br />
<br />
It's useful to look at the residual phase in the amplitude calibration table, i.e. since we applied our best phase-only solution while solving for this new amplitude and phase ('''calmode='ap'''') solution, the phase should already be corrected.<br />
<br />
[[Image:Self_ap_phase.png|thumb|<caption>Residual phase solutions from self_ap.cal.</caption>]]<br />
<br />
[[Image:Self_ap_amp.png|thumb|<caption>Amplitude solutions from self_ap.cal.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='self_ap.cal',xaxis='time',yaxis='phase',<br />
spw='',field='',antenna='1~8',iteration='antenna',<br />
subplot=421,plotrange=[0,0,-1,1],figfile='self_ap_phase.png',showgui=True)<br />
</source><br />
<br />
The residuals are very small as expected. (If you see large residuals here, it means that the phase-only solution is suspect, and you were mostly just moving noise around.) Now look at the amplitude solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotcal(caltable='self_ap.cal',xaxis='time',yaxis='amp',<br />
spw='',field='',antenna='1~8',iteration='antenna',<br />
subplot=421,plotrange=[0,0,0.6,1.4],figfile='self_ap_amp.png',showgui=True)<br />
</source><br />
<br />
For the most part these look very good except one rather large deviation on DV06. Apply both the final phase and the amplitude calibration tables.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='TWHydra_cont.ms',field='',gaintable=['self_2.pcal','self_ap.cal'],calwt=False)<br />
</source><br />
<br />
Plot the corrected data as a function of both time and uv-distance. It is always a good idea to check these after an amplitude self-calbration to be sure that it worked properly. Occasionally, a few spurious bits of data will get blown up by the amplitude self-cal if it is not well-constrained. <br />
<br />
<br />
[[Image:Selfcal_time.png|thumb|<caption>Post self-calibration amplitude vs. time</caption>]]<br />
<br />
<br />
[[Image:Selfcal_uvdist.png|thumb|<caption>Post self-calibration amplitude vs. uv-distance</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_cont.ms',xaxis='time',yaxis='amp',<br />
avgchannel='38',ydatacolumn='corrected',coloraxis='spw',<br />
plotfile='selfcal_time.png',showgui=True)<br />
</source><br />
<br />
The time where we saw a large amplitude correction on DV06 looks consistent here so the large correction we saw in the solutions corrected a real issue. <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_cont.ms',xaxis='uvdist',yaxis='amp',<br />
avgchannel='38',ydatacolumn='corrected',coloraxis='spw',<br />
plotfile='selfcal_uvdist.png',showgui=True)<br />
</source><br />
<br />
You will see that this plot looks much better than before. <br />
<br />
<br />
[[Image:Viewer_apcal.png|thumb|<caption>Clean residuals after final amplitude and phase self-cal.</caption>]]<br />
<br />
<br />
Now make the final continuum image.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_contall_apcal.*')<br />
clean(vis='TWHydra_cont.ms',imagename='TWHydra_contall_apcal',<br />
mode='mfs',imagermode='csclean',<br />
imsize=100,cell=['0.3arcsec'],spw='',<br />
weighting='briggs',robust=0.5,usescratch=False,<br />
mask='TWHydra_contall_2pcal.mask',<br />
interactive=True,threshold='1mJy',niter=10000)<br />
</source><br />
<br />
We know we need to clean at least 250 iterations (i.e. 100+100+50=250), so set iterations to 300 in the gui and hit the round green arrow. After this notice how much better the residual looks. Clean another 100 iterations. The "circular pattern", and 4 bright spots at each corner you see now in the residual is likely due to non-closing amplitude errors that cannot be removed with an antenna based self-calibration, so this is as good as it gets with pure self-cal. Hit the Red X to stop clean.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Clean used 100 iterations to approach a threshhold of 0.00105507<br />
0.0061808 Jy <- cleaned in this cycle for model 0<br />
Final maximum residual = 0.00242926<br />
Model 0: max, min residuals = 0.00238278, -0.00242926 clean flux 1.48201<br />
</pre><br />
<br />
Now repeat the display, blinking, and image statistics described above for the 2nd phase only and the new amplitude & phase self-cal'ed images.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview(raster=[{'file':'TWHydra_contall_2pcal.image','range':[-0.01,0.05]},<br />
{'file':'TWHydra_contall_apcal.image','range':[-0.01,0.05]}])<br />
</source><br />
<br />
The amplitude and phase self cal'ed image is notably better than phase-only. Quantitatively:<br />
<pre style="background-color: #fffacd;"><br />
<br />
TWHydra_contall_2pcal.image<br />
Stokes Velocity Frame Doppler Frequency <br />
I 0km/s TOPO RADIO 3.50845e+11 <br />
BrightnessUnit BeamArea Npts Sum Flux <br />
Jy/beam 33.0039 3542 2.053068e-01 6.220676e-03 <br />
Mean Rms Std dev Minimum Maximum <br />
5.796351e-05 3.955967e-03 3.956101e-03 -1.383034e-02 1.078456e-02 <br />
<br />
TWHydra_contall_apcal.image<br />
Stokes Velocity Frame Doppler Frequency <br />
I 0km/s TOPO RADIO 3.50845e+11 <br />
BrightnessUnit BeamArea Npts Sum Flux <br />
Jy/beam 33.0039 3542 6.044652e-04 1.831495e-05 <br />
Mean Rms Std dev Minimum Maximum <br />
1.706565e-07 1.270183e-03 1.270362e-03 -4.980673e-03 4.137159e-03 <br />
</pre> <br />
<br />
and indeed, we've gained another factor of 3 in sensitivity. <br />
<br />
'''Overall with this self-calibration we've improved the noise by a factor of 6.7! With a peak flux density of 0.98 Jy, the S/N increased from 115 initially to 770 after. So, well worth the trouble.'''<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you have used the '''standard='Butler-JPL-Horizons 2012' ''' model during the calibration process, you should see an increase in peak flux density of about 3%.<br />
</div><br />
<br />
It is notable that we are still a factor of more than 4 from the apriori noise estimate of 0.3 mJy/beam. This is likely due to residual sources of non-closing (not antenna based) errors in the data like baseline errors and baseline-dependent bandpass errors. Additionally, polarization errors (which we did not calibrate at all) can contribute, as well as imperfect uv-coverage with only 8 antennas. A dynamic range of 770 is actually quite good in the submillimeter.<br />
<br />
==Apply self-calibration to Full Dataset and Split Line Data==<br />
<br />
Now apply the self-calibration derived from the continuum emission to the full unchannel-averaged data.<br />
<br />
<source lang="python"><br />
# In CASA<br />
applycal(vis='TWHydra_corrected.ms',gaintable=['self_2.pcal','self_ap.cal'],calwt=False)<br />
</source><br />
<br />
Split off the CO(3-2) spectral line data in spw=2 and the HCO+(4-3) data in spw=0 for further processing.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2.ms*')<br />
split(vis='TWHydra_corrected.ms',outputvis='TWHydra_CO3_2.ms',datacolumn='corrected',spw='2')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplus.ms*')<br />
split(vis='TWHydra_corrected.ms',outputvis='TWHydra_HCOplus.ms',datacolumn='corrected',spw='0')<br />
</source><br />
<br />
==Continuum Subtraction== <br />
<br />
Next we need to subtract the continuum emission from the spectral line data. It is best to do this in the uv-plane. The first step is to make a channel plot of the spectral data to get the line-free channel ranges for both spectral lines. Since the line emission is simple (one spectral line per spw) and there are more than enough line-free channels we do not need to be very precise just stay well away from the line and whenever possible it is best to chose line-free channel ranges on either side of the line for the best possible baseline subtraction. <br />
<br />
[[Image:CO3_2_channel.png|thumb|<caption>CO(3-2) line in channel space.</caption>]]<br />
<br />
[[Image:HCOp4_3_channel.png|thumb|<caption>HCO+(4-3) in channel space.</caption>]]<br />
<br />
<br />
'''Note''': after the split above, the original spw ids will be relabeled as 0 in each file because there is only one spectral window in each file.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_CO3_2.ms',spw='0',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,coloraxis='spw',plotfile='CO3_2_channel.png',showgui=True)<br />
</source><br />
<br />
It is a good idea to stay a little away from the very edges of the baseband which can be noisy. The channel ranges <br />
20~2000 and 2400~3800 look like good choices for CO(3-2).<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_HCOplus.ms',spw='0',xaxis='channel',yaxis='amp',<br />
avgtime='1e8',avgscan=True,coloraxis='spw',plotfile='HCOp4_3_channel.png',showgui=True)<br />
</source><br />
<br />
The channel ranges 20~1500 and 1900~3800 look like good choices for HCO+(4-3).<br />
<br />
Next use the identified line-free channel ranges and {{uvcontsub}} to subtract the continuum model from the line emission.<br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='TWHydra_CO3_2.ms',fitorder=1,fitspw='0:20~2000,0:2400~3800')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
uvcontsub(vis='TWHydra_HCOplus.ms',fitorder=1,fitspw='0:20~1500,0:1900~3800')<br />
</source><br />
<br />
Note: If you do not have line-free channels on both sides of the line emission, it is safer to use '''fitorder=0''', or the fit may diverge.<br />
<br />
==Spectral Line Imaging==<br />
<br />
Now we are ready to make cubes of the line emission. The imaging parameters are similar to the continuum except for <br />
those dealing with the spectral setup: '''mode''', '''start''', '''width''', '''nchan''', '''restfreq''', and '''outframe''' parameters. When making spectral images you have three choices for the '''mode''' parameter: '''channel''', '''velocity''', and '''frequency'''. Data are taken using constant frequency channels. For spectral line analysis it's often more useful to have constant velocity channels, and this is also the best way to make images of multiple lines with the exact same channelization for later comparison. For '''mode='velocity'''', the desired '''start''' and '''width''' also need to be given in velocity units for the desired output frame. <br />
<br />
It is important to note that ALMA does not do on-line Doppler Tracking and the native frame of the data is TOPO. If you do not specify '''outframe''' the output cube will also be in TOPO, which is not very useful for spectral line work. The Doppler Shift is taken out during the regridding to the desired outframe in {{clean}} or alternatively it can be done separately by the {{cvel}} task which would need to be run before {{clean}}. <br />
<br />
<br />
[[Image:CO3_2_vel.png|thumb|<caption>CO(3-2) spectrum in LSRK velocity space.</caption>]]<br />
<br />
[[Image:HCOp4_3_vel.png|thumb|<caption>HCO+(4-3) spectrum in LSRK velocity space.</caption>]]<br />
<br />
<br />
To see what velocity parameters you want to set in clean it is useful to make a plot in {{plotms}} in the desired output frame. To make it easier to see, we set a '''plotrange''' since we know the LSRK velocity of TW Hya is about 2.88 km/s. Note these plots take a little longer because of the frame shift.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_CO3_2.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz',plotrange=[-20,23,0,0],plotfile='CO3_2_vel.png',showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='TWHydra_HCOplus.ms.contsub',xaxis='velocity',yaxis='amp',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='356.7342GHz',plotrange=[-20,23,0,0],plotfile='HCOp4_3_vel.png',showgui=True) <br />
</source><br />
<br />
From these plots it looks like imaging the cubes from about -5 to +8 km/s will encompass the line, but still provide several line-free channels on either side so the noise level can be estimated. The channel width is 122 kHz, which at <br />
345.79599 GHz is 0.106 km/s. Recall that the spectral ''resolution'' is a factor of two poorer. We will use a velocity channel width of 0.12 km/s for a little oversampling, but the spectral resolution remains 0.2 km/s.<br />
<br />
As before, it is very important that you make a clean mask. There are many ways to do this ranging from the complicated to simple. For this example we will make a single clean mask that encompasses the line emission in every channel and apply it to all channels. This is much better than no clean mask, though not quite as good as making an individual mask for each channel. For some sources, one can use the continuum mask to clean the line, however TW Hya is a bit curious in that the continuum emission is not as extended as the line emission (Hughes et al. 2008).<br />
<br />
=== CO(3-2) Imaging ===<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2line.*')<br />
clean(vis='TWHydra_CO3_2.ms.contsub',imagename='TWHydra_CO3_2line',<br />
imagermode='csclean',spw='',<br />
imsize=100,cell=['0.3arcsec'],<br />
mode='velocity',start='-4km/s',width='0.12km/s',nchan=118,<br />
restfreq='345.79599GHz',outframe='LSRK',<br />
weighting='briggs',robust=0.5,<br />
mask='',usescratch=False,<br />
interactive=True,threshold='1mJy',niter=100000)<br />
</source><br />
<br />
<br />
[[Image:Viewer_55.png|thumb|<caption>Interactive Viewer window with CO(3-2) clean mask, before cleaning starts.</caption>]]<br />
<br />
[[Image:Viewer_500.png|thumb|<caption>Interactive Viewer window for CO(3-2), after 500 iterations, more cleaning is needed.</caption>]]<br />
<br />
<br />
It will take a little while to grid the cube, when the interactive viewer opens, use the tapedeck to cycle through the channels. You should see a progress of line emission in channels 40 to 70 or so.<br />
*Go to channel 57 (about line center)<br />
*Select the polygon tool with the left mouse button<br />
*Select the "All Channels" toggle in the green area<br />
*Draw a polygon around the channel 57 emission, double click inside to activate.<br />
*Now go to channel 60, this is the channel with the most extent to the SE, draw another polygon that encompasses this emission if necessary,double click inside to activate.<br />
*Now go to channel 55, this is the channel with the most extent to the NW, draw another polygon that encompasses this emission if necessary,double click inside to activate. <br />
*Now double check the mask by cycling through channels with real line emission to be sure that your clean box encompasses the emission, but does not extend far beyond it. Again the "erase" toggle can come in handy if things go awry or if you want to just delete a small portion of an existing mask (just draw a polygon around what you want to erase and double click inside), be sure to switch back to "add" before going on.<br />
<br />
The figure shows what the final clean box should look like. Now clean using the green circle arrow. You can watch the progress in the logger. When 100 iterations are done, the viewer will show the residual map for each channel. Cycle through the channels and see whether you are still happy with the clean box in each signal channel. The "erase button" can help you fix mistakes. If necessary adjust. It is often useful to adjust the colorscale with the "plus" symbol icon. Keep going until the residual emission looks like the surrounding noise. If it seems to be going slowly, increase the iterations to 300 for a round or two. I stopped cleaning (Red x) when the logger showed (about 900 iterations). <br />
<pre style="background-color: #fffacd;"><br />
12.9846 Jy <- cleaned in this cycle for model 0<br />
Final maximum residual = 0.157312<br />
Model 0: max, min residuals = 0.157312, -0.0871295 clean flux 279.304<br />
</pre><br />
<br />
NOTE: The residuals are still a bit high in the strongest emission channels after it is well cleaned in the weaker channels. This is an indication that the sensitivity is "dynamic range" limited. In other words even with more <br />
intrinsic sensitivity per integration (i.e. better system temperature), the dynamic range limit would remain the same unless you get more uv-coverage. The fundamental dynamic range limitations of ALMA will be considerably better with 16 antennas in Early Science and MUCH better with the full 50-antenna array.<br />
<br />
=== HCO+(4-3) Imaging ===<br />
<br />
Now image and clean the HCO+(4-3) line using what you learned above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplusline.*')<br />
clean(vis='TWHydra_HCOplus.ms.contsub',imagename='TWHydra_HCOplusline',<br />
imagermode='csclean',spw='',<br />
imsize=100,cell=['0.3arcsec'],<br />
mode='velocity',start='-4km/s',nchan=118,width='0.12km/s',<br />
restfreq='356.7342GHz',outframe='LSRK',<br />
weighting='briggs',robust=0.5,<br />
mask='',usescratch=False,<br />
interactive=True,threshold='1mJy',niter=100000)<br />
</source><br />
<br />
I stopped cleaning when: <br />
<pre style="background-color: #fffacd;"><br />
4.79833 Jy <- cleaned in this cycle for model 0<br />
Final maximum residual = 0.0977348<br />
Model 0: max, min residuals = 0.0977348, -0.060484 clean flux 192.066<br />
</pre><br />
<br />
==Image Analysis==<br />
<br />
After all that hard work you now have deconvolved images of the 350 GHz continuum, the CO (3-2) line, and the HCO+ (4-3) line. CASA includes tools to help you understand the contents of these images and make basic scientific measurements. These tools are divided between graphical exploration tools, accessible via the CASA viewer, and command line analysis tasks. To access the viewer, simply type '''viewer()''' and to see the list of analysis tasks available in CASA type '''tasklist''' at the CASA prompt and look under the Analysis sub-heading.<br />
<br />
If you want to do this portion of the guide and have downloaded the (.FITS) reference images, you can import them to be CASA images using '''importfits'''. If you are re-importing the images, use the commented remove command, here, to delete the previous import first.<br />
<br />
<source lang="python"><br />
# In CASA<br />
# os.system("rm -rf TWHydra_CO3_2line.image")<br />
importfits(fitsimage="TWHydra_CO3_2line.image.fits", imagename="TWHydra_CO3_2line.image")<br />
</source><br />
<br />
where '''fitsimage''' refers to the file to imported and '''imagename''' gives the output filename. If you are proceeding directly from the imaging step, your data are already CASA images and you may proceed without any importing.<br />
<br />
You can get the basic properties of any CASA image using the '''imhead''' command. For example, type <br />
<br />
<source lang="python"><br />
# In CASA<br />
imhead("TWHydra_CO3_2line.image")<br />
</source><br />
<br />
and look at the CASA logger. You will see an image summary like the following<br />
<br />
<pre style="background-color: #fffacd;"><br />
INFO imhead ##########################################<br />
INFO imhead ##### Begin Task: imhead #####<br />
INFO imhead imhead(imagename="TWHydra_CO3_2line.image",mode="summary",hdkey="",hdvalue="",<br />
INFO imhead hdtype="",hdcomment="",verbose=False)<br />
INFO ImageAnalysis <br />
INFO ImageAnalysis Image name : TWHydra_CO3_2line.image<br />
INFO ImageAnalysis Object name : TW Hya<br />
INFO ImageAnalysis Image type : PagedImage<br />
INFO ImageAnalysis Image quantity : Intensity<br />
INFO ImageAnalysis Pixel mask(s) : None<br />
INFO ImageAnalysis Region(s) : None<br />
INFO ImageAnalysis Image units : Jy/beam<br />
INFO ImageAnalysis Restoring Beam : 1.68065 arcsec, 1.53193 arcsec, 22.3015 deg<br />
INFO ImageAnalysis <br />
INFO ImageAnalysis Direction reference : J2000<br />
INFO ImageAnalysis Spectral reference : LSRK<br />
INFO ImageAnalysis Velocity type : RADIO<br />
INFO ImageAnalysis Rest frequency : 3.45796e+11 Hz<br />
INFO ImageAnalysis Pointing center : 11:01:51.844983 -34.42.17.160885<br />
INFO ImageAnalysis Telescope : ALMA<br />
INFO ImageAnalysis Observer : Unknown<br />
INFO ImageAnalysis Date observation : 2011/04/22/00:15:42<br />
INFO ImageAnalysis Telescope position: [2.22514e+06m, -5.44031e+06m, -2.48103e+06m] (ITRF)<br />
INFO ImageAnalysis <br />
INFO ImageAnalysis Axis Coord Type Name Proj Shape Tile Coord value at pixel Coord incr Units<br />
INFO ImageAnalysis -------------------------------------------------------------------------------------------------- <br />
INFO ImageAnalysis 0 0 Direction Right Ascension SIN 100 25 11:01:51.845 50.00 -3.000000e-01 arcsec<br />
INFO ImageAnalysis 1 0 Direction Declination SIN 100 25 -34.42.17.161 50.00 3.000000e-01 arcsec<br />
INFO ImageAnalysis 2 2 Spectral Frequency 118 59 3.45801e+11 0.00 -1.38414152e+05 Hz<br />
INFO ImageAnalysis Velocity -4 0.00 1.200000e-01 km/s<br />
INFO ImageAnalysis 3 1 Stokes Stokes 1 1 I<br />
INFO imhead ##### End Task: imhead #####<br />
INFO imhead ##########################################<br />
<br />
</pre><br />
<br />
This report tells you about the target, shape, and coordinates of the image. It also notes the rest frequency and some details of the observation. '''imhead''' may be used more generally to retrieve header keywords as python variables or to manipulate the header of an image. Consult the help for details.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
You may want to pay particular attention to the third and fourth axis of the CASA image. These are <br />
usually frequency (velocity) and Stokes (polarization) in some order. An occasional source of confusion <br />
is that this order is not always the same. Often CLEAN will output images with the Stokes axis third, <br />
while IMPORTFITS usually yields images with the frequency axis third. If you encounter problems with<br />
image analysis tasks, you may want to check whether this results from conflicting axis definitions. You<br />
can adjust the order of the axes using the IMTRANS task.<br />
</pre><br />
<br />
Before proceeding, make sure that you have all three data sets as CASA images: '''TWHydra_CO3_2line.image''', '''TWHydra_HCOPlusline.image''', and '''TWHydra_contall_apcal.image'''. If you do not, then import them from the provided reference images. If you are re-importing the FITS images, run the commented remove commands before running importfits.<br />
<br />
<source lang="python"><br />
# In CASA<br />
# os.system("rm -rf TWHydra_CO3_2line.image")<br />
importfits(fitsimage="TWHydra_CO3_2line.image.fits", imagename="TWHydra_CO3_2line.image")<br />
# os.system("rm -rf TWHydra_HCOplusline.image")<br />
importfits(fitsimage="TWHydra_HCOplusline.image.fits", imagename="TWHydra_HCOplusline.image")<br />
# os.system("rm -rf TWHydra_contall_apcal.image")<br />
importfits(fitsimage="TWHydra_contall_apcal.image.fits", imagename="TWHydra_contall_apcal.image")<br />
</source><br />
<br />
In the rest of this guide we will construct ''moment maps'' showing the integrates, velocity field, and line width of the two lines. We will carry out Gaussian fits to the continuum and line maps. We will also illustrate some of the additional analysis capabilities available in CASA: subcube extraction, retrieval of data into python arrays, cube alignment, smoothing, and overlays of multiple spectral lines.<br />
<br />
===Moment Maps===<br />
<br />
The line "images" of CO(3-2) and HCO+(4-3) are, of course, cubes rather than images. Two dimensional ''moment maps'' are a standard way to present such data. These are maps showing the result of various moments (or similar calculations) carried out along the frequency axis of the cube. For example, the "moment 0" map is the sum of emission along the frequency axis at each pixel. The first moment is the intensity-weighted velocity (or frequency). The second moment is the intensity-weighted velocity (or frequency) dispersion about the mean. These and a number of other calculations (for example the maximum intensity) can be calculated via the '''immoments''' task. Consult the help or the CASA cookbook for more details.<br />
<br />
In this section, we will calculate moments for our two spectral line data cubes. To make the highest quality moment maps, we will first identify the planes of the data cube that contain emission, so that we can carry out the sum only over these channels. Doing otherwise adds noise to the moment map without adding signal and so lowers the signal-to-noise ratio of the final image. For the first and second moments we will identify an intensity threshold above which the data are statistically very likely to correspond to real astronomical emission. We will restrict the calculation to consider only data above this threshold, yielding a cleaner calculation (the second moment, in particular, is not at all robust against the inclusion of outlying noise). Finally, we will inspect the resulting moment maps with the viewer and output them to .FITS files.<br />
<br />
====Estimate the Spectral Extent====<br />
<br />
First, we want to figure out what planes appear to contain real emission. Open the CO data cube with the CASA viewer via:<br />
<br />
<source lang="python"><br />
# In CASA<br />
viewer("TWHydra_CO3_2line.image")<br />
</source><br />
<br />
This will open the CO(3-2) data cube as a raster image. Use the Animator pane to scroll through the cube, noting that the emission is mostly confined to ~40 channels near the center of the cube. There are several ways to make a more rigorous estimate. <br />
<br />
''By-Plane Inspection:'' First, you could increase the stretch of the image to identify by hand the first and last channels with visible emission. To do open the Data Display Options panel in the viewer (the wrench icon) and manually set the data range and/or scaling power cycles to highlight faint emission (note that you can also do this using the mouse and the color stretch - sun - icon). I left the scaling power cycles at 0 and set the data range to [-0.15, 0.15]. Then scroll through the cube and note the first and last channels with obvious emission. If I do this, I find emission from about plane 30 through plane 86.<br />
<br />
<center><br />
<gallery widths=250px heights=250px><br />
File:DataDisplayPanelStretch.png|The Data Display Panel used to set the color stretch, here from [-0.15, 0.15] with a linear (0) stretch. Setting the stretch to a negative value will highlight lower emission (a more logarithmic stretch). We got to this panel by clicking the wrench icon. You can also get from the dropdown menus (Data->Adjust).<br />
File:TWHydraCOChan33.png|Channel 33 of the CO(3-2) cube on this stretch. This is about the first channel to show clear emission.<br />
File:TWHydraCOChan80.png|Channel 80 of the CO(3-2) cube on this stretch. This is about the last channel to show clear emission.<br />
</gallery><br />
</center><br />
<br />
To make this more rigorous, you can work out a noise estimate before hand (we'll find it to be about 0.025 Jy/beam) and overplot a contour to indicate significant emission. You can do this by hand by using the Data Manager (folder icon) to open TWHydra_CO3_2line.image as a "contour map" and then using the Data Display Options to manipulate the contour levels. To show a 3-sigma contour with sigma=0.025 Jy/beam set the Relative Contour Levels to [3], the Base Contour Level to 0, and the Unit Contour Level to 0.025. You can display a 5-sigma contour or change the RMS analogously. Scrolling through the cube, I see 3-sigma emission from channels 33 through 80.<br />
<br />
<center><br />
<gallery widths=250px heights=250px><br />
File:DataDisplayPanelContour.png|The Data Display Panel now being used to set the contour levels. We set the contour unit to 0.025 and ask for a single contour at 3 times this value. Notice that we clicked the tab along the top of the window to select the contour map.<br />
File:TWHydraCOChan33Contour.png|Channel 33 of the CO(3-2) cube on this stretch with a 3-sigma contour overlaid.<br />
File:TWHydraCOChan80Contour.png|Channel 80 of the CO(3-2) cube on this stretch with a 3-sigma contour overlaid.<br />
</gallery><br />
</center><br />
<br />
''Look at a Spectrum:'' You can also look at a spectrum of the source. To do this, scroll to a channel with a lot of emission (I went to 57) and then use the mouse to drag out a rectangle. To do this, left click on the rectangle box in the mouse toolbar (this assigns the left mouse button to define rectangle regions). Then drag out a box around the emission using the same button. Now click the spectrum in the toolbar to open the spectral profile tool. You will see the average profile for the selected region. You can set the bottom axis to show channels. You can zoom in within the spectral profile by dragging out a rectangle. When I do this, I find significant emission from channels 40-80.<br />
<br />
<center><br />
<gallery widths=250px heights=250px><br />
File:TWHydraRectangle.png|Defining a rectangular region around the main emission from TW Hydra. First we clicked the rectangle button on the mouse toolbar, then we held that button and dragged out the purple rectangle. After defining the rectangle, you can drag it around or delete it with <ESC>.<br />
File:TWHydraSpectrum.png|The spectrum of this rectangular region viewed in the spectral profile browser. Note that the bottom axis has been set to show channels. To get this spectrum, we defined the rectangular region and then clicked the spectrum button on the toolbar.<br />
File:TWHydraSpectrumZoom.png|A zoom in on the spectrum. The line extends from channels ~40-80.<br />
</gallery><br />
</center><br />
<br />
''Look at a Declination-Frequency Projection (or an R.A.-Frequency Projection):'' This is probably the least reliable way to do this, but illustrates a neat capability of the viewer. Open the Data Display Options (click the wrench icon) and open the "display axes" and "hidden axes" tabs. Use the dropdown menus to change the display so that the x-axis shows Declination, the y-axis shows Frequency, and the z-axis shows "Right Ascension." Now you are scrolling through planes of intensity as a function of Declination and Frequency. Use the animator to move to R.A. plane ~ 50 (the center of the source) and pull the stretch up (in the wrench tool; I used [-0.15,0.15] again). Mouse over the image and look at the "Position Tracking" pane in the bottom of the image, noting the pixel extent of emission in the frequency axis of this plane. I see traces of emission from channel ~30-90.<br />
<br />
<center><br />
<gallery widths=250px heights=250px><br />
File:DataDisplayPanelAxes.png|The Data Display Panel, here used to set which axes are displayed. We have adjusted the y-axis to show frequency and the x-axis to show Declination. The z-axis is now Right Ascension, so that we will scroll through planes of Right Ascension.<br />
File:DataDisplayPanelPixels.png|The Data Display Panel, here we are setting coordinates to show "pixels" rather than world coordinates.<br />
File:TWHydraDecFreqPixels.png|A Frequency-Declination plane of the cube near the central R.A. pixel (50). The rough extent of emission may be simply read off the y-coordinate. As above, we could overplot a 3-sigma contour to add more rigor.<br />
</gallery><br />
</center><br />
<br />
<br />
You can repeat this exercise for the HCO+ (4-3) emission. Using the Spectral Profile Browser, I find that the line extends from about channel 40 through channel 75.<br />
<br />
====Make Moment 0 Maps====<br />
<br />
With an estimate of the spectral extent of emission, we are now ready to make moment 0 maps of the CO(3-2) and HCO+(4-3) lines. It is possible to do this using the 'Calculate Moments/Spectral Profile tool' in the viewer, but here we do this using the '''immmoments''' command,<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2line.image.mom0')<br />
immoments(imagename='TWHydra_CO3_2line.image',moments=[0],<br />
outfile='TWHydra_CO3_2line.image.mom0',<br />
chans='30~90')<br />
</source><br />
<br />
and<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom0')<br />
immoments(imagename='TWHydra_HCOplusline.image',moments=[0],<br />
outfile='TWHydra_HCOplusline.image.mom0',<br />
chans='40~75')<br />
</source><br />
<br />
immoments will sum across the channels specified by '''chans''' and produce new CASA images with the names specified by '''outfile'''. These can be inspected using the CASA viewer just like any other image. Note that unlike higher-order moments, the moment-0 map is robust to the inclusion of noisy signal-free channels (to within reason, you just decrease your signal-to-noise ratio) and indeed may be biased to miss faint emission by imposing a threshold. For that reason, we will eschew the '''includepix''' keyword as we make the moment 0 maps.<br />
<br />
Let's look at the output using the viewer. We can specify some of the display options in the call to start the viewer by using '''imview''' instead of '''viewer'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview( raster=[ {'file':'TWHydra_CO3_2line.image.mom0',<br />
'range':[0.0,10.],<br />
'scaling':-1},<br />
{'file':'TWHydra_HCOplusline.image.mom0',<br />
'range':[0.0,10.],<br />
'scaling':-1}], <br />
contour={'file':'TWHydra_contall_apcal.image', 'base':0, 'unit':0.0025, 'levels':[3,100]} )<br />
</source><br />
<br />
This does several things: It opens the viewer with raster images of the moment 0 maps of CO(3-2) and HCO+(4-3), sets the range of the color stretch from [0, 10] for each image and sets the scaling to be -1, which highlights faint emission compared to the normal (0) stretch. It also opens the continuum image as a contour and defines two contour levels. Once the viewer is open, you can blink between the images by going into the Animator pane, checking the white box next to "Images" to enable animation between images, and then sliding the bar back and forth. See the name of the image being shown change. You can also show two images side by side by going to the Viewer Canvas Manager (wrench+P icon) and opening the "Number of Panels" tab. Set the "Number of panels in x" to 2 and make sure that the Images box is checked in the Animator pane. You will see both images side by side.<br />
<br />
<center><br />
<gallery widths=250px heights=250px><br />
File:CanvasManagerPanels.png|The Viewer Canvas Manager, accessed via the Wrech+P icon (not to be confused with the plain old wrench). Here we have adjusted the number of x-panels from 1 to 2 so that we show the two moment maps side by side.<br />
File:TWHydraMoment0Maps.png|The moment-0 maps of CO(3-2) and HCO+(4-3) plotted side by side on the same color stretch, both with the continuum contours overlaid. To get the two images side by side, run the imview call above then set the number of panels to 2 and make sure that the Images box in the Animator Pane (here at the bottom of the window) is checked.<br />
File:DataDisplayImview.png|Data Display Panel after our imview call. Imview allowed us to set the stretch, range, and contour levels from the command line.<br />
</gallery><br />
</center><br />
<br />
====Estimate the Noise====<br />
<br />
We have already made a few offhand comparisons to the RMS noise. This quantity is often critical to know when exploring the data. Because the higher-order moments (especially the dispersion) are not at all robust to the inclusion of noise, we will calculate them only from regions of the cube that have intensity several times the RMS noise value or higher and are thus very likely to contain real astronomical emission. To proceed, we need to estimate the RMS noise for our data.<br />
<br />
The easiest way to do this is to calculate the RMS scatter of the data about zero from a part of the data that you suspect to be (mostly) free of astronomical signal. In a data cube the easiest way to do this will often be to identify signal-free planes (channels) of the cube and to simply calculate the RMS in these regions. We can do this straightforwardly using the '''imstat''' command in CASA.<br />
<br />
IMSTAT COMMAND<br />
<br />
IMSTAT OUTPUT<br />
<br />
Alternatively, you can get similar statistics on a region of an image or data cube selected manually inside the viewer. Drag out a region, for example a rectangle, using one of the mouse tools. Then <br />
<br />
IMAGES OF REGION SELECTION, OUTPUT, REGION MANAGER<br />
<br />
Suggested values for CO(3-2), HCO+(4-3), and continuum. Short discussion of dynamic range limited vs. line-free channels.<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
A note on primary beam corrections and noise.<br />
</pre><br />
<br />
====Higher Order Moment Maps for CO(3-2)====<br />
<br />
For higher order moments it is very important to set a conservative flux <br />
threshold. Typically something like 6 sigma, using sigma from peak line channel, works well.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2line.image.mom')<br />
os.system('rm -rf TWHydra_CO3_2line.image.mom.weighted_coord')<br />
os.system('rm -rf TWHydra_CO3_2line.image.mom.weighted_dispersion_coord')<br />
immoments(imagename='TWHydra_CO3_2line.image',moments=[1,2],<br />
outfile='TWHydra_CO3_2line.image.mom',<br />
chans='40~76',includepix=[0.3,100])<br />
</source><br />
<br />
Display all three moment maps in the viewer, overlaid with continuum contours.<br />
<br />
<source lang="python"><br />
# In CASA<br />
imview( raster=[ {'file':'TWHydra_CO3_2line.image.mom0'},<br />
{'file':'TWHydra_CO3_2line.image.mom.weighted_coord'},<br />
{'file':'TWHydra_CO3_2line.image.mom.weighted_dispersion_coord'} ], <br />
contour={'file':'TWHydra_contall_apcal.image', 'base':0, 'unit':0.0025, 'levels':[3,100]} )<br />
</source><br />
<br />
To see all three raster images simultaneously, open the viewer's Panel Display Options (the "P-wrench" icon in the viewer tool bar), click "Number of panels", and change "Number of panels in x" to 3. Then select the "Blink" toggle (radio button) from the Animator panel.<br />
<br />
<br />
[[Image:TWHya_CO3_2_moments.png|center|800px|frame|<caption>ALMA CO(3-2) moment maps, with white continuum contours at 3 and 100 sigma. From left to right: integrated intensity, intensity weighted velocity field, intensity weighted velocity dispersion are shown.</caption>]]<br />
<br />
<br />
Repeat for HCO+(4-3) which has a bit narrower velocity extent than the CO(3-2), but similar rms noise.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom0')<br />
immoments(imagename='TWHydra_HCOplusline.image',moments=[0],<br />
outfile='TWHydra_HCOplusline.image.mom0',<br />
chans='43~74')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom')<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom.weighted_coord')<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom.weighted_dispersion_coord')<br />
immoments(imagename='TWHydra_HCOplusline.image',moments=[1,2],<br />
outfile='TWHydra_HCOplusline.image.mom',<br />
chans='43~74',includepix=[0.3,100])<br />
</source><br />
<br />
Display all three moment maps in the viewer using the same method described above for the CO(3-2) maps.<br />
<br />
[[Image:TWHya_HCOp4_3_moments.png|center|800px|frame|<caption>ALMA HCO+(4-3) moment maps, with white continuum contours at 3 and 100 sigma. From left to right: integrated intensity, intensity weighted velocity field, intensity weighted velocity dispersion are shown.</caption>]]<br />
<br />
=====Exporting Fits Images=====<br />
<br />
If you want to analyze the data using another software package it is easy to convert from CASA format to FITS. <br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2line.image.fits')<br />
exportfits(imagename='TWHydra_CO3_2line.image',fitsimage='TWHydra_CO3_2line.image.fits')<br />
</source><br />
<br />
Although "FITS format" is supposed to be a standard, in fact most packages expect slightly different things from a FITS image. If you are having difficulty, try setting '''velocity=True''' and/or '''dropstokes=True'''.<br />
<br />
<source lang="python"><br />
# In CASA<br />
<br />
os.system('rm -rf TWHydra_HCOplusline.image.fits')<br />
exportfits(imagename='TWHydra_HCOplusline.image',fitsimage='TWHydra_HCOplusline.image.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2line.image.mom0.fits')<br />
exportfits(imagename='TWHydra_CO3_2line.image.mom0',<br />
fitsimage='TWHydra_CO3_2line.image.mom0.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2line.image.mom.weighted_coord.fits')<br />
exportfits(imagename='TWHydra_CO3_2line.image.mom.weighted_coord',<br />
fitsimage='TWHydra_CO3_2line.image.mom.weighted_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_CO3_2line.image.mom.weighted_dispersion_coord.fits')<br />
exportfits(imagename='TWHydra_CO3_2line.image.mom.weighted_dispersion_coord',<br />
fitsimage='TWHydra_CO3_2line.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom0.fits')<br />
exportfits(imagename='TWHydra_HCOplusline.image.mom0',<br />
fitsimage='TWHydra_HCOplusline.image.mom0.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom.weighted_coord.fits')<br />
exportfits(imagename='TWHydra_HCOplusline.image.mom.weighted_coord',<br />
fitsimage='TWHydra_HCOplusline.image.mom.weighted_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_HCOplusline.image.mom.weighted_dispersion_coord.fits')<br />
exportfits(imagename='TWHydra_HCOplusline.image.mom.weighted_dispersion_coord',<br />
fitsimage='TWHydra_HCOplusline.image.mom.weighted_dispersion_coord.fits')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_contall_apcal.image.fits')<br />
exportfits(imagename='TWHydra_contall_apcal.image',fitsimage='TWHydra_contall_apcal.image.fits')<br />
</source><br />
<br />
===Display Channel Maps===<br />
<br />
Using the Viewer you can make channel map figures. Start the Viewer and then open the CO(3-2) cube as a raster image and then the continuum as a contour image. Then we use the "wrench" icon and "P wrench" icons to set up the channel images, contour levels etc. (If you need help setting up the viewer for this image, see a screen shot of the viewer setup windows, below.)<br />
<br />
[[Image:TWHya_channel_co3_2.png|center|frame|<caption>Channel maps of the CO(3-2) emission with white continuum contours at 3 and 100 sigma.</caption>]]<br />
<br />
<br />
<br />
[[Image:Viewer_setup.png|center|thumb|500px|<caption>The setup windows looked like this</caption>]]<br />
<br />
<br />
Repeat for HCO+(4-3)<br />
<br />
[[Image:TWHya_channel_HCOp4_3.png|center|thumb|800px|<caption>Channel maps of the HCO+(4-3) emission with white continuum contours at 3 and 100 sigma. The color intensity scale is the same as that used for CO(3-2).</caption>]]<br />
<br />
<br />
<!--<br />
====Examine CO Contours on an HCO+ Raster====<br />
<br />
====Matched Movies====<br />
<br />
===Image Plane and Spectral Fits===<br />
<br />
====Fit a Gaussian to the Continuum====<br />
<br />
=====Fit In the Viewer=====<br />
<br />
=====Fit Using the Command Line=====<br />
<br />
====Fit a Gaussian to the CO Line====<br />
<br />
=====Fit In the Viewer=====<br />
<br />
=====Fit Using the Command Line=====<br />
<br />
===Extract a Sub-Image===<br />
<br />
===Align Image Cubes===<br />
--><br />
<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=TWHydraBand7_Calibration_4.3&diff=30879TWHydraBand7 Calibration 4.32021-08-19T20:55:25Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Spectral Line]]<br />
*'''This script assumes that you have downloaded TWHYA_BAND7_UnCalibratedMSAndTablesForReduction.tgz from [[TWHydraBand7#Getting_the_Data]]'''<br />
<br />
*'''An introduction to this data set is available at [[TWHydraBand7]].<br />
<br />
*'''This guide is designed for CASA 4.3. If you are using an older version of CASA please see [[TWHydraBand7_Calibration_4.2]].<br />
<br />
==Preparation==<br />
<br />
Once you have downloaded TWHYA_BAND7_UnCalibratedMSAndTablesForReduction.tgz, in a terminal unpack it with<br />
<br />
<source lang="bash"><br />
tar -zxvf TWHYA_BAND7_UnCalibratedMSAndTablesForReduction.tgz<br />
</source><br />
<br />
Please be aware that you will need about 100 GB of free space to do the data reduction. It is a good idea to check that now before you begin. Some of the plots described here require 9 GB of RAM; as little as 4 GB may be sufficient if additional data averaging is used when the plots are generated. When you are satisfied with the available drive space and memory on your machine:<br />
<br />
<source lang="bash"><br />
cd TWHYA_BAND7_UnCalibratedMSAndTablesForReduction<br />
</source><br />
<br />
'''NOTE: the calibration tables included in TWHYA_BAND7_UnCalibratedMSAndTablesForReduction CANNOT be used with CASA 4.3, they should be discarded, or moved out of the working directory.'''<br />
<br />
Then start CASA. <br />
<br />
<source lang="bash"><br />
casapy<br />
</source><br />
<br />
== Confirm your version of CASA==<br />
<br />
This guide has been written for CASA release 4.3. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In CASA<br />
version = casadef.casa_version<br />
print "You are using " + version<br />
if (version < '4.3.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING."<br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Install Analysis Utilities==<br />
<br />
Analysis Utilities (or analysisUtils for short) is a small set of Python scripts that provide a number of analysis and plotting utilities for ALMA data reduction. This guide uses a few of these utilities. They are very easy to install (just download and untar). See <br />
<br />
http://casaguides.nrao.edu/index.php?title=Analysis_Utilities<br />
<br />
for a full description and download instructions. If you do not wish to do this, see the [http://casaguides.nrao.edu/index.php?title=TWHydraBand7_Calibration_for_CASA_3.3 CASA 3.3 version of the guide] for alternative (but slow) plotting options. Analysis Utilities are updated frequently so if its been a while since you installed it, its probably worth doing it again. If you are at an ALMA site or ARC, the analysis utilities are probably already installed and up to date.<br />
<br />
==Fix Titan's coordinates==<br />
<br />
'''Note that in the near future this step will be unnecessary.''' <br />
<br />
It is temporarily necessary to correct the positions of ephemeris objects observed during ALMA Cycle 0, because the control software revisions earlier than 9.0 erroneously write the data with coordinates of 00:00:00.0000 +00.00.00.0000 (J2000). For this dataset, we need to correct the position of the absolute flux calibrator Titan, as we can<br />
see from the first {{listobs}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f X3c1.ms.listobs')<br />
listobs('X3c1.ms', verbose=True, listfile='X3c1.ms.listobs') <br />
</source><br />
<br />
You can view the listobs output files using command "less" from the CASA prompt.<br />
<br />
<pre style="background-color: #fffacd;"><br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 23625<br />
1 none Titan 00:00:00.00000 +00.00.00.0000 J2000 1 4842<br />
2 none TW Hya 11:01:51.84498 -34.42.17.1609 J2000 2 175176<br />
3 none J1147-382=QSO 11:47:01.38151 -38.12.11.1179 J2000 3 14832<br />
4 none J1037-295=QSO 10:37:16.08989 -29.34.02.9888 J2000 4 59877<br />
</pre><br />
<br />
To correct Titan's position, we will use the task [http://casa.nrao.edu/stable/docs/TaskRef/fixplanets-task.html fixplanets], which writes the actual coordinates at the instant of the first observation. This step takes a while because we are recalculating the uvw's accordingly.<br />
<source lang="python"><br />
# In CASA<br />
fixplanets(vis='X3c1.ms', field='Titan', fixuvw=True)<br />
</source><br />
<br />
Check the header information again with {{listobs}} to confirm that the information for Titan has changed:<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -f X3c1.ms.listobs')<br />
listobs('X3c1.ms', verbose=True, listfile='X3c1.ms.listobs') <br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 23625<br />
1 none Titan 12:49:25.93349 -02.22.41.5638 J2000 1 4842<br />
2 none TW Hya 11:01:51.84498 -34.42.17.1609 J2000 2 175176<br />
3 none J1147-382=QSO 11:47:01.38151 -38.12.11.1179 J2000 3 14832<br />
4 none J1037-295=QSO 10:37:16.08989 -29.34.02.9888 J2000 4 59877<br />
</pre><br />
<br />
Now fix the other two measurement sets similarly:<br />
<source lang="python"><br />
# In CASA<br />
fixplanets(vis='X5d8.ms', field='Titan', fixuvw=True)<br />
fixplanets(vis='X7ef.ms', field='Titan', fixuvw=True)<br />
</source><br />
<br />
==Observing Log and Priors==<br />
<br />
Below is some information that may not be obvious from the data<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
Like at most telescopes, the ALMA operators and astronomers at the site log any significant issues <br />
that were noticed from the online system during the observation. For these Band 7 observations of <br />
TW Hya on April 22, 2011, the only issue of note was that antenna DV04 could not observe at Band 7 <br />
and will need to be flagged. <br />
</pre><br />
<br />
First let's examine the basic properties of the data using {{listobs}}. Among other information we <br />
will be able to deduce what the source ids are and what each source was used for:<br />
<br />
<pre style="background-color: #E0FFFF;"><br />
3c279 is the bandpass calibrator (field id=0)<br />
Titan is the absolute flux calibrator (field id=1)<br />
J1147-382 is the secondary phase calibrator (field id=3)<br />
J1037-295 is the primary phase calibrator (field id=4)<br />
</pre><br />
<br />
<source lang="python"><br />
# In CASA<br />
data=['X3c1.ms','X5d8.ms','X7ef.ms']<br />
for vis in data:<br />
os.system('rm '+vis+'.listobs')<br />
listobs(vis, verbose=True, listfile=vis+'.listobs') <br />
</source><br />
<br />
You can view the {{listobs}} output files using command '''less''' from the CASA prompt. <br />
Below we copy select parts of the {{listobs}} output files for reference. <br />
<br />
<pre style="background-color: #fffacd;"><br />
X3c1.ms:<br />
Data records: 278352 Total integration time = 6109.06 seconds<br />
Observed from 22-Apr-2011/00:01:52.9 to 22-Apr-2011/01:43:42.0 (UTC)<br />
<br />
X5d8.ms:<br />
Data records: 278406 Total integration time = 5995.01 seconds<br />
Observed from 22-Apr-2011/01:48:05.8 to 22-Apr-2011/03:28:00.8 (UTC)<br />
<br />
X7ef.ms:<br />
Data records: 255717 Total integration time = 5407.49 seconds<br />
Observed from 22-Apr-2011/03:30:39.7 to 22-Apr-2011/05:00:47.2 (UTC)<br />
<br />
Fields: 5<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 23625<br />
1 none Titan 12:49:25.97588 -02.22.41.3024 J2000 1 4842<br />
2 none TW Hya 11:01:51.84498 -34.42.17.1609 J2000 2 175176<br />
3 none J1147-382=QSO 11:47:01.38151 -38.12.11.1179 J2000 3 14832<br />
4 none J1037-295=QSO 10:37:16.08989 -29.34.02.9888 J2000 4 59877 <br />
<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs<br />
0 4 TOPO 184550 1500000 7500000 I<br />
1 128 TOPO 355740.062 15625 2000000 XX YY<br />
2 1 TOPO 356716.625 1796875 1796875 XX YY<br />
3 128 TOPO 356507.813 15625 2000000 XX YY<br />
4 1 TOPO 357484.375 1796875 1796875 XX YY<br />
5 128 TOPO 346792.187 15625 2000000 XX YY<br />
6 1 TOPO 345784.375 1796875 1796875 XX YY<br />
7 128 TOPO 345182.438 15625 2000000 XX YY<br />
8 1 TOPO 344174.625 1796875 1796875 XX YY<br />
9 128 TOPO 344386.763 15625 2000000 XX YY<br />
10 1 TOPO 343378.95 1796875 1796875 XX YY<br />
11 128 TOPO 346324.263 15625 2000000 XX YY<br />
12 1 TOPO 345316.45 1796875 1796875 XX YY<br />
13 128 TOPO 354402.388 15625 2000000 XX YY<br />
14 1 TOPO 355378.95 1796875 1796875 XX YY<br />
15 128 TOPO 356402.388 15625 2000000 XX YY<br />
16 1 TOPO 357378.95 1796875 1796875 XX YY<br />
17 3840 TOPO 356497.936 122.070312 468750 XX YY<br />
18 1 TOPO 356732.189 468750 468750 XX YY<br />
19 3840 TOPO 357734.314 122.070312 468750 XX YY<br />
20 1 TOPO 357499.939 468750 468750 XX YY<br />
21 3840 TOPO 346034.314 122.070312 468750 XX YY<br />
22 1 TOPO 345799.939 468750 468750 XX YY<br />
23 3840 TOPO 343955.936 122.070312 468750 XX YY<br />
24 1 TOPO 344190.189 468750 468750 XX YY<br />
<br />
Antennas: 9:<br />
Name Station Diam. Long. Lat.<br />
0 DV04 J505 12.0 m -067.45.18.0 -22.53.22.8<br />
1 DV06 T704 12.0 m -067.45.16.2 -22.53.22.1<br />
2 DV07 J510 12.0 m -067.45.17.8 -22.53.23.5<br />
3 DV08 T703 12.0 m -067.45.16.2 -22.53.23.9<br />
4 DV09 N602 12.0 m -067.45.17.4 -22.53.22.3<br />
5 DV10 N606 12.0 m -067.45.17.1 -22.53.23.6<br />
6 PM01 T702 12.0 m -067.45.18.6 -22.53.24.1<br />
7 PM02 T701 12.0 m -067.45.18.8 -22.53.22.2<br />
8 PM03 J504 12.0 m -067.45.17.0 -22.53.23.0<br />
</pre><br />
<br />
Things to Notice from above: <br />
*There are three different data sets acquired close together in time. We will calibrate each dataset independently and then combine the calibrated data prior to imaging. <br />
*Titan appears to have slightly different positions in each dataset because it is a moving body. In fact, at the time of observation, due to temporary data capture issue in the ALMA system, the position was written has (0,0). We have pre-corrected this error using fixplanets. Later in the tutorial, we will run fixplanets again to force all three datasets to have the same fixed position for Titan.<br />
*There are a lot of spectral windows! Don't be alarmed. In ALMA data, <br />
**spw=0 is always reserved for the water vapor radiometry data.<br />
**spw=9, 11, 13, 15 are the wideband (TDM) correlator mode with 128 channels per spw default settings for doing pointing and this is all they are used for in these data.<br />
**Currently Tsys data can only be taken in the wide (TDM) correlator mode with 128 channels per spw, these correspond to spws=1, 3, 5, 7 above. The frequencies of these TDM spws were carefully chosen so the the narrowband FDM spws fall within them.<br />
** The 0.5 GHz, high spectral resolution spws that contain all the real source data (science target and calibrators) are 17, 19, 21, and 23<br />
**For quicklook purposes, the ALMA online system produces a channel averaged spw for each channelized spw, these correspond to spw=2,4,6,8,10,12,14,16,18,20,22,24 and shouldn't be used for anything in post-processing as bandpass calibration etc has not been applied.<br />
<br />
In the portion of the {{listobs}} output that shows the timerange, the final column gives the scan intent. It is important to have a look at this because you can flag and do other operations in CASA using the scan intent as a selection option. Also, these intents will be used in the future for pipeline processing. You will usually want to use them to flag the CALIBRATE_POINTING* and CALIBRATE_ATMOS* (the latter correspond to the hot and cold load measurements used to calculate Tsys) data. This is essential for science or calibration data taken only in TDM (wide bandwidth) mode, i.e. the same mode that the pointing and Tsys data are taken in. For the current data however, that calibration data is recorded in different spws from the useful FDM (narrow bandwidth) data, so we'll be able to separate them by splitting them off at the final step prior to imaging.<br />
<br />
<br />
[[Image:Ant_pos.png|thumb|<caption>Antenna locations for the first dataset.</caption>]]<br />
<br />
<br />
Before we move on, its also nice to get an idea where the antennas are with respect to each other. We can use {{plotants}} to plot their positions. It is a good idea to check each file since an antenna may have entered or exited the array between observations.<br />
<source lang="python"><br />
# In CASA<br />
for vis in data:<br />
plotants(vis=vis,figfile=vis+'.plotants.png')<br />
</source><br />
<br />
You can view the output images using any standard image viewer.<br />
<br />
==Create the Tsys and WVR Tables, and Examine ==<br />
<br />
Each ms include tables that contain the Tsys and WVR measurements associated with the visibility data.<br />
We must create calibration tables from these measurements. First we create the WVR calibration tables <br />
by running the task [http://casa.nrao.edu/stable/docs/TaskRef/wvrgcal-task.html wvrgcal]. This task <br />
examines the data for each ms as a whole and creates a calibration<br />
table containing only phase corrections for each antenna and spw. <br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all = ['X3c1','X5d8','X7ef']<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.wvr')<br />
wvrgcal(vis=asdm+'.ms', caltable=asdm+'.wvr', segsource=True, toffset=-1)<br />
</source><br />
<br />
Next we create the Tsys tables by running {{gencal}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all = ['X3c1','X5d8','X7ef']<br />
for asdm in basename_all:<br />
os.system('rm -rf '+asdm+'.tsys')<br />
gencal(vis=asdm+'.ms', caltable=asdm+'.tsys', caltype='tsys')<br />
</source><br />
<br />
It is very important that Tsys measurements taken as close in time and elevation as possible to a particular source are applied. To save telescope time, Tsys measurements are sometimes intentionally skipped in the scheduling block for specific sources. So the first step is to establish which sources have their own Tsys measurements and which do not. To do this, you may use the intent parameter of {{listobs}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all = ['X3c1','X5d8','X7ef']<br />
for asdm in basename_all:<br />
listobs(vis=asdm+'.ms', verbose=False, selectdata=True, intent='CALIBRATE_ATMOS*')<br />
</source><br />
<br />
<br />
[[File:X3c1.tsys_vs_time_3.4.png|thumb|<caption>Tsys vs. time for spw 1 of the X3c1.ms data.</caption>]]<br />
<br />
[[File:X5d8.tsys_vs_time_3.4.png|thumb|<caption>Tsys vs. time for spw 1 of the X5d8.ms data.</caption>]]<br />
<br />
[[File:X7ef.tsys_vs_time_3.4.png|thumb|<caption>Tsys vs. time for spw 1 of the X7ef.ms data.</caption>]]<br />
<br />
<br />
<pre style="background-color: #fffacd;"><br />
Fields: 4<br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 1116 <br />
1 none Titan 12:49:25.97588 -02.22.41.3024 J2000 1 1125 <br />
2 none TW Hya 11:01:51.84498 -34.42.17.1609 J2000 2 6795 <br />
4 none J1037-295=QSO 10:37:16.08989 -29.34.02.9888 J2000 4 6777 <br />
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)<br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 4 TOPO 184550 1500000 7500000 I <br />
1 128 TOPO 355740.062 15625 2000000 XX YY <br />
2 1 TOPO 356716.625 1796875 1796875 XX YY <br />
3 128 TOPO 356507.813 15625 2000000 XX YY <br />
4 1 TOPO 357484.375 1796875 1796875 XX YY <br />
5 128 TOPO 346792.187 15625 2000000 XX YY <br />
6 1 TOPO 345784.375 1796875 1796875 XX YY <br />
7 128 TOPO 345182.438 15625 2000000 XX YY <br />
8 1 TOPO 344174.625 1796875 1796875 XX YY <br />
</pre><br />
<br />
Because field '3' (the secondary phase calibrator) does not appear in the list, it has no tsys. Since the secondary phase calibrator is close on the sky to the primary phase calibrator and observed near in time, we will transfer the Tsys from the primary to the secondary phase calibrator. <br />
Also notice that spws 1,3,5,7 are the only multi-channel spws that contain Tsys data. At present, Tsys measurements can only be obtained with TDM spectral setups (128 channels/dual pol). Because our science target is observed in FDM (3840 channels/dual pol), we will need to interpolate the measurements from TDM to FDM. But first, to inspect the Tsys vs. time, we use {{plotcal}}:<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename_all = ['X3c1','X5d8','X7ef']<br />
tsysfields=['0','1','2','4']<br />
os.system('mkdir cal_plots; mkdir cal_plots/Tsys_plots') <br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. time for "+asdm<br />
plotcal(caltable=asdm+'.tsys', xaxis='time',yaxis='tsys',<br />
antenna='1~8',plotrange=[0,0,100,500],<br />
iteration='antenna',subplot=421,poln='',spw='1:50~50',<br />
figfile='cal_plots/Tsys_plots/'+asdm+'.tsys_vs_time.png',showgui=True)<br />
if (asdm != basename_all[-1]):<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
To inspect Tsys vs. frequency, we use [[plotbandpass]]. There are many different options for plotbandpass, and we demonstrate some of the most useful ones. The option '''showtsky=True''' will overlay the effective temperature curve from the atmospheric model. This shows us the location of any atmospheric lines which will appear as emission lines in the autocorrelation data (from which the Tsys spectra are derived) from all antennas. In the<br />
example below, we set '''interactive=False''', which will produce many pages of plots as<br />
individual pngs but will only show the final page in the graphics window. To view all the pages we use the image viewer '''eog''' (Eye of GNOME) available on Linux. The alternative is to set '''interactive=True''' and examine the plots in the casa graphics window as they are produced.<br />
<br />
<br />
[[File:X3c1.tsys.field0.DV04.spw1.t1.png|thumb|<caption>Tsys vs. frequency on 3C279 for the X3c1.ms data, spw=1.</caption>]]<br />
<br />
<br />
[[File:X3c1.tsys.field0.DV04.spw3.t1.png|thumb|<caption>Tsys vs. frequency on 3C279 for the X3c1.ms data, spw=3.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
basename_all = ['X3c1','X5d8','X7ef']<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. frequency for "+asdm<br />
for spw in [1,3,5,7]:<br />
for field in ['0','1']:<br />
plotbandpass(caltable=asdm+'.tsys', xaxis='freq',yaxis='amp',<br />
showtsky=True,subplot=42,field=field,<br />
spw=spw, interactive=False, plotrange=[0,0,0,0],<br />
figfile='cal_plots/Tsys_plots/'+asdm+'.tsys.field%s.png'%field,<br />
buildpdf=False)<br />
</source><br />
<br />
In '''Tsys vs. frequency on 3C279 for the X3c1.ms data, spw=1''' and '''Tsys vs. frequency on 3C279 for the X3c1.ms data, spw=3''', we see the effect of atmospheric ozone lines on the Tsys measurements associated with 3C279 in spw 1 and 3.<br />
<br />
For sources that have multiple Tsys measurements (typically the primary phase calibrator and/or the science target) one can use '''overlay="time"''' to overlay all the measurements for that source. This can help to identify any anomalous Tsys scans. As a fraction of the plots produced by this loop, here we show the Tsys recorded on spw=1 for TW Hya from the first dataset.<br />
<br />
[[File:X3c1.tsys.field2.DV04.spw1.png|thumb|<caption>Tsys vs. freq for field 2, spw 1 of the X3c1.ms data (antennas 0..3).</caption>]]<br />
<br />
<br />
[[File:X3c1.tsys.field2.DV09.spw1.png|thumb|<caption>Tsys vs. freq for field 2, spw 1 of the X3c1.ms data (antennas 4..7).</caption>]]<br />
<br />
[[File:X3c1.tsys.field2.PM03.spw1.png|thumb|<caption>Tsys vs. freq for field 2, spw 1 of the X3c1.ms data (antenna 8).</caption>]]<br />
<br />
<br />
<source lang="python"><br />
basename_all = ['X3c1','X5d8','X7ef']<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. frequency for "+asdm<br />
for spw in [1,3,5,7]:<br />
for field in ['2','4']:<br />
plotbandpass(caltable=asdm+'.tsys', xaxis='freq',yaxis='amp',<br />
showtsky=True,subplot=22,field=field,<br />
overlay='time',spw=spw, interactive=False, plotrange=[0,0,0,0],<br />
figfile='cal_plots/Tsys_plots/'+asdm+'.tsys.field%s.png'%field,<br />
buildpdf = False)<br />
</source><br />
<br />
Alternatively, the '''overlay="antenna"''' option will quickly reveal any Antennas with discrepant Tsys values, as they<br />
will be obvious outliers. For FDM projects like this one, using the '''showfdm=True''' option can be helpful, as it<br />
will show where in the TDM bandpass each FDM spw was located. <br />
<br />
[[Image:X3c1.tsys.field0.spw1.t1.png|thumb|<caption>Tsys vs Freq for field 0, spw 1 of the X3c1.ms data with all antennas overlaid.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
tsysfields=['0','1','2','4'] <br />
basename_all = ['X3c1','X5d8','X7ef']<br />
for asdm in basename_all:<br />
print "Plotting Tsys vs. frequency for "+asdm<br />
for spw in [1,3,5,7]:<br />
for field in tsysfields:<br />
plotbandpass(caltable=asdm+'.tsys', xaxis='freq',yaxis='amp',<br />
showtsky=True,subplot=11,field=field,<br />
overlay='antenna',spw=spw, interactive=False,<br />
figfile='cal_plots/Tsys_plots/'+asdm+'.tsys.field%s.png'%field,<br />
buildpdf=False,showfdm=True)<br />
</source><br />
<br />
<br />
Note that DV04 consistently has a significantly higher Tsys than the rest of the antennas. We already know from the Observation log that<br />
we need to flag that antenna, but had we not had that information, examining this plot would have demonstrated the situation.<br />
<br />
==Apply the Tsys and WVR Tables, and Split ==<br />
<br />
Next we apply the calibrations. For the (4) sources with Tsys measurements, we use the '''field''' and '''gainfield''' parameters of {{applycal}} to apply solutions only to themselves. {{applycal}} has the ability to interpolate gain tables from one spw onto another spw. In this case, we want to interpolate the TDM Tsys values in spws 1,3,5,7 onto the FDM spws 17,19,21,23. We use the '''interp''' parameter to request linear interpolation in the time dimension and spline interpolation in the channel/frequency dimension. To map the solutions for each spw, we use the '''spwmap''' parameter, which requires a list of all spws for each gain table to be applied, or a blank list ([]) which is a shorthand notation to apply the solution for each spw to itself. While applying Tsys, we will also apply the wvr calibration tables; wvr data is usually always present for all sources. Below we first set some definitions so we can loop over the correct parameters:<br />
<br />
<source lang="python"><br />
# In CASA<br />
fdm='17,19,21,23'<br />
nocal='3' # This source had no Tsys measurements associated with it <br />
tsysfields=['0','1','2','4'] <br />
tsysmap = range(25)<br />
tsysmap[17] = 1<br />
tsysmap[19] = 3<br />
tsysmap[21] = 5<br />
tsysmap[23] = 7<br />
for asdm in basename_all:<br />
for field in tsysfields:<br />
applycal(vis=asdm+'.ms', spw=fdm, field=field, gainfield=field,<br />
gaintable=[asdm+'.tsys', asdm+'.wvr'], spwmap=[tsysmap,[]],<br />
interp=['linear,spline','nearest'], flagbackup=False, calwt=True)<br />
</source><br />
<br />
For the sources without Tsys (i.e. field=3), couple them with the best available source with Tsys (i.e. field=4).<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
applycal(vis=asdm+'.ms', spw=fdm, field=nocal,<br />
gaintable=[asdm+'.tsys', asdm+'.wvr'], spwmap=[tsysmap,[]],<br />
gainfield=['4',nocal], interp=['linear,spline','nearest'],flagbackup=False,<br />
calwt=True)<br />
</source><br />
<br />
Check Tsys application with plotms. First we make plots with '''ydatacolumn='data'''' to look at the raw data. For <br />
simplicity we'll just look at the two strongest sources 3C279 and Titan.<br />
<br />
<br />
[[Image:X5d8_wvrtsys_corrected_spw19.png|thumb|<caption>Spectral plots of 3C279 (black) and Titan (magenta) for spw=19 after Tsys and wvr correction.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X5d8.ms',spw='17,19,21,23',xaxis='frequency',yaxis='amp',field='0,1',<br />
antenna='*&*',avgtime='1e8',avgscan=True,coloraxis='field',iteraxis='spw',<br />
xselfscale=True,ydatacolumn='data',showgui=True)<br />
</source><br />
<br />
[[Image:X5d8_wvrtsys_corrected_spw21.png|thumb|<caption>Spectral plots of 3C279 (black) and Titan (magenta) for spw=21 after Tsys and wvr correction.</caption>]]<br />
<br />
<br />
'''Note''': the antenna='*&*' is specifying that we only want to see the cross-correlation data (i.e. not the autocorrelations).<br />
<br />
Use the '''green arrows''' on the {{plotms}} display to cycle through the 4 FDM spws. <br />
<br />
Now change '''Data Column''' to '''corrected''' on the "Axes" tab (this is the same as ydatacolumn in the task) and hit '''Plot''' button on the GUI. Flip through again.<br />
<br />
'''Things to notice:'''<br />
* The amplitude scale has changed due to application of Tsys<br />
* spw=17 is clean and flat<br />
* spw=19 shows a strong dip around channel 3000 (channels can be plotted by going to the Axes tab, and changing X Axis to Channel, then hit the Plot button)-- this is an atmospheric ozone absorption feature at 357.6 GHz. In the future, high spectral resolution Tsys measurements may help to remove the shape of the absorption, but until then we will need to flag this part of the spectrum.<br />
* spw=21 Titan shows both a narrow and a broad spectral feature. The broad feature is CO(3-2) and it completely fills the 0.5 GHz band so there are no line-free channels! Titan must be handled carefully when it is used as a calibrator because it contains strong line emission. Indeed, we will not be able to use the absolute flux transfer from this spw for other targets. When possible Titan should be avoided for absolute flux calibration. For these observations, no other suitable absolute flux calibrator was available.<br />
* spw=23 shows another strong line from Titan.<br />
<br />
Beyond these things to notice, the application of Tsys and wvr seem to have gone well. We can now split off only the narrow band FDM spectral windows for further processing.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename_all:<br />
newvis = asdm + '_wvrtsys.ms'<br />
os.system('rm -rf ' + newvis)<br />
split(vis=asdm+'.ms',outputvis=newvis,datacolumn='corrected',spw=fdm)<br />
</source><br />
<br />
'''Note''': After {{split}}, the spectral windows will be renumbered to 0, 1, 2, 3 corresponding to the old spw=17, 19, 21, 23, see {{listobs}} excerpt below.<br />
<br />
<pre style="background-color: #fffacd;"><br />
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs <br />
0 3840 TOPO 356497.936 122.070312 468750 XX YY <br />
1 3840 TOPO 357734.314 122.070312 468750 XX YY <br />
2 3840 TOPO 346034.314 122.070312 468750 XX YY <br />
3 3840 TOPO 343955.936 122.070312 468750 XX YY <br />
</pre><br />
<br />
==Initial Inspection and Flagging==<br />
<br />
From here we begin to operate on the *wvrtsys.ms files. <br />
<br />
Before we begin flagging data, we unflag all data in the measurement sets. If this is your first pass through this section of the guide, no data in the measurement sets will be flagged, and you can skip this step. If you are rerunning the flagging commands below, this will reset your flag columns to their original state.<br />
<br />
<source lang="python"><br />
# In CASA<br />
splitdata=['X3c1_wvrtsys.ms','X5d8_wvrtsys.ms','X7ef_wvrtsys.ms']<br />
for vis in splitdata:<br />
flagdata(vis=vis, mode='unflag', action='apply', flagbackup=False)<br />
</source><br />
<br />
ALMA data contains both the cross correlation and autocorrelation data. Presently nothing more can be done with the autocorrelation data so we flag it. Additionally, for smaller configurations of the array, and northerly sources one antenna can ''shadow'' another, blocking its view. This data also needs to be flagged. Finally, the observing log told us DV04 was not behaving properly at Band 7 and shouldn't be used so we flag that as well.<br />
<br />
<source lang="python"><br />
# In CASA<br />
splitdata=['X3c1_wvrtsys.ms','X5d8_wvrtsys.ms','X7ef_wvrtsys.ms']<br />
for vis in splitdata:<br />
flagdata(vis=vis, mode='manual', autocorr=True, action = 'apply', flagbackup=True)<br />
flagdata(vis=vis, mode='shadow',tolerance=12.0, action='apply', flagbackup=False)<br />
flagdata(vis=vis, mode='manual', antenna='DV04', action='apply', flagbackup=False)<br />
</source><br />
<br />
Next we inspect the data with {{plotms}} for additional issues that need flagging, first looking at amplitude vs. time for each of the three datasets. Use the '''green''' arrows on the plotms gui to cycle through spws. <br />
<br />
[[Image:Tsyswvr_time_spw0_X3x1.png|thumb|<caption>Amplitude as a function of time for X3c1_wvrtsys.ms, spw=0</caption>]]<br />
<br />
<br />
'''Note''': when '''iteraxis''' is invoked in {{plotms}}, unzoom does not always work properly. If you have trouble with this, flip to next page and then back to current one. This usually fixes the problem.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X3c1_wvrtsys.ms',spw='',xaxis='time',yaxis='amp',field='',avgchannel='3840',<br />
coloraxis='field',iteraxis='spw',showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X5d8_wvrtsys.ms',spw='',xaxis='time',yaxis='amp',field='',avgchannel='3840',<br />
coloraxis='field',iteraxis='spw',showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X7ef_wvrtsys.ms',spw='',xaxis='time',yaxis='amp',field='',avgchannel='3840',<br />
coloraxis='field',iteraxis='spw',showgui=True)<br />
</source><br />
<br />
[[Image:Tsyswvr_time_spw2corr_X3x1.png|thumb|<caption>Amplitude as a function of time for X3x1_wvrtsys.ms, spw=2 with colorize='corr'.</caption>]]<br />
<br />
<br />
'''Things to Notice:'''<br />
* In spw=2 there are some low points in all three data files. Use the "Mark Regions" box (left of center on bottom row of icons) to draw small box on low points. Then click the magnifying glass icon to the right of center. The output will be sent to the logger window. '''Note''': keep the marked region small or you can overwhelm the buffer. Locate suggests correlation YY on PM03 is the culprit. <br />
<br />
To see this more clearly you can rerun {{plotms}} with '''coloraxis='field'''' switched to '''coloraxis='corr''''<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X3c1_wvrtsys.ms',spw='2',xaxis='time',yaxis='amp',field='',avgchannel='3840',<br />
coloraxis='corr',showgui=True)<br />
</source><br />
<br />
To further check, you can exclude PM03 from the plot using '''antenna='!PM03'.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X3c1_wvrtsys.ms',spw='2',xaxis='time',yaxis='amp',field='',avgchannel='3840',<br />
coloraxis='corr',antenna='!PM03',showgui=True)<br />
</source><br />
<br />
Having confirmed the problem antenna, we flag it below. <br />
<br />
<source lang="python"><br />
# In CASA<br />
splitdata=['X3c1_wvrtsys.ms','X5d8_wvrtsys.ms','X7ef_wvrtsys.ms']<br />
for vis in splitdata:<br />
flagdata(vis=vis, mode='manual', action= 'apply', flagbackup=True,<br />
spw='2',<br />
antenna='PM03',<br />
correlation='YY')<br />
</source><br />
<br />
'''Next we need to inspect the spectral properties of the data to look for issues.'''<br />
<br />
First we look at phase as a function of frequency for a well behaved antenna like DV06. <br />
'''coloraxis='baselines'''' (note this parameter is called '''Colorize''' in the GUI "Display" tab) will show each baseline as a different color. Significant delay errors or baseline errors will show up as phase slopes in these plots. Again you will want to page through the spectral windows.<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X3c1_wvrtsys.ms',spw='',xaxis='frequency',yaxis='phase',field='0',antenna='DV06',<br />
avgtime='1e8',avgscan=True,coloraxis='baseline',iteraxis='spw',xselfscale=True,showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X5d8_wvrtsys.ms',spw='',xaxis='frequency',yaxis='phase',field='0',antenna='DV06',<br />
avgtime='1e8',avgscan=True,coloraxis='baseline',iteraxis='spw',xselfscale=True,showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X7ef_wvrtsys.ms',spw='',xaxis='frequency',yaxis='phase',field='0',antenna='DV06',<br />
avgtime='1e8',avgscan=True,coloraxis='baseline',iteraxis='spw',xselfscale=True,showgui=True)<br />
</source><br />
<br />
Overall these plots look good with no large delays or baseline errors apparent.<br />
<br />
<br />
[[Image:Birdies_spw2_X3x1.png|thumb|<caption>Phase calibrators (brown and green) and TW Hya (orange) showing both real CO(3-2) emission in TW Hya and weak ''birdie'' spectral features in all three sources for spw=2.</caption>]]<br />
<br />
<br />
With such high spectral resolution data, one often sees occasional "birdies" in the data. These are typically very narrow weak spectral features that are internally generated in the system. Over time as we track down issues the number of birdies should decrease, but you should always check. This is easiest to do by looking at amplitude vs. frequency for sources observed over a long time range (for sensitivity). It is helpful to look at more than one source to verify the nature of the emission. Below we look at both calibrators (brown and green) and TW Hya (orange). If you have enough sensitivity birdies should be seen on all sources, however, be careful not to mistake real line emission for a birdie even on a calibrator - they can be present as we've seen for Titan.<br />
<br />
[[Image:Birdies_spw3_X3x1.png|thumb|<caption>Phase calibrators (brown and green) and TW Hya (orange) showing only weak birdie spectral features in spw=3.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X3c1_wvrtsys.ms',spw='',xaxis='channel',yaxis='amp',field='2,3,4',<br />
avgtime='1e8',avgscan=True,coloraxis='field',iteraxis='spw',xselfscale=True,yselfscale=True,showgui=True)<br />
</source><br />
<br />
Do a quick check that other two datasets are the same (they are usually similar if the source of internal resonance hasn't changed in the meantime).<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X5d8_wvrtsys.ms',spw='',xaxis='channel',yaxis='amp',field='2,3,4',<br />
avgtime='1e8',avgscan=True,coloraxis='field',iteraxis='spw',xselfscale=True,yselfscale=True,showgui=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X7ef_wvrtsys.ms',spw='',xaxis='channel',yaxis='amp',field='2,3,4',<br />
avgtime='1e8',avgscan=True,coloraxis='field',iteraxis='spw',xselfscale=True,yselfscale=True,showgui=True)<br />
</source><br />
<br />
Now you can zoom in on each region to get channels for flagging, or to save time just go ahead and run the flagging command below where this has been done for you.<br />
<br />
<source lang="python"><br />
# In CASA<br />
splitdata=['X3c1_wvrtsys.ms','X5d8_wvrtsys.ms','X7ef_wvrtsys.ms']<br />
for vis in splitdata:<br />
flagdata(vis=vis, mode='manual', action='apply', flagbackup=True,<br />
spw='0:1067~1068;1279~1280;2367~2368;3775~3776, 1:1279~1280;2367~2368;3775~3776, 2:1279~1280;3775~3776, 3:831~832;1535~1536;2367~2368;3775~3776;3839~3839')<br />
</source><br />
<br />
Replot in {{plotms}} to check that flagging has been done as anticipated.<br />
<br />
==Flag Calibrator Spectral Features==<br />
<br />
In this section, we will flag spectral features in the calibrators so they do not skew the calibration solutions. First make plots of 3C279 and Titan in channel space to get channels for flagging. <br />
<br />
[[Image:3C279_Titan_spw1.png|thumb|<caption>Spectral plots of 3C279 (black) and Titan (magenta) for spw=1 after Tsys and wvr correction.</caption>]]<br />
<br />
<br />
[[Image:3c279_meso_freq.png|thumb|<caption>Zoomed spectral plot of 3C279 showing mesospheric absorption of CO(3-2) in frequency space.</caption>]]<br />
<br />
[[Image:3c279_meso_chann.png|thumb|<caption>Zoomed spectral plot of 3C279 showing mesospheric absorption of CO(3-2) in channel space.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='channel',yaxis='amp',field='0,1',<br />
avgtime='1e8',coloraxis='field',iteraxis='spw',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
The figure to the right shows a broad absorption line in spw 1 at channel 3000. This is the ozone absorption line at 357.62982 GHz. However, there is another narrow absorption feature in the spectra (spw 2) that is most easily seen on 3C279 (because it is the strongest calibrator). Let's look at it first in frequency space.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='2:1600~2300',xaxis='frequency',yaxis='amp',<br />
field='0',avgtime='1e8',coloraxis='field',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
The absorption feature lies exactly at the rest frequency for CO 3-2 (345.79599 GHz). Because it is so narrow (< 1 MHz), it must originate from a very low pressure region (i.e. high altitude) in the Earth's atmosphere, otherwise it would be more pressure broadened. Indeed, the first detection of CO in the mesosphere was made over 35 years ago (see, e.g. [http://adsabs.harvard.edu/abs/1976Sci...191.1174W Waters et al. 1976] or [http://adsabs.harvard.edu/abs/1979JGR....84..416G Goldsmith et al. 1979]). Be aware that the line is stronger as you go up the CO ladder (strength ~ J^3 for low J) because the temperature at that altitude (~50km) is about 250 K. In April 2011, the apparent depth of the feature was about 10% in the 2-1 line and 35% in the 3-2 line. However, the depth of the absorption feature is exaggerated due to the way ALMA normalizes the cross-correlation spectra by the autocorrelation spectra prior to storage. While this technique generally leads to a flat bandpass shape, it will accentuate telluric features because they are emission lines in the single dish spectrum (e.g. JCMT observations of mesospheric CO 3-2 can be found in [http://adsabs.harvard.edu/abs/1993MNRAS.264..673P Preston et al. 1993]). Finally, note that large seasonal variations in the line strength have been reported from observations at [http://adsabs.harvard.edu/abs/2003GeoRL..30j..39F Onsala]. <br />
<br />
Now, let's re-plot the line with channel number as the x-axis in order to get the channel range for flagging. <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='2:1600~2300',xaxis='channel',yaxis='amp',<br />
field='0',avgtime='1e8',coloraxis='field',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Now lets look at Titan in more detail.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='channel',yaxis='amp',field='1',<br />
avgtime='1e8',coloraxis='field',iteraxis='spw',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Explicitly back up the flag tables so that flagged regions can be restored later.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
flagmanager(vis=asdm+'.ms',mode='save',versionname=asdm+'.before_calspectralflags')<br />
</source><br />
<br />
<br />
[[Image:Titan_spw2.png|thumb|<caption>Spectral plot of Titan (magenta) for spw=2.</caption>]]<br />
<br />
<br />
Flag absorption features for all sources.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
print "Flagging absorption features in all sources for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', spw='1:2000~3839, 2:1945~1960')<br />
</source><br />
<br />
<br />
[[Image:Titan_spw3.png|thumb|<caption>Spectral plot of Titan (magenta) for spw=3.</caption>]]<br />
<br />
<br />
Now we need to flag emission. The very broad CO(3-2) in spw=2 of Titan will prevent us from using it as a calibrator for this spectral window since there are no line-free channels (we will transfer, phase, amplitude, and absolute flux from another Titan spw below). We do want to flag the narrower Titan line in spw=3.<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
flagmanager(vis=asdm+'.ms',mode='save',versionname=asdm+'.before_emissionflags')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
print "Flagging emission for "+asdm<br />
flagdata(vis=asdm+'.ms', mode='manual', <br />
field='1',<br />
spw='3:1000~3000')<br />
</source><br />
<br />
If you like, check that the right things have been flagged.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='channel',yaxis='amp',field='0,1',<br />
avgtime='1e8',coloraxis='field',iteraxis='spw',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
==Set Up the Flux Calibrator Model==<br />
<br />
Fill the model data column for Titan with a model for the flux density as a function of baseline. The model is a uniformly illuminated disk with the size obtained from the JPL Horizons ephemeris. For CASA 5.3 and later, usescratch should be explicitly set to True for any ephemeris calibrator. <br />
<br />
<source lang="python"><br />
# In CASA<br />
# make sure usescratch = True is using CASA version 5.3 or later<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
setjy(vis=asdm+'.ms',field='1',usescratch=True,<br />
standard='Butler-JPL-Horizons 2010',scalebychan=False)<br />
</source><br />
<br />
<br />
[[Image:Titan amp vs uvdist.png|thumb|<caption>Plot of Titan model for each of the 4 spws. We used the Butler-JPL-Horizons 2010 standard to compute this model.</caption>]]<br />
<br />
<br />
After running setjy, it is a good idea to plot the model in order to confirm that it makes sense. In this case, Titan is partially resolved, so we expect the flux density to drop with uv distance, but not so much as to pass through a null. We also expect it to be brighter at higher frequencies since it has a thermal blackbody spectrum.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',field='1',xaxis='uvdist',yaxis='amp',coloraxis='spw',<br />
ydatacolumn='model',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
[[File:twhydra_X3c1_wvrtsys_2012_titan_model.png|thumb|right|<caption>Titan model produced by 'Butler-JPL-Horizons 2012'.</caption>]]<br />
<br />
<br/><br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
The setjy invocation, above, uses the flux density standard '''standard='Butler-JPL-Horizons 2010' '''. A more updated standard is also available: '''standard='Butler-JPL-Horizons 2012' '''. '''Titan model''' shows the Titan model produced using the updated standard. The motivated reader may wish to rerun {{setjy}} using the updated model. We will highlight later in this tutorial, and in the imaging tutorial that follows, the effects of using the updated model.<br />
</div><br />
<br />
==Bandpass Calibration==<br />
<br />
First we need to check how the amplitude and phase of 3C279 behave as a function of time. In order to set<br />
a fixed scale on the y-axis, we first make a plot of all the data, excluding antenna DV04. (We must specify '''ydatacolumn='data' ''' explicitly here because the default value of '''ydatacolumn''' equals the value used in the previous execution of {{plotms}}. In the previous execution '''ydatacolumn''' was set to '''model'''. See the online help, <tt>help plotms</tt>, for details.)<br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='X3c1_wvrtsys.ms',xaxis='time',yaxis='amp',coloraxis='corr',<br />
field='0',avgchannel='3840',ydatacolumn='data',showgui=True)<br />
</source><br />
<br />
<br />
[[Image:ThreeScansAmp.png|thumb|<caption>Amplitude variation for three 3C279 scans, spw 0 (for one representative baseline).</caption>]]<br />
<br />
<br />
We see that the uncalibrated amplitudes range between 0.2-0.4, so let's set this as our<br />
scale and examine a representative baseline (DV06-DV07) to see how the amplitude changes with time. <br />
<br />
<source lang="python"><br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for spw in ['0','1','2','3']:<br />
for asdm in basename:<br />
print "Now showing spw %s from %s" % (spw, asdm+'.ms')<br />
plotms(vis=asdm+'.ms',spw=spw,xaxis='time',yaxis='amp',coloraxis='corr',iteraxis='spw',<br />
field='0',avgchannel='3840',antenna='DV06&DV07',plotrange=[0,0,0.2,0.4],showgui=True)<br />
user_check=raw_input('press enter to go to the next plot\n')<br />
</source><br />
<br />
<br />
[[Image:ThreeScansPhase.png|thumb|<caption>Phase variation for three 3C279 scans, spw 0 (for one representative baseline).</caption>]]<br />
<br />
<br />
Page through the measurement sets for each spw. This shows significant variations in the amplitude across the three measurement sets.<br />
Now do the same with phase.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for spw in ['0','1','2','3']:<br />
for asdm in basename:<br />
print "Now showing spw %s from %s" % (spw, asdm+'.ms')<br />
plotms(vis=asdm+'.ms',spw=spw,xaxis='time',yaxis='phase',coloraxis='corr',iteraxis='spw',<br />
field='0',avgchannel='3840',antenna='DV06&DV07',plotrange=[0,0,-180,180],showgui=True)<br />
user_check=raw_input('press enter to go to the next plot\n')<br />
</source><br />
<br />
This examination reveals two findings: (1) significant variation in the phase across the three scans, as well as (2) smaller but significant variations within a scan. The first finding suggests that we do not want to combine the scans for bandpass calibration, but instead derive an independent bandpass for each scan. Actually what is more important than a DC offset is if the '''shape''' of the amplitude or phase changes from scan to scan. There isn't enough S/N to tell for sure so we will keep them separate. This situation is often the case, which is why we did not concatenate the three datasets prior to calibration. The second finding suggests that we need to correct the phase vs. time behavior of the bandpass calibrator (3C279; field=0) within each scan, before doing the bandpass. We want to chose a relatively narrow range of channels near the center of the spws so that the bandpass phase slopes (to be corrected with the bandpass calibration itself) do not decorrelate the signal. You need to avoid spectral regions that were completely flagged above.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
os.system('rm -rf ' + asdm + '.bpphase.gcal')<br />
gaincal(vis=asdm+'.ms',caltable=asdm+'.bpphase.gcal',<br />
field='0',spw='0~3:900~1100',refant='DV06',<br />
calmode='p',solint='int',minsnr=2.0,minblperant=4)<br />
</source><br />
<br />
<br />
[[Image:X7ef_wvrtsys.bpphase.X.png|thumb|<caption>Phase only solutions for correlation X on the third scan of the bandpass calibrator 3C279.</caption>]]<br />
<br />
<br />
Inspect the phase-only calibration solutions for the bandpass calibrator. Look for any noisy solutions. We do XX and YY separately to reduce confusion in the plot. It will help if you make your plotcal window wide.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.bpphase.gcal',xaxis='time',yaxis='phase',spw='',antenna='1~8', <br />
iteration='antenna',subplot=421,plotrange=[0,0,-180,180],<br />
figfile=asdm+'.bpphase.X.png',poln='X',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
In the plotcal gui you can use the magnifying glass symbol to zoom in on one of the scans to see the phase variations. The colors are the 4 different spws.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.bpphase.gcal',xaxis='time',yaxis='phase',spw='',antenna='1~8', <br />
iteration='antenna',subplot=421,plotrange=[0,0,-180,180],<br />
figfile=asdm+'.bpphase.Y.png',poln='Y',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Next we apply this phase-only correction on the fly while calculating the bandpass solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
os.system('rm -rf ' + asdm + '.bandpass.bcal')<br />
bandpass(vis=asdm+'.ms',caltable=asdm+'.bandpass.bcal',<br />
field='0',spw='',combine='',refant='DV06',<br />
solint='inf',solnorm=True,minblperant=4, bandtype='B', fillgaps=17,<br />
gaintable=asdm+'.bpphase.gcal')<br />
</source><br />
<br />
You will see some error messages in the terminal, these are just related to the flagged channels. We set fillgaps=17 to interpolate across the smaller regions of flagged channels (mesospheric features and birdies).<br />
<br />
<br />
[[Image:Bandpass.ampspw0.png|thumb|<caption>Bandpass amplitude solution plots.</caption>]]<br />
<br />
[[Image:Bandpass.phasespw0.png|thumb|<caption>Bandpass phase solution plots.</caption>]]<br />
<br />
<br />
'''A few words about solint and combine:'''<br />
<br />
In {{bandpass}}, the use of '''solint='inf'''' (as in "infinite") will derive a bandpass<br />
solution for each 3C279 scan, '''unless''' '''combine='scan'''' (which is the default). Here we set combine=' ' explicitly so that it does not combine the scans into one bandpass. Likewise, field boundaries are only crossed if combine='field', and spw boundaries are only crossed if combine='spw'. In some cases it can be helpful to combine fields if you suffer from lack of S/N, but it is never a good idea to combine spws. <br />
<br />
Now inspect the phase and amplitude solutions. You are looking for excursions from smooth fits, or particularly noisy solutions compared to the others. <br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
for spw in [0,1,2,3]:<br />
plotbandpass(asdm+'.bandpass.bcal',xaxis='freq',yaxis='amp', spw=spw,<br />
antenna='1~8', subplot=42, figfile=asdm+'.bandpass.amp', showatm=True,<br />
interactive=True)<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
for spw in [0,1,2,3]:<br />
plotbandpass(asdm+'.bandpass.bcal',xaxis='freq',yaxis='phase',spw=spw,<br />
antenna='1~8',subplot=42,figfile=asdm+'.bandpass.phs', showatm=True,<br />
interactive=True)<br />
</source><br />
<br />
The values of phase on DV06 are very close to zero because it was used as the reference antenna.<br />
<br />
==Gain Calibration==<br />
<br />
Now that we have a bandpass solution to apply we can solve for the antenna-based phase and amplitude gain calibration. Since the phase changes on a much shorter timescale than the amplitude, we will solve for them separately. In particular, if the phase changes significantly over a scan time, the amplitude would be decorrelated, if the un-corrected phase were averaged over this timescale. Note that we re-solve for the gain solutions of the bandpass calibrator, so we can derive new solutions that are corrected for the bandpass shape. Since the bandpass calibrator will not be used again, this is not strictly necessary, but is useful to check its calibrated flux density for example. <br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
os.system('rm -rf ' + asdm + '.intphase.gcal')<br />
gaincal(vis=asdm+'.ms',caltable=asdm+'.intphase.gcal',<br />
field='0,1,3,4',spw='0~3:40~3800',refant='DV06',<br />
calmode='p',solint='int',minsnr=2.0,minblperant=4,<br />
gaintable=[asdm+'.bandpass.bcal'])<br />
</source><br />
<br />
Here '''solint='int' ''' coupled with '''calmode='p' ''' will derive a single phase solution for each 10 second integration. Note that the bandpass table is applied on-the-fly before solving for the phase solutions, however the bandpass is NOT applied to the data permanently until applycal is run later on. <br />
<br />
Although '''solint='int' ''' (i.e. the integration time of 10 seconds) is the best choice to apply before solving for the amplitude solutions, it is not a good idea to use this to apply to the target. This is because the phase-scatter within a scan can dominate the interpolation between calibrator scans. Instead, we also solve for the phase on the scan time, '''solint='inf' ''' (but '''combine=' ' ''', since we want one solution per scan) for application to the target later on. Unlike the bandpass task, for {{gaincal}}, the default of the combine parameter is '''combine=' ' '''. <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
os.system('rm -rf ' + asdm + '.scanphase.gcal')<br />
gaincal(vis=asdm+'.ms',caltable=asdm+'.scanphase.gcal',<br />
field='0,1,3,4',spw='0~3:40~3800',refant='DV06',<br />
calmode='p',solint='inf',minsnr=2.0,minblperant=4,<br />
gaintable=[asdm+'.bandpass.bcal'])<br />
</source><br />
<br />
Alternatively, instead of making a separate phase solution for application to the target, one can also run {{smoothcal}} to smooth the solutions derived on the integration time. <br />
<br />
Next we apply the bandpass and '''solint='int' ''' phase-only calibration solutions on-the-fly to derive amplitude solutions. Here the use of '''solint='inf' ''', but '''combine=' ' ''' will result in one solution per scan interval. <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
os.system('rm -rf ' + asdm + '.amp.gcal')<br />
gaincal(vis=asdm+'.ms',caltable=asdm+'.amp.gcal',<br />
field='0,1,3,4',spw='0~3:40~3800',refant='DV06',<br />
calmode='ap',solint='inf',minsnr=2.0,minblperant=4,<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'])<br />
</source><br />
<br />
Now carefully inspect all these solutions looking for discrepant solutions. If you see any, you will need to flag them and rerun the calibration tasks above. <br />
<br />
We make plots in X and Y separately for clarity. The colors are showing the spectral windows.<br />
<br />
[[Image:X3c1_wvrtsys.intphase_X.png|thumb|<caption>Phase solutions for the X polarization for every integration time of the first dataset.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.intphase.gcal',xaxis='time',yaxis='phase',antenna='1~8', <br />
spw='',field='0,1,3,4',iteration='antenna',subplot=421,<br />
plotrange=[0,0,-180,180],poln='X',figfile=asdm+'.intphase_X.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.intphase.gcal',xaxis='time',yaxis='phase',antenna='1~8', <br />
spw='',field='0,1,3,4',iteration='antenna',subplot=421,<br />
plotrange=[0,0,-180,180],poln='Y',figfile=asdm+'.intphase_Y.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
[[Image:X3c1_wvrtsys.scanphase_X.png|thumb|<caption>Phase solutions for the X polarization for each scan of the first dataset.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.scanphase.gcal',xaxis='time',yaxis='phase',antenna='1~8', <br />
spw='',field='0,1,3,4',iteration='antenna',subplot=421,<br />
plotrange=[0,0,-180,180],poln='X',figfile=asdm+'.scanphase_X.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.scanphase.gcal',xaxis='time',yaxis='phase',antenna='1~8', <br />
spw='',field='0,1,3,4',iteration='antenna',subplot=421,<br />
plotrange=[0,0,-180,180],poln='Y',figfile=asdm+'.scanphase_Y.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Since we have taken out the phase as best we can by applying the '''solint='int' ''' phase-only solution, this plot will give a good idea of the residual phase error. If you see scatter of more than a few degrees here, you should consider going back and looking for more data to flag, particularly bad timeranges etc. <br />
<br />
<br />
[[Image:X3c1_wvrtsys.amp_phase.png|thumb|<caption>Residual phase after applying intphase.gcal for both correlations in the first dataset.</caption>]]<br />
<br />
<br />
[[Image:X3c1_wvrtsys.amp_X.png|thumb|<caption>Amplitude solutions on a scan interval for correlation X in the first dataset.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.amp.gcal',xaxis='time',yaxis='phase',antenna='1~8', <br />
spw='',field='0,1,3,4',plotrange=[0,0,-1,1],<br />
iteration='antenna',subplot=421,figfile=asdm+'.amp_phase.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
These are very small, as they should be. Now look at the amplitude solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.amp.gcal',xaxis='time',yaxis='amp',antenna='1~8', <br />
iteration='antenna',subplot=421,spw='',poln='X',<br />
plotrange=[0,0,0.0,0.3],figfile=asdm+'.amp_X.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.amp.gcal',xaxis='time',yaxis='amp',antenna='1~8', <br />
iteration='antenna',subplot=421,spw='',poln='Y',<br />
plotrange=[0,0,0.0,0.3],figfile=asdm+'.amp_Y.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
==Flux Calibration==<br />
<br />
Next we use the flux calibrator (whose flux density was set in {{setjy}} above) to derive the flux density of the other calibrators. Note that the flux table REPLACES the amp.gcal in terms of future application of the calibration to the data, i.e. the flux table contains both the amp.gcal and flux scaling. Unlike the gain calibration steps, this is not an incremental table.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
os.system('rm -rf ' + asdm + '.flux.cal')<br />
fluxscale(vis=asdm+'.ms',caltable=asdm+'.amp.gcal',<br />
fluxtable=asdm+'.flux.cal',reference='1',refspwmap=[0,1,3,3],<br />
listfile=asdm+'.fluxscale.txt')<br />
</source><br />
<br />
Its a good idea to record the derived fluxscale values using the listfile option. The values can also be viewed in the Log Messages window.<br />
<br />
<pre style="background-color: #fffacd;"><br />
Found reference field(s): Titan<br />
Found transfer field(s): 3c279 J1147-382=QSO J1037-295=QSO<br />
Spw=2 will be referenced to spw=3<br />
Flux density for 3c279 in SpW=0 (freq=3.56498e+11 Hz) is: 10.537 +/- 0.0183518 (SNR = 574.169, N = 16)<br />
Flux density for 3c279 in SpW=1 (freq=3.57734e+11 Hz) is: 10.6933 +/- 0.0264134 (SNR = 404.842, N = 16)<br />
Flux density for 3c279 in SpW=2 (freq=3.46034e+11 Hz) (ref SpW=3) is: 10.251 +/- 0.0477949 (SNR = 214.479, N = 14)<br />
Flux density for 3c279 in SpW=3 (freq=3.43956e+11 Hz) is: 9.91798 +/- 0.020167 (SNR = 491.791, N = 16)<br />
Flux density for J1147-382=QSO in SpW=0 (freq=3.56498e+11 Hz) is: 1.03399 +/- 0.0167344 (SNR = 61.7887, N = 16)<br />
Flux density for J1147-382=QSO in SpW=1 (freq=3.57734e+11 Hz) is: 0.844817 +/- 0.0191013 (SNR = 44.2283, N = 16)<br />
Flux density for J1147-382=QSO in SpW=2 (freq=3.46034e+11 Hz) (ref SpW=3) is: 0.987193 +/- 0.024391 (SNR = 40.4737, N = 14)<br />
Flux density for J1147-382=QSO in SpW=3 (freq=3.43956e+11 Hz) is: 1.28859 +/- 0.0194364 (SNR = 66.298, N = 16)<br />
Flux density for J1037-295=QSO in SpW=0 (freq=3.56498e+11 Hz) is: 0.968281 +/- 0.00895224 (SNR = 108.161, N = 16)<br />
Flux density for J1037-295=QSO in SpW=1 (freq=3.57734e+11 Hz) is: 0.75916 +/- 0.015366 (SNR = 49.4053, N = 16)<br />
Flux density for J1037-295=QSO in SpW=2 (freq=3.46034e+11 Hz) (ref SpW=3) is: 0.948551 +/- 0.0165421 (SNR = 57.3416, N = 14)<br />
Flux density for J1037-295=QSO in SpW=3 (freq=3.43956e+11 Hz) is: 1.21905 +/- 0.0139967 (SNR = 87.0956, N = 16)<br />
Fitted spectrum for 3c279 with fitorder=1: Flux density = 10.2962 +/- 0.0450319 (freq=351.002 GHz) spidx=1.70757 +/- 0.245737<br />
Fitted spectrum for J1147-382=QSO with fitorder=1: Flux density = 1.06853 +/- 0.0709817 (freq=351.002 GHz) spidx=-6.77353 +/- 3.71867<br />
Fitted spectrum for J1037-295=QSO with fitorder=1: Flux density = 1.01822 +/- 0.0654586 (freq=351.002 GHz) spidx=-6.28977 +/- 3.71453<br />
Storing result in X7ef_wvrtsys.flux.cal<br />
Writing solutions to table: X7ef_wvrtsys.flux.cal<br />
</pre><br />
<br />
[[Image:x3c1_wvrtsys.flux_X.png|thumb|<caption>Absolute flux calibration solutions for correlation X for the first dataset.</caption>]]<br />
<br />
<br />
<div class="noprint" style="float:none; border:0px solid blue;background-color:#F6CEF5;padding:2px;"><br />
If you used the updated (Butler-JPL-Horizons 2012) flux density standard when you ran {{setjy}} earlier in this tutorial, the derived flux values for the calibrators will be slightly different. For example, when averaged over the 3 measurement sets, 3c279 in spectral window 0 has a flux of 10.77 Jy using the 2010 standard and 10.63 Jy using the 2012 standard; as a second example, J1147-382 in SPW 3 has an average flux of 1.13 Jy (2010 standard) and 1.15 Jy (2012 standard). <br />
</div><br />
<br />
Now look at the flux solutions.<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.flux.cal',xaxis='time',yaxis='amp',antenna='1~8', <br />
iteration='antenna',subplot=421,spw='',poln='X',<br />
plotrange=[0,0,0.0,0.3],figfile=asdm+'.flux_X.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotcal(caltable=asdm+'.flux.cal',xaxis='time',yaxis='amp',antenna='1~8', <br />
iteration='antenna',subplot=421,spw='',poln='Y',<br />
plotrange=[0,0,0.0,0.3],figfile=asdm+'.flux_Y.png',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
==Apply Calibration and Inspect==<br />
<br />
Now we can use the {{flagmanager}} if we want to restore any of the spectral flagging we did prior to bandpass calibration. <br />
In this case, we choose to restore the flags of the emission lines on Titan, because we are interested in seeing how<br />
strong they are compared to the continuum. However, we leave the strong ozone line and mesospheric CO line flagged on all <br />
sources, because we want to get the most accurate continuum flux density for the science target. <br />
<br />
<source lang="python"><br />
# In CASA<br />
basename=["X3c1_wvrtsys","X5d8_wvrtsys","X7ef_wvrtsys"]<br />
for asdm in basename:<br />
flagmanager(vis=asdm+'.ms',mode='restore',versionname=asdm+'.before_emissionflags')<br />
</source><br />
<br />
Next we apply the calibration solutions to each source individually, using the '''gainfield''' parameter to specify which calibrator's solutions should be applied from each of the '''gaintable''' calibration tables. <br />
<br />
First, we do 3C279 (the bandpass calibrator), applying all 3 of its solutions to itself (that's why gainfield contains three zeros):<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
applycal(vis=asdm+'.ms',field='0',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['0','0','0'], flagbackup=True, calwt=False)<br />
</source><br />
<br />
Next, we do Titan (the absolute flux calibrator) using the bandpass solution from 3C279:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
applycal(vis=asdm+'.ms',field='1',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['0','1','1'], flagbackup=True, calwt=False)<br />
</source><br />
<br />
For the secondary phase calibrator, we apply the phase and flux solutions from the primary phase calibrator. This will allow us to check how well the phase transfer to the science target (TW Hya) has worked:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
applycal(vis=asdm+'.ms',field='3',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','linear','linear'],<br />
gainfield=['0','4','4'], flagbackup=True, calwt=False)<br />
</source><br />
<br />
Next is the Primary phase calibrator:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
applycal(vis=asdm+'.ms',field='4',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['0','4','4'], flagbackup=True, calwt=False)<br />
</source><br />
<br />
Finally, for the Science Target TW Hya, we apply the phase solutions from both the primary and secondary phase calibrators:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
applycal(vis=asdm+'.ms',field='2',<br />
interp=['nearest','linear','linear'],<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],<br />
gainfield=['0','3,4','3,4'], flagbackup=True, calwt=False)<br />
</source><br />
<br />
Now we can check the results of applying the calibration solutions.<br />
<br />
<br />
[[Image:Corrected_vs_time_1.png|thumb|<caption>The calibrated data for the first dataset (amplitude vs. time).</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='time',yaxis='amp',field='',avgchannel='3840',<br />
coloraxis='field',iteraxis='spw',ydatacolumn='corrected',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
In amplitude, it looks good but there are some noisy points toward the end of the first scheduling block, but it mostly only affects the secondary phase calibrator (green). In phase, we expect all the points to cluster about zero. The large scatter in phase on the secondary phase calibrator is readily apparent: <br />
<br />
[[Image:Corrected_phase_vs_time_1.png|thumb|<caption>The calibrated data on the calibrators for the first dataset (phase vs. time).</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='time',yaxis='phase',field='0,1,3,4',avgchannel='3840',<br />
coloraxis='field',iteraxis='spw',ydatacolumn='corrected',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Let's check whether the situation improves when we apply this calibrator's solutions to itself rather than transferring the solutions from the primary phase calibrator (which is 17 degrees away, and observed at a different time). <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
applycal(vis=asdm+'.ms',field='3',<br />
gaintable=[asdm+'.bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],<br />
interp=['nearest','nearest','nearest'],<br />
gainfield=['0','3','3'],flagbackup=True, calwt=True)<br />
</source><br />
<br />
<br />
[[Image:Corrected_vs_time_1_field3.png|thumb|<caption>The calibrated data for the first dataset (amplitude vs. time), but now with field 3, the secondary phase calibrator, calibrated with itself.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='time',yaxis='amp',field='',avgchannel='3840',<br />
coloraxis='field',iteraxis='spw',ydatacolumn='corrected',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
[[Image:Corrected_phase_vs_time_1_field3.png|thumb|<caption>The calibrated data from the first dataset (phase vs. time), but now with field 3, the secondary phase calibrator, calibrated with itself.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='time',yaxis='phase',field='0,1,3,4',<br />
avgchannel='3840', coloraxis='field',iteraxis='spw',<br />
ydatacolumn='corrected',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Indeed, the secondary calibrator looks better now in amplitude and phase. This is an indication that the calibration transfer is not working well in this timerange. Because the effect is not catastrophic and we know we can self-calibrate on TW Hya later, we will let this go for now. Or, if you want you can delve deeper into the data and try to solve the mystery... <br />
<br />
In any case, we will re-issue the original {{applycal}} so that it is this calibration that our subsequent {{split}} will use. This will allow us to check the effect of correcting the secondary phase calibrator with the primary phase calibrator later. <br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
applycal(vis=asdm+'.ms',field='3',<br />
gaintable=[asdm+'.bandpass.bcal', asdm+'.scanphase.gcal', asdm+'.flux.cal'],<br />
interp=['nearest','linear','linear'],<br />
gainfield=['0','4','4'], flagbackup=True, calwt=False)<br />
</source><br />
<br />
<br />
Let's check the spectral calibration now. First 3C279 and Titan:<br />
<br />
<br />
[[Image:X3c1_spw2_corrected_3.4.png|thumb|<caption>Spectral UV-plots of 3C279 and Titan of spw=2 for the first dataset.</caption>]]<br />
<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='frequency',yaxis='amp',field='0,1',<br />
avgtime='1e8',avgscan=True,coloraxis='field',ydatacolumn='corrected',<br />
iteraxis='spw',xselfscale=True,showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Flip through the spws. These look good. Next the two phase calibrators:<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='frequency',yaxis='amp',field='3,4',<br />
avgtime='1e8',avgscan=True,coloraxis='field',ydatacolumn='corrected',<br />
iteraxis='spw',xselfscale=True,showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Flip through the spws. This also looks good, the secondary phase calibrator is a bit noisier but this is normal. The upswing at the lower frequency end of spw=3 will probably need to be flagged on the science target as well. It corresponds to the edge of an atmospheric feature.<br />
<br />
Now for the exciting part, what does the science target look like...<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='',xaxis='frequency',yaxis='amp',field='2',<br />
avgtime='1e8',avgscan=True,coloraxis='field',ydatacolumn='corrected',<br />
iteraxis='spw',xselfscale=True,showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
We see HCO+(4-3) in spw=0 and CO(3-2) in spw=2 as expected. Because the shape of the <br />
line profiles varies with baseline, you can already tell that the line emission is resolved.<br />
<br />
<br />
[[Image:X3c1_CO3_2_uvplot_3.4.png|thumb|<caption>UV-Plot of the TW Hya CO(3-2) emission from the first dataset.</caption>]]<br />
<br />
[[Image:X3c1_HCOp4_3_uvplot_3.4.png|thumb|<caption>UV-Plot of the TW Hya HCO+(4-3) emission from the first dataset.</caption>]]<br />
<br />
<br />
Now check that the velocity of TWHydra is correct in the LSRK frame, it should be around 2.88 km/s.<br />
* CO 3-2 rest freq 345.79599 GHz<br />
* HCO+ 4-3 rest freq 356.7342 GHz<br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='2',xaxis='velocity',yaxis='amp',field='2',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='345.79599GHz', plotrange=[-10,15,0,0],coloraxis='spw',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
for asdm in basename:<br />
plotms(vis=asdm+'.ms',spw='0',xaxis='velocity',yaxis='amp',field='2',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',<br />
restfreq='356.7342GHz',plotrange=[-10,15,0,0],coloraxis='spw',showgui=True)<br />
print('When you are done with the graphics window,')<br />
print('quit that window, and')<br />
user_check=raw_input('press enter to continue script\n')<br />
</source><br />
<br />
Both lines show up at the expected velocity.<br />
<br />
<br />
==Concatenate the Data==<br />
<br />
Here we will concatenate the three datasets prior to imaging.<br />
<br />
<source lang="python"><br />
# In CASA<br />
splitdata=['X3c1_wvrtsys.ms','X5d8_wvrtsys.ms','X7ef_wvrtsys.ms']<br />
os.system('rm -rf Band7multi_april22.ms')<br />
concat(vis=splitdata,concatvis='Band7multi_april22.ms')<br />
</source><br />
<br />
If you like you can run listobs on new combined dataset<br />
<br />
<source lang="python"><br />
# In CASA<br />
listfile = 'Band7multi_april22.listobs.txt'<br />
os.system('rm ' + listfile)<br />
listobs(vis='Band7multi_april22.ms',verbose=False, listfile=listfile)<br />
</source><br />
<br />
<br />
[[Image:All_spw2_corrected_3.4.png|thumb|<caption>Spectral UV-plots of 3C279 and Titan of spw=2 for the combined datasets.</caption>]]<br />
<br />
<br />
You can also examine the spectral uv plots for the combined dataset to see the improvement in S/N: <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Band7multi_april22.ms',spw='',xaxis='frequency',yaxis='amp',field='0,1',<br />
avgtime='1e8',avgscan=True,coloraxis='field',ydatacolumn='corrected',<br />
iteraxis='spw',xselfscale=True,showgui=True)<br />
</source><br />
<br />
[[Image:All_CO3_2_uvplot_3.4.png|thumb|<caption>UV-Plot of the TW Hya CO(3-2) emission from the combined datasets.</caption>]]<br />
<br />
<br />
...and the CO line profile for the combined dataset: <br />
<br />
<source lang="python"><br />
# In CASA<br />
plotms(vis='Band7multi_april22.ms',spw='2',xaxis='velocity',yaxis='amp',field='2',<br />
avgtime='1e8',avgscan=True,transform=True,freqframe='LSRK',restfreq='345.79599GHz',<br />
plotrange=[-10,15,0,0],coloraxis='spw',showgui=True)<br />
</source><br />
<br />
==Split Calibrated Data==<br />
<br />
Now we split off the calibrated data for the Science Target as well as the interesting parts of the calibrators for imaging.<br />
<br />
When we {{split}} off the Titan data, a few dozen warnings like the following are written to the log:<br />
<pre><br />
2012-05-15 22:00:35 WARN split::SubMS::copySource() Invalid SOURCE ID in SOURCE table row 20 <br />
</pre><br />
These can be ignored, because they do not indicate any problem that will prevent you from imaging Titan correctly at a single position using all the data in the output ms.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf TWHydra_corrected.ms')<br />
split(vis='Band7multi_april22.ms',outputvis='TWHydra_corrected.ms',<br />
datacolumn='corrected',spw='',field='2')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf J1037_corrected.ms')<br />
split(vis='Band7multi_april22.ms',outputvis='J1037_corrected.ms',<br />
datacolumn='corrected',spw='',field='4')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf J1147_corrected.ms')<br />
split(vis='Band7multi_april22.ms',outputvis='J1147_corrected.ms',<br />
datacolumn='corrected',spw='',field='3')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf 3C279_CO3_2.ms')<br />
split(vis='Band7multi_april22.ms',outputvis='3C279_CO3_2.ms',<br />
datacolumn='corrected',spw='2:1900~2000',field='0')<br />
</source><br />
<br />
==Optional: Reconcile Titan's Coordinates==<br />
<br />
If you do not care to image Titan, then you can skip this step and simply continue on to the [http://casaguides.nrao.edu/index.php?title=TWHydraBand7_Imaging_4.0 imaging guide].<br />
<br />
Our earlier treatment of Titan's coordinates using [http://casa.nrao.edu/stable/docs/TaskRef/fixplanets-task.html fixplanets] replaced the zeros written by the ALMA control system with the correct position of the body at the instant of the first observation of that SB. As a result, it has slightly different positions in all three datasets, as seen by listobs:<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='Band7multi_april22.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
ID Code Name RA Decl Epoch SrcId nRows <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 70866 <br />
1 none Titan 12:49:25.97588 -02.22.41.3024 J2000 1 4842 <br />
2 none TW Hya 11:01:51.84498 -34.42.17.1609 J2000 2 515151 <br />
3 none J1147-382=QSO 11:47:01.38151 -38.12.11.1179 J2000 3 40797 <br />
4 none J1037-295=QSO 10:37:16.08989 -29.34.02.9888 J2000 4 171153 <br />
5 none Titan 12:49:26.53729 -02.22.27.1521 J2000 0 4833 <br />
6 none Titan 12:49:26.53059 -02.22.18.7878 J2000 0 4833 <br />
</pre><br />
<br />
If we want to image such moving objects observed across multiple SBs, we need to bring the positions into agreement, otherwise {{clean}} will try to make a mosaic. Task [http://casa.nrao.edu/stable/docs/TaskRef/fixplanets-task.html fixplanets] can be used to do this. Here, we set the direction to the direction of the first dataset as reported by listobs. Note that the RA/Dec format is slightly different from the {{listobs}} output format. Also, we do not<br />
fix the uvw coordinates because they are already more correct now than they would be if we "fixed" them.<br />
<br />
<source lang="python"><br />
# In CASA<br />
fixplanets(vis='Band7multi_april22.ms', field='Titan', direction='J2000 12h49m25.97588 -02d22m41.3024', fixuvw=False)<br />
</source><br />
<br />
Check the header information again to see that the information for Titan has changed for the second<br />
and third datasets. They will appear as fields 5 and 6.<br />
<br />
<source lang="python"><br />
# In CASA<br />
listobs(vis='Band7multi_april22.ms',verbose=False)<br />
</source><br />
<br />
<pre style="background-color: #fffacd;"><br />
ID Code Name RA Decl Epoch SrcId <br />
0 none 3c279 12:56:11.16657 -05.47.21.5247 J2000 0 70866 <br />
1 none Titan 12:49:25.97588 -02.22.41.3024 J2000 1 4842 <br />
2 none TW Hya 11:01:51.84498 -34.42.17.1609 J2000 2 515151 <br />
3 none J1147-382=QSO 11:47:01.38151 -38.12.11.1179 J2000 3 40797 <br />
4 none J1037-295=QSO 10:37:16.08989 -29.34.02.9888 J2000 4 171153 <br />
5 none Titan 12:49:25.97588 -02.22.41.3024 J2000 0 4833 <br />
6 none Titan 12:49:25.97588 -02.22.41.3024 J2000 0 4833 <br />
</pre><br />
<br />
==Optional: Split Calibrated Titan Data==<br />
<br />
When we {{split}} off the Titan data, a few dozen warnings like the following are written to the log:<br />
<pre><br />
2012-05-15 22:00:35 WARN split::SubMS::copySource() Invalid SOURCE ID in SOURCE table row 20 <br />
</pre><br />
These can be ignored, because they do not indicate any problem that will prevent you from imaging Titan correctly at a single position using all the data in the output ms.<br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Titan_cont.ms')<br />
split(vis='Band7multi_april22.ms',outputvis='Titan_cont.ms',<br />
datacolumn='corrected',spw='0',field='1')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Titan_CO3_2.ms')<br />
split(vis='Band7multi_april22.ms',outputvis='Titan_CO3_2.ms',<br />
datacolumn='corrected',spw='2',field='1')<br />
</source><br />
<br />
<source lang="python"><br />
# In CASA<br />
os.system('rm -rf Titan_unknown.ms')<br />
split(vis='Band7multi_april22.ms',outputvis='Titan_unknown.ms',<br />
datacolumn='corrected',spw='3',field='1')<br />
</source><br />
<br />
==Continue on to Imaging of the Science Target==<br />
<br />
Now you can continue on to the [http://casaguides.nrao.edu/index.php?title=TWHydraBand7_Imaging_4.3 imaging guide].<br />
<br />
{{Checked 4.3.0}}</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=Sunspot_Band6_Feathering_for_CASA_5.1&diff=30878Sunspot Band6 Feathering for CASA 5.12021-08-19T20:42:44Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Imaging]][[Category:Sun]]<br />
<br />
==Overview==<br />
<br />
This portion of the Sunspot Band6 CASA Guide for CASA 5.1 covers the combine process between the synthesized solar image and the full-sun image obtained with TP array. It begins where the calibration of the visibility data and single-dish data, image synthesis, and creating of the full-sun map are completed. If you do not complete them, at first, <br />
<br />
1) Download the file of the un-calibrated data package ‘Sunspot_Band6_UncalibratedData.tgz’, do the calibration of the data, and create the images from the interferometric data and single-dish data, along with the ‘Sunspot_Band6_*_for_CASA_5.1’ tutorials.<br />
<br />
or <br />
<br />
2) Download the file of the calibrated data package ‘Sunspot_Band6_CalibratedData.tgz’, and synthesize the image from interferometric data along the ‘Sunspot_Band6_Imaging_for_CASA_5.1’ tutorial.<br />
<br />
In case 1), we assume that you are working on the working directory ‘Sunspot_Band6_UncalibratedData’. On the other hand, when you started the tutorial from the calibrated package (case 2), we assume that the working directory is ‘Sunspot_Band6_CalibratedData.’<br />
<br />
From next, we will show all commands for combining the images.<br />
<br />
==Obtaining the RA/Dec coordinate of the sun center at the reference time==<br />
Before starting the combination process on CASA, we need to know the RA/Dec coordinate of the sun center at the reference time that is used for the image synthesis, because the observing time of the TP-array is not the same as the reference time exactly. The Sun was moving on the RA/Dec coordinate frame during the gap between the times. To combine the synthesized image with the full-sun image obtained from the TP array, we need to shift the coordinate of the full-sun image based on the coordinate of the sun center at the reference time. Hence, we obtain the solar coordinate from the JPL’s HORIZONS system[http://ssd.jpl.nasa.gov/horizons.cgi].<br />
<br />
The reference time in this tutorial is 19:49:00UT, December 18 2015. At the time, the RA/Dec coordinate of the sun center estimated by JPL’s HORIZONS system is RA: 266.01898 degree, Dec: -23.38648 degrees.<br />
<br />
==Preparation for Feathering task: The Single-Dish data==<br />
We start the process on CASA by defining some names of the data.<br />
<source lang='python'><br />
# In Casa<br />
int_base = 'AR12470_B6AllSpw_I' # basename of the synthesized image<br />
sd_img_base = 'uid___A002_Xae00c5_X2e6b.PM03.StkI.Spw3' <br />
# basename of the full-sun map<br />
out_base = 'AR12470_B6AllSpe_I_Feather' # basename of the final products<br />
</source><br />
<br />
[[File:sunspot_fullsun2.png|thumb|right|'''Fig. 1.''' The full-sun map with ‘Jy/beam’ unit.]]<br />
<br />
<br />
[[File:sunspot_sdspot.png|thumb|right|'''Fig. 2.''' The trimmed map from the full-sun map.]]<br />
<br />
We shift the coordinate of the full-sun map created from the single-dish data, based on the RA/Dec coordinate obtained in the previous section, as follows.<br />
<source lang='python'><br />
# In Casa<br />
sun_ra = str(266.01898 * pi/180.) <br />
sun_dec = str(-23.38648 * pi/180.) <br />
imhead(imagename = sd_img_base+'.rescl', mode = 'put', hdkey = 'crval1', hdvalue = sun_ra)<br />
imhead(imagename = sd_img_base+'.rescl', mode = 'put', hdkey = 'crval2', hdvalue = sun_dec)<br />
</source><br />
The unit of the full-sun map is Kelvin. On the other hand, the unit of the synthesized image is Jy/beam. So, the conversion of the unit has to be done. In the tutorial, we convert the unit of the full-sun image to Jy/beam, using the following commands.<br />
<source lang='python'><br />
# In Casa<br />
crval4S= imhead(sd_img_base+'.rescl', mode='get',hdkey='crval4')<br />
freqSD = crval4S.values()[0]<br />
sbema = imhead(sd_img_base+'.rescl', mode='get',hdkey='beammajor')<br />
sbemi = imhead(sd_img_base+'.rescl', mode='get',hdkey='beamminor')<br />
sbmajor = sbema.values()[0]<br />
sbminor = sbemi.values()[0]<br />
convKJb = str(13.6 * (300./(freqSD/1.e9))**2. * (1./sbmajor)*(1./sbminor))<br />
<br />
immath(imagename=sd_img_base+'.rescl', expr='IM0/'+convKJb, outfile=sd_img_base+'.jyb.im')<br />
<br />
imhead(imagename = sd_img_base+'.jyb.im', mode = 'put', hdkey = 'crval3', hdvalue = ['I'])<br />
imhead(imagename = sd_img_base+'.jyb.im', mode = 'put', hdkey = 'bunit', hdvalue = 'Jy/beam')<br />
</source><br />
<br />
The result of the process is shown in Figure 1. A look of the image is not changed, but you can find the changing of the values and unit in the ‘Sunspot_Band6_SingleDish_for_CASA_4.7’ tutorial.<br />
<br />
Then we do the re-gridding and trimming to fit the map area to that of the synthesized image. The area of the trimming is set to the same as the CLEAN box, in the tutorial.<br />
<source lang='python'><br />
# In Casa<br />
#re-griding of SD data<br />
imregrid(imagename=sd_img_base+'.jyb.im', template=int_base+'.image', axes=[0, 1], output=sd_img_base+'.jyb.regrid')<br />
<br />
# Trimming to exclude noisy edge regions in the TP image<br />
imsubimage(imagename=sd_img_base+'.jyb.regrid', outfile=sd_img_base+'.jyb.regrid.subim', box='555,565,1505,1490')<br />
</source><br />
As the final process of the preparation of the single-dish data, we multiply the trimmed image by the primary beam response, as follows.<br />
<source lang='python'><br />
# In Casa<br />
imsubimage(imagename=int_base+'.flux',outfile=int_base+'.flux.subim',box='555,565,1505,1490')<br />
<br />
immath(imagename=[sd_img_base+'.jyb.regrid.subim',int_base+'.flux.subim'], expr='IM0*IM1', outfile=sd_img_base+'.jyb.regrid.subim.depb')<br />
</source><br />
<br />
==Preparation for Feathering task: The Synthesized image==<br />
<br />
[[File:sunspot_syntrim.png|thumb|right|'''Fig. 3''' The trimmed map from the synthesized image.]]<br />
<br />
To fit the map area to that of the trimmed single-dish image, we do the trimming for the synthesized image.<br />
<source lang='python'><br />
# In Casa<br />
imsubimage(imagename=int_base+'.image', outfile=int_base+'.image.subim', box='555,565,1505,1490')<br />
</source><br />
<br />
==Combining, primary beam correction, and creating of the FITS file.==<br />
The combining of the maps can be done using ‘feathering’ task, as the same as the non-solar data.<br />
<source lang='python'><br />
# In CASA<br />
feather(imagename=out_base+'.image', highres=int_base+'.image.subim',lowres=sd_img_base+'.jyb.regrid.subim.depb')<br />
</source><br />
<br />
CAUTION: In this tutorial and the packages of the solar SV data released on 2017/01/18, we use the default parameters of the task, as shown above. As a result, the averaged brightness temperature of the combined image is always larger a few % than the temperature brightness at the same position in the single-dish map, although the values have to be the same basically. It means that you need to tune the “sdfactor" parameter of the task for obtaining consistent images, before using combined images for your science. The detail of the parameter is described in Section 5.6 of “CASA User Reference & Cookbook Release 4.7.0”[https://casa.nrao.edu/docs/cookbook/index.html].<br />
<br />
After the feathering process, we do the primary beam correction, as follows.<br />
<source lang='python'><br />
# In Casa<br />
impbcor(imagename=out_base+'.image', pbimage=int_base+'.flux.subim', outfile=out_base+'.pbcor', mode='divide')<br />
</source><br />
To convert the unit from Jy/beam to Kelvin, we use the following commands.<br />
<br />
[[File:sunspot_final.png|thumb|right|'''Fig. 4.''' The trimmed map from the synthesized image.]]<br />
<br />
<source lang='python'><br />
# In Casa<br />
crval4I= imhead(int_base+'.image', mode='get',hdkey='crval4')<br />
freq = crval4I.values()[0]<br />
fbema = imhead(out_base+'.image', mode='get',hdkey='beammajor')<br />
fbemi = imhead(out_base+'.image', mode='get',hdkey='beamminor')<br />
fbmajor = fbema.values()[0]<br />
fbminor = fbemi.values()[0]<br />
<br />
convKJb2 = str(13.6 * (300./(freq/1.e9))**2. * (1./fbmajor)*(1./fbminor))<br />
<br />
immath(imagename=out_base+'.pbcor',expr='IM0*'+convKJb2, outfile= out_base+'_K.pbcor')<br />
<br />
imhead(imagename = out_base+'_K.pbcor', mode = 'put', hdkey = 'bunit', hdvalue = 'K')<br />
</source><br />
Finally, we create the FITS file of the combined and primary corrected map.<br />
<source lang='python'><br />
# In Casa<br />
exportfits(imagename=out_base+'_K.pbcor', fitsimage=out_base+'_K.fits')<br />
</source><br />
<br />
The result of the combining is shown in Figure 4.<br />
<br />
----<br />
<br />
'''Caution: The single-dish images and feathering images in the solar SV data packages (ex. Sunspot_Band6_ReferenceImages.tgz) were not applied the re-scaling to the single-dish data (see [[Sunspot_Band6_SingleDish_for_CASA_5.1#Calibration_.28re-scaling.29_after_Imaging]]). Please do not use the reference image for your science.'''</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=Sunspot_Band6_Imaging_for_CASA_5.1&diff=30877Sunspot Band6 Imaging for CASA 5.12021-08-19T20:37:59Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Imaging]][[Category:Sun]]<br />
<br />
==Overview==<br />
This portion of the Sunspot Band6 CASA Guide for CASA 5.1 will cover the image synthesis of a sunspot. It begins where [[Sunspot_Band6_Calibration_for_CASA_5.1]] is completed. In the case, we assume that you are working on the working directory ‘Sunspot_Band6_UncalibratedData’.<br />
<br />
If you did not complete the Calibration portion of the guide, the you can download the calibrated visibility data by click on the region closed to your location:<br />
<br />
[https://alma-dl.mtk.nao.ac.jp/ftp/alma/sciver/2015AR149ptBand6/ East Asia]<br />
<br />
[https://almascience.eso.org/almadata/sciver/2015AR149ptBand6/ Eurpoe]<br />
<br />
[https://almascience.nrao.edu/almadata/sciver/2015AR149ptBand6/ North Amrica]<br />
<br />
Once the download has finished, unpack the file:<br />
<br />
<source lang='bash'><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf Sunspot_Band6_CalibratedData.tgz<br />
<br />
cd Sunspot_Band6_CalibratedData<br />
<br />
#Start CASA<br />
casa<br />
</source><br />
<br />
From next, we will show all commands for solar image synthesis.<br />
<br />
==Confirm your version of CASA==<br />
This guide has been written for CASA release 4.7. Please confirm your version before proceeding.<br />
<source lang='python'><br />
# In Casa<br />
import casadef<br />
version = casadef.casa_version <br />
print "You are using " + version <br />
if (version < '5.1.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING." <br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
<br />
==Flagging of the surplus scans==<br />
<br />
<br />
[[File:sunspot_fieldid.png|thumb|right|'''Fig. 1.''' The plot of the FieldID of the scans as a function of the time.]]<br />
<br />
<br />
The actual duration of the observations does not exactly equal to the required duration of the 149-poinitng MOSAIC. Hence, some fields were observed twice, as shown in Figure 1. Although the visibility data of the surplus scans are valid, it is better that they are not used for the image synthesis, considering the uniformity of the sensitivity in a map. For the reason, we flagged the surplus scans by the following command.<br />
<source lang='python'><br />
# In Casa<br />
msc = ‘uid___A002_Xae00c5_X2a8d_split.ms.cal’<br />
flagdata(vis = msc, mode = 'manual', timerange = '2015/12/18/20:02:35~20:08:20', flagbackup = False)<br />
</source><br />
<br />
==Continuum imaging==<br />
<br />
[[File:sunspot_mos2.png|thumb|right|'''Fig. 2.''' The pattern of mosaic AFTER the re-calculation of the direction.]]<br />
<br />
<br />
[[File:sunspot_res.png|thumb|right|'''Fig. 3.''' The residual map (color), effective responsibility (black contours) and CLEAN mask (red box).]]<br />
<br />
[[File:sunspot_synimg.png|thumb|right|'''Fig. 4.''' The CLEANed image of a sunspot observed with Band6]]<br />
<br />
<br />
Now we are ready to synthesize a sunspot image from the visibilities. For the image synthesis, we execute the following command.<br />
<source lang='python'><br />
# In Casa<br />
clean(vis = msc, spw = '0,1,2,3', stokes = 'I', field='0,3~150',phasecenter='0', imagename='AR12470_B6AllSpw_I', cell = '0.15 arcsec', imsize = [2048, 2048], interactive = False,mask = 'box[[555pix,565pix],[1505pix,1490pix]]', weighting = 'briggs', robust = 1., niter = 5000000000, psfmode = 'clark', imagermode='mosaic', mosweight = True, minpb=0.3, pbcor=False, threshold='1.0Jy', gain=0.1)<br />
</source><br />
<br />
To obtain good SNR of the image, we synthesize an image from the data of all spectrum windows. From here, we will comment only about the options that are the special measures for solar image synthesis.<br />
<br />
<pre style="background-color: #fffacd;"><br />
mask = 'box[[555pix,565pix],[1505pix,1490pix]]<br />
</pre><br />
In most cases of solar observations, the field of view (FoV) of the map is filled up with solar structures. Therefore, ideally, the CLEAN box for the solar image synthesis is the same as the FoV. Since the observation was done with the 7m + 12m heterogeneous array, the FoV of the 7m-array is not the same as that of the 12m-array, and the area near the outer edge of the FoV is observed only with 7m-array, as shown in Figure 2. If the CLEAN box includes the area, the synthesized map will include remarkable artificial structures. To avoid the artificial structure, we need to make a CLEAN box that is smaller than the FoV of the 12m-array. In the tutorial, the CLEAN box covers the area that the effective response is larger than 0.6, as shown in Figure 3.<br />
<br />
<pre style="background-color: #fffacd;"><br />
weighting = 'briggs', robust = 1.,<br />
</pre><br />
The ‘briggs’ weighting, which is the default weighting option of the CLEAN task, is used for the solar image synthesis. The longest baseline of the 12m-array for the observation is shorter than that of C40-1. However, the visibility data includes the data of over 200m baselines, because the center of the 12m-array is about 200 m distant from the center of the 7m-array. When the robust parameter is set to 0 as a default, the contribution of the longer baseline data is too large and the artificial stripes appear in the synthesized map. For the reason, we set the ‘robust’ value to 1, and the weighting function is set to near the ‘natural’ weighting. The ‘robust’ parameter is not fixed based on the deep optimization. There is still room to examine that.<br />
<br />
<pre style="background-color: #fffacd;"><br />
imagermode='mosaic', mosweight = True<br />
</pre><br />
To deal the data obtain with a heterogeneous array in the CLEAN task, we need to set the ‘mosaic’ option to ‘imagemode’, even when you synthesize an image from the single-pointing data. In most solar cases, the average brightness of a field might be significantly different from that of the other fields. Therefore, we use ‘mosweight’ function.<br />
(see https://casaguides.nrao.edu/index.php/CARMA_spectral_line_mosaic_M99_3.2#Deconvolution_and_Imaging)<br />
<br />
Figure 4 is the synthesized image from the SV data.<br />
<br />
==Primary beam correction and creating of the FITS file==<br />
To do the primary beam correction and create the FITS file of the corrected map, the following commands.<br />
<source lang='python'><br />
# In Casa<br />
impbcor(imagename='AR12470_B6AllSpw_I.image', pbimage='AR12470_B6AllSpw_I.flux', outfile='AR12470_B6AllSpw_I.pbcor', mode='divide')<br />
exportfits(imagename='AR12470_B6AllSpw_I.pbcor', fitsimage='AR12470_B6AllSpw_I.fits')<br />
</source><br />
<br />
Note: The FITS file can be imported to the data-analysis environment based on SolarSoftWare (SSW) + IDL without any modification, but the coordinate system is the RA/Dec coordinate. You have to convert the coordinate from the RA/Dec coordinate system to the heliocentric coordinate system (X-cen/Y-cen) for the co-alignment with the other solar instrument’s data.</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=Sunspot_Band6_SingleDish_for_CASA_5.1&diff=30875Sunspot Band6 SingleDish for CASA 5.12021-08-19T20:33:56Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Sun]][[Category:Continuum]][[Category:Calibration]]<br />
<br />
==Overview==<br />
This portion of the Sunspot Band6 CASA Guide for CASA 5.1 will cover the calibration and imaging of the data obtained with the TP array. <br />
<br />
From next, we will show all commands for the calibration and be creating the map of a full Sun with Band6.<br />
<br />
==Import the data and split into the files for each antenna==<br />
We assume that you already complete the calibration of the visibility data along the Sunspot Band6 Calibration for CASA 5.1, and you are working in the directory ‘Sunspot_Band6_UncalibratedData’. After starting CASA, we start by defining the directory name of the ASDM and Measurement set for the calibration.<br />
<br />
<source lang='python'><br />
#In Casa<br />
sduid ='uid___A002_Xae00c5_X2e6b'<br />
sdmsd = sduid + '.ms'<br />
sdmsc = sdmsd + '.split.cal'<br />
ant='PM03'<br />
</source><br />
In the tutorial, we will use the data obtained with the PM03 antenna. <br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task {{importasdm}}.<br />
<source lang='python'><br />
#In Casa<br />
importasdm(asdm=sduid,with_pointing_correction=True, verbose=True,vis=sdmsd,overwrite=True)<br />
</source><br />
The usual first step is then to get some basic information about the data. <br />
<source lang='python'><br />
#In Casa<br />
listobs(sdmsd, listfile = sduid + '.listobs')<br />
</source><br />
<br />
==Calibration ==<br />
In the observation, the atmospheric calibration was done before the fast-scanning of the Sun. We use the calibration date to apply the solar data using the option ‘ps, tsys, apply’ for the ‘calmode’.<br />
<source lang='python'><br />
#In Casa<br />
sdcal(infile=sdmsd, calmode = 'ps,tsys,apply', spw='0,1,2,3')<br />
split(vis = sdmsd, spw = '0,1,2,3', outputvis = sdmsc, scan = '2', datacolumn = 'corrected', keepflags = True)<br />
</source><br />
Unlike non-solar single-dish observations, the flux calibrator is not observed in solar single-dish observations in Cycle 4. Therefore, we have to calibrate the data using a correction factor which includes various antenna efficiencies of the PM antenna. <br />
<br />
Based on the commissioning observations, the average correction factor of the PM antennas with Band6 is 0.862, which enables us to calibrate the observed antenna temperature to the absolute brightness temperature. We will calibrate using the value, as follows.<br />
<source lang='python'><br />
#In Casa<br />
gencal(vis=sdmsc,caltable=sduid+'.cal.eff.tbl', caltype='amp', spw='',parameter=[sqrt(0.862)])<br />
applycal(vis=sdmsc, gaintable=sduid+'.cal.eff.tbl')<br />
</source><br />
<br />
Caution: The correction factor in this document is written in White et al. (2017)<br />
<br />
==Imaging==<br />
<br />
<br />
[[File:sunspot_fullsun1.png|thumb|right|'''Fig. 1.''' The full-sun image with Band6]]<br />
<br />
To create the full sun map, we use the data of the spectrum window #3 obtained with the PM03 antenna. The command for the process is as follows.<br />
<source lang='python'><br />
#In Casa<br />
sd_img =sduid+ '.'+ant+'.StkI.Spw3' <br />
sdimaging(infiles = sdmsc, spw = '3', antenna =ant, field='0', nchan = 1, cell = ['3.0arcsec','3.0arcsec'], outfile = sd_img, imsize=[800,800], gridfunction = 'SF', convsupport = 6, intent ='*ON_SOURCE*', stokes = 'I', ephemsrcname = 'sun')<br />
<br />
</source><br />
After finishing the process successfully, you can see the full-sun image as shown in Figure 1.<br />
When you carry out the imaging of Band3 data, we recommend the following parameters for the {{sdimaging}} task.<br />
<source lang='python'><br />
#In Casa<br />
sdimaging(infiles = sdmsc, spw = '3', antenna =ant, field='0', nchan = 1, cell = ['6.0arcsec','6.0arcsec'], outfile = sd_img, imsize=[400,400], gridfunction = 'SF', convsupport = 6, intent ='*ON_SOURCE*', stokes = 'I', ephemsrcname = 'sun')<br />
</source><br />
<br />
==Calibration (re-scaling) after Imaging ==<br />
<br />
White et al. (2017) recommended that the standard ALMA single-dish image products be scaled to match their best determination of quiet-Sun temperatures at disk center (specifically, 5900 K averaged over an 80″-square region centered on the apparent disk center at Band 6, and 7300 K averaged over a 120″-square region centered on the apparent disk center at Band 3, with uncertainties of about 100 K). For the re-scaling, the following commands are executed.<br />
<br />
<source lang='python'><br />
#In Casa<br />
res = imstat(sd_img, region='circle[[400pix,400pix],80pix]') <br />
immath(imagename=sd_img, expr = 'IM0*'+str(5900./res['mean'][0]), outfile = sd_img+'.rescl')<br />
</source><br />
<br />
For Band3<br />
<source lang='python'><br />
#In Casa<br />
res = imstat(sd_img, region='circle[[200pix,200pix],40pix]') <br />
immath(imagename=sd_img, expr = 'IM0*'+str(7300./res['mean'][0]), outfile = sd_img+'.rescl')<br />
</source><br />
<br />
'''Note: All full-sun images that are included in the SV data package (ex. Sunspot_Band6_ReferenceImages.tgz) were not applied the re-scaling because the SV data were released before publishing White et al. (2017). When you use the full-sun image of the SV data for your studies, you should be done the re-scaling process. While the archive data obtained in Cycles were applied the re-scaling process.'''</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=Sunspot_Band6_Calibration_for_CASA_5.1&diff=30874Sunspot Band6 Calibration for CASA 5.12021-08-19T20:31:38Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Calibration]][[Category:Sun]]<br />
<br />
==Overview==<br />
<br />
The portion of the Sunspot_Band6 CASA Guide will cover the calibration of the raw visibility data. '''To follow this guide you must have downloaded the file Sunspot_Band6_UncalibratedData.tgz from [[Sunspot_Band6#Obtaining the Data]].'''<br />
<br />
Detail of the ALMA observations are provided at [[Sunspot_Band6]]<br />
<br />
To skip to the image synthesis portion of the guide, see [[Sunspot_Band6_Imaging_for_CASA_5.1]].<br />
<br />
'''This guide is designed for CASA 5.1'''<br />
<br />
From next, we will show all commands for the calibration. <br />
<br />
==Before Starting the Calibration of Visibility Data==<br />
<br />
The “Analysis Utilities” package must be used for the calibration of solar raw visibility data. Therefore, before starting the tutorial, you need to install the package to your data-analysis environment. The documents and software of the package can be obtained from the Analysis Utilities page[https://casaguides.nrao.edu/index.php/Analysis_Utilities].<br />
<br />
==Unpack the Data==<br />
<br />
Once the file Sunspot_Band6_UncalibratedData.tgz had been download, unpack it as follows:<br />
<source lang="bash"><br />
# in bash<br />
# In a terminal outside CASA<br />
tar -xvzf Sunspot_Band6_UncalibratedData.tgz<br />
<br />
cd Sunspot_Band6_UncalibratedData<br />
<br />
<br />
#Start CASA<br />
casa<br />
</source><br />
<br />
==Confirm your version of CASA==<br />
This guide has been written for CASA release 4.7. Please confirm your version before proceeding.<br />
<source lang="python"><br />
# In Casa<br />
import casadef<br />
version = casadef.casa_version <br />
print "You are using " + version <br />
if (version < '5.1.0'):<br />
print "YOUR VERSION OF CASA IS TOO OLD FOR THIS GUIDE."<br />
print "PLEASE UPDATE IT BEFORE PROCEEDING." <br />
else:<br />
print "Your version of CASA is appropriate for this guide."<br />
</source><br />
We need to import some scripts we will use during the calibration.<br />
<source lang="python"><br />
import analysisUtils as aU<br />
es = aU.stuffForScienceDataReduction() <br />
execfile('Sun_reduction_util.py')<br />
</source><br />
“sun_reduction_util.py” is here [[File:sun_reduction_util.py]].<br />
<br />
==Initial Inspection, A priori calibration==<br />
We start by defining the directory name of the ASDM and some directory names of the Measurement Sets (MS) for the calibration.<br />
<source lang="python"><br />
#In Casa<br />
asdm ='uid___A002_Xae00c5_X2a8d' <br />
mso = asdm + '.ms'<br />
mss = asdm + '_split.ms'<br />
msc = mss + '.cal'<br />
</source><br />
The raw data have been provided to you in the ASDM format. It is the native format of the data produced by the ALMA observatory.<br />
<br />
Before we can proceed to the calibration, we will need to convert those data to the CASA MS format. This is done simply with the task {{importasdm}}.<br />
<source lang="python"><br />
#In Casa<br />
importasdm(asdm = asdm, vis = mso, asis='Antenna Station Receiver Source CalAtmosphere CalWVR CorrelatorMode SBSummary CalDevice ')<br />
</source><br />
<br />
To fix the bug in the SYSCAL table times, the following commands are executed.<br />
<source lang="python"><br />
#In Casa<br />
from recipes.almahelpers import fixsyscaltimes<br />
fixsyscaltimes(vis = mso)<br />
</source><br />
<br />
The usual first step is then to get some basic information about the data. We do this using the task {{listobs}}, which will output a detailed summary of each dataset supplied.<br />
<source lang="python"><br />
#In Casa<br />
listobs(mso, listfile = asdm + '_listobs.txt')<br />
</source><br />
The output will be sent to the CASA {{logger}}, or saved in a text file. Here is a snippet extracted from the {{listobs}} output:<br />
<br />
<pre style="background-color: #fffacd;"><br />
================================================================================<br />
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent<br />
18-Dec-2015/19:15:42.3 - 19:16:47.5 1 0 Sun 236530 [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24] [0.016, 0.016, 0.016, 0.016, 1.15, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.016, 0.016, 0.016, 0.016, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576] [CALIBRATE_ATMOSPHERE#AMBIENT,CALIBRATE_ATMOSPHERE#HOT,CALIBRATE_ATMOSPHERE#OFF_SOURCE,CALIBRATE_ATMOSPHERE#REFERENCE,CALIBRATE_WVR#AMBIENT,CALIBRATE_WVR#HOT,CALIBRATE_WVR#OFF_SOURCE,CALIBRATE_WVR#REFERENCE]<br />
19:17:31.8 - 19:19:26.4 2 1 J1924-2914 1081497 [4,25,26,27,28,29,30,31,32,33,34,35,36] [1.15, 0.016, 0.016, 0.016, 0.016, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_POINTING#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
19:20:14.2 - 19:21:17.7 3 1 J1924-2914 895435 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576] [CALIBRATE_SIDEBAND_RATIO#IMAGE,CALIBRATE_SIDEBAND_RATIO#SIGNAL,CALIBRATE_WVR#IMAGE,CALIBRATE_WVR#SIGNAL]<br />
19:22:06.0 - 19:22:23.1 4 1 J1924-2914 177382 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576] [CALIBRATE_ATMOSPHERE#AMBIENT,CALIBRATE_ATMOSPHERE#HOT,CALIBRATE_ATMOSPHERE#OFF_SOURCE,CALIBRATE_WVR#AMBIENT,CALIBRATE_WVR#HOT,CALIBRATE_WVR#OFF_SOURCE]<br />
19:23:11.6 - 19:28:27.9 5 1 J1924-2914 3244522 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_BANDPASS#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
19:29:14.5 - 19:31:09.2 6 2 nrao530 1081528 [4,25,26,27,28,29,30,31,32,33,34,35,36] [1.15, 0.016, 0.016, 0.016, 0.016, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_POINTING#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
19:31:54.6 - 19:32:11.4 7 2 nrao530 177413 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576] [CALIBRATE_ATMOSPHERE#AMBIENT,CALIBRATE_ATMOSPHERE#HOT,CALIBRATE_ATMOSPHERE#OFF_SOURCE,CALIBRATE_WVR#AMBIENT,CALIBRATE_WVR#HOT,CALIBRATE_WVR#OFF_SOURCE]<br />
19:32:59.1 - 19:35:36.6 8 2 nrao530 1622261 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_FLUX#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
19:36:21.9 - 19:36:52.1 9 2 nrao530 324446 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_PHASE#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
19:37:41.4 - 19:37:57.8 10 0 Sun 177413 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576] [CALIBRATE_ATMOSPHERE#AMBIENT,CALIBRATE_ATMOSPHERE#HOT,CALIBRATE_ATMOSPHERE#OFF_SOURCE,CALIBRATE_WVR#AMBIENT,CALIBRATE_WVR#HOT,CALIBRATE_WVR#OFF_SOURCE]<br />
19:39:23.9 - 19:49:36.0 11 0 Sun 3760610 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#OFF_SOURCE,CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#OFF_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
19:39:23.9 - 19:49:36.0 11 3 Sun 18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
19:39:23.9 - 19:49:36.0 11 4 Sun 18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
[...]<br />
19:39:23.9 - 19:49:36.0 11 79 Sun 18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
19:50:21.7 - 19:50:52.0 12 2 nrao530 324446 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_PHASE#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
19:51:41.2 - 19:51:58.0 13 0 Sun 177413 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576] [CALIBRATE_ATMOSPHERE#AMBIENT,CALIBRATE_ATMOSPHERE#HOT,CALIBRATE_ATMOSPHERE#OFF_SOURCE,CALIBRATE_WVR#AMBIENT,CALIBRATE_WVR#HOT,CALIBRATE_WVR#OFF_SOURCE]<br />
19:53:26.0 - 20:03:37.0 14 0 Sun 3742537 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#OFF_SOURCE,CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#OFF_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
19:53:26.0 - 20:03:37.0 14 3 Sun 18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
19:53:26.0 - 20:03:37.0 14 4 Sun 18042 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
[...]<br />
19:53:26.0 - 20:03:37.0 14 150 Sun 18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
20:04:22.7 - 20:04:53.7 15 2 nrao530 324446 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_PHASE#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
20:05:43.3 - 20:06:00.2 16 0 Sun 177413 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576, 0.576] [CALIBRATE_ATMOSPHERE#AMBIENT,CALIBRATE_ATMOSPHERE#HOT,CALIBRATE_ATMOSPHERE#OFF_SOURCE,CALIBRATE_WVR#AMBIENT,CALIBRATE_WVR#HOT,CALIBRATE_WVR#OFF_SOURCE]<br />
20:06:47.8 - 20:07:30.5 17 0 Sun 227168 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#OFF_SOURCE,CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#OFF_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
20:06:47.8 - 20:07:30.5 17 10 Sun 18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
20:06:47.8 - 20:07:30.5 17 11 Sun 18042 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
20:06:47.8 - 20:07:30.5 17 12 Sun 18011 [4,5,6,7,8,9,10,11,12] [1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_WVR#ON_SOURCE,OBSERVE_TARGET#ON_SOURCE]<br />
20:08:16.5 - 20:08:47.1 18 2 nrao530 324446 [0,1,2,3,4,5,6,7,8,9,10,11,12] [0.016, 0.016, 0.016, 0.016, 1.15, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01, 2.02, 1.01] [CALIBRATE_PHASE#ON_SOURCE,CALIBRATE_WVR#ON_SOURCE]<br />
(nRows = Total number of rows per scan) <br />
Fields: 151<br />
ID Code Name RA Decl Epoch SrcId nRows<br />
0 none Sun 17:44:06.944771 -23.19.30.42665 ICRS 0 8499084<br />
1 none J1924-2914 19:24:51.055957 -29.14.30.12103 ICRS 1 5398836<br />
2 none nrao530 17:33:02.705760 -13.04.49.54800 ICRS 2 4178986<br />
3 none Sun 17:44:06.379967 -23.20.38.51230 ICRS 0 36022<br />
4 none Sun 17:44:07.341071 -23.20.38.51849 ICRS 0 36053<br />
5 none Sun 17:44:08.302174 -23.20.38.52434 ICRS 0 36053<br />
6 none Sun 17:44:09.263278 -23.20.38.52983 ICRS 0 36053<br />
7 none Sun 17:44:10.224382 -23.20.38.53498 ICRS 0 36053<br />
[...]<br />
149 none Sun 17:44:18.760424 -23.18.24.86295 ICRS 0 18011<br />
150 none Sun 17:44:19.721223 -23.18.24.86558 ICRS 0 18011<br />
Spectral Windows: (37 unique spectral windows and 2 unique polarization setups)<br />
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs <br />
0 BB_1#SQLD 1 TOPO 230000.000 2000000.000 2000000.0 230000.0000 1 XX YY<br />
1 BB_2#SQLD 1 TOPO 232000.000 2000000.000 2000000.0 232000.0000 2 XX YY<br />
2 BB_3#SQLD 1 TOPO 246000.000 2000000.000 2000000.0 246000.0000 3 XX YY<br />
3 BB_4#SQLD 1 TOPO 248000.000 2000000.000 2000000.0 248000.0000 4 XX YY<br />
4 WVR#NOMINAL 4 TOPO 184550.000 1500000.000 7500000.0 187550.0000 0 XX<br />
5 X241538345#ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 230992.188 -15625.000 2000000.0 230000.0000 1 XX YY<br />
6 X241538345#ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 229984.375 1796875.000 1796875.0 229984.3750 1 XX YY<br />
7 X241538345#ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 232992.188 -15625.000 2000000.0 232000.0000 2 XX YY<br />
8 X241538345#ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 231984.375 1796875.000 1796875.0 231984.3750 2 XX YY<br />
9 X241538345#ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 245007.813 15625.000 2000000.0 246000.0000 3 XX YY<br />
10 X241538345#ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 245984.375 1796875.000 1796875.0 245984.3750 3 XX YY<br />
11 X241538345#ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 247007.813 15625.000 2000000.0 248000.0000 4 XX YY<br />
12 X241538345#ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 247984.375 1796875.000 1796875.0 247984.3750 4 XX YY<br />
13 BB_1#SQLD 1 TOPO 219559.000 2000000.000 2000000.0 219559.0000 1 XX YY<br />
14 BB_2#SQLD 1 TOPO 219559.000 2000000.000 2000000.0 219559.0000 2 XX YY<br />
15 BB_3#SQLD 1 TOPO 219559.000 2000000.000 2000000.0 219559.0000 3 XX YY<br />
16 BB_4#SQLD 1 TOPO 219559.000 2000000.000 2000000.0 219559.0000 4 XX YY<br />
17 X241538345#ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 220551.188 -15625.000 2000000.0 219559.0000 1 XX YY<br />
18 X241538345#ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 219543.375 1796875.000 1796875.0 219543.3750 1 XX YY<br />
19 X241538345#ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 220551.188 -15625.000 2000000.0 219559.0000 2 XX YY<br />
20 X241538345#ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 219543.375 1796875.000 1796875.0 219543.3750 2 XX YY<br />
21 X241538345#ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 220551.188 -15625.000 2000000.0 219559.0000 3 XX YY<br />
22 X241538345#ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 219543.375 1796875.000 1796875.0 219543.3750 3 XX YY<br />
23 X241538345#ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 220551.188 -15625.000 2000000.0 219559.0000 4 XX YY<br />
24 X241538345#ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 219543.375 1796875.000 1796875.0 219543.3750 4 XX YY<br />
25 BB_1#SQLD 1 TOPO 221538.000 2000000.000 2000000.0 221538.0000 1 XX YY<br />
26 BB_2#SQLD 1 TOPO 223538.000 2000000.000 2000000.0 223538.0000 2 XX YY<br />
27 BB_3#SQLD 1 TOPO 237538.000 2000000.000 2000000.0 237538.0000 3 XX YY<br />
28 BB_4#SQLD 1 TOPO 239538.000 2000000.000 2000000.0 239538.0000 4 XX YY<br />
29 X0000000000#ALMA_RB_06#BB_1#SW-01#FULL_RES 128 TOPO 222530.188 -15625.000 2000000.0 221538.0000 1 XX YY<br />
30 X0000000000#ALMA_RB_06#BB_1#SW-01#CH_AVG 1 TOPO 221514.562 1781250.000 1781250.0 221514.5625 1 XX YY<br />
31 X0000000000#ALMA_RB_06#BB_2#SW-01#FULL_RES 128 TOPO 224530.188 -15625.000 2000000.0 223538.0000 2 XX YY<br />
32 X0000000000#ALMA_RB_06#BB_2#SW-01#CH_AVG 1 TOPO 223514.562 1781250.000 1781250.0 223514.5625 2 XX YY<br />
33 X0000000000#ALMA_RB_06#BB_3#SW-01#FULL_RES 128 TOPO 236545.813 15625.000 2000000.0 237538.0000 3 XX YY<br />
34 X0000000000#ALMA_RB_06#BB_3#SW-01#CH_AVG 1 TOPO 237514.563 1781250.000 1781250.0 237514.5625 3 XX YY<br />
35 X0000000000#ALMA_RB_06#BB_4#SW-01#FULL_RES 128 TOPO 238545.813 15625.000 2000000.0 239538.0000 4 XX YY<br />
36 X0000000000#ALMA_RB_06#BB_4#SW-01#CH_AVG 1 TOPO 239514.563 1781250.000 1781250.0 239514.5625 4 XX YY<br />
Antennas: 31:<br />
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m) <br />
East North Elevation x y z<br />
0 CM01 N602 7.0 m -067.45.17.4 -22.53.22.3 8.8042 -527.8587 22.2034 2225080.354846 -5440132.955920 -2481524.789784<br />
1 CM02 J502 7.0 m -067.45.17.7 -22.53.23.0 2.1073 -549.4461 22.1460 2225070.957857 -5440127.670516 -2481544.655003<br />
2 CM03 J503 7.0 m -067.45.17.4 -22.53.23.2 9.2482 -555.0637 22.1304 2225076.734430 -5440122.931505 -2481549.824201<br />
3 CM06 N606 7.0 m -067.45.17.1 -22.53.23.6 19.1995 -566.5684 22.1011 2225084.240492 -5440114.997537 -2481560.411621<br />
4 CM07 N601 7.0 m -067.45.17.0 -22.53.22.5 21.0633 -532.5817 22.1865 2225090.999805 -5440126.600430 -2481529.134327<br />
5 CM08 J505 7.0 m -067.45.18.0 -22.53.22.8 -7.2123 -541.3466 22.1685 2225063.532652 -5440134.133528 -2481537.202006<br />
6 CM09 N603 7.0 m -067.45.17.7 -22.53.22.3 -0.0497 -527.8657 22.1913 2225072.154896 -5440136.294753 -2481524.791487<br />
7 CM10 J501 7.0 m -067.45.17.4 -22.53.22.9 10.0863 -545.4959 22.1606 2225078.929507 -5440126.084513 -2481541.021572<br />
8 CM11 N604 7.0 m -067.45.17.8 -22.53.23.7 -0.2657 -571.8966 22.0829 2225065.433709 -5440120.432522 -2481565.313207<br />
9 DA41 A004 12.0 m -067.45.15.9 -22.53.28.0 52.6609 -704.4171 21.7726 2225094.796703 -5440052.421785 -2481687.277348<br />
10 DA49 A002 12.0 m -067.45.16.3 -22.53.27.6 40.6333 -690.2503 21.8023 2225085.761255 -5440062.100754 -2481674.237730<br />
11 DA50 A038 12.0 m -067.45.18.5 -22.53.29.4 -22.4285 -745.7518 22.0606 2225019.312629 -5440066.210860 -2481725.468896<br />
12 DA52 A018 12.0 m -067.45.17.2 -22.53.28.1 16.8264 -706.6065 21.7531 2225061.301483 -5440065.182346 -2481689.286759<br />
13 DA54 A005 12.0 m -067.45.14.8 -22.53.28.7 83.3315 -725.0764 21.7237 2225120.123751 -5440033.331382 -2481706.290680<br />
14 DA55 A047 12.0 m -067.45.16.4 -22.53.30.3 38.4542 -775.2187 21.5966 2225071.160250 -5440032.158745 -2481752.434689<br />
15 DA57 A025 12.0 m -067.45.18.7 -22.53.27.4 -26.4296 -685.5228 22.2053 2225024.528985 -5440089.533369 -2481670.039352<br />
16 DA59 A001 12.0 m -067.45.16.9 -22.53.27.7 24.1880 -693.3966 21.7925 2225070.073933 -5440067.185105 -2481677.132442<br />
17 DA61 A006 12.0 m -067.45.15.0 -22.53.28.0 79.0341 -702.0939 21.7778 2225119.549746 -5440043.278703 -2481685.139167<br />
18 DA62 A035 12.0 m -067.45.16.6 -22.53.28.1 32.0366 -706.8051 21.7640 2225075.353584 -5440059.362225 -2481689.473934<br />
19 DA63 A039 12.0 m -067.45.18.0 -22.53.29.6 -6.1070 -751.7850 22.0696 2225033.533416 -5440057.867935 -2481731.030436<br />
20 DA65 A030 12.0 m -067.45.18.1 -22.53.27.2 -10.3876 -679.1070 21.8267 2225040.189208 -5440085.447750 -2481663.981462<br />
21 DV02 A007 12.0 m -067.45.15.1 -22.53.27.3 74.0140 -681.2928 21.3255 2225117.808983 -5440052.282474 -2481665.800190<br />
22 DV07 A027 12.0 m -067.45.19.0 -22.53.28.7 -35.0460 -726.6032 21.5989 2225010.293430 -5440077.487712 -2481707.648731<br />
23 DV11 A049 12.0 m -067.45.14.6 -22.53.29.6 88.4465 -754.5446 20.1464 2225119.968124 -5440019.440510 -2481732.824666<br />
24 DV12 A017 12.0 m -067.45.15.9 -22.53.26.8 51.3634 -665.5893 21.3604 2225099.169786 -5440066.540560 -2481651.346952<br />
25 DV13 A064 12.0 m -067.45.14.7 -22.53.31.4 85.6567 -808.0278 21.0176 2225109.813656 -5440001.983403 -2481782.434609<br />
26 DV17 A031 12.0 m -067.45.19.1 -22.53.27.1 -37.8149 -675.5186 21.7325 2225015.299768 -5440097.041914 -2481660.638998<br />
27 DV18 A009 12.0 m -067.45.16.1 -22.53.26.1 48.2542 -644.4621 21.0152 2225099.282791 -5440075.029589 -2481631.749120<br />
28 DV20 A033 12.0 m -067.45.19.4 -22.53.29.0 -47.3621 -735.6360 21.8836 2224997.663712 -5440079.140555 -2481716.080921<br />
29 DV22 A014 12.0 m -067.45.15.1 -22.53.26.4 74.5090 -654.2102 20.9880 2225122.137554 -5440061.557701 -2481640.719077<br />
30 DV23 A003 12.0 m -067.45.16.5 -22.53.27.0 35.5295 -672.6352 21.3390 2225083.469962 -5440069.979758 -2481657.829615<br />
</pre><br />
<br />
The first section of the output describes the detail of each scan, and the second section (from line 34) shows the information of the observing targets. This second section shows that three targets with 151 fields were observed: The Sun, J1924-2914, nrao350(J1733-1304). From the section, the J1924-2914 was observed for the calibrations of pointing [2]4, sideband ratio [3], atmosphere [4], and bandpass [5]. nrao350 was observed for the calibration of pointing [6], atmosphere [7], flux [8], and phase [9,12,15,18]. The Sun was observed for scientific observations [11,14,17], the calibration of atmosphere [10,16] and the measurement of zero-signal level [1].<br />
<br />
[[File:sunspot_antplots.png|thumb|right|'''Fig. 1.''' “plotants” output for the dataset uid___A002_Xae00c5_X2a8d]]<br />
<br />
<br />
The third section of the listobs output (from line 47) shows the information of the spectrum windows (Spw) in the dataset. From first and this sections, the scientific observations are done with the Spw 0~12, and the IDs of the Spw with 128 channels, which are used for image synthesis, are 5, 7, 9, and 11. Therefore, we will calibrate the data of SpwID 5, 7, 9, and 11 only. The data of spwID 0,1,2,3 are the data from the square-law detectors of the basebands. The data will be used for creating Tsys+Tant tables, and are archived as the auto-correlation data in the dataset.<br />
Thirty-one antennas were used for the dataset. Note that numbering in python always begins with "0", so the antennas have IDs 0-30. To see what the antenna configuration looked like at the time of this observation, we use the task {{plotants}} (Figure 1).<br />
<br />
===Flagging before creating Tsys and Tsys+Tant tables===<br />
<br />
Some scans in the data were used by the online system for pointing and sideband ratio calibration. These scans are no longer needed, and we can flag them easily with {{flagdata}} by selecting on 'intent'.<br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mso, mode = 'manual', intent = '*POINTING*,*SIDEBAND_RATIO*', flagbackup = False)<br />
</source><br />
The averaged data of each spectrum window is not used, so we flagged the averaged data as a follow. <br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mso, mode = 'manual', spw = '6,8,10,12', flagbackup = False)<br />
</source><br />
We will then store the current flagging state for each dataset using the {{flagmanager}}:<br />
<source lang="python"><br />
#In Casa<br />
flagmanager(vis = mso, mode = 'save', versionname = 'priori1')<br />
</source><br />
<br />
===Tsys calibration of the visibilities of the calibrators===<br />
<br />
[[File:sunspot_tsys.png|thumb|right|'''Fig. 2.''' “plotants” output for the dataset uid___A002_Xae00c5_X2a8d]]<br />
<br />
The Tsys calibration gives a first-order correction for the atmospheric opacity as a function of time and frequency, and associates weights with each visibility that persist through imaging. The MS dataset contains Tsys measurements; the task {{gencal}} is used to generate a calibration table.<br />
<source lang="python"><br />
#In Casa<br />
gencal(vis = mso, caltable = mso + '.tsys', caltype = 'tsys')<br />
flagdata(vis = mso + '.tsys', mode = 'manual', spw = '5:0~9;116~127,7:0~9;116~127,9:0~9;116~127,11:0~9;116~127', flagbackup = False)<br />
es.checkCalTable(mso+'.tsys', msName=mso, interactive=False)<br />
</source><br />
Since the dataset obtained with TDM, the data in the channels near the both edges of the spectrum window (~10 channels) are flagged. Then, the plots for checking are created by the subroutine of Analysis Utilities package (Figure 2). The Tsys of DA54 antennas are significant large, from the plot. In the later part, we will flag the data of the antenna. Please ignore the data of the scan #1, because the scan is not done for ATM cal. The scan is used for the measurement of zero-signal level .<br />
<br />
We will apply the Tsys calibration table to the data of the calibrators with the task {{applycal}}, which reads the specified gain calibration tables, applies them to the (raw) data column, and writes the calibrated results into the corrected column. For non-solar observations, we also apply the WVR (Water Vapor Radiometer) calibration table to data. However, we must NOT apply the WVR table to the solar data, because the WVR receivers at the Sun occur the saturation.<br />
We apply the Tsys calibration table to the data of the bandpass calibrator:<br />
<source lang="python"><br />
#In Casa<br />
applycal(vis = mso, field = '1', spw = '5,7,9,11', gaintable = mso + '.tsys', gainfield = '1', interp = 'linear,linear', calwt = True, flagbackup = False)<br />
</source><br />
In the observations, we do not the atmospheric calibration of the phase calibrator between the scientific scans, because a long suspension of scientific observations has a bad influence on science. Therefore, we apply the Tsys calibration table, which is created from the data of the atmosphere calibration at the flux calibrator or the Sun, to the phase calibrator, as follows.<br />
<source lang="python"><br />
#In Casa<br />
applycal(vis = mso, field = '2', spw = '5,7,9,11', gaintable = mso + '.tsys', gainfield = '2', interp = 'linear,linear', calwt = True, flagbackup = False)<br />
</source><br />
You can use {{plotms}} to plot channel-averaged amplitudes as a function of time, comparing the DATA and CORRECTED columns after applying the Tsys correction. This way you can check that calibration has done what was expected, which is put the data onto the Kelvin temperature scale.<br />
<br />
===Tsys+Tant calibration of the visibilities of the Sun===<br />
<br />
The standard method of Tsys calibration cannot be apply to the data of the Sun, because the antenna temperature of the Sun cannot be neglected for estimating the system equivalent flux density (SEFD). To estimate correct SEFD at the Sun, the solar observing sequence includes some special measurements, like the measurement of zero-signal level. The subroutines for creating and applying Tsys+Tant calibration tables are prepared by the ALMA solar development team. The subroutines are already imported at the section [[#Confirm your version of CASA]]. For the Tsys+Tant calibration of the solar data, you will execute only the following command.<br />
<br />
<source lang="python"><br />
#In Casa<br />
sol_ampcal_2(mso, mso + '.tsys', exisTbl=False, outCSV=True)<br />
</source><br />
<br />
'''The process takes long time, about one night or day.''' If you have already carried out the process before and there are the Tsys+Tant calibration tables (The directory name of the table includes “tsystant”), you can skip the generating of the tables using the following command.<br />
<br />
<source lang="python"><br />
#In Casa<br />
sol_ampcal_2(mso, mso + '.tsys', exisTbl=True, outCSV=False)<br />
</source><br />
<br />
===Flagging after Tsys and Tsys+Tant calibration===<br />
Since we completed the Tsys and Tsys+Tant calibration, the data that are not used the image synthesis are flagged. At first, the data of auto-correlation and atmosphere calibration are flagged as follows:<br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mso, mode = 'manual', autocorr = True, flagbackup = False)<br />
flagdata(vis = mso, mode = 'manual', intent = '*ATMOSPHERE*', flagbackup = False)<br />
</source><br />
Next is the spectrum windows that are not used in later:<br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mso, mode = 'manual', spw = '0~4', flagbackup = False)<br />
flagdata(vis = mso, mode = 'manual', spw = '13~36', flagbackup = False)<br />
</source><br />
The data in the channels near the both edges of the spectrum window (~10 channels) are flagged:<br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mso, mode = 'manual', flagbackup = False, spw='5:0~9;116~127,7:0~9;116~127,9:0~9;116~127,11:0~9;116~127')<br />
</source><br />
The some sub-scans at the start and end of the scientific scans are used to measure the sky with the setting of optimized for the Sun. Since the data are used only for estimating the antenna temperatures at the Sun, we flag the data:<br />
<source lang="python"><br />
#In Casa<br />
mymsmd = createCasaTool(msmdtool)<br />
mymsmd.open(mso)<br />
sciScan = mymsmd.scansforintent('*OBSERVE_TARGET*')<br />
mymsmd.done()<br />
<br />
for i in range(0, len(sciScan)):<br />
subInf=aU.computeDurationOfScan(sciScan[i], vis=mso, returnSubscanTimes=True)<br />
subNum = subInf[1]<br />
flagdata(vis = mso, timerange = subInf[3][1], mode = 'manual', flagbackup = False) <br />
flagdata(vis = mso, timerange = subInf[3][2],mode = 'manual', flagbackup = False) <br />
flagdata(vis = mso, timerange = subInf[3][subNum-1], mode = 'manual', flagbackup = False) <br />
flagdata(vis = mso, timerange = subInf[3][subNum], mode = 'manual', flagbackup = False)<br />
</source><br />
<br />
As mentioned in the section [[#Tsys calibration of the visibilities of the calibrators]], the Tsys values of DA54 antennas are significantly large. Therefore, we flag the data of the antenna as follows:<br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mso, antenna = 'DA54', mode = 'manual', flagbackup = False)<br />
</source><br />
<br />
Then, we store the current flagging state for each dataset using the {{flagmanager}}:<br />
<source lang="python"><br />
#In Casa<br />
flagmanager(vis = mso, mode = 'save', versionname = 'priori2')<br />
</source><br />
<br />
Now split out the CORRECTED data column, retaining spectral windows 5,7,9, and 11. This will get rid of the extraneous spectral windows.<br />
<source lang="python"><br />
#In Casa<br />
split(vis = mso, outputvis = mss, datacolumn = 'corrected', intent ='*BANDPASS*,*FLUX*,*PHASE*,*TARGET*', spw = '5,7,9,11', keepflags = True)<br />
</source><br />
<br />
==Additional Data Inspection==<br />
<br />
We will do some additional inspection. <br />
<br />
The solar data frequently include zero values in some channels. To avoid their influence, we flag the data using the “clip” mode with “clipzeros=True” option.<br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mss, mode = 'clip', clipzeros = True, flagbackup = False)<br />
</source><br />
The valleys appear in the plot of amplitude vs channel of the Spw #3. It shows the effect of the lines of the earth atmosphere. We need to flag the data.<br />
<source lang="python"><br />
#In Casa<br />
flagdata(vis = mss, spw = '3:68~85', mode = 'manual', flagbackup = False)<br />
flagdata(vis = mss, spw = '3:20~40', mode = 'manual', flagbackup = False)<br />
</source><br />
Then, we store the current flagging state for each dataset using the {{flagmanager}}:<br />
<source lang="python"><br />
#In Casa<br />
flagmanager(vis = mss, mode = 'save', versionname = 'Initial')<br />
</source><br />
<br />
==Set up the Flux Calibration Model==<br />
It is very rare that a useful planet for the flux calibration is located near the Sun. Therefore, we usually use a quasar near the Sun as a flux calibrator. In this observations, there is no quasar that flux is enough for the observations with the Mixer De-tuning mode, except nrao350. Therefore, the flux calibrator is the same as the phase calibrator in this observation.<br />
To fill the model data column for nrao350 with a model for the flux density, we obtain the flux density and spectral index of the nro350 from the ALMA calibrator database.<br />
<br />
<source lang="python"><br />
#In Casa<br />
intentSources=es.getIntentsAndSourceNames(mss)<br />
ampCalId = intentSources['CALIBRATE_AMPLI']['id'] + intentSources['CALIBRATE_FLUX']['id']<br />
calFieldNames = intentSources['CALIBRATE_AMPLI']['name'] + intentSources['CALIBRATE_FLUX']['name']<br />
<br />
amp_cal_name=calFieldNames[1]<br />
spwInfo=es.getSpwInfo(mss)<br />
obs_freq="%fGHz"%(spwInfo[0]['refFreq']/1e9)<br />
<br />
date=aU.getObservationStartDate(mss)<br />
date_obs=date.split()[0]<br />
spw1_flux=aU.getALMAFlux(sourcename=amp_cal_name, date=date_obs,frequency=obs_freq)<br />
</source><br />
<br />
Then, fill the model data column for nrao350 with a model<br />
<source lang="python"><br />
#In Casa<br />
setjy(vis = mss, field = '2', spw = '0,1,2,3', standard = 'manual', fluxdensity = [spw1_flux['fluxDensity'], 0, 0, 0], spix= spw1_flux['spectralIndex'], reffreq = obs_freq)<br />
</source><br />
<br />
==Creating the Bandpass Calibration Table==<br />
''The bandpass and gain calibrations are the same as the calibrations of non-solar data basically. Please refer to the other ALMA tutorials of the CASA guide too. The calibration method presented below is the same as that used in the QA2 process of the ALMA observatory.''<br />
<br />
At first, we determine the reference antenna, and run {{gaincal}} on the bandpass calibrator to determine phase-only gain solutions. We will use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time.<br />
<br />
<source lang="python"><br />
#In Casa<br />
ref_ant = 'DA41'<br />
gaincal(vis = mss, caltable = mss + '.ap_pre_bandpass', field = '1', scan = '5', solint = 'int', refant = ref_ant, calmode = 'p')<br />
</source><br />
The plots are created for the checking.<br />
<source lang="python"><br />
#In Casa<br />
es.checkCalTable(mss+'.ap_pre_bandpass', msName=mss, interactive=False)<br />
</source><br />
The plots look good. Next, create the bandpass calibration table: <br />
<source lang="python"><br />
#In Casa<br />
bandpass(vis = mss, caltable = mss+'.bandpass', field = '1', scan = '5', solint = 'inf', refant = ref_ant, solnorm = True, bandtype = 'B', gaintable = mss+'.ap_pre_bandpass')<br />
</source><br />
Check the bandpass calibration table using the plots created from the following command.<br />
<source lang="python"><br />
#In Casa<br />
es.checkCalTable(mss+'.bandpass', msName=mss, interactive=False)<br />
</source><br />
<br />
==Creating the Gain Calibration Table==<br />
At first, we determine phase-only gain solutions of the calibrators, using the bandpass calibration table and solint='int' option, and create the plots for checking.<br />
<br />
<source lang='python'><br />
#In Casa<br />
gaincal(vis = mss, caltable = mss + '.phase_int', field = '1~2', solint = 'int', refant = ref_ant,gaintype = 'G',calmode = 'p',minsnr = 3.0, gaintable = mss + '.bandpass')<br />
es.checkCalTable(mss + '.phase_int', msName=mss, interactive=False)<br />
</source><br />
Using the bandpass and phase calibration tables, we obtain the amplitude-only gain solutions on the scan time, '''solint='inf''''.<br />
<source lang='python'><br />
# In CASA<br />
gaincal(vis = mss, caltable = mss + '.ampli_inf', field = '1~2', solint = 'inf', refant = ref_ant, gaintype = 'G', calmode = 'a', minsnr = 3.0, gaintable = [mss + '.bandpass', mss + '.phase_int'])<br />
es.checkCalTable(mss + '.ampli_inf', msName=mss, interactive=False) <br />
</source><br />
Usually, the gaintype is set to 'T' for nominal ALMA observations . Nevetheless, the gaintype is set to 'G' for solar data, because the diffrence between XX and YY images is used for for estimating the noise level (see Section 4.2, Shimojo et al. 2017).<br />
<br />
Next we use the flux calibrator (whose flux density was set in {{setjy}} above) to derive the flux density of the other calibrators. Note that the flux table REPLACES the amp.gcal in terms of future application of the calibration to the data, i.e. the flux table contains both the amp.gcal and flux scaling. Unlike the gain calibration steps, this is not an incremental table.<br />
<source lang='python'><br />
#In Casa<br />
fluxscaleDict = fluxscale(vis = mss, caltable = mss + '.ampli_inf', fluxtable = mss + '.flux_inf', reference = '2') <br />
es.fluxscale2(caltable = mss+'.ampli_inf', removeOutliers=True, msName=mss, writeToFile=True, preavg=10000)<br />
</source><br />
Finally, we create the gain calibration table of phase-only gain solutions on the scan time, '''solint='inf''''.<br />
<source lang='python'><br />
#In Casa<br />
gaincal(vis = mss, caltable = mss+'.phase_inf', field = '1~2', solint = 'inf', refant = ref_ant, gaintype = 'G', calmode = 'p', minsnr = 3.0, gaintable = mss+'.bandpass')<br />
es.checkCalTable(mss+'.phase_inf', msName=mss, interactive=False)<br />
</source><br />
<br />
==Applying the Calibration Tables==<br />
We apply the calibration solutions to each source individually, using the gainfield parameter to specify which calibrator's solutions should be applied from each of the gaintable calibration tables.<br />
Applying the tables to the bandpass, flux and phase calibrators:<br />
<source lang='python'><br />
#In Casa<br />
for i in ['1', '2']: <br />
applycal(vis = mss,field = str(i), gaintable = [mss + '.bandpass', mss+'.phase_int', mss+'.flux_inf'], gainfield = ['', i, i], interp = 'linear,linear', calwt = True, flagbackup = False)<br />
</source><br />
Next, applying them to the solar data:<br />
<source lang='python'><br />
#In Casa<br />
applycal(vis = mss, field = '0, 3~150', gaintable = [mss+'.bandpass', mss+'.phase_inf', mss+'.flux_inf'], gainfield = ['', '2', ''], interp = 'linear,linear', calwt = True, flagbackup = False)<br />
</source><br />
Finally, split out the CORRECTED data column<br />
<source lang='python'><br />
#In Casa<br />
split(vis = mss, outputvis = msc, datacolumn = 'corrected', keepflags = True)<br />
</source><br />
<br />
==Re-calculation of the direction==<br />
<br />
<br />
[[File:sunspot_mos1.png|thumb|right|'''Fig. 3.''' The pattern of mosaic BEFORE the re-calculation of the direction.]]<br />
<br />
<br />
[[File:sunspot_mos2.png|thumb|right|'''Fig. 4.''' The pattern of mosaic AFTER the re-calculation of the direction.]]<br />
<br />
<br />
During most solar observations, the antennas are tracking a structure on the Sun according to the solar differential rotation. The image frame is fixed on the solar frame, but the frame is moving on the RA/Dec coordinate frame. If we do not do any measures, the pattern of pointing in the MOSAIC observation is shown in Figure 3 , and the shape of the pattern is a rhombus though the correct shape is the square.<br />
<br />
To correct the MOSAIC pattern, we re-calculate the coordinate of each field. At first, we modify the coordinate of the field “0” from the reference time using {{fixplanets}} task. The reference time has to be the time when the antennas are directed to the field “0”.<br />
<source lang='python'><br />
#In Casa<br />
reftime = '2015/12/18/19:49:00'<br />
fixplanets(msc, field ='0', fixuvw = False, refant=ref_ant, reftime = reftime)<br />
</source><br />
We define that the modified coordinate of field ’0’ is as the reference coordinate, and re-calculate the coordinate of each field, as follows.<br />
<source lang='python'><br />
#In Casa<br />
tb.open(msc+'/FIELD', nomodify=True)<br />
phsCenOff = tb.getcol("PHASE_DIR")<br />
tb.close()<br />
refRaDec = aU.rad2radec(phsCenOff[0][0][0], phsCenOff[1][0][0], prec=1, hmsdms=True, delimiter=' ') <br />
for i in range(3, 151):<br />
raOff = phsCenOff[0][0][i] * 180./pi * 60. *60.<br />
deOff = phsCenOff[1][0][i] * 180./pi * 60. *60.<br />
offRaDec = aU.radec2deg(aU.radecOffsetToRadec(refRaDec, raOff, deOff, prec=1))<br />
offRaDecF = 'J2000 ' + aU.deg2radec(offRaDec[0], offRaDec[1], prec=1, hmsdms=True, delimiter=' ') <br />
fixplanets(msc, field =str(i), fixuvw = False, direction = offRaDecF)<br />
<br />
tb.open(msc+'/FIELD', nomodify=False)<br />
tgt_refdir = tb.getcol("RefDir_Ref")<br />
for id in range(3, len(tgt_refdir)):<br />
tb.putcell("RefDir_Ref", id, 21)<br />
tb.putcell("DelayDir_Ref", id, 21)<br />
tb.putcell("PhaseDir_Ref", id, 21)<br />
tb.close()<br />
</source><br />
Moreover, the direction in the pointing table has a bad influence to the coordinate system of the image synthesis. We erase the pointing table as follows:<br />
<source lang='python'><br />
#In Casa<br />
tb.open(msc+'/POINTING', nomodify = False)<br />
a = tb.rownumbers()<br />
tb.removerows(a)<br />
</source><br />
As a result, the pattern of the pointing in the MOSAIC observation becomes the map, as shown in Figure 4.</div>Jbertonehttps://casaguides.nrao.edu/index.php?title=Sunspot_Band6&diff=30873Sunspot Band62021-08-19T20:25:19Z<p>Jbertone: </p>
<hr />
<div>[[Category:ALMA]][[Category:Sun]][[Category:Continuum]]<br />
<br />
==Science Target Overview==<br />
<br />
<br />
[[File:sunspot_SolarMonitor.png|thumb|right|The multi-wavelength images of AR12470 observed with HMI/SDO, AIA/SDO, SWAP/PROBA2 and XRT/Hinode. The images are quoted from “SolarMonitor”]]<br />
<br />
<br />
These SV observations have been taken with the aim of testing ALMA solar capability at Band6 (239 GHz 1.25mm), on 18 December 2015 in 6th solar commissioning campaign.<br />
The target of the observations is the preceding sunspot of the active region “AR 12470”. The summary of the active region can be seen on the webpage in “SolarMonitor”[https://www.solarmonitor.org/index.php?date=20151218&region=12470]. The preceding sunspot of AR12470 is the biggest sunspot in the solar disk on the observing day, and its magnetic polarity is negative. In the active region, two “C” GOES-class flares occurred on the observing day, but the time range of the observations does not cover the flares.<br />
<br />
==ALMA Data Overview==<br />
<br />
The CASA Guide shows the calibration, imaging and feathering steps for the ALMA Science Verification data of a sunspot.<br />
<br />
The ALMA scientific commissioning observations on the sunspot in Band6 were carried out on December 18th, 2015. The observations consist of two individual datasets (ASDMs) as follows:<br />
<br />
* uid___A002_Xae00c5_X2a8d<br />
* uid___A002_Xae00c5_X2e6b<br />
<br />
The first dataset (uid___A002_Xae00c5_X2a8d) was taken with the heterogeneous array that consists from 21 12m-antennas and 9 7m-antennas. At the solar observation, the all antennas were connected to the 64-channel correlator. Hence, the combine process between the data obtained with 12m- and 7m- arrays is not needed. The spectrum setup for the observation is the Time Domain Mode (TDM) with 128 channels per one spectrum window (2GHz), and the frequency of the first Local Oscillator (LO1) is set at 239 GHz as the same as the default observing frequency of the solar observations with Band6 in Cycle 4. For the observation, the bandpass calibrator is J1924-2914, and the flux and phase calibrators are J1733-1304 (NRAO 530).<br />
<br />
The second dataset (uid___A002_Xae00c5_X2e8b) was taken with the TotalPower (TP)-array using the fast-scanning mode. The field of view of the map is a circle with 2400” diameter (Full Sun). The frequency of the LO1 is the same as the previous dataset, but any correlator is not used for taking the dataset. The dataset consists from the output of the square-law detectors at the four basebands. Therefore, the number of the channels per one spectrum window is one.<br />
<br />
'''Using the data for publication:'''<br />
The following statement should be included in the acknowledgment of papers using the datasets listed above: "This paper makes use of the following ALMA data: ADS/JAO.ALMA#2011.0.00020.SV. ALMA is a partnership of ESO (representing its member states), NSF (USA) and NINS (Japan), together with NRC (Canada) and NSC and ASIAA (Taiwan), and KASI (Republic of Korea), in cooperation with the Republic of Chile. The Joint ALMA Observatory is operated by ESO, AUI/NRAO and NAOJ."<br />
<br />
==Obtaining the Data==<br />
<br />
To download the data, click on the regional closest to your location:<br />
<br />
[https://alma-dl.mtk.nao.ac.jp/ftp/alma/sciver/2015AR149ptBand6/ East Asia]<br />
<br />
[https://almascience.eso.org/almadata/sciver/2015AR149ptBand6/ Europe]<br />
<br />
[https://almascience.nrao.edu/almadata/sciver/2015AR149ptBand6/ North America]<br />
<br />
Here you find three gzipped tar files, which after unpacking, will create three directories:<br />
<br />
* '''Sunspot_Band6_UncalibratedData''' - This file contains the raw data files in ALMA Science Data Model (ASDM) format, and the reduction and imaging scripts for CASA 4.7.<br />
* '''Sunspot_Band6_CalibratedData''' - The fully-calibrated u-v data and SD data, ready for image synthesis, and the imaging scripts for CASA 4.7.<br />
* '''Sunspot_Band6_ReferenceImages''' - The file includes the FITS files of the synthesized image obtained with the 12m+7m array, full-sun map obtained with TP-array, and combined image from the synthesized image and full-sun map.<br />
<br />
You can work with these data products at any of the three stages. To see which files you will need, read on below. The downloads to your local computer will take some time, so you may wish to begin them now.<br />
<br />
==Sunspot Data Reduction Tutorial==<br />
<br />
The tutorial (or casaguide) for reduction these data using CASA version 5.1 has been split into calibration imaging and feathering pages:<br />
<br />
1) '''[[Sunspot_Band6_Calibration]]''' This section of the tutorial steps you through inspection and calibration of the visibility (u-v) data. To complete this part, you will need the data in the Sunspot_Band6_UncalibratedData directory.<br />
<br />
2) '''[[Sunspot_Band6_SingleDish]]''' This section of the tutorial steps you through calibration and imaging from the TP-array data. To complete this section, you will need the data in the Sunspot_Band6_UncalibratedData directory.<br />
<br />
3) '''[[Sunspot_Band6_Imaging]]''' This section of the tutorial focuses on constructing images from the fully calibrated visibility data. If you wish to skip calibration and proceed directly to this part of the tutorial, you will need the fully-calibrated visibility data in the Sunspot_Band6_CalibratedData directory.<br />
<br />
4) '''[[Sunspot_Band6_Feathering]]''' This section of tutorial focuses on the “feathering” process from the synthesized image and single-dish map. To complete this section, you will need the fully-calibrated visibility data in the Sunspot_Band6_CalibratedData directory, and carry out the process described in 3).<br />
<br />
'''For the tutorials, both CASA 5.1 (or later) and the Analysis Utilities [https://casaguides.nrao.edu/index.php/Analysis_Utilities] package are required.'''<br />
<br />
==How to use this casaguide==<br />
<br />
The scripts that include all commands shown in the above tutorials are provided from the download sites described in [[#Obtaining the Data]]. After downloading the scripts, you can carry out the reduction processes along the tutorial. The detail and usage of the scripts are described in each above tutorial.<br />
-----<br />
<br />
'''Caution: The single-dish images and feathering images in the solar SV data packages (ex. Sunspot_Band6_ReferenceImages.tgz) were not applied the re-scaling to the single-dish data (see [[Sunspot_Band6_SingleDish_for_CASA_5.1#Calibration_.28re-scaling.29_after_Imaging]]). Please do not use the reference image for your science.'''</div>Jbertone