EVLA Continuum Tutorial 3C391-CASA3.4: Difference between revisions

From CASA Guides
Jump to navigationJump to search
← Older edit
Smyers (talk | contribs)
727 edits
Jott (talk | contribs)
operators
4,947 edits
No edit summary
 
(22 intermediate revisions by one other user not shown)
Line 268: Line 268:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms',flagbackup=T,  
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', flagbackup=T,  
        mode='manualflag',selectdata=T,scan='1')
          mode='manual', scan='1')
</source>
</source>


[[Image:3C391_flagdata.png|200px|thumb|right|flagdata inputs]]
[[Image:3C391_tflagdata.png|200px|thumb|right|tflagdata inputs]]
* <tt>flagbackup=T</tt> : A comment is warranted on the setting of flagbackup (here set to T or True).  If set to True, {{flagdata}} will save a copy of the existing set of flags <em>before</em> entering any new flags.  The setting of flagbackup is therefore a matter of some taste.  One could choose not to save any flags or only save "major" flags, or one could save every flag.  (One of the authors of this document was glad that flagbackup was set to True as he recently ran {{flagdata}} with a typo in one of the entries.)
 
* <tt>flagbackup=T</tt> : A comment is warranted on the setting of flagbackup (here set to T or True).  If set to True, {{tflagdata}} will save a copy of the existing set of flags <em>before</em> entering any new flags.  The setting of flagbackup is therefore a matter of some taste.  One could choose not to save any flags or only save "major" flags, or one could save every flag.  (One of the authors of this document was glad that flagbackup was set to True as he recently ran {{tflagdata}} with a typo in one of the entries.)
* <tt>mode='manualflag'</tt> : Specific data are going to be selected to be edited.   
* <tt>mode='manualflag'</tt> : Specific data are going to be selected to be edited.   
* <tt>selectdata=T</tt> : In order to select the specific data to be flagged, selectdata has to be set to True.  Once selectdata is set to True, then the various data selection options become visible (use ''help flagdata'' to see the possible options).  In this case, scan='1' is chosen to select only the first scan.  Note that scan expects an entry in the form of a <em>string</em>.  (scan=1 would generate an error.)
<!--* <tt>selectdata=T</tt> : In order to select the specific data to be flagged, selectdata has to be set to True.  Once selectdata is set to True, then the various data selection options become visible (use ''help flagdata'' to see the possible options).  In this case, -->
* scan='1' is chosen to select only the first scan.  Note that scan expects an entry in the form of a <em>string</em>.  (scan=1 would generate an error.)


If satisfied with the inputs, run this task.  The initial display in the logger will include  
If satisfied with the inputs, run this task.  The initial display in the logger will include  
<pre style="background-color: #ffe4b5;">
<pre style="background-color: #ffe4b5;">
##########################################
##########################################
##### Begin Task: flagdata          #####
##### Begin Task: tflagdata          #####
flagdata::::casa
 
attached MS [...]
Table type is Measurement Set
Saving current flags to manualflag_1 before applying new flags
...
Creating new backup flag file called manualflag_1
Backup original flags before applying new flags
Saving current flags to tflagdata_1 before applying new flags
Creating new backup flag file called tflagdata_1
</pre>
</pre>
which indicates that, among other things, the flags that existed in the data set prior to this run will be saved to another file called manualflag_1.  Should one ever desire to revert to the data prior to this run, the task {{flagmanager}} could be used.
which indicates that, among other things, the flags that existed in the data set prior to this run will be saved to another file called tflagdata_1.  Should one ever desire to revert to the data prior to this run, the task {{flagmanager}} could be used.
 




Line 293: Line 296:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms',flagbackup=T,mode='manualflag',  
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', flagbackup=T, mode='manual',  
        selectdata=T,antenna='ea13,ea15')
          antenna='ea13,ea15')
</source>
</source>
* <tt>antenna='ea13,ea15'</tt> : Once again, this parameter requires a string input.  Remember that antenna='ea13' and 'antenna='13' are <em>not</em> the same antenna.  (See the discussion after our call to {{listobs}} above.)
* <tt>antenna='ea13,ea15'</tt> : Once again, this parameter requires a string input.  Remember that antenna='ea13' and 'antenna='13' are <em>not</em> the same antenna.  (See the discussion after our call to {{listobs}} above.)
Line 303: Line 306:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms',mode='quack',  
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', mode='quack',  
        quackinterval=10.0,quackmode='beg')
          quackinterval=10.0, quackmode='beg')
</source>
</source>
* <tt>mode='quack'</tt> : Quack is another mode in which the same edit will be applied to all scans for all baselines.
* <tt>mode='quack'</tt> : Quack is another mode in which the same edit will be applied to all scans for all baselines.
Line 315: Line 318:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
clearstat() # This removes any existing table locks generated by flagdata
clearstat() # This removes any existing table locks generated by tflagdata
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',xaxis='',yaxis='',  
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',xaxis='',yaxis='',  
       averagedata=False,transform=False,extendflag=False,  
       averagedata=False,transform=False,extendflag=False,  
Line 349: Line 352:
As a general data editing and examination strategy, at this stage in the data reduction process, one wants to focus on the calibrators.  The data reduction strategy is to determine various corrections from the calibrators, then apply these correction factors to the science data.  The 3C 286 data look relatively clean.  There are no wildly egregious data (e.g., amplitudes that are 100,000x larger than the rest of the data).  One may notice that there are antenna-to-antenna variations (under the 'Display' tab, select 'Colorize by Antenna1').  These antenna-to-antenna variations are acceptable, that's what calibration will help determine.
As a general data editing and examination strategy, at this stage in the data reduction process, one wants to focus on the calibrators.  The data reduction strategy is to determine various corrections from the calibrators, then apply these correction factors to the science data.  The 3C 286 data look relatively clean.  There are no wildly egregious data (e.g., amplitudes that are 100,000x larger than the rest of the data).  One may notice that there are antenna-to-antenna variations (under the 'Display' tab, select 'Colorize by Antenna1').  These antenna-to-antenna variations are acceptable, that's what calibration will help determine.


'''Do not''' close the plotms GUI after running {{plotms}}, or you will need to exit casapy and restart if at any point you wish to run plotms again, otherwise the GUI will not come up a second time.
<b>Warning:</b> '''Do not''' close the plotms GUI after running {{plotms}}, or you will need to exit casapy and restart if at any point you wish to run plotms again, otherwise the GUI will not come up a second time.
 
[[Image:plotms_3c391-datastream.png|200px|right|thumb|datastream view of MS]]
One final useful plot we will make is a "datastream" plot of the antenna2 in a baseline for the data versus ea01. This shows (assuming ea01 is there the entire run) when various antennas drop out.
<source lang="python">
# In CASA
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='',correlation='RR,LL',
      timerange='',antenna='ea01',spw='0:31',
      xaxis='time',yaxis='antenna2',
      plotrange=[-1,-1,0,26],coloraxis='field')
</source>
See the plot to right (after fiddling with the symbol sizes to improve visibility). This plot was actually made from the MS later on after ea05 (index 4) was flagged. From this display you can see that antenna 3 (ea04) is missing the last scan on 3C84 (which will have some implications later on), antennas 1,6,13 (ea02,ea08,ea16) have some missing blocks of time in the middle of the run, and 23 (ea26) is missing a scan or two near the end.


== Calibrating the Data ==
== Calibrating the Data ==
Line 534: Line 548:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.G0',  
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.G0all',  
         field='J1331+3030',refant='ea21',spw='0:27~36',
         field='0,1,9', refant='ea21', spw='0:27~36',
         gaintype='G',calmode='p',solint='int',minsnr=5,
         gaintype='G',calmode='p', solint='int',  
        gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'])
        minsnr=5, gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'])
</source>
</source>


Line 571: Line 585:
# In CASA
# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',
         xaxis='time',yaxis='phase',poln='L',iteration='antenna',plotrange=[-1,-1,-180,180])
         xaxis='time', yaxis='phase', poln='L',  
        iteration='antenna', plotrange=[-1,-1,-180,180])
</source>
</source>
Antennas other than ea05 look OK.
Antennas other than ea05 look OK.
Line 578: Line 593:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
flagdata(vis='3c391_ctm_mosaic_10s_spw0.ms',
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms',
        flagbackup=T,mode='manualflag',selectdata=T,antenna='ea05')
          flagbackup=T, mode='manual', antenna='ea05')
</source>
</source>


Line 638: Line 653:
# In CASA
# In CASA
bandpass(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.B0',
bandpass(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.B0',
         field='J1331+3030',spw='',refant='ea01',solnorm=True,combine='scan',  
         field='J1331+3030',spw='',refant='ea21',solnorm=True,combine='scan',  
         solint='inf',bandtype='B',
         solint='inf',bandtype='B',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
Line 820: Line 835:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.Kcross',
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.Kcross',
         field='J1331+3030',spw='0:5~58',
         field='J1331+3030', spw='0:5~58',
         gaintype='KCROSS',solint='inf',combine='scan',refant='ea21',
         gaintype='KCROSS', solint='inf', combine='scan', refant='ea21',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                   '3c391_ctm_mosaic_10s_spw0.K0',
                   '3c391_ctm_mosaic_10s_spw0.K0',
Line 828: Line 843:
                   '3c391_ctm_mosaic_10s_spw0.G1'],
                   '3c391_ctm_mosaic_10s_spw0.G1'],
         gainfield=['','','','J1331+3030'],
         gainfield=['','','','J1331+3030'],
         interp=['linear','nearest','nearest','linear'])
         interp=['linear','nearest','nearest','linear'], parang=T)
</source>
</source>


Line 838: Line 853:
         figfile='plotcal_3c391-Kcross-delay.png')
         figfile='plotcal_3c391-Kcross-delay.png')
</source>
</source>
As expected there is a single value for R versus L (with L delay set to zero) across all antennas. In our case it is 7.157 ns.
As expected there is a single value for R versus L (with L delay set to zero) across all antennas. The solution is reported in the logger, and is 7.149 ns. <!-- was 7.157 ns for parang=F -->


Note that if we did not solve for this delay, it would be absorbed into the phases per channel of the following Df and Xf solutions.  This would not cause us problems, as we are not solving for the Q+iU polarization of our D-term calibrator (we are using unpolarized 3C84 for that) but if we were (e.g. using our gain calibrator J1822-0938 with <tt>poltype='Df+QU'</tt>) then this step would be essential.
Note that if we did not solve for this delay, it would be absorbed into the phases per channel of the following Df and Xf solutions.  This would not cause us problems, as we are not solving for the Q+iU polarization of our D-term calibrator (we are using unpolarized 3C84 for that) but if we were (e.g. using our gain calibrator J1822-0938 with <tt>poltype='Df+QU'</tt>) then this step would be essential.
Line 926: Line 941:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
 
polcal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.X1',
      field='J1331+3030',refant='ea21',
      poltype='Xf',solint='inf',combine='scan',
      gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                  '3c391_ctm_mosaic_10s_spw0.K0',
                  '3c391_ctm_mosaic_10s_spw0.B0',
                  '3c391_ctm_mosaic_10s_spw0.G1',
                  '3c391_ctm_mosaic_10s_spw0.Kcross',
                  '3c391_ctm_mosaic_10s_spw0.D2'],
      gainfield=['','','','J1331+3030','',''],
      interp=['linear','nearest','nearest','linear','nearest','nearest'])
</source>
</source>


Line 962: Line 987:
{{fluxscale}} will print to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument.  These are also captured in the return variable from the task. You should examine the output to ensure that it looks sensible.  If one's data set has more than 1 spectral window, depending upon where they are spaced and the spectrum of the source, it is quite possible to find (quite) different flux densities at the different frequencies for the secondary calibrators.  Example output would be
{{fluxscale}} will print to the CASA logger the derived flux densities of all calibrator sources specified with the ''transfer'' argument.  These are also captured in the return variable from the task. You should examine the output to ensure that it looks sensible.  If one's data set has more than 1 spectral window, depending upon where they are spaced and the spectrum of the source, it is quite possible to find (quite) different flux densities at the different frequencies for the secondary calibrators.  Example output would be
<pre style="background-color: #fffacd;">
<pre style="background-color: #fffacd;">
  CASA <172>: myscale['1']
CASA <314>: myscale['1']
    Out[172]:  
  Out[314]:  
  {'fieldName': 'J1822-0938',
{'fieldName': 'J1822-0938',
    'fluxd': array([ 2.33172054]),
'fluxd': array([ 2.33283596]),
    'fluxdErr': array([ 0.00638675]),
'fluxdErr': array([ 0.0063605]),
    'numSol': array([46], dtype=int32)}
'numSol': array([46], dtype=int32)}


  CASA <173>: myscale['9']
CASA <315>: myscale['9']
    Out[173]:  
  Out[315]:  
  {'fieldName': 'J0319+4130',
{'fieldName': 'J0319+4130',
    'fluxd': array([ 13.75809616]),
'fluxd': array([ 13.76485928]),
    'fluxdErr': array([ 0.03865178]),
'fluxdErr': array([ 0.03852698]),
    'numSol': array([44], dtype=int32)}
'numSol': array([44], dtype=int32)}
</pre>
</pre>
or in the logger
or in the logger
Line 983: Line 1,008:
INFO    fluxscale::::    Found reference field(s): J1331+3030
INFO    fluxscale::::    Found reference field(s): J1331+3030
INFO    fluxscale::::    Found transfer field(s):  J1822-0938 J0319+4130
INFO    fluxscale::::    Found transfer field(s):  J1822-0938 J0319+4130
INFO    fluxscale::::    Flux density for J1822-0938 in SpW=0 is: 2.33172 +/- 0.00638675 (SNR = 365.087, N = 46)
INFO    fluxscale::::    Flux density for J1822-0938 in SpW=0 is: 2.33284 +/- 0.0063605 (SNR = 366.769, N = 46)
INFO    fluxscale::::    Flux density for J0319+4130 in SpW=0 is: 13.7581 +/- 0.0386518 (SNR = 355.95, N = 44)
INFO    fluxscale::::    Flux density for J0319+4130 in SpW=0 is: 13.7649 +/- 0.038527 (SNR = 357.278, N = 44)
INFO    fluxscale::Calibrater::fluxscale        Storing result in 3c391_ctm_mosaic_10s_spw0.fluxscale1
INFO    fluxscale::Calibrater::fluxscale        Storing result in 3c391_ctm_mosaic_10s_spw0.fluxscale1
INFO    fluxscale::::  Writing solutions to table: 3c391_ctm_mosaic_10s_spw0.fluxscale1
INFO    fluxscale::::  Writing solutions to table: 3c391_ctm_mosaic_10s_spw0.fluxscale1
Line 990: Line 1,015:


The [http://www.vla.nrao.edu/astro/calib/manual/csource.html VLA calibrator manual] can be used to check whether the derived flux densities look sensible.  Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.
The [http://www.vla.nrao.edu/astro/calib/manual/csource.html VLA calibrator manual] can be used to check whether the derived flux densities look sensible.  Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.
We plot the rescaled amplitudes from this table:
<source lang="python">
# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.fluxscale1',xaxis='time',yaxis='amp',
        poln='R',figfile='plotcal_3c391-fluxscale1-amp-R.png')
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.fluxscale1',xaxis='time',yaxis='amp',
        poln='L',figfile='plotcal_3c391-fluxscale1-amp-L.png')
</source>
You can see (in the plots shown below) that the amplitude gain factors are now similar across sources, compared to the raw factors in the G1 table.
{|
| [[Image:plotcal_3c391-fluxscale1-amp-R.png|200px|thumb|left|post-fluxscale amp solutions, R pol]]
| [[Image:plotcal_3c391-fluxscale1-amp-L.png|200px|thumb|center|post-fluxscale amp solutions, L pol]]
|}


== Applying the Calibration ==
== Applying the Calibration ==
Line 1,003: Line 1,043:
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.B0',  
                    '3c391_ctm_mosaic_10s_spw0.K0',
                     '3c391_ctm_mosaic_10s_spw0.D1',
                     '3c391_ctm_mosaic_10s_spw0.B0',
                     '3c391_ctm_mosaic_10s_spw0.Kcross',
                    '3c391_ctm_mosaic_10s_spw0.D2',
                     '3c391_ctm_mosaic_10s_spw0.X1'],
                     '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J1331+3030','','',''],  
         gainfield=['','J1331+3030','','','','',''],  
         interp=['','nearest','','',''],
         interp=['','nearest','','','','',''],
         parang=True,calwt=F)
         parang=True,calwt=F)
#
#
Line 1,014: Line 1,056:
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.B0',  
                    '3c391_ctm_mosaic_10s_spw0.K0',
                     '3c391_ctm_mosaic_10s_spw0.D1',
                     '3c391_ctm_mosaic_10s_spw0.B0',
                     '3c391_ctm_mosaic_10s_spw0.Kcross',
                    '3c391_ctm_mosaic_10s_spw0.D2',
                     '3c391_ctm_mosaic_10s_spw0.X1'],
                     '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J0319+4130','','',''],  
         gainfield=['','J0319+4130','','','','',''],  
         interp=['','nearest','','',''],
         interp=['','nearest','','','','',''],
         parang=True,calwt=F)
         parang=True,calwt=F)
#
#
Line 1,025: Line 1,069:
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.B0',  
                    '3c391_ctm_mosaic_10s_spw0.K0',
                     '3c391_ctm_mosaic_10s_spw0.D1',
                     '3c391_ctm_mosaic_10s_spw0.B0',
                     '3c391_ctm_mosaic_10s_spw0.Kcross',
                    '3c391_ctm_mosaic_10s_spw0.D2',
                     '3c391_ctm_mosaic_10s_spw0.X1'],
                     '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J1822-0938','','',''],  
         gainfield=['','J1822-0938','','','','',''],  
         interp=['','nearest','','',''],
         interp=['','nearest','','','','',''],
         parang=True,calwt=F)
         parang=True,calwt=F)
</source>
</source>


* <tt>gaintable</tt> : We provide a Python list of the calibration tables to be applied.  This list must contain the antenna position corrections (in 3c391_ctm_mosaic_10s_spw0.antpos), the properly-scaled gain calibration for the amplitudes and phases (in 3c391_ctm_mosaic_10s_spw0.fluxscale1) which were just made using {{fluxscale}}, the bandpass solutions (in 3c391_ctm_mosaic_10s_spw0.B0), the leakage calibration (in 3c391_ctm_mosaic_10s_spw0.D1), and the R-L phase corrections (in 3c391_ctm_mosaic_10s_spw0.X1).  While the latter three tables were derived using a particular calibrator source, the table containing the gain solutions for amplitude and phase was derived separately for each individual calibrator.
* <tt>gaintable</tt> : We provide a Python list of the calibration tables to be applied.  This list must contain the antenna position corrections (in .antpos), the properly-scaled gain calibration for the amplitudes and phases (in .fluxscale1) which were just made using {{fluxscale}}, the parallel-hand delays (in .K0), the bandpass solutions (in .B0), the cross-hand delays (in .Kcross), the leakage calibration (in .D2 derived by our second solution), and the R-L phase corrections (in .X1).
* <tt>gainfield, interp</tt> : To ensure that we use the correct gain amplitudes and phases for a given calibrator (those derived on that same calibrator), then for each calibrator source, we need to specify the particular subset of gain solutions to be applied. This requires use of the ''gainfield'' and ''interp'' arguments; these are both Python lists, and for the list item corresponding to the calibration table made by {{fluxscale}}, we set ''gainfield'' to the field name corresponding to that calibrator, and the desired interpolation type (''interp'') to ''nearest''.
* <tt>gainfield, interp</tt> : To ensure that we use the correct gain amplitudes and phases for a given calibrator (those derived on that same calibrator), then for each calibrator source, we need to specify the particular subset of gain solutions to be applied. This requires use of the ''gainfield'' and ''interp'' arguments; these are both Python lists, and for the list item corresponding to the calibration table made by {{fluxscale}}, we set ''gainfield'' to the field name corresponding to that calibrator, and the desired interpolation type (''interp'') to ''nearest''.
* <tt>parang</tt> : Since we have performed polarization calibration, we '''must''' set ''parang=True'', or we will discard all that hard work we did earlier.  However, if you skipped the [[http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391#Polarization_Calibration Polarization_Calibration]] section, the tables 3c391_ctm_mosaic_10s_spw0.D1 and 3c391_ctm_mosaic_10s_spw0.X1 will not exist.  In this case, you should leave out the final two tables in the ''gaintable'' list, and the final two sets of empty elements in the ''gainfield'' list each time you run {{applycal}} above.  You should also set ''parang=False''.
* <tt>parang</tt> : Since we have performed polarization calibration, we '''must''' set ''parang=True'', or we will discard all that hard work we did earlier.  However, if you skipped the [[http://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391#Polarization_Calibration Polarization_Calibration]] section, the .Kcross, .D2 and .X1 will not exist.  In this case, you should leave out these tables from the ''gaintable'' list, and the corresponding sets of elements in the ''gainfield'' list each time you run {{applycal}} above.  You should also set ''parang=False''.
* <tt>calwt=F</tt> : At the time of writing, the EVLA is not yet recording real weights, thus trying to calibrate them can produce nonsensical results.  In particular, experience has shown that calibrating the weights will lead to problems especially in the self-calibration steps.
* <tt>calwt=F</tt> : At the time of writing, we are not yet using system calibration data to compute real (1/Jy<sup>2</sup>) weights, thus trying to calibrate them can produce nonsensical results.  In particular, experience has shown that calibrating the weights will lead to problems especially in the self-calibration steps.


Finally, we apply the calibration to the target fields in the mosaic, linearly interpolating the gain solutions from the secondary calibrator, J1822-0938.  In this case however, we want to apply the amplitude and phase gains derived from the secondary calibrator, J1822-0938, since that is close to the target source on the sky, and we assume that the gains applicable to the target source are very similar to those derived in the direction of the secondary calibrator.  Of course, this is not strictly true, since the gains on J1822-0938 were derived at a different time and in a different position on the sky from the target.  However, assuming that the calibrator was sufficiently close to the target, and the weather was sufficiently well-behaved, then this is a reasonable approximation, and should get us a sufficiently good calibration that we can later use self-calibration to correct for the small inaccuracies thus introduced.
Finally, we apply the calibration to the target fields in the mosaic, linearly interpolating the gain solutions from the secondary calibrator, J1822-0938.  In this case however, we want to apply the amplitude and phase gains derived from the secondary calibrator, J1822-0938, since that is close to the target source on the sky, and we assume that the gains applicable to the target source are very similar to those derived in the direction of the secondary calibrator.  Of course, this is not strictly true, since the gains on J1822-0938 were derived at a different time and in a different position on the sky from the target.  However, assuming that the calibrator was sufficiently close to the target, and the weather was sufficiently well-behaved, then this is a reasonable approximation, and should get us a sufficiently good calibration that we can later use self-calibration to correct for the small inaccuracies thus introduced.
Line 1,047: Line 1,093:
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',  
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                     '3c391_ctm_mosaic_10s_spw0.B0',  
                    '3c391_ctm_mosaic_10s_spw0.K0',
                     '3c391_ctm_mosaic_10s_spw0.D1',
                     '3c391_ctm_mosaic_10s_spw0.B0',
                     '3c391_ctm_mosaic_10s_spw0.Kcross',
                    '3c391_ctm_mosaic_10s_spw0.D2',
                     '3c391_ctm_mosaic_10s_spw0.X1'],
                     '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J1822-0938','','',''],  
         gainfield=['','J1822-0938','','','','',''],  
         interp=['','linear','','',''],
         interp=['','linear','','','','',''],
         parang=True,calwt=F)
         parang=True,calwt=F)
</source>
</source>
Line 1,103: Line 1,151:
</source>
</source>


Inspecting the data at this stage may well show up previously-unnoticed bad data.  Plotting up the '''corrected''' amplitude against UV distance, or against time is a good way to find such issues.  If you find bad data, you can remove them via interactive flagging in {{plotms}}, or via manual flagging in {{flagdata}} once you have identified the offending antennas/baselines/channels/times.  When you are happy that all data (particularly on your target source) look good, you may proceed.
Inspecting the data at this stage may well show up previously-unnoticed bad data.  Plotting up the '''corrected''' amplitude against UV distance, or against time is a good way to find such issues.  If you find bad data, you can remove them via interactive flagging in {{plotms}}, or via manual flagging in {{tflagdata}} once you have identified the offending antennas/baselines/channels/times.  When you are happy that all data (particularly on your target source) look good, you may proceed.


Now that the calibration has been applied to the target data, we can split off the science targets, creating a new, calibrated measurement set containing all the target fields.
Now that the calibration has been applied to the target data, we can split off the science targets, creating a new, calibrated measurement set containing all the target fields.
Line 1,117: Line 1,165:
* <tt>field</tt> : We wish to put all the mosaic pointings into a single measurement set, for imaging and joint deconvolution.
* <tt>field</tt> : We wish to put all the mosaic pointings into a single measurement set, for imaging and joint deconvolution.


== Imaging ==
== Initial Imaging ==


Now that we have split off the target data into a separate measurement set with all the calibration applied, it's time to make an image.  Recall from the lectures that the visibility data and the sky brightness distribution (a.k.a. image) are Fourier transform pairs
Now that we have split off the target data into a separate measurement set with all the calibration applied, it's time to make an image.  Recall from the lectures that the visibility data and the sky brightness distribution (a.k.a. image) are Fourier transform pairs
Line 1,127: Line 1,175:
The <math>u</math> and <math>v</math> coordinates are the baselines, measured in units of the observing wavelength while the <math>l</math> and <math>m</math> coordinates are the direction cosines on the sky.  For generality, the sky coordinates are written in terms of direction cosines, but for most EVLA (and ALMA) observations they can be related simply to the right ascension (<math>l</math>) and declination (<math>m</math>).  Also recall from the lectures that this equation is valid only if the <math>w</math> coordinate of the baselines can be neglected.  This assumption is almost always true at high frequencies and smaller EVLA configurations (such as the 4.6 GHz, D-configuration observations here); the <math>w</math> coordinate cannot be neglected at lower frequencies and larger configurations (e.g., 0.33 GHz, A-configuration observations).  This expression also neglects other factors, such as the shape of the primary beam.  For more information on imaging, see [[http://casa.nrao.edu/docs/userman/UserManch5.html#x236-2330005 Synthesis Imaging]] within the CASA Reference Manual.
The <math>u</math> and <math>v</math> coordinates are the baselines, measured in units of the observing wavelength while the <math>l</math> and <math>m</math> coordinates are the direction cosines on the sky.  For generality, the sky coordinates are written in terms of direction cosines, but for most EVLA (and ALMA) observations they can be related simply to the right ascension (<math>l</math>) and declination (<math>m</math>).  Also recall from the lectures that this equation is valid only if the <math>w</math> coordinate of the baselines can be neglected.  This assumption is almost always true at high frequencies and smaller EVLA configurations (such as the 4.6 GHz, D-configuration observations here); the <math>w</math> coordinate cannot be neglected at lower frequencies and larger configurations (e.g., 0.33 GHz, A-configuration observations).  This expression also neglects other factors, such as the shape of the primary beam.  For more information on imaging, see [[http://casa.nrao.edu/docs/userman/UserManch5.html#x236-2330005 Synthesis Imaging]] within the CASA Reference Manual.


[[Image:3c391_clean_param.png|200px|thumb|left|clean parameters]]
CASA has a single task, {{clean}} which both Fourier transforms the data and deconvolves the resulting image. For the purposes of this tutorial Part 1, we will make a simple mosaic clean image in Stokes I only.  In Part 2 we will proceed to make multiscale mosaic images in IQUV and to analyze them.


CASA has a single task, {{clean}} which both Fourier transforms the data and deconvolves the resulting image.
To make this simple clean image interactively:
Assuming you did the polarization calibration earlier, a command line call to image and deconvolve the dataset would be:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
clean(vis='3c391_ctm_mosaic_spw0.ms',imagename='3c391_ctm_spw0_IQUV',
clean(vis='3c391_ctm_mosaic_spw0.ms',
      imagename='3c391_ctm_spw0_noms_I',
       field='',spw='',
       field='',spw='',
       mode='mfs',
       mode='mfs',
       niter=5000,
       niter=5000,
       gain=0.1, threshold='1.0mJy',
       gain=0.1, threshold='1.0mJy',
       psfmode='clarkstokes',
       psfmode='clark',
       imagermode='mosaic', ftmachine='mosaic',
       imagermode='mosaic', ftmachine='mosaic',
       multiscale=[0, 6, 18, 54], smallscalebias=0.9,
       multiscale=[0],  
       interactive=True,
       interactive=True,
       imsize=[576,576], cell=['2.5arcsec','2.5arcsec'],
       imsize=[480,480], cell=['2.5arcsec','2.5arcsec'],
       stokes='IQUV',
       stokes='I',
       weighting='briggs',robust=0.0,
       weighting='briggs',robust=0.5,
       usescratch=True)
       usescratch=True)
</source>
</source>
If you previously skipped the polarization calibration, you should instead set ''stokes='I' '' and ''psfmode='clark' ''.


{{clean}} is a powerful task, with many inputs, and a certain amount of experimentation may be (likely is) required.
{{clean}} is a powerful task, with many inputs, and a certain amount of experimentation may be (likely is) required.
* <tt>vis='3c391_ctm_mosaic_spw0.ms'</tt> : this split MS contains our 7-pt mosaic fields, now indexed as 0-6. Field 0 is the central field of the mosaic (you can use <tt>listobs</tt> to verify this).
* <tt>imagename='3c391_ctm_spw0_noms_I'</tt> : our output images will all start with this, e.g. 3c391_ctm_spw0_noms_I.image
* <tt>mode='mfs'</tt> : Use multi-frequency synthesis imaging.  The fractional bandwidth of these data is non-zero (128 MHz at a central frequency of 4.6 GHz).  Recall that the <math>u</math> and <math>v</math> coordinates are defined as the baseline coordinates, measured in wavelengths.  Thus, slight changes in the frequency from channel to channel result in slight changes in <math>u</math> and <math>v</math>.  There is a concomitant improvement in <math>u</math>-<math>v</math> coverage if the visibility data from the multiple spectral channels are gridded separately onto the <math>u</math>-<math>v</math> plane, as opposed to treating all spectral channels as having the same frequency.
* <tt>mode='mfs'</tt> : Use multi-frequency synthesis imaging.  The fractional bandwidth of these data is non-zero (128 MHz at a central frequency of 4.6 GHz).  Recall that the <math>u</math> and <math>v</math> coordinates are defined as the baseline coordinates, measured in wavelengths.  Thus, slight changes in the frequency from channel to channel result in slight changes in <math>u</math> and <math>v</math>.  There is a concomitant improvement in <math>u</math>-<math>v</math> coverage if the visibility data from the multiple spectral channels are gridded separately onto the <math>u</math>-<math>v</math> plane, as opposed to treating all spectral channels as having the same frequency.
* <tt>niter=5000,gain=0.1,threshold='1.0mJy'</tt> : Recall that the CLEAN gain is the amount by which a CLEAN component is subtracted during the CLEANing process.  <tt>niter</tt> and <tt>threshold</tt> are (coupled) means of determining when to stop the CLEANing process, with <tt>niter</tt> specifying to find and subtract that many CLEAN components while <tt>threshold</tt> specifies a minimum flux density threshold a CLEAN component can have before CLEAN stops.  See also <tt>interactive</tt> below.  Imaging is an iterative process, and to set the threshold and number of iterations, it is usually wise to CLEAN interactively in the first instance, stopping when spurious emission from sidelobes (arising from gain errors) dominates the residual emission in the field.  Here, we have used our experience in interactive mode to set a threshold level based on the rms noise in the resulting image.  The number of iterations should then be set high enough to reach this threshold.
* <tt>niter=25000,gain=0.1,threshold='1.0mJy'</tt> : Recall that the CLEAN gain is the amount by which a CLEAN component is subtracted during the CLEANing process.  <tt>niter</tt> and <tt>threshold</tt> are (coupled) means of determining when to stop the CLEANing process, with <tt>niter</tt> specifying to find and subtract that many CLEAN components while <tt>threshold</tt> specifies a minimum flux density threshold a CLEAN component can have before CLEAN stops.  See also <tt>interactive</tt> below.  Imaging is an iterative process, and to set the threshold and number of iterations, it is usually wise to CLEAN interactively in the first instance, stopping when spurious emission from sidelobes (arising from gain errors) dominates the residual emission in the field.  Here, we have used our experience in interactive mode to set a threshold level based on the rms noise in the resulting image.  The number of iterations should then be set high enough to reach this threshold.
* <tt>interactive=T</tt> : Very often, particularly when one is exploring how a source appears for the first time, it can be valuable to interact with the CLEANing process.  If True, <tt>interactive</tt> causes a {{viewer}} window to appear.  One can then set CLEAN regions, restricting where CLEAN searches for CLEAN components, as well as monitor the CLEANing process.  A standard procedure is to set a large value for <tt>niter</tt>, and stop the CLEANing when it visually appears to be approaching the noise level.  This procedure also allows one to change the CLEANing region, in cases when low-level intensity becomes visible as the CLEANing process proceeds.  For more details, see [[http://casa.nrao.edu/docs/userman/UserMansu254.html#x292-2870005.3.14 Interactive Cleaning]], and the discussion below.
* <tt>interactive=True</tt> : Very often, particularly when one is exploring how a source appears for the first time, it can be valuable to interact with the CLEANing process.  If True, <tt>interactive</tt> causes a {{viewer}} window to appear.  One can then set CLEAN regions, restricting where CLEAN searches for CLEAN components, as well as monitor the CLEANing process.  A standard procedure is to set a large value for <tt>niter</tt>, and stop the CLEANing when it visually appears to be approaching the noise level.  This procedure also allows one to change the CLEANing region, in cases when low-level intensity becomes visible as the CLEANing process proceeds.  For more details, see [[http://casa.nrao.edu/docs/userman/UserMansu254.html#x292-2870005.3.14 Interactive Cleaning]], and the discussion below.
* <tt>imsize=[576],cell=['2.5arcsec']</tt> : See the discussion below regarding the setting of the image size and cell size.
* <tt>imsize=[480,480],cell=['2.5arcsec']</tt> : See the discussion below regarding the setting of the image size and cell size.
* <tt>stokes='IQUV',psfmode='clarkstokes'</tt> : Separate images will be made in all four polarizations (total intensity I, linear polarizations Q and U, and circular polarization V), and, with <tt>psfmode='clarkstokes'</tt>, the Clark CLEAN algorithm will deconvolve each Stokes plane separately thereby making the polarization image more independent of the total intensity.
* <tt>stokes='I',psfmode='clark'</tt> : A single image will be made for total intensity I.
* <tt>weighting='briggs',robust=0.0</tt> : 3C 391 has diffuse, extended emission that is (at least partially) resolved out by the interferometer owing to a lack of short spacings.  A naturally-weighted image would show large-scale patchiness in the noise.  In order to suppress this effect, Briggs weighting is used (intermediate between natural and uniform weighting), with a default robust factor of 0.
* <tt>weighting='briggs',robust=0.5</tt> : 3C 391 has diffuse, extended emission that is (at least partially) resolved out by the interferometer owing to a lack of short spacings.  A naturally-weighted image would show large-scale patchiness in the noise.  In order to suppress this effect, Briggs weighting is used (intermediate between natural and uniform weighting), with a default robust factor of 0.5 (which corresponds to something between natural and uniform weighting).
* <tt>imagermode='mosaic', ftmachine='mosaic'</tt> : The data consist of a 7-pointing mosaic, since the supernova remnant fills almost the full primary beam at 4.6 GHz.  A mosaic combines the data from all of the fields, with imaging and deconvolution being done jointly on all 7 fields.  A mosaic both helps compensate for the shape of the primary beam and reduces the amount of large (angular) scale structure that is resolved out by the interferometer.
* <tt>imagermode='mosaic', ftmachine='mosaic'</tt> : The data consist of a 7-pointing mosaic, since the supernova remnant fills almost the full primary beam at 4.6 GHz.  A mosaic combines the data from all of the fields, with imaging and deconvolution being done jointly on all 7 fields.  A mosaic both helps compensate for the shape of the primary beam and reduces the amount of large (angular) scale structure that is resolved out by the interferometer.
* <tt>multiscale=[0, 6, 18, 54], smallscalebias=0.9</tt> :  A multi-scale CLEANing algorithm is used because the supernova remnant contains both diffuse, extended structure on large spatial scales and finer filamentary structure on smaller scales. The settings for <tt>multiscale</tt> are in units of pixels, with 0 pixels equivalent to the traditional delta-function CLEAN.  The scales here are chosen to provide delta functions and then three logarithmically scaled sizes to fit to the data.  The first scale (6 pixels) is chosen to be comparable to the size of the beam.  The <tt>smallscalebias</tt> attempts to balance the weight given to larger scales, which often have more flux density, and the smaller scales, which often are brighter.  Considerable experimentation is likely to be necessary; one of the authors of this document found that it was useful to CLEAN several rounds with this setting, change to <tt>multiscale=[]</tt> and remove much of the smaller scale structure, then return to this setting.
* <tt>multiscale=[0]</tt> :  For now, we will clean with a single scale of size 0 (a classic clean delta-function)
* <tt>usescratch=True</tt> As in calibration, we will direct the imaging to make use of the MODEL_DATA scratch column.  There is the option in 3.4 to use scratchless model operation for some cases but is not yet recommended for all cases.


Setting the appropriate pixel depends upon basic optics aspects of interferometry.  Using [[plotms]] to look at the newly-calibrated, target-only data set,
Setting the appropriate pixel size depends upon basic optics aspects of interferometry.  Using [[plotms]] to look at the newly-calibrated, target-only data set,
<source lang="python">
<source lang="python">
# In CASA
# In CASA
plotms(vis='3c391_ctm_mosaic_spw0.ms',xaxis='uvwave',yaxis='amp')
plotms(vis='3c391_ctm_mosaic_spw0.ms',xaxis='uvwave',yaxis='amp',
      field='0',avgtime='30s',correlation='RR',
      plotfile='plotms_3c391-mosaic0-uvwave.png',overwrite=True)
</source>
</source>
[[Image:3c391 ctm spw0 uvplt.jpg|thumb|{{plotms}} GUI showing Amplitude vs UV Distance in wavelengths for 3C 391 at 4600 MHz]]
[[Image:plotms_3c391-mosaic0-uvwave.png|thumb|{{plotms}} plot showing Amplitude vs UV Distance in wavelengths for 3C 391 at 4600 MHz]]
one should obtain a plot similar to the one shown at the right with the (calibrated) visibility amplitude as a function of <math>u</math>-<math>v</math> distance.
one should obtain a plot similar to the one shown at the right with the (calibrated) visibility amplitude as a function of <math>u</math>-<math>v</math> distance.
The maximum baseline is about 16,000 wavelengths, i.e., an angular scale of 12 arcseconds (<math>\lambda/D=1/16000</math>).  The most effective CLEANing occurs with 3--5 pixels across the synthesized beam.  Above, a cell size of 2.5 arcseconds (just under 5 pixels per beam) is specified.  If only one value for the cell size is specified (as done here), the same value is used in both directions.
The maximum baseline is about 16,000 wavelengths, i.e., an angular scale of 12 arcseconds (<math>\lambda/D=1/16000</math>).  The most effective CLEANing occurs with 3--5 pixels across the synthesized beam.  Above, a cell size of 2.5 arcseconds (just under 5 pixels per beam) is specified.  If only one value for the cell size is specified (as done here), the same value is used in both directions.


The supernova remnant itself is known to have a diameter of order 9 arcminutes, corresponding to about 216 pixels for the chosen cell size.  The mosaic was set up with 7 fields, 1 centered on the remnant and 6 flanking fields; the spacing of the fields was chosen based on the size of the (antenna) primary beam.  In order to prevent image artifacts arising from aliasing due to the mosaicing, the image should be sized such that the supernova remnant is restricted to the inner quarter of the image. CASA also has the feature that its Fourier transform engine does <em>not</em> require a strict power of 2 for the number of pixels in the image (i.e., <math>2^n \times 2^n</math> pixel image).
The supernova remnant itself is known to have a diameter of order 9 arcminutes, corresponding to about 216 pixels for the chosen cell size.  The mosaic was set up with 7 fields, 1 centered on the remnant and 6 flanking fields; the spacing of the fields was chosen based on the size of the (antenna) primary beam.  For our choice of <tt>ftmachine='mosaic'</tt>, you do not have to fit the mosaic inside the inner quarter of the total image in order to prevent image artifacts arising from aliasing, we just want to have a bit of padding around the outside. Although CASA has the feature that its Fourier transform engine (FFTW) does <em>not</em> require a strict power of 2 for the number of linear pixels in a given image axis, it is somewhat more efficient if the number of pixels on a side is a composite number divisible by <em>any pair of</em> 2 and 3 and/or 5.  Because {{clean}} internally applies a padding of 1.2 (=3x2/5) choose 480, which is 2<sup>5</sup> x 3 x 5 (so 480 x 1.2 = 576 = 2<sup>6</sup> x 3<sup>2</sup>).  We therefore set <tt>imsize=[480,480]</tt> and our mosaic fits comfortably inside the image.
<!--  The Fourier transform is most efficient if the number of pixels on a side is a composite number divisible by 2 and 3 and/or 5.  We choose 576, which is <math>2^6\times3^2</math>, and is close to <math>2\times216</math>.  We therefore set ''imsize=[576,576]''.
-->


[[Image:3C391 interactive clean.png|thumb|Example of interactive cleaning]]
[[Image:screenshot_3c391_clean_interactive_noms.png|thumb|Interactive clean at beginning, zoomed in and having set polygon region]]
As mentioned above, we can guide the clean process by allowing it to find clean components only within a user-specified region.  The easiest way to do this is via interactive clean.  When {{clean}} runs in interactive mode, a viewer window will pop up as shown right.  To get a more detailed view of the central regions containing the emission, zoom in by tracing out a rectangle with the left mouse button and <em>double-clicking</em> inside the zoom box you just made.  Play with the color scale to bring out the emission better, by holding down the middle mouse button and moving it around.  To create a clean box (a region within which components may be found), hold down the right mouse button and trace out a rectangle around the source, then <em>double click</em> inside that rectangle to set it as a box.  Note that the CLEAN box must turn <em>white</em> for it to be registered; if the box is not white, it has not been set!  Alternatively, you can trace out a more generic shape to better enclose the irregular outline of the supernova remnant.  To do that, right-click on the polygonal icon highlighted in red (the oval on the right) shown in the figure.  Then trace out a shape by right-clicking where you want the corners of that shape.  Once you have come full circle, the shape will be traced out in green, with small squares at the corners.  <em>Double-click</em> inside this region and the green outline will turn white.  You have now set the clean region. To toggle back to the rectangle tracer again, right-click on the icon circled in green (the oval on the left) shown in the figure.  If you have made a mistake with your clean box, click on the "Erase" button, trace out a rectangle around your erroneous region, and <em>double click</em> inside that rectangle.  You can also set multiple clean regions.  By default, all clean regions will apply only to the plane shown.  To change this to select all planes, click the "All Channels" button at the top.
[[Image:screenshot_3c391_clean_interactive_noms_final.png|thumb|Interactive clean near end of cleaning (after 10000 iterations)]]
As mentioned above, we can guide the clean process by allowing it to find clean components only within a user-specified region.  The easiest way to do this is via interactive clean.  When {{clean}} runs in interactive mode, a viewer window will pop up as shown right.  To get a more detailed view of the central regions containing the emission, zoom in by tracing out a rectangle with the left mouse button and <em>double-clicking</em> inside the zoom box you just made.  Play with the color scale to bring out the emission better, by holding down the middle mouse button and moving it around.  To create a clean box (a region within which components may be found), hold down the right mouse button and trace out a rectangle around the source, then <em>double click</em> inside that rectangle to set it as a box.  Note that the CLEAN box must turn <em>white</em> for it to be registered; if the box is not white, it has not been set!  Alternatively, you can trace out a more generic shape to better enclose the irregular outline of the supernova remnant.  To do that, right-click on the closed polygonal icon.  Then trace out a shape by right-clicking where you want the corners of that shape.  Once you have come full circle, the shape will be traced out in green, with small squares at the corners.  <em>Double-click</em> inside this region and the green outline will turn white.  You have now set the clean region. If you have made a mistake with your clean box, click on the "Erase" button, trace out a rectangle around your erroneous region, and <em>double click</em> inside that rectangle.  You can also set multiple clean regions.  By default, all clean regions will apply only to the plane shown.  To change this to select all planes, click the "All Channels" button at the top (we have only a single mfs channel in the image cube), and likewise for Stokes planes using the "All Polarizations" (you will need to set this when we image all the Stokes).


When you are happy with the clean regions, press the green circular arrow button on the far right to continue deconvolution.  After completing a cycle, a revised image will come up.  As the brightest points are removed from the image ("cleaned" off), fainter emission may show up.  You can adjust the clean boxes each cycle, to enclose all real emission.  After many cycles, once only noise is left, you can hit the red and white cross icon to stop cleaning.
At any stage in the cleaning, you can adjust the number of iterations that clean will do before returning to the GUI.  By default this is set to 100 (see the "iterations" window in mid-upper left of panel).  You probably want to set this to a high number for this mosaic due to the complicated structure, values from 1000 to 5000 later on seem to work.  Note that this will override the <tt>niter</tt> that was set when you started the clean task. Clean will keep going until it reaches threshold or runs out of cycles (the "cycles" window to the right of the "iterations").


When you are happy with the clean regions, press the green circular arrow button on the far right to continue deconvolution.  After completing a cycle, a revised image will come up.  As the brightest points are removed from the image ("cleaned" off), fainter emission may show up.  You can adjust the clean boxes each cycle, to enclose all real emission.  After many cycles, once only noise is left, you can hit the red and white cross icon to stop cleaning.  The second figure to the right shows the interactive clean panel late in the process (after cleaning around 10000 iterations). At this point we have used the polygon tool to draw around the emission.  There is still signal left in the residual, and it would take forever to clean all that out using our standard clean!  You should just quit after doing around 10000-20000 iterations total.  We'll improve this in the next part of the tutorial.


[[Image:3c391_ctm_i_image.jpg|thumb|{{viewer}} display of the Stokes I mosaic of 3C 391 at 4600 MHz]]
[[Image:screenshot_3c391_viewer_noms_final.png|thumb|{{viewer}} display of the Stokes I mosaic of 3C 391]]
{{clean}} will make several output files, all named with the prefix given as <tt>imagename</tt>.  These include:
{{clean}} will make several output files, all named with the prefix given as <tt>imagename</tt>.  These include:
* <tt>.image</tt> - final restored image, with the clean components convolved with a restoring beam and added to the remaining residuals at the end of the imaging process
* <tt>.image</tt> - final restored image, with the clean components convolved with a restoring beam and added to the remaining residuals at the end of the imaging process
Line 1,192: Line 1,244:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
viewer('3c391_ctm_spw0_IQUV.image')
viewer('3c391_ctm_spw0_noms_I.image')
</source>
</source>


This will bring up a viewer window containing the image, which should look similar to that shown at right. The tape deck buttons that you see under the image can be used to step through the different Stokes parameters (I,Q,U,V).  You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do).
This will bring up a viewer window containing the image, which should look similar to that shown at right. You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do).


Note that the image is cut off in a circular fashion at the edges, corresponding to the default minimum primary beam response (<tt>minpb</tt>)within {{clean}} of 0.2.
Note that the image is cut off in a circular fashion at the edges, corresponding to the default minimum primary beam response (<tt>minpb</tt>)within {{clean}} of 0.2.


The example above illustrates multi-scale CLEAN.  Not all sources or fields will require multi-scale CLEAN; for reference, here is the same data set, but without multi-scale CLEANing.
As we saw during the interactive cleaning, this single-scale delta-function cleaning is having problems cleaning out the large-scale emission. You can see the large "clean bowl" of negative emission surrounding the object, and the heavy pixelization of the emission at low levels. We'll try to improve on this in the next part of the tutorial.
 
<source lang="python">
# In CASA
clean(vis='3c391_ctm_mosaic_spw0.ms',imagename='3c391_ctm_spw0_no_multiscale_IQUV',
      field='',spw='',
      mode='mfs',
      niter=5000,
      gain=0.1, threshold='1.0mJy',
      psfmode='clarkstokes',
      imagermode='mosaic', ftmachine='mosaic',
      interactive=True,
      imsize=[576,576], cell=['2.5arcsec','2.5arcsec'],
      stokes='IQUV',
      weighting='briggs',robust=0.0,
      calready=True)
</source>


== Next Steps ==
== Next Steps ==


There are a variety of additional analyses that could be done, including extracting the statistics of the images just produced, continuing with the polarization imaging, and self-calibration of the data. Continue with [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]  
In Part 2 of the tutorial we will carry out the polarization imaging, self-calibration, and polarization analysis and display of the data.
Continue with [http://casaguides.nrao.edu/index.php?title=EVLA_Advanced_Topics_3C391-CASA3.4 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration]  


{{Checked 3.4.0}}
{{Checked 3.4.0}}

Latest revision as of 21:11, 22 October 2012

This is the first part of the EVLA 3C391 data reduction tutorial. The second is 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration which follows on from this one.

Overview

This article describes the calibration and imaging of a multiple-pointing EVLA continuum dataset on the supernova remnant 3C 391. The data were taken in EVLA early science OSRO1 mode, with 128 MHz of bandwidth in each of two widely spaced spectral windows, centered at 4.6 and 7.5 GHz, and were set up for full polarization calibration.

There are a number of possible ways to run CASA, described in more detail in Getting Started in CASA. In brief, there are at least three different ways to run CASA:

  • Interactively examining task inputs. In this mode, one types default taskname to load the task, inp to examine the inputs, and go once those inputs have been set to your satisfaction. Allowed inputs are shown in blue, and bad inputs are colored red. The inputs themselves are changed one by one, e.g., selectdata=T. Screenshots of the inputs to various tasks used in the data reduction below are provided, to illustrate which parameters need to be set.

More detailed help can be obtained on any task by typing help taskname. Once a task is run, the set of inputs are stored and can be retrieved via tget taskname; subsequent runs will overwrite the previous tget file.

  • Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set.
  • Non-interactively via a script. A series of task function calls can be combined together into a script, and run from within CASA via execfile('scriptname.py'). This (and other) CASAguide has been designed to be extracted into a script using the script extractor. Should one use the script generated by the script extractor for this CASAguide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is in fact unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered), and in one case (that of plotms), you must leave the graphics window open, as the GUI cannot be reopened without first exiting from CASA.

If you are a relative novice (and particularly for this tutorial), it is strongly recommended that you start with the interactive mode, graduating to the pseudo- or non-interactive mode as you gain experience. Work at your own pace, look at the inputs to the tasks to see what other options exist, and read the help files.

Obtaining the Data

For the purposes of this tutorial, we have created a "starting" data set, upon which several initial processing steps have already been conducted. This data set may already be present on the machine that you are using; if not, obtain it from here http://casa.nrao.edu/Data/EVLA/3C391/3c391_ctm_mosaic_10s_spw0.ms.tgz (dataset size: 3.1GB)

We are providing this "starting" data set, rather than the "true" initial data set for (at least) two reasons. First, many of these initial processing steps can be rather time consuming (> 1 hr), and the time for the data reduction tutorial is limited. Second, while necessary, many of these steps are not fundamental to the calibration and imaging process, upon which we want to focus today. For completeness, however, here are the steps that were taken from the initial data set to produce the "starting" data set:

  • The data loaded into CASA, converting the initial Science Data Model (SDM) file into a measurement set.
  • Basic data flagging was applied, to account for "shadowing" of the antennas. These data are from the D configuration, in which antennas are particularly susceptible to being blocked or "shadowed" by other antennas in the array, depending upon the elevation of the source.
  • The data were averaged to 10-second samples, from the initial 1-second correlator sample time. In the D configuration, the fringe rate is relatively slow and time-average smearing is less of a concern.
  • The data were acquired with two spectral windows (around 4.6 and 7.5 GHz). Because of disk space concerns on some machines, the focus will be on only one of the two spectral windows.

We emphasize that, were this a real science observation, all of these steps would need to be run. Detailed instructions on obtaining the data from the archive and creating this "starting" data set may be found in the Obtaining EVLA Data: 3C 391 Example tutorial.

The Observation

Before starting the calibration process, we want to get some basic information about the data set. To examine the observing conditions during the observing run, and to find out any known problems with the data, download the observer log. Simply fill in the known observing date (in our case 2010-Apr-24) as both the Start and Stop date, and click on the "Show Logs" button. The relevant log is labeled with the project code, TDEM0001, and can be downloaded as a PDF file. From this, we find the following: 

Information from observing log:
There is no C-band receivers on ea13
Antenna ea06 is out of the array
Antenna ea15 has some corrupted data
Antennas ea10, ea12, ea22 do not have good baseline positions
Gusty winds, mixed clouds, API rms up to 11.5.

Before beginning our data reduction, we must start CASA. If you have not used CASA before, some helpful tips are available on the Getting Started in CASA page.

Once you have CASA up and running in the directory containing the data, then start your data reduction by getting some basic information about the data. The task listobs can be used to get a listing of the individual scans comprising the observation, the frequency setup, source list, and antenna locations. 

# In CASA
listobs(vis='3c391_ctm_mosaic_10s_spw0.ms',verbose=T)

listobs should now produce output similar to the following in the casa logger. (Note that the listing shown is for both spectral windows, whereas the data set actually being used contains only one spectral window.)

One will note that there are nine sources observed. Here the various sources are introduced briefly, with more detail contained in the sections below in which they are used.

  • J1331+3030 = 3C 286, which will later serve as a calibrator for the visibility amplitudes, i.e., it is assumed to have a precisely known flux density; the spectral bandpass; and the polarization position angle;
  • J1822-0938, which will serve as a calibrator for the visibility phases;
  • J0319+4130 = 3C 84, which will serve as a polarization calibrator; and
  • 3C391 C1--C7, which are 7 fields centered on and surrounding the supernova remnant.

This observation was set up as a 7-pointing mosaic because the supernova remnant is so large that it essentially fills the primary beam. 


INFO	listobs::::casa	##########################################
INFO	listobs::::casa	##### Begin Task: listobs            #####
INFO	listobs::::casa	
INFO	listobs::ms::summary	================================================================================
INFO	listobs::ms::summary+	           MeasurementSet Name:  /export/home/hamal/jmiller/TDEM0001_sb1218006/3c391_mosaic_fullres.ms      MS Version 2
INFO	listobs::ms::summary+	================================================================================
INFO	listobs::ms::summary+	   Observer: Dr. James Miller-Jones     Project: T.B.D.  
INFO	listobs::ms::summary+	Observation: EVLA
INFO	listobs::ms::summary	Data records: 18666050       Total integration time = 28716 seconds
INFO	listobs::ms::summary+	   Observed from   24-Apr-2010/08:01:34.5   to   24-Apr-2010/16:00:10.5 (UTC)
INFO	listobs::ms::summary	
INFO	listobs::ms::summary+	   ObservationID = 0         ArrayID = 0
INFO	listobs::ms::summary+	  Date        Timerange (UTC)          Scan  FldId FieldName    nVis   Int(s)   SpwIds
INFO	listobs::ms::summary+	  24-Apr-2010/08:01:34.5 - 08:02:28.5     1      0 J1331+3030   35750  1        [0, 1]
INFO	listobs::ms::summary+	              08:02:29.5 - 08:09:27.5     2      0 J1331+3030   272350 1        [0, 1]
INFO	listobs::ms::summary+	              08:09:28.5 - 08:16:26.5     3      0 J1331+3030   272350 1        [0, 1]
INFO	listobs::ms::summary+	              08:16:27.5 - 08:24:25.5     4      1 J1822-0938   311350 1        [0, 1]
INFO	listobs::ms::summary+	              08:24:26.5 - 08:29:44.5     5      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              08:29:45.5 - 08:34:43.5     6      3 3C391 C2     194350 1        [0, 1]
INFO	listobs::ms::summary+	              08:34:44.5 - 08:39:42.5     7      4 3C391 C3     194350 1        [0, 1]
INFO	listobs::ms::summary+	              08:39:43.5 - 08:44:41.5     8      5 3C391 C4     194350 1        [0, 1]
INFO	listobs::ms::summary+	              08:44:42.5 - 08:49:40.5     9      6 3C391 C5     194350 1        [0, 1]
INFO	listobs::ms::summary+	              08:49:41.5 - 08:54:40.5    10      7 3C391 C6     195000 1        [0, 1]
INFO	listobs::ms::summary+	              08:54:41.5 - 08:59:39.5    11      8 3C391 C7     194350 1        [0, 1]
INFO	listobs::ms::summary+	              08:59:40.5 - 09:01:29.5    12      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              09:01:30.5 - 09:06:48.5    13      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              09:06:49.5 - 09:11:47.5    14      3 3C391 C2     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:11:48.5 - 09:16:46.5    15      4 3C391 C3     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:16:47.5 - 09:21:45.5    16      5 3C391 C4     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:21:46.5 - 09:26:44.5    17      6 3C391 C5     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:26:45.5 - 09:31:44.5    18      7 3C391 C6     195000 1        [0, 1]
INFO	listobs::ms::summary+	              09:31:45.5 - 09:36:43.5    19      8 3C391 C7     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:36:44.5 - 09:38:32.5    20      1 J1822-0938   70850  1        [0, 1]
INFO	listobs::ms::summary+	              09:38:33.5 - 09:43:52.5    21      2 3C391 C1     208000 1        [0, 1]
INFO	listobs::ms::summary+	              09:43:53.5 - 09:48:51.5    22      3 3C391 C2     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:48:52.5 - 09:53:50.5    23      4 3C391 C3     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:53:51.5 - 09:58:49.5    24      5 3C391 C4     194350 1        [0, 1]
INFO	listobs::ms::summary+	              09:58:50.5 - 10:03:48.5    25      6 3C391 C5     194350 1        [0, 1]
INFO	listobs::ms::summary+	              10:03:49.5 - 10:08:47.5    26      7 3C391 C6     194350 1        [0, 1]
INFO	listobs::ms::summary+	              10:08:48.5 - 10:13:47.5    27      8 3C391 C7     195000 1        [0, 1]
INFO	listobs::ms::summary+	              10:13:48.5 - 10:15:36.5    28      1 J1822-0938   70850  1        [0, 1]
INFO	listobs::ms::summary+	              10:15:37.5 - 10:20:55.5    29      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              10:20:56.5 - 10:25:55.5    30      3 3C391 C2     195000 1        [0, 1]
INFO	listobs::ms::summary+	              10:25:56.5 - 10:30:54.5    31      4 3C391 C3     194350 1        [0, 1]
INFO	listobs::ms::summary+	              10:30:55.5 - 10:35:53.5    32      5 3C391 C4     194350 1        [0, 1]
INFO	listobs::ms::summary+	              10:35:54.5 - 10:40:52.5    33      6 3C391 C5     194350 1        [0, 1]
INFO	listobs::ms::summary+	              10:40:53.5 - 10:45:51.5    34      7 3C391 C6     194350 1        [0, 1]
INFO	listobs::ms::summary+	              10:45:52.5 - 10:50:51.5    35      8 3C391 C7     195000 1        [0, 1]
INFO	listobs::ms::summary+	              10:50:52.5 - 10:52:40.5    36      1 J1822-0938   70850  1        [0, 1]
INFO	listobs::ms::summary+	              10:52:41.5 - 10:57:39.5    37      0 J1331+3030   194350 1        [0, 1]
INFO	listobs::ms::summary+	              10:57:40.5 - 11:02:39.5    38      1 J1822-0938   195000 1        [0, 1]
INFO	listobs::ms::summary+	              11:02:40.5 - 11:07:58.5    39      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              11:07:59.5 - 11:12:47.5    40      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:12:48.5 - 11:17:36.5    41      4 3C391 C3     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:17:37.5 - 11:22:25.5    42      5 3C391 C4     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:22:26.5 - 11:27:15.5    43      6 3C391 C5     188500 1        [0, 1]
INFO	listobs::ms::summary+	              11:27:16.5 - 11:32:04.5    44      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:32:05.5 - 11:36:53.5    45      8 3C391 C7     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:36:54.5 - 11:38:43.5    46      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              11:38:44.5 - 11:44:02.5    47      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              11:44:03.5 - 11:48:51.5    48      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:48:52.5 - 11:53:40.5    49      4 3C391 C3     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:53:41.5 - 11:58:29.5    50      5 3C391 C4     187850 1        [0, 1]
INFO	listobs::ms::summary+	              11:58:30.5 - 12:03:19.5    51      6 3C391 C5     188500 1        [0, 1]
INFO	listobs::ms::summary+	              12:03:20.5 - 12:08:08.5    52      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              12:08:09.5 - 12:12:57.5    53      8 3C391 C7     187850 1        [0, 1]
INFO	listobs::ms::summary+	              12:12:58.5 - 12:14:47.5    54      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              12:14:48.5 - 12:20:06.5    55      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              12:20:07.5 - 12:24:55.5    56      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              12:24:56.5 - 12:29:44.5    57      4 3C391 C3     187850 1        [0, 1]
INFO	listobs::ms::summary+	              12:29:45.5 - 12:34:34.5    58      5 3C391 C4     188500 1        [0, 1]
INFO	listobs::ms::summary+	              12:34:35.5 - 12:39:23.5    59      6 3C391 C5     187850 1        [0, 1]
INFO	listobs::ms::summary+	              12:39:24.5 - 12:44:12.5    60      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              12:44:13.5 - 12:49:01.5    61      8 3C391 C7     187850 1        [0, 1]
INFO	listobs::ms::summary+	              12:49:02.5 - 12:50:51.5    62      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              12:50:52.5 - 12:56:10.5    63      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              12:56:11.5 - 13:00:59.5    64      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:01:00.5 - 13:05:48.5    65      4 3C391 C3     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:05:49.5 - 13:10:38.5    66      5 3C391 C4     188500 1        [0, 1]
INFO	listobs::ms::summary+	              13:10:39.5 - 13:15:27.5    67      6 3C391 C5     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:15:28.5 - 13:20:16.5    68      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:20:17.5 - 13:25:05.5    69      8 3C391 C7     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:25:06.5 - 13:26:55.5    70      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              13:26:56.5 - 13:32:14.5    71      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              13:32:15.5 - 13:37:03.5    72      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:37:04.5 - 13:41:52.5    73      4 3C391 C3     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:41:53.5 - 13:46:42.5    74      5 3C391 C4     188500 1        [0, 1]
INFO	listobs::ms::summary+	              13:46:43.5 - 13:51:31.5    75      6 3C391 C5     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:51:32.5 - 13:56:20.5    76      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              13:56:21.5 - 14:01:09.5    77      8 3C391 C7     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:01:10.5 - 14:02:59.5    78      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              14:03:00.5 - 14:08:18.5    79      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              14:08:19.5 - 14:13:07.5    80      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:13:08.5 - 14:17:57.5    81      4 3C391 C3     188500 1        [0, 1]
INFO	listobs::ms::summary+	              14:17:58.5 - 14:22:46.5    82      5 3C391 C4     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:22:47.5 - 14:27:35.5    83      6 3C391 C5     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:27:36.5 - 14:32:24.5    84      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:32:25.5 - 14:37:13.5    85      8 3C391 C7     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:37:14.5 - 14:39:03.5    86      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              14:39:04.5 - 14:44:22.5    87      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              14:44:23.5 - 14:49:11.5    88      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:49:12.5 - 14:54:01.5    89      4 3C391 C3     188500 1        [0, 1]
INFO	listobs::ms::summary+	              14:54:02.5 - 14:58:50.5    90      5 3C391 C4     187850 1        [0, 1]
INFO	listobs::ms::summary+	              14:58:51.5 - 15:03:39.5    91      6 3C391 C5     187850 1        [0, 1]
INFO	listobs::ms::summary+	              15:03:40.5 - 15:08:28.5    92      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              15:08:29.5 - 15:13:17.5    93      8 3C391 C7     187850 1        [0, 1]
INFO	listobs::ms::summary+	              15:13:18.5 - 15:15:07.5    94      1 J1822-0938   71500  1        [0, 1]
INFO	listobs::ms::summary+	              15:15:08.5 - 15:20:26.5    95      2 3C391 C1     207350 1        [0, 1]
INFO	listobs::ms::summary+	              15:20:27.5 - 15:25:15.5    96      3 3C391 C2     187850 1        [0, 1]
INFO	listobs::ms::summary+	              15:25:16.5 - 15:30:05.5    97      4 3C391 C3     188500 1        [0, 1]
INFO	listobs::ms::summary+	              15:30:06.5 - 15:34:54.5    98      5 3C391 C4     187850 1        [0, 1]
INFO	listobs::ms::summary+	              15:34:55.5 - 15:39:43.5    99      6 3C391 C5     187850 1        [0, 1]
INFO	listobs::ms::summary+	              15:39:44.5 - 15:44:32.5   100      7 3C391 C6     187850 1        [0, 1]
INFO	listobs::ms::summary+	              15:44:33.5 - 15:49:22.5   101      8 3C391 C7     188500 1        [0, 1]
INFO	listobs::ms::summary+	              15:49:23.5 - 15:51:11.5   102      1 J1822-0938   70850  1        [0, 1]
INFO	listobs::ms::summary+	              15:51:12.5 - 16:00:10.5   103      9 J0319+4130   350350 1        [0, 1]
INFO	listobs::ms::summary	           (nVis = Total number of time/baseline visibilities per scan) 
INFO	listobs::ms::summary	Fields: 10
INFO	listobs::ms::summary+	  ID   Code Name         RA            Decl           Epoch   SrcId nVis   
INFO	listobs::ms::summary+	  0    N    J1331+3030   13:31:08.2880 +30.30.32.9589 J2000   0     774800 
INFO	listobs::ms::summary+	  1    J    J1822-0938   18:22:28.7042 -09.38.56.8350 J2000   1     1361750
INFO	listobs::ms::summary+	  2    NONE 3C391 C1     18:49:24.2440 -00.55.40.5800 J2000   2     2488850
INFO	listobs::ms::summary+	  3    NONE 3C391 C2     18:49:29.1490 -00.57.48.0000 J2000   3     2280850
INFO	listobs::ms::summary+	  4    NONE 3C391 C3     18:49:19.3390 -00.57.48.0000 J2000   4     2282150
INFO	listobs::ms::summary+	  5    NONE 3C391 C4     18:49:14.4340 -00.55.40.5800 J2000   5     2282150
INFO	listobs::ms::summary+	  6    NONE 3C391 C5     18:49:19.3390 -00.53.33.1600 J2000   6     2281500
INFO	listobs::ms::summary+	  7    NONE 3C391 C6     18:49:29.1490 -00.53.33.1600 J2000   7     2281500
INFO	listobs::ms::summary+	  8    NONE 3C391 C7     18:49:34.0540 -00.55.40.5800 J2000   8     2282150
INFO	listobs::ms::summary+	  9    Z    J0319+4130   03:19:48.1601 +41.30.42.1030 J2000   9     350350 
INFO	listobs::ms::summary+	   (nVis = Total number of time/baseline visibilities per field) 
INFO	listobs::ms::summary	Spectral Windows:  (2 unique spectral windows and 1 unique polarization setups)
INFO	listobs::ms::summary+	  SpwID  #Chans Frame Ch1(MHz)    ChanWid(kHz)TotBW(kHz)  Ref(MHz)    Corrs           
INFO	listobs::ms::summary+	  0          64 TOPO  4536        2000        128000      4536        RR  RL  LR  LL  
INFO	listobs::ms::summary+	  1          64 TOPO  7436        2000        128000      7436        RR  RL  LR  LL  
INFO	listobs::ms::summary	Sources: 20
INFO	listobs::ms::summary+	  ID   Name         SpwId RestFreq(MHz)  SysVel(km/s) 
INFO	listobs::ms::summary+	  0    J1331+3030   0     -              -            
INFO	listobs::ms::summary+	  0    J1331+3030   1     -              -            
INFO	listobs::ms::summary+	  1    J1822-0938   0     -              -            
INFO	listobs::ms::summary+	  1    J1822-0938   1     -              -            
INFO	listobs::ms::summary+	  2    3C391 C1     0     -              -            
INFO	listobs::ms::summary+	  2    3C391 C1     1     -              -            
INFO	listobs::ms::summary+	  3    3C391 C2     0     -              -            
INFO	listobs::ms::summary+	  3    3C391 C2     1     -              -            
INFO	listobs::ms::summary+	  4    3C391 C3     0     -              -            
INFO	listobs::ms::summary+	  4    3C391 C3     1     -              -            
INFO	listobs::ms::summary+	  5    3C391 C4     0     -              -            
INFO	listobs::ms::summary+	  5    3C391 C4     1     -              -            
INFO	listobs::ms::summary+	  6    3C391 C5     0     -              -            
INFO	listobs::ms::summary+	  6    3C391 C5     1     -              -            
INFO	listobs::ms::summary+	  7    3C391 C6     0     -              -            
INFO	listobs::ms::summary+	  7    3C391 C6     1     -              -            
INFO	listobs::ms::summary+	  8    3C391 C7     0     -              -            
INFO	listobs::ms::summary+	  8    3C391 C7     1     -              -            
INFO	listobs::ms::summary+	  9    J0319+4130   0     -              -            
INFO	listobs::ms::summary+	  9    J0319+4130   1     -              -            
INFO	listobs::ms::summary	Antennas: 26:
INFO	listobs::ms::summary+	  ID   Name  Station   Diam.    Long.         Lat.         
INFO	listobs::ms::summary+	  0    ea01  W09       25.0 m   -107.37.25.2  +33.53.51.0  
INFO	listobs::ms::summary+	  1    ea02  E02       25.0 m   -107.37.04.4  +33.54.01.1  
INFO	listobs::ms::summary+	  2    ea03  E09       25.0 m   -107.36.45.1  +33.53.53.6  
INFO	listobs::ms::summary+	  3    ea04  W01       25.0 m   -107.37.05.9  +33.54.00.5  
INFO	listobs::ms::summary+	  4    ea05  W08       25.0 m   -107.37.21.6  +33.53.53.0  
INFO	listobs::ms::summary+	  5    ea07  N06       25.0 m   -107.37.06.9  +33.54.10.3  
INFO	listobs::ms::summary+	  6    ea08  N01       25.0 m   -107.37.06.0  +33.54.01.8  
INFO	listobs::ms::summary+	  7    ea09  E06       25.0 m   -107.36.55.6  +33.53.57.7  
INFO	listobs::ms::summary+	  8    ea11  E04       25.0 m   -107.37.00.8  +33.53.59.7  
INFO	listobs::ms::summary+	  9    ea12  E08       25.0 m   -107.36.48.9  +33.53.55.1  
INFO	listobs::ms::summary+	  10   ea13  N07       25.0 m   -107.37.07.2  +33.54.12.9  
INFO	listobs::ms::summary+	  11   ea14  E05       25.0 m   -107.36.58.4  +33.53.58.8  
INFO	listobs::ms::summary+	  12   ea15  W06       25.0 m   -107.37.15.6  +33.53.56.4  
INFO	listobs::ms::summary+	  13   ea16  W02       25.0 m   -107.37.07.5  +33.54.00.9  
INFO	listobs::ms::summary+	  14   ea17  W07       25.0 m   -107.37.18.4  +33.53.54.8  
INFO	listobs::ms::summary+	  15   ea18  N09       25.0 m   -107.37.07.8  +33.54.19.0  
INFO	listobs::ms::summary+	  16   ea19  W04       25.0 m   -107.37.10.8  +33.53.59.1  
INFO	listobs::ms::summary+	  17   ea20  N05       25.0 m   -107.37.06.7  +33.54.08.0  
INFO	listobs::ms::summary+	  18   ea21  E01       25.0 m   -107.37.05.7  +33.53.59.2  
INFO	listobs::ms::summary+	  19   ea22  N04       25.0 m   -107.37.06.5  +33.54.06.1  
INFO	listobs::ms::summary+	  20   ea23  E07       25.0 m   -107.36.52.4  +33.53.56.5  
INFO	listobs::ms::summary+	  21   ea24  W05       25.0 m   -107.37.13.0  +33.53.57.8  
INFO	listobs::ms::summary+	  22   ea25  N02       25.0 m   -107.37.06.2  +33.54.03.5  
INFO	listobs::ms::summary+	  23   ea26  W03       25.0 m   -107.37.08.9  +33.54.00.1  
INFO	listobs::ms::summary+	  24   ea27  E03       25.0 m   -107.37.02.8  +33.54.00.5  
INFO	listobs::ms::summary+	  25   ea28  N08       25.0 m   -107.37.07.5  +33.54.15.8  
INFO	listobs::::casa	
INFO	listobs::::casa	##### End Task: listobs              #####
INFO	listobs::::casa	##########################################

Note that the antenna IDs (which are numbered sequentially up to the total number of antennas in the array; 0 through 25 in this instance) do not correspond to the actual antenna names (ea01 through ea28; these numbers correspond to those painted on the side of the dishes). During our data reduction, we can refer to the antennas using either convention; antenna='22' would correspond to ea25, whereas antenna='ea22' would correspond to ea22. Note that the antenna numbers in the observer log correspond to the actual antenna names, i.e. the 'ea??' numbers given in listobs.

Both to get a sense of the array, as well as identify an antenna for later use in calibration, use the task plotants. In general, for calibration purposes, one would like to select an antenna that is close to the center of the array (and that is not listed in the operator's log as having had problems!). 

# In CASA
plotants(vis='3c391_ctm_mosaic_10s_spw0.ms',figfile='plotants_3c391_antenna_layout.png')
clearstat() # This removes the table lock generated by plotants in script mode
plotants parameters
plotants figure

Examining and Editing the Data

It is always a good idea, particularly with a new system like the EVLA, to examine the data. Moreover, from the observer's log, we already know that one antenna will need to be flagged because it does not have a C-band receiver. Start by flagging data known to be bad, then examine the data.

In its current operation, it is common to insert a dummy scan as the first scan. (From the listobs output above, one may have noticed that the first scan is less than 1 minute long.) This first scan can safely be deleted. 

# In CASA
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', flagbackup=T, 
          mode='manual', scan='1')
tflagdata inputs
  • flagbackup=T : A comment is warranted on the setting of flagbackup (here set to T or True). If set to True, tflagdata will save a copy of the existing set of flags before entering any new flags. The setting of flagbackup is therefore a matter of some taste. One could choose not to save any flags or only save "major" flags, or one could save every flag. (One of the authors of this document was glad that flagbackup was set to True as he recently ran tflagdata with a typo in one of the entries.)
  • mode='manualflag' : Specific data are going to be selected to be edited.
  • scan='1' is chosen to select only the first scan. Note that scan expects an entry in the form of a string. (scan=1 would generate an error.)

If satisfied with the inputs, run this task. The initial display in the logger will include 

##########################################
##### Begin Task: tflagdata          #####
   
Table type is Measurement Set
...
Backup original flags before applying new flags
Saving current flags to tflagdata_1 before applying new flags
 	Creating new backup flag file called tflagdata_1

which indicates that, among other things, the flags that existed in the data set prior to this run will be saved to another file called tflagdata_1. Should one ever desire to revert to the data prior to this run, the task flagmanager could be used.


From the observer's log, we know that antenna ea13 does not have a C band receiver and antenna ea15 had some corrupted data, so they should be flagged as well. The parameters are similar as before. 

# In CASA
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', flagbackup=T, mode='manual', 
          antenna='ea13,ea15')
  • antenna='ea13,ea15' : Once again, this parameter requires a string input. Remember that antenna='ea13' and 'antenna='13' are not the same antenna. (See the discussion after our call to listobs above.)


Finally, it is common for the array to require a small amount of time to "settle down" at the start of a scan. Consequently, it has become standard practice to edit out the initial samples from the start of each scan. 

# In CASA
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms', mode='quack', 
          quackinterval=10.0, quackmode='beg')
  • mode='quack' : Quack is another mode in which the same edit will be applied to all scans for all baselines.
  • quackmode='beg' : In this case, data from the start of each scan will be flagged. Other options include flagging data at the end of the scan.
  • quackinterval=10 : In this data set, the sampling time is 10 seconds, so this choice flags the first sample from all scans on all baselines.


Having now done some basic editing of the data, based in part on a priori information, it is time to look at the data to determine if there are any other obvious problems. One task to examine the data themselves is plotms. 

# In CASA
clearstat() # This removes any existing table locks generated by tflagdata
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',xaxis='',yaxis='', 
       averagedata=False,transform=False,extendflag=False, 
       plotfile='',selectdata=True,field='0')
plotms inputs
  • xaxis=' ', yaxis=' ' : One can choose the axes of the plot, i.e., the way of visualizing the data, by using the GUI display once the task is executed.
  • averagedata=F : It is possible to average the data in time, frequency, etc.
  • transform=F : It is possible to change the velocity reference frame of the data.
  • extendflag=F : It is possible to "extend" a flag, i.e., flag data surrounding bad data. For example, one might want to flag spectral channels surrounding a bad spectral channel or one might want to flag cross-polarization data if one flags the parallel polarization data.
  • plotfile=' ' : It is possible to produce a hard copy (e.g., for a paper, report, or Web site) by specifying a file.
  • selectdata=T : One can choose to plot only subsets of the data.
  • field='0' : The entire dataset is rather large, and different sources have very different amplitudes, so it is advisable to start by loading a subset of the data. One can later loop through the different fields (i.e. sources) and spectral windows using the GUI interface.

In this case, many other values have been left to defaults as it is also possible to select them from within the plotms GUI. Review the inputs, then run the task.

plotms should produce a GUI, with the default view being to show the visibility amplitude as a function of time. The figure at right shows the result of running plotms without the field selection discussed above (using field=' ' ).

plotms default GUI view, having loaded all fields at once

plotms allows one to select and view the data in many ways. Across the top of the left panel are a set of tabs labeled 'Plots', 'Flagging', 'Tools', 'Annotator', and 'Options'. If one selects the 'Flagging' tab, the option is to 'Extend flags'. Thus, even though plotms was started with extendflag=F, if one decides that it does make sense to extend the flags, one can still do so here.

In the default view, the 'Plots' tab is visible, and there are a number of tabs running down the side of the left hand panel, including 'Data', 'Axes', 'Trans', 'Cache', 'Display', 'Canvas', and 'Export'. Once again, one can make changes on the fly. Thus, supposing that one wants to save a hard copy, even if plotms was started with plotfile=' ', one can select 'Export' and enter a file name in which to save a copy of a plot.

One should spend several minutes displaying the data in various formats. For instance, one could select the 'Data' tab and specify field 0 (source J1331+3030, a.k.a. 3C 286) to display data associated with the amplitude calibrator, then select the 'Axes' tab and change the x axis to be UVDist (baseline length, in meters), and plot the data. The result should be that of the first thumbnail image shown below. The amplitude distribution is relatively constant as a function of u-v distance or baseline length (i.e., [math]\displaystyle{ \sqrt{u^2+v^2} }[/math]). From the various lectures, one should recognize that a relatively constant visibility amplitude as a function of baseline length means that the source is very nearly a point source. (The Fourier transform of a constant is a delta function, a.k.a. a point source.)

By contrast, if one selects field 3 (one of the 3C 391 fields) in the 'Data' tab and plots these data, one sees a visibility function that falls rapidly with increasing baseline length. Such a visibility function indicates a highly resolved source. By noting the baseline length at which the visibility function falls to some fiducial value (e.g., 1/2 of its peak value), one can obtain a rough estimate of the angular scale of the source. (From the lectures, angular scale [in radians] ~ 1/baseline [in wavelengths]. To plot baseline length in wavelengths rather than meters, one needs to select UVwave as the x-axis in the plotms GUI.)


plotms view of 3C 286
plotms view of 3C 391


As a general data editing and examination strategy, at this stage in the data reduction process, one wants to focus on the calibrators. The data reduction strategy is to determine various corrections from the calibrators, then apply these correction factors to the science data. The 3C 286 data look relatively clean. There are no wildly egregious data (e.g., amplitudes that are 100,000x larger than the rest of the data). One may notice that there are antenna-to-antenna variations (under the 'Display' tab, select 'Colorize by Antenna1'). These antenna-to-antenna variations are acceptable, that's what calibration will help determine.

Warning: Do not close the plotms GUI after running plotms, or you will need to exit casapy and restart if at any point you wish to run plotms again, otherwise the GUI will not come up a second time.

datastream view of MS

One final useful plot we will make is a "datastream" plot of the antenna2 in a baseline for the data versus ea01. This shows (assuming ea01 is there the entire run) when various antennas drop out. 

# In CASA
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='',correlation='RR,LL',
       timerange='',antenna='ea01',spw='0:31',
       xaxis='time',yaxis='antenna2',
       plotrange=[-1,-1,0,26],coloraxis='field')

See the plot to right (after fiddling with the symbol sizes to improve visibility). This plot was actually made from the MS later on after ea05 (index 4) was flagged. From this display you can see that antenna 3 (ea04) is missing the last scan on 3C84 (which will have some implications later on), antennas 1,6,13 (ea02,ea08,ea16) have some missing blocks of time in the middle of the run, and 23 (ea26) is missing a scan or two near the end.

Calibrating the Data

It is now time to begin calibrating the data. The general data reduction strategy is to derive a series of scaling factors or corrections from the calibrators, which are then collectively applied to the science data. For much more discussion of the philosophy, strategy, and implementation of calibration of synthesis data within CASA, see Synthesis Calibration in the CASA Reference Manual.

Recall that the observed visibility [math]\displaystyle{ V^{\prime} }[/math] between two antennas [math]\displaystyle{ (i,j) }[/math] is related to the "true" visibility [math]\displaystyle{ V }[/math] by

[math]\displaystyle{ V^{\prime}_{i,j}(u,v,f) = b_{ij}(t)\,[B_i(f,t) B^{*}_j(f,t)]\,g_i(t) g_j(t)\,V_{i,j}(u,v,f)\,e^{i [\theta_i(t) - \theta_j(t)]} }[/math]

Here, for generality, we show the visibility as a function of frequency [math]\displaystyle{ f }[/math] and spatial wavenumbers [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math]. The other terms are

  • [math]\displaystyle{ g_i }[/math] and [math]\displaystyle{ \theta_i }[/math] are the amplitude and phase portions of what is commonly termed the complex gain. They are shown separately here because they are usually determined separately. For completeness, these are shown as a function of time [math]\displaystyle{ t }[/math] to indicate that they can change with temperature, atmospheric conditions, etc.
  • [math]\displaystyle{ B_i }[/math] is the complex bandpass, the instrumental response as a function of frequency, [math]\displaystyle{ f }[/math]. As shown here, the bandpass may also vary as a function of time.
  • [math]\displaystyle{ b(t) }[/math] is the often-neglected baseline term. It can be important to include for the highest dynamic range images or shortly after a configuration change at the [E]VLA, when antenna positions may not be known well.

Strictly, the equation above is a simplification of a more general measurement equation formalism, but it is a useful simplification in many cases.

For safety or sanity, one can begin by "clearing the calibration." In CASA, the data structure is that the observed data are stored in a DATA column, estimates of the data (e.g., a priori models for the calibrators, and those derived from the self-calibration process to be done later) are stored in the MODEL_DATA column, and the calibrated data are stored in the CORRECTED_DATA column. The task clearcal initializes the MODEL_DATA and CORRECTED_DATA and sets up some scratch data columns as well. For a pristine data set, straight from the Archive, clearcal probably should not be required; clearcal could be quite important if one decides later that a horrible mistake has been made in the calibration process and one wishes to start over. If you have started with the 10s-averaged dataset suggested at the top of this tutorial, this step has already been done for you, so may be omitted. 

# In CASA
clearcal(vis='3c391_ctm_mosaic_10s_spw0.ms',field='',spw='')

All parameters are set to blank so that the initialization occurs for all sources and spectral windows.

Note: In this release 3.4 we now have the option to bypass the creation and use of the scratch columns in most of the steps, and instead calculate the model "on the fly" during calibration. This is effected by setting usescratch = False in tasks like setjy. As this conversion to full "scratchless" operation is not yet complete and fully tested, in this tutorial we will explicitly force use of the scratch columns.

A priori Antenna Position Corrections

As mentioned in the observing log above, antennas ea10, ea12, and ea22 do not have good baseline positions. Antenna ea10 was not in the array, but, for the other two antennas, any improved baseline positions need to be incorporated. The importance of this step is that the visibility function is a function of [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math]. If the baseline positions are incorrect, then [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] will be calculated incorrectly, and there will be errors in the image. (These corrections could also be determined later by a baseline-based calibration incorporating the [math]\displaystyle{ b_{ij} }[/math] term from the equation above, but since they are known a priori it makes sense to incorporate them now.)

Any corrections can be ascertained from the EVLA/VLA Baseline Corrections site. For future reference, be sure to read to the bottom of that document to see how to calculate the additive corrections. Fortunately, the current case is simple as there is only a single correction for each antenna. The calculations are inserted via gencal which now (as of Version 3.4) allows automated lookup of the corrections. 

# In CASA
gencal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.antpos',caltype='antpos')

In the logger you can see the corrections reported: 

INFO    gencal::::casa  ##########################################
INFO    gencal::::casa  ##### Begin Task: gencal             #####
INFO    gencal::::casa  
INFO    calibrater::open        Opening MS: 3c391_ctm_mosaic_10s_spw0.ms for calibration.
INFO    Calibrater::    Initializing nominal selection to the whole MS.
INFO    gencal::::casa   Determine antenna position offests from the baseline correction database
INFO    gencal::::casa  offsets for antenna ea01 :  0.00000   0.00300   0.00000
INFO    gencal::::casa  offsets for antenna ea02 : -0.00080   0.00000   0.00000
INFO    gencal::::casa  offsets for antenna ea03 : -0.00280   0.00000   0.00000
INFO    gencal::::casa  offsets for antenna ea05 :  0.00000   0.00280   0.00000
INFO    gencal::::casa  offsets for antenna ea11 :  0.00090   0.00000   0.00000
INFO    gencal::::casa  offsets for antenna ea12 : -0.01000   0.00450  -0.00170
INFO    gencal::::casa  offsets for antenna ea13 :  0.00000  -0.00080   0.00000
INFO    gencal::::casa  offsets for antenna ea17 : -0.00120   0.00000   0.00000
INFO    gencal::::casa  offsets for antenna ea18 :  0.00040  -0.00080   0.00040
INFO    gencal::::casa  offsets for antenna ea22 : -0.02570   0.00270  -0.01900
INFO    gencal::::casa  offsets for antenna ea23 : -0.00140   0.00000   0.00000
INFO    gencal::::casa  offsets for antenna ea24 : -0.00150   0.00000   0.00000
INFO    gencal::::casa  offsets for antenna ea26 : -0.00190   0.00000   0.00210
INFO    gencal::::casa  offsets for antenna ea27 :  0.00000   0.00190  -0.00160
INFO    calibrater::specifycal  Beginning specifycal-----------------------
INFO            Creating KAntPos Jones table from specified parameters.
INFO            Writing solutions to table: 3c391_ctm_mosaic_10s_spw0.antpos
INFO    gencal::::casa  
INFO    gencal::::casa  ##### End Task: gencal               #####
INFO    gencal::::casa  ##########################################

This corresponds to what is found in the database for 2010: 

;                2010 BASELINE CORRECTIONS IN METERS
;ANT
;MOVED OBSDATE   Put_In_ MC(IAT)  ANT  PAD    Bx      By      Bz
;
JAN27   FEB12     FEB21  01:57    11   E04  0.0000  0.0000  0.0000
JAN27   FEB12     FEB21  01:57    26   W03 -0.0170  0.0204  0.0041
MAR24   MAR25     MAR26  18:28    17   W07 -0.0061 -0.0069 -0.0055
APR21   MAY02     MAY04  23:25    12   E08 -0.0072  0.0045 -0.0017
MAR09   MAY02     MAY04  23:25    22   N04 -0.0220  0.0040 -0.0190
JUN08   JUN20     JUN22  03:00    10   N03  0.0046 -0.0196  0.0090
        JUL17     JUL18  21:44     1   W09  0.0000  0.0030  0.0000
        JUL17     JUL18  21:44     2   E02 -0.0008  0.0000  0.0000
        JUL17     JUL18  21:44     3   E09 -0.0028  0.0000  0.0000
        JUL17     JUL18  21:44     5   W08  0.0000  0.0028  0.0000
JUL01   JUL17     JUL18  21:44     6   N06  0.0022  0.0010  0.0059
        JUL17     JUL18  21:44    10   N03  0.0008  0.0030 -0.0014
        JUL17     JUL18  21:44    11   E04  0.0009  0.0000  0.0000
        JUL17     JUL18  21:44    12   E08 -0.0028  0.0000  0.0000
        JUL17     JUL18  21:44    13   N07  0.0000 -0.0008  0.0000
        JUL17     JUL18  21:44    17   W07 -0.0012  0.0000  0.0000 
        JUL17     JUL18  21:44    18   N09  0.0004 -0.0008  0.0004
        JUL17     JUL18  21:44    22   N04 -0.0037 -0.0013  0.0000
        JUL17     JUL18  21:44    23   E07 -0.0014  0.0000  0.0000
        JUL17     JUL18  21:44    24   W05 -0.0015  0.0000  0.0000
        JUL17     JUL18  21:44    26   W03 -0.0019  0.0000  0.0021
        JUL17     JUL18  21:44    27   E03  0.0000  0.0019 -0.0016
AUG05   SEP15     SEP16  00:11    07   E05  0.0087  0.0137  0.0000
SEP13   SEP15     SEP16  00:11    08   N10  0.0098 -0.0063  0.0059
SEP13   SEP15     SEP16  00:11    10   N14  0.0124 -0.0050  0.0110
SEP14   SEP15     SEP16  00:11    13   N12  0.0020 -0.0060 -0.0077
SEP13   SEP15     SEP16  00:11    18   N16 -0.0035  0.0000 -0.0027
SEP13   SEP15     SEP16  00:11    20   N18  0.0044 -0.0020  0.0000 
OCT04   OCT06     OCT11  05:03     1   W18 -0.0087  0.0077 -0.0041
OCT05   OCT06     OCT11  05:03     3   E18 -0.0131 -0.0143  0.0099
OCT04   OCT06     OCT11  05:03     4   W10  0.0041  0.0032  0.0005
OCT04   OCT06     OCT11  05:03     7   E14  0.0027  0.0154  0.0049
OCT05   OCT06     OCT11  05:03    17   W14 -0.0141  0.0095 -0.0046
OCT04   OCT06     OCT11  05:03    23   E16  0.0044  0.0045 -0.0076
OCT05   OCT06     OCT11  05:03    24   W16 -0.0017  0.0004  0.0038
OCT05   OCT06     OCT11  05:03    27   E10  0.0039  0.0188 -0.0060
OCT06   OCT07     OCT12  20:12    21   E12 -0.0125 -0.0140 -0.0100
OCT20   OCT22     OCT23  19:54    14   W12  0.0191 -0.0126  0.0032
        OCT22     OCT23  19:54    21   E12  0.0196 -0.0062  0.0051 
DEC01   DEC04     DEC08  20:28    26   W02 -0.0130  0.0265  0.0187

Flux Density Scale

The next step is to provide a flux density value for the amplitude calibrator J1331+3030 (a.k.a. 3C 286). For the VLA, the ultimate flux density scale at most frequencies was set by 3C 295, which was then transferred to a small number of "primary flux density calibrators," including 3C 286. For the EVLA, at the time of this writing, the flux density scale at most frequencies will be determined from WMAP observations of the planet Mars, in turn then transferred to a small number of primary flux density calibrators. Thus, the procedure is to assume that the flux density of a primary calibrator source is known and, by comparison with the observed data for that calibrator, determine the [math]\displaystyle{ g_i }[/math] values.

To start, let's find the available calibrator models with setjy and the option listmodels=True: 

# In CASA
setjy(vis='3c391_ctm_mosaic_10s_spw0.ms', listmodels=T)

This command will show all available calibrator models: 

No candidate modimages matching '*.im* *.mod*' found in .

Candidate modimages (*) in /usr/lib64/casapy/stable/data/nrao/VLA/CalModels:
3C138_C.im  3C138_Q.im  3C147_C.im  3C147_Q.im  3C286_C.im  3C286_Q.im  3C48_C.im  3C48_Q.im  README
3C138_K.im  3C138_U.im  3C147_K.im  3C147_U.im  3C286_K.im  3C286_U.im  3C48_K.im  3C48_U.im
3C138_L.im  3C138_X.im  3C147_L.im  3C147_X.im  3C286_L.im  3C286_X.im  3C48_L.im  3C48_X.im

Since any image could be a potential calibrator model setjy will list all *.im and *.mod images in the working directory. In addition, it will list all models that are provided by NRAO with the CASA package, and they will be picked by their names. We will be using the C-band EVLA standard model for 3C286 which is aptly named '3C286_C.im': 

# In CASA
setjy(vis='3c391_ctm_mosaic_10s_spw0.ms',field='J1331+3030',modimage='3C286_C.im', 
      standard='Perley-Butler 2010',usescratch=True,scalebychan=True,spw='')
  • field='J1331+3030' : Clearly one has to specify what the flux density calibrator is, otherwise all sources will be assumed to have the same flux density.
  • modimage='3C286_C.im' : Although above, from plotms, it was estimated that 3C 286 is roughly a point source, depending upon the frequency and configuration, the source may be slightly resolved. Fiducial model images have been determined from a painstaking set of observations, and, if one is available, it should be used to compensate for slight resolution effects. In this case, spectral window 0 (at 4.536 GHz) is in the C band, so the C-band model image is used.
  • standard='Perley-Butler 2010' : Periodically, the flux density scale at the VLA was revised, updated, or expanded. The specified value represents the most recent determination of the flux density scale (by R. Perley and B. Butler in 2010); older scales can also be specified, and might be important if, for example, one was attempting to conduct a careful comparison with a previously published result.
  • usescratch=True : We are forcing the writing of the model visibilities to the MODEL_DATA scratch column, for the reasons given above.
  • scalebychan=True : In order to take out the intrinsic spectral index of our flux density calibrator 3C286 when we use it as our bandpass calibrator, we let it calculate a flux density value per channel.
  • spw=' ' : The original data contained two spectral windows. Having split off spectral window 0, it is not necessary to specify spw. Had the spectral window 0 not been split off, as has been done here, we might wish to specify the spectral window because, in this observation, the spectral windows were sufficiently separated that two different model images for 3C 286 would be appropriate; 3C286_C.im at 4.6 GHz and 3C286_X.im at 7.5 GHz. This would require two separate runs of setjy, one for each spectral window. If the spectral windows were much closer together, it might be possible to calibrate both using the same model.

In this case, a model image of a primary flux density calibrator exists. However, for some kinds of polarization calibration or in extreme situations (e.g., there are problems with the scan on the flux density calibrator), it can be useful or required to set the flux density of the source explicitly.

The output from setjy should look similar to the following. 

INFO    setjy::::casa   Using /usr/lib64/casapy/stable/data/nrao/VLA/CalModels/3C286_C.im for modimage.
INFO    imager::setjy() Using channel dependent flux densities
INFO    imager::data selection  Selected 31964 out of 845379 rows.
INFO    imager::setjy()   J1331+3030 (fld ind 0) spw 0  [I=7.8169, Q=0, U=0, V=0] Jy, (Perley-Butler 2010)
INFO    imager::setjy() Using model image /usr/lib64/casapy/stable/data/nrao/VLA/CalModels/3C286_C.im
INFO    imager::setjy() Scaling spw 0's model image by channel to I = [7.81694, 7.81481, 7.81267, 7.81054, 
 7.80841, 7.80628, 7.80416, 7.80203, 7.7999, 7.79778, 7.79566, 7.79354, 7.79142, 7.7893, 7.78718, 7.78507, 
 7.78295, 7.78084, 7.77873, 7.77662, 7.77451, 7.7724, 7.7703, 7.76819, 7.76609, 7.76398, 7.76188, 7.75978,
 7.75769, 7.75559, 7.75349, 7.7514, 7.74931, 7.74721, 7.74512, 7.74303, 7.74095, 7.73886, 7.73677, 7.73469,
 7.73261, 7.73053, 7.72845, 7.72637, 7.72429, 7.72221, 7.72014, 7.71807, 7.71599, 7.71392, 7.71185, 7.70978,
 7.70772, 7.70565, 7.70359, 7.70152, 7.69946, 7.6974, 7.69534, 7.69328, 7.69123, 7.68917, 7.68712, 7.68506]
 Jy (ch 0) for visibility prediction.
INFO    imager::setjy() The model image's reference pixel is 0.00302169 arcsec from J1331+3030's phase center.
INFO    imager::data selection  Selected 31964 out of 845379 rows.
INFO    imager::createFTMachine()       Performing interferometric gridding...

As set, the flux density scale is being set only for spectral window 0 (spw='0' ). The flux density in each Stokes (IQUV) for the reference channel 0 is reported, followed by the I flux density in each channel of the spectral window that will be used to scale the data. This value is determined from an analytical formula for the spectrum of the source as a function of frequency; this value must be determined so that the flux density in the image can be scaled to it, as it is unlikely that the observation was taken at exactly the same frequency as the model image.

Initial Phase Calibration

Before solving for the bandpass, we first do an initial phase calibration. Depending upon frequency and configuration, there could be gain variations between the different scans of the bandpass calibrator, particularly if the scans happen at much different elevations. One can solve for an initial set of antenna-based gains, which will later be discarded, in order to moderate the effects of variations from scan to scan on the bandpass calibrator. While amplitude variations will have little effect on the bandpass solutions, it is important to solve for any phase variations with time to prevent decorrelation when vector averaging the data in computing the bandpass solutions.

We solve for phase versus time for the central channels on our three calibrators: 

# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.G0all', 
        field='0,1,9', refant='ea21', spw='0:27~36',
        gaintype='G',calmode='p', solint='int', 
        minsnr=5, gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'])
  • caltable='3c391_ctm_mosaic_10s_spw0.G0all' : The gain solutions will be stored in an external table.
  • field='0,1,9' : Specify the calibrators. We will use this opportunity to inspect solutions for all calibrators to weed out bad data.
  • refant='ea21' : Earlier, by looking at the output from plotants, a reference antenna near the center of the array was noted. Here is the first time that that choice will be used. Strictly, all of the gain corrections derived will be relative to this reference antenna.
  • spw='0:27~36' : One wants to choose a subset of the channels from which to determine the gain corrections. These should be near the center of the band, and there should be enough channels chosen so that a reasonable signal-to-noise ratio can be obtained. (See the output of plotms above.) Particularly at lower frequencies where RFI can manifest itself, one should choose RFI-free frequency channels. Also note that, even though these data have only a single spectral window, the syntax requires specifying the spectral window in order to specify the spectral channels.
  • gaintype='G' : Compute G solutions, one per antenna per spw per polarization per solution interval
  • calmode='p' : Solve for only the phase portion of the gain.
  • solint='int' : One wants to be able to track the phases, so a short solution interval is chosen. (A single integration time or 10 seconds for this case)
  • minsnr=5 : One probably wants to restrict the solutions to be at relatively high signal-to-noise ratios, although this parameter may need to be varied depending upon the source and frequency.
  • gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'] : Having produced antenna position corrections, they should now be applied.
initial gain phases, R polarization, ea05

To really see what is going on, we use plotcal to inspect the solutions for a single antenna at a time, iterating through each antenna in sequence, starting with the R polarization: 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',xaxis='time',yaxis='phase',
        poln='R',iteration='antenna',plotrange=[-1,-1,-180,180])

You use the Next button on the GUI to advance the displayed antenna.

NOTE: Sometimes plotcal will "lock" a table and/or keep it in the table cache beyond the end of plotting. This can hang up further use of that table, or cause errors if you delete the table (outside of CASA) and want to re-create it. To deal with that issue, simply close the plotcal GUI using the Quit button when you are done looking at the plot.

Antennas that have been flagged will show a blank plot, as there are no solutions for these antennas. For most antennas, we see a fairly smooth variation with time, so we expect to be able to calibrate our data nicely. However, when you get top ea05, ote the phase jumps where it appears to be bouncing between two states. We can plot this antenna separately (see the figure to the right): 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',xaxis='time',yaxis='phase',antenna='ea05',
        poln='R',iteration='antenna',plotrange=[-1,-1,-180,180],figfile='plotcal_3c391-G0all-phase-R-ea05.png')

We see the same thing for the L polarization solutions: 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0all',
        xaxis='time', yaxis='phase', poln='L', 
        iteration='antenna', plotrange=[-1,-1,-180,180])

Antennas other than ea05 look OK.

We will not be able to transfer calibration for antenna ea05 so we will delete it from the data: 

# In CASA
tflagdata(vis='3c391_ctm_mosaic_10s_spw0.ms',
          flagbackup=T, mode='manual', antenna='ea05')

For the following bandpass solution we need only solve for our bandpass calibrator, and we will do so now after flagging: 

# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.G0', 
        field='J1331+3030', refant='ea21', spw='0:27~36', calmode='p', solint='int', 
        minsnr=5, gaintable=['3c391_ctm_mosaic_10s_spw0.antpos'])

You can inspect this with plotcal as we did above. For example, plot R for the first block of 3C286 data only 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G0',
        xaxis='time',yaxis='phase',poln='R',field='J1331+3030',iteration='antenna',
        plotrange=[-1,-1,-180,180],timerange='08:02:00~08:17:00')

You can do the same for L if you like.

Delay Calibration

The first stage of bandpass calibration involves solving for the antenna-based delays which put a phase ramp versus frequency channel in each spectral window. The K gain type in gaincal solves for the relative delays of each antenna relative to the refant (so be sure you pick one that is there for this entire scan and good). This is not a full global delay, but gives one value per spw per polarization. 

# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.K0', 
        field='J1331+3030',refant='ea21',spw='0:5~58',gaintype='K', 
        solint='inf',combine='scan',minsnr=5,
        gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                   '3c391_ctm_mosaic_10s_spw0.G0'])
  • field='J1331+3030' : For the bandpass calibrator
  • refant='ea21' : Delays will be relative to this antenna, make sure it is there!
  • spw='0:5~58' : Widest possible frequency range in the spw, avoiding edge channels
  • gaintype='K' : Compute K solutions, one per antenna per spw per polarization per solution interval
  • solint='inf',combine='scan' : Only need one solution averaged over all times and scans
  • minsnr=5 : One probably wants to restrict the solutions to be at relatively high signal-to-noise ratios, although this parameter may need to be varied depending upon the source and frequency.
  • gaintable : Use the antpos and G0 tables.
delay solutions

We can plot these solutions (in nanoseconds) as a function of antenna: 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.K0',xaxis='antenna',yaxis='delay',
        figfile='plotcal_3c391-K0-delay.png')

These are within about 4 nanoseconds, as expected for the EVLA early science observations.

Bandpass Calibration

In this step one solves for the complex bandpass, [math]\displaystyle{ B_i }[/math].

bandpass illustration

For the VLA, in its old continuum modes, this step could be skipped. With the EVLA, all data are spectral line, even if the science that one is conducting is continuum. Solving for the bandpass won't hurt for continuum data, and, for moderate or high dynamic range image, it is essential. To motivate the need for solving for the bandpass, consider the image to the right. It shows the right circularly polarized data (RR polarization) for the source J1331+3030, which will serve as the bandpass calibrator. The data are color coded by scan, and they are averaged over all baselines, as earlier plots from plotms indicated that the visibility data are nearly constant with baseline length. Ideally, the visibility data would be constant as a function of frequency as well. The variations with frequency are a reflection of the (slightly) different antenna bandpasses. (Exercise for the reader, reproduce this plot using plotms.)

Now form the bandpass, using the phase solutions just derived. 

# In CASA
bandpass(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.B0',
         field='J1331+3030',spw='',refant='ea21',solnorm=True,combine='scan', 
         solint='inf',bandtype='B',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                    '3c391_ctm_mosaic_10s_spw0.G0',
                    '3c391_ctm_mosaic_10s_spw0.K0'])
  • caltable='3c391_ctm_mosaic_10s_spw0.B0' : Specify where to store the bandpass corrections.
  • solnorm=T : Make sure that the average of the amplitudes of the bandpass corrections are normalized to unity (across the spw)
  • solint='inf', combine='scan' : This observation contains multiple scans on the bandpass calibrator, J1331+3030. Because these are continuum observations, it is probably acceptable to combine all the scans and compute one bandpass correction per antenna, which is achieved by the combination of solint='inf' and combine='scan'. Had combine=' ', then there would have been a bandpass correction derived per scan, which might be necessary for the highest dynamic range spectral line observations.
  • bandtype='B' : The bandpass solution will be derived on a channel-by-channel basis. There is an alternate, somewhat experimental option of bandtype='BPOLY' that will attempt to fit an n-th order polynomial to the bandpass.
  • gaintable : Pre-apply antenna positions, phase solutions, and delays.

Once again, one can use plotcal to display the bandpass solutions. Note that in the plotcal inputs below, the amplitudes are being displayed as a function of frequency channel and, for compactness, subplot=221 is used to display multiple plots per page. One could use yaxis='phase' to view the phases as well. We use iteration='antenna' to step through separate plots for each antenna. 

# In CASA
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='R', 
        xaxis='chan',yaxis='amp',field= 'J1331+3030',subplot=221, 
        iteration='antenna',figfile='plotcal_3c391-3C286-B0-R-amp.png')
#
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='L', 
        xaxis='chan',yaxis='amp',field= 'J1331+3030',subplot=221, 
        iteration='antenna',figfile='plotcal_3c391-3C286-B0-L-amp.png')
#
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='R', 
        xaxis='chan',yaxis='phase',field= 'J1331+3030',subplot=221, 
        iteration='antenna',plotrange=[-1,-1,-180,180],
        figfile='plotcal_3c391-3C286-B0-R-phase.png')
#
plotcal(caltable= '3c391_ctm_mosaic_10s_spw0.B0',poln='L', 
        xaxis='chan',yaxis='phase',field= 'J1331+3030',subplot=221, 
        iteration='antenna',plotrange=[-1,-1,-180,180],
        figfile='plotcal_3c391-3C286-B0-L-phase.png')

As expected, the bandpass phases (rightmost plots below) are relatively flat, with the slopes removed by the delay calibration. Residual excursions are on the order of a few degrees.

bandpass amp for 3C 286 R
bandpass amp for 3C 286 L
bandpass phase for 3C 286 R
bandpass phase for 3C 286 L

Gain Calibration

The next step is to derive corrections for the complex antenna gains, [math]\displaystyle{ g_i }[/math] and [math]\displaystyle{ \theta_i }[/math]. As discussed in the lectures and above, the absolute magnitude of the gain amplitudes [math]\displaystyle{ g_i }[/math] are determined by reference to a standard flux density calibrator. In order to determine the appropriate complex gains for the target source, one wants to observe a so-called phase calibrator that is much closer to the target, in order to minimize differences through the atmosphere (neutral and/or ionized) between the lines of sight to the phase calibrator and the target source. If we determine the relative gain amplitudes and phases for different antennas using the phase calibrator, we can later determine the absolute flux density scale by comparing the gain amplitudes [math]\displaystyle{ g_i }[/math] derived for 3C 286 with those derived for the phase calibrator. This will eventually be done using the task fluxscale. Since there is no such thing as absolute phase, we determine a zero phase by selecting a reference antenna for which the gain phase is defined to be zero.

In principle, one could determine the complex antenna gains for all sources with a single invocation of gaincal; for clarity here, two separate invocations will be used.

In the first step, we derive the appropriate complex gains [math]\displaystyle{ g_i }[/math] and [math]\displaystyle{ \theta_i }[/math] for the flux density calibrator 3C 286. 

# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.G1',
        field='J1331+3030',spw='0:5~58',
        solint='inf',refant='ea21',gaintype='G',calmode='ap',solnorm=F,
        gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                   '3c391_ctm_mosaic_10s_spw0.K0',
                   '3c391_ctm_mosaic_10s_spw0.B0'])
  • caltable='3c391_ctm_mosaic_10s_spw0.G1' : Produce a new calibration table containing these gain solutions. In order to make the bookkeeping easier, a '1' is appended to the file name to distinguish it from the earlier set of gain solutions, which are effectively being "thrown away."
  • spw='0:5~58' : From the inspection of the bandpass, one can determine the range of edge channels that are affected by the bandpass filter rolloff. Because the amplitude is dropping rapidly in these channels, one does not want to include them in the solution.
  • gaintype='G', calmode='ap', solnorm=F : Solve for the complex antenna gains for 3C 286. The objective is to relate the measured data values to the (assumed known) flux density of 3C 286, thus the solution is both amplitude and phase ('ap') and the solutions should not be normalized to unity amplitude.
  • solint='inf' : Produce a solution for each scan. Phase coherence for these observations is good.
  • gaintable: Use the antenna position corrections, delays, and bandpass solutions determined earlier before solving for the gain amplitudes.

In the second step, the appropriate complex gains for a direction on the sky close to the target source will be determined from the phase calibrator J1822-0938. We also determine the complex gains for the polarization calibrator source J0319+4130. These will be solved separately, but in practice could be solved together as there are no gaintables that are time dependent at this point (and thus would risk having cross-source interpolation issues) nor are we doing different solution intervals per source. 

# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.G1',
        field='J1822-0938',
        spw='0:5~58',solint='inf',refant='ea21',gaintype='G',calmode='ap',
        gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                   '3c391_ctm_mosaic_10s_spw0.K0',
                   '3c391_ctm_mosaic_10s_spw0.B0'],
        append=True)
#
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.G1',
        field='J0319+4130',
        spw='0:5~58',solint='inf',refant='ea21',gaintype='G',calmode='ap',
        gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                   '3c391_ctm_mosaic_10s_spw0.K0',
                   '3c391_ctm_mosaic_10s_spw0.B0'],
        append=True)
  • caltable='3c391_ctm_mosaic_10s_spw0.G1', append=True: In all previous invocations of gaincal, append has been set to False. Here, the gain solutions from the phase calibrators are going to be appended to the existing set from 3C 286. In following steps, all of these gain solutions will then be used together to derive a set of complex gains that are applied to the science data for the target source.

If one checks the gain phase solutions using plotcal, one should see smooth solutions for each antenna as a function of time. 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='phase',
        poln='R',plotrange=[-1,-1,-180,180],figfile='plotcal_3c391-G1-phase-R.png')
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='phase',
        poln='L',plotrange=[-1,-1,-180,180],figfile='plotcal_3c391-G1-phase-L.png')
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='amp',
        poln='R',figfile='plotcal_3c391-G1-amp-R.png')
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.G1',xaxis='time',yaxis='amp',
        poln='L',figfile='plotcal_3c391-G1-amp-L.png')
gain phase solutions, R polarization
gain phase solutions, L polarization
gain amp solutions, R polarization
gain amp solutions, L polarization

Polarization Calibration

[If time is running short, skip this step and proceed to [Applying the Calibration].] ("Stay on target." Gold Five)

Having set the complex gains, we now need to do the polarization calibration. This should be done prior to running fluxscale, since it has to run using the un-rescaled gains in the MODEL_DATA column of the measurement set. Polarization calibration is done in two steps. First, we solve for the instrumental polarization (the frequency-dependent leakage terms, or 'D-terms'), using either an unpolarized source or a source which has sufficiently good parallactic angle coverage. Second, we solve for the polarization position angle using a source with a known polarization position angle (3C 286 is recommended here).

Our initial run of setjy only set the total intensity of our flux calibrator source, 3C 286. This source is known to have a fairly stable fractional polarization of 11.2% at C-band, and a polarization position angle of 66 degrees. NRAO conducted regular monitoring of a number of polarization calibrators (including 3C 286) from 1999 through 2009. If you go to the polarization calibration webpage and follow the link for a particular year, then search for '1331+305 C band' (1331+305 is better known as 3C 286), you will see in the table the measured values for the percentage polarization and polarization position angle. We are in the process of updating this to include monitoring with the JVLA from 2010 onward, see the JVLA Observational Status Summary guide to polarization for details on this.

In order to calibrate the position angle, we need to set the appropriate values for Stokes Q and U. Examining our casapy.log file to find the output of setjy, we find that the total intensity was set to 7.81694 Jy in channel 0 of spw 0, at 4536 MHz according to the listobs output. We therefore use python to find the polarized flux, P, and the values of Stokes Q and U at this reference channel: 

# In CASA
i0=7.81694 # Stokes I value for spw 0 ch 0
p0=0.112*i0 # Fractional polarization=11.2%
q0=p0*cos(66*pi/180) # Stokes Q for spw 0
u0=p0*sin(66*pi/180) # Stokes U for spw 0

Furthermore, in CASA 3.4, we can introduce a spectral index to the flux densities. We do this by noting that our printout from setjy gave 7.68506 Jy in the last channel 127 (at 4663 MHz). Therefore we use python again to compute a spectral index 

# In CASA
alpha = log(7.68506/7.81694)/log(4663.0/4536.0)

which gives alpha = -0.61618.

We now set the values of Stokes I, Q and U for 3C 286, using setjy using these values: 

# In CASA
setjy(vis='3c391_ctm_mosaic_10s_spw0.ms',field='J1331+3030',modimage='',
      spw='0',fluxdensity=[i0,q0,u0,0],spix=alpha,reffreq='4536.0MHz',
      scalebychan=True,usescratch=True)
  • modimage=' ' : A model image is not used here.
  • fluxdensity=[i0,q0,u0,0] : use the IQUV values for channel 0 calculated above
  • spix=alpha  : Set the spectral index using the formula above (this will also apply to QU)
  • reffreq='4536.0MHz' : The frequency of channel 0 from listobs
  • scalebychan=True : let it compute a value per channel
  • usescratch=True : use the MODEL_DATA column explicitly

Note that the Stokes V flux value is set to zero, corresponding to no circular polarization.

We can see the results in the model column in plotms 

# In CASA
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='RR',
       timerange='08:02:00~08:17:00',antenna='ea01&ea02',
       xaxis='channel',yaxis='amp',ydatacolumn='model',
       plotfile='plotms_3c391-model-amp-RR.png')

showing our model source spectrum. We can see this translates to the spectrum in QU 

# In CASA
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='RL',
       timerange='08:02:00~08:17:00',antenna='ea01&ea02',
       xaxis='channel',yaxis='amp',ydatacolumn='model',
       plotfile='plotms_3c391-model-amp-RL.png')

Finally, our R-L phase difference is constant at 66 degrees as desired: 

# In CASA
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='RL',
       timerange='08:02:00~08:17:00',antenna='ea01&ea02',
       xaxis='channel',yaxis='phase',ydatacolumn='model',
       plotrange=[-1,-1,-180,180],plotfile='plotms_3c391-model-phase-RL.png')
model RR amplitudes
model RL amplitudes
model RL phases

Solving for the Cross-Hand delays

Just as we did for the parallel-hand (RR,LL) delays before bandpass calibration, we solve for the cross-hand (RL,LR) delays due to the residual delay difference between the R and L on the reference antenna (ea21) used for the original delay calibration. In our case we simply use 3C286 which has a strong polarized signal in the RL,LR correlations. 

# In CASA
gaincal(vis='3c391_ctm_mosaic_10s_spw0.ms', caltable='3c391_ctm_mosaic_10s_spw0.Kcross',
        field='J1331+3030', spw='0:5~58',
        gaintype='KCROSS', solint='inf', combine='scan', refant='ea21',
        gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                   '3c391_ctm_mosaic_10s_spw0.K0',
                   '3c391_ctm_mosaic_10s_spw0.B0',
                   '3c391_ctm_mosaic_10s_spw0.G1'],
        gainfield=['','','','J1331+3030'],
        interp=['linear','nearest','nearest','linear'], parang=T)
cross-hand delay solutions

We can plot these: 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.Kcross',xaxis='antenna',yaxis='delay',
        figfile='plotcal_3c391-Kcross-delay.png')

As expected there is a single value for R versus L (with L delay set to zero) across all antennas. The solution is reported in the logger, and is 7.149 ns.

Note that if we did not solve for this delay, it would be absorbed into the phases per channel of the following Df and Xf solutions. This would not cause us problems, as we are not solving for the Q+iU polarization of our D-term calibrator (we are using unpolarized 3C84 for that) but if we were (e.g. using our gain calibrator J1822-0938 with poltype='Df+QU') then this step would be essential.

Solving for the Leakage Terms

The task polcal is used for polarization calibration. In this data set, we observed the unpolarized calibrator J0319+4130 (a.k.a. 3C 84) in order to solve for the instrumental polarization. polcal uses the Stokes I, Q, and U values in the MODEL_DATA column (Q and U being zero for our unpolarized calibrator) to derive the leakage solutions. The function call is 

# In CASA
polcal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.D1',
       field='J0319+4130',spw='0:5~58',
       refant='ea21',poltype='Df',solint='inf',combine='scan',
       gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                  '3c391_ctm_mosaic_10s_spw0.K0',
                  '3c391_ctm_mosaic_10s_spw0.B0',
                  '3c391_ctm_mosaic_10s_spw0.G1',
                  '3c391_ctm_mosaic_10s_spw0.Kcross'],
       gainfield=['','','','J0319+4130',''],
       interp=['linear','nearest','nearest','linear','nearest'])
  • caltable='3c391_ctm_mosaic_10s_spw0.D1' : polcal will create a new calibration table containing the leakage solutions, which we specify with the caltable argument.
  • field='J0319+4130' : The unpolarized source J0319+4130 (a.k.a. 3C 84) is used to solve for the leakage terms.
  • spw='0:5~58' : In this example, the edge channels are not used in finding the solution. Because the bandpass is one of the calibration tables being applied (in gaintable), this restriction is not necessary. However, if one restricts the spectral window here, it must also be restricted in the remainder of the calibration steps, particularly applycal, otherwise the final data set will contain frequency channels for which the leakage terms have not been calibrated.
  • poltype='Df' : We will solve for the leakages (D) on a per-channel basis (f) assuming zero source polarization.
  • solint='inf' : One solution over the entire run (the default)
  • combine='scan' : Combine all scans together (the default)
  • gaintable : All of the previous corrections---antenna positions, K-delay, bandpass, Kcross-delay, and complex gain---are to be applied
  • gainfield : The gain caltable that is being applied on the fly, 3c391_ctm_mosaic_10s_spw0.G1, contains the solutions for multiple sources. Only the solutions from J0319+4130 should be applied to itself in the process of finding the polarization leakage terms.
  • interp : The interpolation to be used for each gaintable.

After polcal has finished running, you are strongly advised to examine the solutions with plotcal, to ensure that everything looks good.

Df amp vs. channel for ea01
Df phase vs. channel for ea01
# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D1',xaxis='chan',yaxis='amp', 
        spw='',field='',iteration='antenna')
#
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D1',xaxis='chan',yaxis='phase', 
        spw='',field='',iteration='antenna',plotrange=[-1,-1,-180,180])

This will produce plots similar to those shown at right. As ever, you can cycle through the antennas by clicking the "Next" button. You should see leakages of between 5 and 15% in most cases. We can also display these in a single plot versus antenna index:

Df solutions for J0319+4130 versus antenna index
# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D1',xaxis='antenna',yaxis='amp', 
        figfile='plotcal_3c391-D1.png')

Note that there are no solutions for antenna ea04 - a bit of sleuthing will turn up that ea04 was missing for the scans on 3C84.

If we want to rescue ea04 (which otherwise seems ok) then we turn to our gain calibrator. If we plot data for field 1 versus ParAngle in plotms then we see that it has sufficient range (>60 deg) in Parallactic Angle so should be useable as a calibrator with unknown polarization. We can make a new set of solutions: 

# In CASA
polcal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.D2',
       field='J1822-0938',spw='0:5~58',
       refant='ea21',poltype='Df+QU',solint='inf',combine='scan',
       gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                  '3c391_ctm_mosaic_10s_spw0.K0',
                  '3c391_ctm_mosaic_10s_spw0.B0',
                  '3c391_ctm_mosaic_10s_spw0.G1',
                  '3c391_ctm_mosaic_10s_spw0.Kcross'],
       gainfield=['','','','J1822-0938',''],
       interp=['linear','nearest','nearest','linear','nearest'])
  • field='J1822-0938' : Our gain calibrator observed throughout the schedule block
  • poltype='Df+QU' : Solve for the leakages using a calibrator with unknown polarization but with good parallactic angle coverage, and simultaneously for the source polarization (averaged over frequency)
  • gainfield : For 3c391_ctm_mosaic_10s_spw0.G1 use only the solutions from J1822-0938 itself.
Df solutions for J1822-0938 versus antenna index

We now plot this as we did before: 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.D2',xaxis='antenna',yaxis='amp', 
        figfile='plotcal_3c391-D2.png')

Comparison of this plot with that for the D1 caltable shows that we get nearly identical results, but now ea04 (index 3) is present! That should give us some confidence in our leakage calibration as well.

Solving for the R-L polarization angle

Having calibrated the instrumental polarization, the total polarization is now correct, but the R-L phase still needs to calibrated in order to obtain an accurate polarization position angle. We use the same task, polcal, but this time set poltype='Xf, which specifies a frequency-dependent (f) position angle (X) calibration, using the source J1331+3030 (a.k.a. 3C 286), whose position angle is known, having set this earlier using setjy. Note that we must correct for the leakages before determining the R-L phase, which we do by adding the calibration table made in the previous step (3c391_ctm_mosaic_10s_spw0.D2) to the gain tables that are applied on-the-fly. Note that we are using the second D table we made as it included ea04. 

# In CASA
polcal(vis='3c391_ctm_mosaic_10s_spw0.ms',caltable='3c391_ctm_mosaic_10s_spw0.X1',
       field='J1331+3030',refant='ea21',
       poltype='Xf',solint='inf',combine='scan',
       gaintable=['3c391_ctm_mosaic_10s_spw0.antpos',
                  '3c391_ctm_mosaic_10s_spw0.K0',
                  '3c391_ctm_mosaic_10s_spw0.B0',
                  '3c391_ctm_mosaic_10s_spw0.G1',
                  '3c391_ctm_mosaic_10s_spw0.Kcross',
                  '3c391_ctm_mosaic_10s_spw0.D2'],
       gainfield=['','','','J1331+3030','',''],
       interp=['linear','nearest','nearest','linear','nearest','nearest'])
Xf solutions versus channel.

Again, it is strongly suggested that you check the calibration worked properly, by plotting up the newly-generated calibration table using plotcal. The results are shown at right. 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.X1',xaxis='chan',yaxis='phase',
        figfile='plotcal_3c391-X1.png')

Because the Xf term captures the residual R-L phase on the reference antenna over the array, there is one value for all antennas. Also, as we took out the RL delays using the Kcross solution, these Xf variations only span about 6 degress across the spectral window.

At this point, you have all the necessary polarization calibration tables.

Scaling the Amplitude Gains

While we know the flux density of our primary calibrator (in our case, J1331+3030[math]\displaystyle{ \equiv }[/math]3C 286), the model assumed for the secondary calibrator (here, J1822-0938) was a point source of 1 Jy located at the phase center. While the secondary calibrator was chosen to be a point source (at least, over some limited range of uv-distance; see the VLA calibrator manual for any u-v restrictions on your calibrator of choice at the observing frequency), its absolute flux density is unknown. Being pointlike, secondary calibrators typically vary on timescales of months to years, in some cases by up to 50--100%. A nice Java Applet is available to track the flux density history of various calibrators over time. Play around with it to see how much some of the calibrators from the manual can vary, and over what sorts of timescales.

We use the primary calibrator (the 'flux calibrator') to determine the system response to a source of known flux density, and assume that the mean gain amplitudes for the primary calibrator are the same as those for the secondary calibrator. This then allows us to find the true flux density of the secondary calibrator. To do this, we use the task fluxscale, which produces a new calibration table containing properly-scaled amplitude gains for the secondary calibrator. 

# In CASA
myscale = fluxscale(vis='3c391_ctm_mosaic_10s_spw0.ms',
                    caltable='3c391_ctm_mosaic_10s_spw0.G1', 
                    fluxtable='3c391_ctm_mosaic_10s_spw0.fluxscale1', 
                    reference=['J1331+3030'],
                    transfer=['J1822-0938,J0319+4130'])
  • myscale = fluxscale() : fluxscale returns a dictionary of results, which we capture in the variable myscale
  • caltable='3c391_ctm_mosaic_10s_spw0.G1' : We provide fluxscale with the calibration table containing the amplitude gain solutions derived earlier.
  • fluxtable='3c391_ctm_mosaic_10s_spw0.fluxscale1' : We specify the name of the new output table to be written, which will contain the properly-scaled amplitude gains.
  • reference='J1331+3030' : We specify the source with the known flux density.
  • transfer=['J1822-0938,J0319+4130'] : We specify the sources whose amplitude gains are to be rescaled.

fluxscale will print to the CASA logger the derived flux densities of all calibrator sources specified with the transfer argument. These are also captured in the return variable from the task. You should examine the output to ensure that it looks sensible. If one's data set has more than 1 spectral window, depending upon where they are spaced and the spectrum of the source, it is quite possible to find (quite) different flux densities at the different frequencies for the secondary calibrators. Example output would be 

CASA <314>: myscale['1']
  Out[314]: 
{'fieldName': 'J1822-0938',
 'fluxd': array([ 2.33283596]),
 'fluxdErr': array([ 0.0063605]),
 'numSol': array([46], dtype=int32)}

CASA <315>: myscale['9']
  Out[315]: 
{'fieldName': 'J0319+4130',
 'fluxd': array([ 13.76485928]),
 'fluxdErr': array([ 0.03852698]),
 'numSol': array([44], dtype=int32)}

or in the logger 

INFO    fluxscale::calibrater::open     Opening MS: 3c391_ctm_mosaic_10s_spw0.ms for calibration.
INFO    fluxscale::Calibrater:: Initializing nominal selection to the whole MS.
INFO    fluxscale::calibrater::fluxscale        Beginning fluxscale--(MSSelection version)-------
INFO    fluxscale::::    Found reference field(s): J1331+3030
INFO    fluxscale::::    Found transfer field(s):  J1822-0938 J0319+4130
INFO    fluxscale::::    Flux density for J1822-0938 in SpW=0 is: 2.33284 +/- 0.0063605 (SNR = 366.769, N = 46)
INFO    fluxscale::::    Flux density for J0319+4130 in SpW=0 is: 13.7649 +/- 0.038527 (SNR = 357.278, N = 44)
INFO    fluxscale::Calibrater::fluxscale        Storing result in 3c391_ctm_mosaic_10s_spw0.fluxscale1
INFO    fluxscale::::   Writing solutions to table: 3c391_ctm_mosaic_10s_spw0.fluxscale1

The VLA calibrator manual can be used to check whether the derived flux densities look sensible. Wildly different flux densities or flux densities with very high error bars should be treated with suspicion; in such cases you will have to figure out whether something has gone wrong.

We plot the rescaled amplitudes from this table: 

# In CASA
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.fluxscale1',xaxis='time',yaxis='amp',
        poln='R',figfile='plotcal_3c391-fluxscale1-amp-R.png')
plotcal(caltable='3c391_ctm_mosaic_10s_spw0.fluxscale1',xaxis='time',yaxis='amp',
        poln='L',figfile='plotcal_3c391-fluxscale1-amp-L.png')

You can see (in the plots shown below) that the amplitude gain factors are now similar across sources, compared to the raw factors in the G1 table.

post-fluxscale amp solutions, R pol
post-fluxscale amp solutions, L pol

Applying the Calibration

Now that we have derived all the calibration solutions, we need to apply them to the actual data, using the task applycal. The measurement set contains three data columns; DATA, MODEL_DATA, and CORRECTED_DATA. The DATA column contains the original data. The MODEL_DATA column contains whatever model we used for the calibration; for J1331+3030, this is what we specified in setjy, and for all other sources, this was set to a point source of 1 Jy at the phase center when the scratch columns were originally created using clearcal. To apply the calibration we have so painstakingly derived, we specify the appropriate calibration tables, which are then applied to the DATA column, with the results being written in the CORRECTED_DATA column.

First, we apply the calibration to each individual calibrator, using the gain solutions derived on that calibrator alone to compute the CORRECTED_DATA. To do this, we iterate over the different calibrators, in each case specifying the source to be calibrated (using the field parameter). The relevant function calls are given below, although as explained presently, the calls to applycal will differ slightly if you skipped the [Polarization Calibration]. 

# In CASA
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',
         field='J1331+3030',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', 
                    '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                    '3c391_ctm_mosaic_10s_spw0.K0',
                    '3c391_ctm_mosaic_10s_spw0.B0',
                    '3c391_ctm_mosaic_10s_spw0.Kcross', 
                    '3c391_ctm_mosaic_10s_spw0.D2',
                    '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J1331+3030','','','','',''], 
         interp=['','nearest','','','','',''],
         parang=True,calwt=F)
#
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',
         field='J0319+4130',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', 
                    '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                    '3c391_ctm_mosaic_10s_spw0.K0',
                    '3c391_ctm_mosaic_10s_spw0.B0',
                    '3c391_ctm_mosaic_10s_spw0.Kcross', 
                    '3c391_ctm_mosaic_10s_spw0.D2',
                    '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J0319+4130','','','','',''], 
         interp=['','nearest','','','','',''],
         parang=True,calwt=F)
#
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',
         field='J1822-0938',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', 
                    '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                    '3c391_ctm_mosaic_10s_spw0.K0',
                    '3c391_ctm_mosaic_10s_spw0.B0',
                    '3c391_ctm_mosaic_10s_spw0.Kcross', 
                    '3c391_ctm_mosaic_10s_spw0.D2',
                    '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J1822-0938','','','','',''], 
         interp=['','nearest','','','','',''],
         parang=True,calwt=F)
  • gaintable : We provide a Python list of the calibration tables to be applied. This list must contain the antenna position corrections (in .antpos), the properly-scaled gain calibration for the amplitudes and phases (in .fluxscale1) which were just made using fluxscale, the parallel-hand delays (in .K0), the bandpass solutions (in .B0), the cross-hand delays (in .Kcross), the leakage calibration (in .D2 derived by our second solution), and the R-L phase corrections (in .X1).
  • gainfield, interp : To ensure that we use the correct gain amplitudes and phases for a given calibrator (those derived on that same calibrator), then for each calibrator source, we need to specify the particular subset of gain solutions to be applied. This requires use of the gainfield and interp arguments; these are both Python lists, and for the list item corresponding to the calibration table made by fluxscale, we set gainfield to the field name corresponding to that calibrator, and the desired interpolation type (interp) to nearest.
  • parang : Since we have performed polarization calibration, we must set parang=True, or we will discard all that hard work we did earlier. However, if you skipped the [Polarization_Calibration] section, the .Kcross, .D2 and .X1 will not exist. In this case, you should leave out these tables from the gaintable list, and the corresponding sets of elements in the gainfield list each time you run applycal above. You should also set parang=False.
  • calwt=F : At the time of writing, we are not yet using system calibration data to compute real (1/Jy2) weights, thus trying to calibrate them can produce nonsensical results. In particular, experience has shown that calibrating the weights will lead to problems especially in the self-calibration steps.

Finally, we apply the calibration to the target fields in the mosaic, linearly interpolating the gain solutions from the secondary calibrator, J1822-0938. In this case however, we want to apply the amplitude and phase gains derived from the secondary calibrator, J1822-0938, since that is close to the target source on the sky, and we assume that the gains applicable to the target source are very similar to those derived in the direction of the secondary calibrator. Of course, this is not strictly true, since the gains on J1822-0938 were derived at a different time and in a different position on the sky from the target. However, assuming that the calibrator was sufficiently close to the target, and the weather was sufficiently well-behaved, then this is a reasonable approximation, and should get us a sufficiently good calibration that we can later use self-calibration to correct for the small inaccuracies thus introduced.

The procedure for applying the calibration to the target source is very similar to what we just did for the calibrator sources. 

# In CASA
applycal(vis='3c391_ctm_mosaic_10s_spw0.ms',
         field='2~8',
         gaintable=['3c391_ctm_mosaic_10s_spw0.antpos', 
                    '3c391_ctm_mosaic_10s_spw0.fluxscale1',
                    '3c391_ctm_mosaic_10s_spw0.K0',
                    '3c391_ctm_mosaic_10s_spw0.B0',
                    '3c391_ctm_mosaic_10s_spw0.Kcross', 
                    '3c391_ctm_mosaic_10s_spw0.D2',
                    '3c391_ctm_mosaic_10s_spw0.X1'],
         gainfield=['','J1822-0938','','','','',''], 
         interp=['','linear','','','','',''],
         parang=True,calwt=F)
  • field : We can calibrate all seven target fields at once by setting field='2~8' .
  • gainfield : In this case, we wish to use the gains derived on the secondary calibrator, for the reasons explained in the previous paragraph.
  • interp : This time, we linearly interpolate between adjacent calibrator scans, to compute the appropriate gains for the intervening observations of the target.
amp vs channel for J1822-0938 RR,RL,LR,LL
phase vs channel for J1822-0938 RR,RL,LR,LL
amp vs channel for J1822-0938 RR,LL
phase vs channel for J1822-0938 RR,LL

We should now have fully-calibrated visibilities in the CORRECTED_DATA column of the measurement set, and it is worthwhile pausing to inspect them, to ensure that the calibration did what we expected it to. We make some standard plots (see above): 

# In CASA
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='',
       timerange='08:02:00~08:17:00',antenna='',avgtime='60s',
       xaxis='channel',yaxis='amp',ydatacolumn='corrected',
       plotfile='plotms_3c391-fld0-corrected-amp.png')
#
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='0',correlation='',
       timerange='08:02:00~08:17:00',antenna='',avgtime='60s',
       xaxis='channel',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotfile='plotms_3c391-fld0-corrected-phase.png')
#
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='1',correlation='RR,LL',
       timerange='',antenna='',avgtime='60s',
       xaxis='channel',yaxis='amp',ydatacolumn='corrected',
       plotfile='plotms_3c391-fld1-corrected-amp.png')
#
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='1',correlation='RR,LL',
       timerange='',antenna='',avgtime='60s',
       xaxis='channel',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotfile='plotms_3c391-fld1-corrected-phase.png')

For 3C286 (leftmost plots) we see the polarized signal in the cross-hands. There is no sign of bad data remaining.

corrected amplitude vs phase for J1822-0938

Another nice display is to use plotms to plot the amplitude and phase of the CORRECTED_DATA column against one another, for one of the parallel-hand correlations (RR or LL; the signal in the cross-hands, RL and LR is much smaller, and will be noiselike for an unpolarized calibrator). This should then show a nice ball of visibilities centered at zero phase (with some scatter) and the amplitude found for that source in fluxscale. An example is shown at right. 

# In CASA
plotms(vis='3c391_ctm_mosaic_10s_spw0.ms',field='1',correlation='RR,LL',
       timerange='',antenna='',avgtime='60s',
       xaxis='phase',xdatacolumn='corrected',yaxis='amp',ydatacolumn='corrected',
       plotrange=[-180,180,0,3],coloraxis='corr',
       plotfile='plotms_3c391-fld1-corrected-ampvsphase.png')

Inspecting the data at this stage may well show up previously-unnoticed bad data. Plotting up the corrected amplitude against UV distance, or against time is a good way to find such issues. If you find bad data, you can remove them via interactive flagging in plotms, or via manual flagging in tflagdata once you have identified the offending antennas/baselines/channels/times. When you are happy that all data (particularly on your target source) look good, you may proceed.

Now that the calibration has been applied to the target data, we can split off the science targets, creating a new, calibrated measurement set containing all the target fields. 

# In CASA
split(vis='3c391_ctm_mosaic_10s_spw0.ms',outputvis='3c391_ctm_mosaic_spw0.ms',
      datacolumn='corrected',field='2~8')
  • outputvis : We give the name of the new measurement set to be written, which will contain the calibrated data on the science targets.
  • datacolumn : We use the CORRECTED_DATA column, containing the calibrated data which we just wrote using applycal.
  • field : We wish to put all the mosaic pointings into a single measurement set, for imaging and joint deconvolution.

Initial Imaging

Now that we have split off the target data into a separate measurement set with all the calibration applied, it's time to make an image. Recall from the lectures that the visibility data and the sky brightness distribution (a.k.a. image) are Fourier transform pairs

[math]\displaystyle{ I(l,m) = \int V(u,v) e^{[2\pi i(ul + vm)]} dudv }[/math]

The [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] coordinates are the baselines, measured in units of the observing wavelength while the [math]\displaystyle{ l }[/math] and [math]\displaystyle{ m }[/math] coordinates are the direction cosines on the sky. For generality, the sky coordinates are written in terms of direction cosines, but for most EVLA (and ALMA) observations they can be related simply to the right ascension ([math]\displaystyle{ l }[/math]) and declination ([math]\displaystyle{ m }[/math]). Also recall from the lectures that this equation is valid only if the [math]\displaystyle{ w }[/math] coordinate of the baselines can be neglected. This assumption is almost always true at high frequencies and smaller EVLA configurations (such as the 4.6 GHz, D-configuration observations here); the [math]\displaystyle{ w }[/math] coordinate cannot be neglected at lower frequencies and larger configurations (e.g., 0.33 GHz, A-configuration observations). This expression also neglects other factors, such as the shape of the primary beam. For more information on imaging, see [Synthesis Imaging] within the CASA Reference Manual.

CASA has a single task, clean which both Fourier transforms the data and deconvolves the resulting image. For the purposes of this tutorial Part 1, we will make a simple mosaic clean image in Stokes I only. In Part 2 we will proceed to make multiscale mosaic images in IQUV and to analyze them.

To make this simple clean image interactively: 

# In CASA
clean(vis='3c391_ctm_mosaic_spw0.ms',
      imagename='3c391_ctm_spw0_noms_I',
      field='',spw='',
      mode='mfs',
      niter=5000,
      gain=0.1, threshold='1.0mJy',
      psfmode='clark',
      imagermode='mosaic', ftmachine='mosaic',
      multiscale=[0], 
      interactive=True,
      imsize=[480,480], cell=['2.5arcsec','2.5arcsec'],
      stokes='I',
      weighting='briggs',robust=0.5,
      usescratch=True)

clean is a powerful task, with many inputs, and a certain amount of experimentation may be (likely is) required.

  • vis='3c391_ctm_mosaic_spw0.ms' : this split MS contains our 7-pt mosaic fields, now indexed as 0-6. Field 0 is the central field of the mosaic (you can use listobs to verify this).
  • imagename='3c391_ctm_spw0_noms_I' : our output images will all start with this, e.g. 3c391_ctm_spw0_noms_I.image
  • mode='mfs' : Use multi-frequency synthesis imaging. The fractional bandwidth of these data is non-zero (128 MHz at a central frequency of 4.6 GHz). Recall that the [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] coordinates are defined as the baseline coordinates, measured in wavelengths. Thus, slight changes in the frequency from channel to channel result in slight changes in [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math]. There is a concomitant improvement in [math]\displaystyle{ u }[/math]-[math]\displaystyle{ v }[/math] coverage if the visibility data from the multiple spectral channels are gridded separately onto the [math]\displaystyle{ u }[/math]-[math]\displaystyle{ v }[/math] plane, as opposed to treating all spectral channels as having the same frequency.
  • niter=25000,gain=0.1,threshold='1.0mJy' : Recall that the CLEAN gain is the amount by which a CLEAN component is subtracted during the CLEANing process. niter and threshold are (coupled) means of determining when to stop the CLEANing process, with niter specifying to find and subtract that many CLEAN components while threshold specifies a minimum flux density threshold a CLEAN component can have before CLEAN stops. See also interactive below. Imaging is an iterative process, and to set the threshold and number of iterations, it is usually wise to CLEAN interactively in the first instance, stopping when spurious emission from sidelobes (arising from gain errors) dominates the residual emission in the field. Here, we have used our experience in interactive mode to set a threshold level based on the rms noise in the resulting image. The number of iterations should then be set high enough to reach this threshold.
  • interactive=True : Very often, particularly when one is exploring how a source appears for the first time, it can be valuable to interact with the CLEANing process. If True, interactive causes a viewer window to appear. One can then set CLEAN regions, restricting where CLEAN searches for CLEAN components, as well as monitor the CLEANing process. A standard procedure is to set a large value for niter, and stop the CLEANing when it visually appears to be approaching the noise level. This procedure also allows one to change the CLEANing region, in cases when low-level intensity becomes visible as the CLEANing process proceeds. For more details, see [Interactive Cleaning], and the discussion below.
  • imsize=[480,480],cell=['2.5arcsec'] : See the discussion below regarding the setting of the image size and cell size.
  • stokes='I',psfmode='clark' : A single image will be made for total intensity I.
  • weighting='briggs',robust=0.5 : 3C 391 has diffuse, extended emission that is (at least partially) resolved out by the interferometer owing to a lack of short spacings. A naturally-weighted image would show large-scale patchiness in the noise. In order to suppress this effect, Briggs weighting is used (intermediate between natural and uniform weighting), with a default robust factor of 0.5 (which corresponds to something between natural and uniform weighting).
  • imagermode='mosaic', ftmachine='mosaic' : The data consist of a 7-pointing mosaic, since the supernova remnant fills almost the full primary beam at 4.6 GHz. A mosaic combines the data from all of the fields, with imaging and deconvolution being done jointly on all 7 fields. A mosaic both helps compensate for the shape of the primary beam and reduces the amount of large (angular) scale structure that is resolved out by the interferometer.
  • multiscale=[0] : For now, we will clean with a single scale of size 0 (a classic clean delta-function)
  • usescratch=True As in calibration, we will direct the imaging to make use of the MODEL_DATA scratch column. There is the option in 3.4 to use scratchless model operation for some cases but is not yet recommended for all cases.

Setting the appropriate pixel size depends upon basic optics aspects of interferometry. Using plotms to look at the newly-calibrated, target-only data set, 

# In CASA
plotms(vis='3c391_ctm_mosaic_spw0.ms',xaxis='uvwave',yaxis='amp',
       field='0',avgtime='30s',correlation='RR',
       plotfile='plotms_3c391-mosaic0-uvwave.png',overwrite=True)
plotms plot showing Amplitude vs UV Distance in wavelengths for 3C 391 at 4600 MHz

one should obtain a plot similar to the one shown at the right with the (calibrated) visibility amplitude as a function of [math]\displaystyle{ u }[/math]-[math]\displaystyle{ v }[/math] distance. The maximum baseline is about 16,000 wavelengths, i.e., an angular scale of 12 arcseconds ([math]\displaystyle{ \lambda/D=1/16000 }[/math]). The most effective CLEANing occurs with 3--5 pixels across the synthesized beam. Above, a cell size of 2.5 arcseconds (just under 5 pixels per beam) is specified. If only one value for the cell size is specified (as done here), the same value is used in both directions.

The supernova remnant itself is known to have a diameter of order 9 arcminutes, corresponding to about 216 pixels for the chosen cell size. The mosaic was set up with 7 fields, 1 centered on the remnant and 6 flanking fields; the spacing of the fields was chosen based on the size of the (antenna) primary beam. For our choice of ftmachine='mosaic', you do not have to fit the mosaic inside the inner quarter of the total image in order to prevent image artifacts arising from aliasing, we just want to have a bit of padding around the outside. Although CASA has the feature that its Fourier transform engine (FFTW) does not require a strict power of 2 for the number of linear pixels in a given image axis, it is somewhat more efficient if the number of pixels on a side is a composite number divisible by any pair of 2 and 3 and/or 5. Because clean internally applies a padding of 1.2 (=3x2/5) choose 480, which is 25 x 3 x 5 (so 480 x 1.2 = 576 = 26 x 32). We therefore set imsize=[480,480] and our mosaic fits comfortably inside the image.

Interactive clean at beginning, zoomed in and having set polygon region
Interactive clean near end of cleaning (after 10000 iterations)

As mentioned above, we can guide the clean process by allowing it to find clean components only within a user-specified region. The easiest way to do this is via interactive clean. When clean runs in interactive mode, a viewer window will pop up as shown right. To get a more detailed view of the central regions containing the emission, zoom in by tracing out a rectangle with the left mouse button and double-clicking inside the zoom box you just made. Play with the color scale to bring out the emission better, by holding down the middle mouse button and moving it around. To create a clean box (a region within which components may be found), hold down the right mouse button and trace out a rectangle around the source, then double click inside that rectangle to set it as a box. Note that the CLEAN box must turn white for it to be registered; if the box is not white, it has not been set! Alternatively, you can trace out a more generic shape to better enclose the irregular outline of the supernova remnant. To do that, right-click on the closed polygonal icon. Then trace out a shape by right-clicking where you want the corners of that shape. Once you have come full circle, the shape will be traced out in green, with small squares at the corners. Double-click inside this region and the green outline will turn white. You have now set the clean region. If you have made a mistake with your clean box, click on the "Erase" button, trace out a rectangle around your erroneous region, and double click inside that rectangle. You can also set multiple clean regions. By default, all clean regions will apply only to the plane shown. To change this to select all planes, click the "All Channels" button at the top (we have only a single mfs channel in the image cube), and likewise for Stokes planes using the "All Polarizations" (you will need to set this when we image all the Stokes).

At any stage in the cleaning, you can adjust the number of iterations that clean will do before returning to the GUI. By default this is set to 100 (see the "iterations" window in mid-upper left of panel). You probably want to set this to a high number for this mosaic due to the complicated structure, values from 1000 to 5000 later on seem to work. Note that this will override the niter that was set when you started the clean task. Clean will keep going until it reaches threshold or runs out of cycles (the "cycles" window to the right of the "iterations").

When you are happy with the clean regions, press the green circular arrow button on the far right to continue deconvolution. After completing a cycle, a revised image will come up. As the brightest points are removed from the image ("cleaned" off), fainter emission may show up. You can adjust the clean boxes each cycle, to enclose all real emission. After many cycles, once only noise is left, you can hit the red and white cross icon to stop cleaning. The second figure to the right shows the interactive clean panel late in the process (after cleaning around 10000 iterations). At this point we have used the polygon tool to draw around the emission. There is still signal left in the residual, and it would take forever to clean all that out using our standard clean! You should just quit after doing around 10000-20000 iterations total. We'll improve this in the next part of the tutorial.

viewer display of the Stokes I mosaic of 3C 391

clean will make several output files, all named with the prefix given as imagename. These include:

  • .image - final restored image, with the clean components convolved with a restoring beam and added to the remaining residuals at the end of the imaging process
  • .flux - effective response of the telescope (the primary beam)
  • .flux.pbcoverage - effective response of the full mosaic image
  • .mask - areas where clean has been allowed to search for emission
  • .model - sum of all the clean components, which also has been stored as the MODEL_DATA column in the measurement set
  • .psf - dirty beam, which is being deconvolved from the true sky brightness during the clean process
  • .residual - what is left at the end of the deconvolution process; this is useful to diagnose whether or not to clean more deeply

After the imaging and deconvolution process has finished, you can use the viewer to look at your image. 

# In CASA
viewer('3c391_ctm_spw0_noms_I.image')

This will bring up a viewer window containing the image, which should look similar to that shown at right. You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do).

Note that the image is cut off in a circular fashion at the edges, corresponding to the default minimum primary beam response (minpb)within clean of 0.2.

As we saw during the interactive cleaning, this single-scale delta-function cleaning is having problems cleaning out the large-scale emission. You can see the large "clean bowl" of negative emission surrounding the object, and the heavy pixelization of the emission at low levels. We'll try to improve on this in the next part of the tutorial.

Next Steps

In Part 2 of the tutorial we will carry out the polarization imaging, self-calibration, and polarization analysis and display of the data. Continue with 3C391 Tutorial Part 2: Image Analysis, Polarization, Self-calibration

Last checked on CASA Version 3.4.0.

Retrieved from "https://casaguides.nrao.edu/index.php?title=EVLA_Continuum_Tutorial_3C391-CASA3.4&oldid=13018"