CASA Guides:Polarization Calibration based on CASA pipeline standard reduction: The radio galaxy 3C75-CASA6.2.1: Difference between revisions

From CASA Guides
Jump to navigationJump to search
← Older edit
Akapinsk (talk | contribs)
689 edits
Alawson (talk | contribs)
321 edits
m pi rotation
 
(132 intermediate revisions by 4 users not shown)
Line 5: Line 5:
== Overview ==
== Overview ==
This CASA guide describes the calibration and imaging of a single-pointing continuum data set taken with the Karl G. Jansky Very Large Array (VLA) of the binary black hole system 3C 75 in Abell 400 cluster of galaxies.  
This CASA guide describes the calibration and imaging of a single-pointing continuum data set taken with the Karl G. Jansky Very Large Array (VLA) of the binary black hole system 3C 75 in Abell 400 cluster of galaxies.  
[http://simbad.u-strasbg.fr/simbad/sim-id?Ident=3C75].  The data were taken as a demonstration for the VLA data reduction workshops under project code TDRW0001. To reduce the dataset size, the data was recorded with a single 1 GHz baseband centered at 3.0 GHz, resulting in 8x128 MHz wide spectral windows with 64 channels each. The observation was set up to allow for full polarization calibration. The calibration part of this tutorial can be performed on a laptop type computer with sufficient storage, however the imaging part will require more computing power and memory. This CASA guide was also used as basis for a presentation on polarization calibration at the 7th VLA data reduction workshop: [https://science.nrao.edu/science/meetings/2019/vla-data-reduction/Schinzel_Polarization.pdf]
[http://simbad.u-strasbg.fr/simbad/sim-id?Ident=3C75].  The data were taken as a demonstration for the VLA data reduction workshops under project code TDRW0001. To reduce the dataset size, the data was recorded with a single 1 GHz baseband centered at 3.0 GHz, resulting in 8x128 MHz wide spectral windows with 64 channels each. The observation was set up to allow for full polarization calibration. The calibration part of this tutorial can be performed on a laptop computer with sufficient storage. The imaging part, however, will require more computing power and memory. This CASA guide was also used as basis for a presentation on polarization calibration at the 7th VLA data reduction workshop: [https://science.nrao.edu/science/meetings/2019/vla-data-reduction/Schinzel_Polarization.pdf]


== How to Use This CASA Guide ==
== How to Use This CASA Guide ==


Here are a number of possible ways to run CASA, described in more detail in [[Getting Started in CASA]]. In brief, there are at least three different ways to run CASA:
Here are a number of possible ways to run CASA, described in more detail in [https://casadocs.readthedocs.io/en/v6.2.1/notebooks/usingcasa.html Getting Started in CASA]. In brief, there are at least three different ways to run CASA:


* Interactively examining task inputs. In this mode, one types '''default taskname''' to load the task (this will also set all the task parameters to default values), '''inp''' to examine the inputs, and '''go''' once those inputs have been set to your satisfaction. Allowed inputs are colored blue and bad inputs are colored red. The input parameters themselves are changed one by one, e.g., ''selectdata=True''. Screenshots of the inputs to various tasks used in the data reduction are provided to illustrate which parameters need to be set. More detailed help can be obtained on any task by typing '''help ''taskname'''''. Once a task is run, the set of inputs are stored and can be retrieved via '''tget ''taskname'''''; subsequent runs will overwrite the previous '''tget''' file.
* Interactively examining task inputs. In this mode, one types '''default taskname''' to load the task (this will also set all the task parameters to default values), '''inp''' to examine the inputs, and '''go''' once those inputs have been set to your satisfaction. Allowed inputs are colored blue and bad inputs are colored red. The input parameters themselves are changed one by one, e.g., ''selectdata=True''. Screenshots of the inputs to various tasks used in the data reduction are provided to illustrate which parameters need to be set. More detailed help can be obtained on any task by typing '''help ''taskname'''''. Once a task is run, the set of inputs are stored and can be retrieved via '''tget ''taskname'''''; subsequent runs will overwrite the previous '''tget''' file.
Line 15: Line 15:
* Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set.
* Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set.


* Non-interactively via a script. A series of task function calls can be combined together into a script and run from within CASA via '''execfile('scriptname.py')'''. This and other CASA Tutorial Guides have been designed to be extracted into a script via the script extractor by using the method described at the [[Extracting_scripts_from_these_tutorials]] page. Should you decide to use the script generated by the script extractor for this CASA Guide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is, in fact, unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered).
* Non-interactively via a script. A series of task function calls can be combined together into a script and run from within CASA via '''execfile('scriptname.py')'''. This and other CASA Tutorial Guides have been designed to be extracted into a script via the script extractor by using the method described at the [[Extracting scripts from these tutorials]] page. Should you decide to use the script generated by the script extractor for this CASA Guide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is, in fact, unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered).


If you are a relative novice or just new to CASA, it is strongly recommended to work through this tutorial by cutting and pasting the task function calls provided below after you have read all the associated explanations. Work at your own pace, look at the inputs to the tasks to see what other options exist, and read the help files. Later, when you are more comfortable, you might try to extract the script, modify it for your purposes, and begin to reduce other data.
If you are a relative novice or just new to CASA, it is strongly recommended to work through this tutorial by cutting and pasting the task function calls provided below after you have read all the associated explanations. Work at your own pace, look at the inputs to the tasks to see what other options exist, and read the help files. Later, when you are more comfortable, you might try to extract the script, modify it for your purposes, and begin to reduce other data.
Line 23: Line 23:
If starting from scratch, you can obtain the dataset from the [https://data.nrao.edu/ NRAO archive] and search for the Archive File ID: 'TDRW0001.sb35624494.eb35628826.58395.23719237269'. The uncalibrated visibilities have a size of 12.5 GB. Make sure to select to download the SDM-BDF dataset, if you want to start from the lowest level, because by default a .ms file will be provided by the archive.
If starting from scratch, you can obtain the dataset from the [https://data.nrao.edu/ NRAO archive] and search for the Archive File ID: 'TDRW0001.sb35624494.eb35628826.58395.23719237269'. The uncalibrated visibilities have a size of 12.5 GB. Make sure to select to download the SDM-BDF dataset, if you want to start from the lowest level, because by default a .ms file will be provided by the archive.


For those that want to skip the step of obtaining a continuum Stokes I calibrated measurement set, we have created a starting dataset on which the polarization calibration steps and final imaging can be performed: ['https://casa.nrao.edu/Data/VLA/Polarization/TDRW0001_calibrated.ms.tgz '] (size: 11 GB). It is recommended to use the command line tool '''wget''' to download the calibrated data or directly download through the browser. You will need to untar and unzip the file using the command: 'tar -xzvf TDRW0001_calibrated.ms.tgz'. Then you can skip ahead to the section 'The Observation'.
For those who want to skip the step of obtaining a continuum Stokes I calibrated measurement set, we have created a starting dataset on which the polarization calibration steps and final imaging can be performed: ['https://casa.nrao.edu/Data/VLA/Polarization/TDRW0001_calibrated_CASA6.2.1.ms.tgz '] (size: 10 GB). It is recommended to use the command line tool '''wget''' to download the calibrated data or directly download through the browser. You will need to untar and unzip the file using the command: 'tar -xzvf TDRW0001_calibrated_CASA6.2.1.ms.tgz'. Then you can skip ahead to the section 'The Observation'.


== Pipeline Calibration of Parallel Hands (RR/LL) ==
== Pipeline Calibration of Parallel Hands (RR/LL) ==


If you start with the uncalibrated visibilities obtained from the archive, you will need to first perform a standard continuum calibration of the parallel-hand (RR/LL) cross-correlation visibilities. In this guide we use the standard VLA pipeline that is packaged with the CASA release. You can find more information on the latest release of the VLA pipeline here: [https://science.nrao.edu/facilities/vla/data-processing/pipeline https://science.nrao.edu/facilities/vla/data-processing/pipeline].
If you start with the uncalibrated visibilities obtained from the archive, you will need to first perform a standard continuum calibration of the parallel-hand (RR/LL) cross-correlation visibilities. In this guide we use the standard VLA pipeline that is packaged with the CASA release. You can find more information on the latest release of the VLA pipeline at: [https://science.nrao.edu/facilities/vla/data-processing/pipeline https://science.nrao.edu/facilities/vla/data-processing/pipeline].


In this example, we will not run the pipeline in its standard way but tweak it to force a certain reference antenna. The pipeline typically tries to pick a reference antenna at the center of the array; however this dataset was observed in D array configuration with very short baselines. It was found to be better to use one of the outer antennas for reference, which provides more longer baselines and more stable phase solutions. To set the reference antenna, we specify the ''refantignore'' parameter in some of the pipeline tasks to exclude all but the reference antenna, and use a pipeline execution script ('casa_pipescript.py'). Take the script given below and paste it into a text file inside your working directory that also contains the dataset you downloaded from the NRAO archive and name it casa_pipescript.py.  
In this example, we will not run the pipeline in its standard way but tweak it to force a certain reference antenna. The pipeline typically tries to pick a reference antenna at the center of the array; however, this dataset was observed in D array configuration with very short baselines. It is better to use one of the outer antennas for reference, which provides longer baselines and more stable phase solutions. To set the reference antenna, we specify the ''refantignore'' parameter in some of the pipeline tasks to exclude all but the reference antenna, and use a pipeline execution script ('casa_pipescript.py'). Take the script given below and paste it into a text file inside your working directory that also contains the dataset you downloaded from the NRAO archive and name it casa_pipescript.py.  
    
    
<source lang="python">
<source lang="python">
Line 74: Line 74:
</source>
</source>


Now you can go and get a cup of coffee or lunch. This is going to take a while. On a beefy computer expect about two hours. Once the pipeline has successfully finished you will see some similar messages on the command line prompt.
Now you can go and get a cup of coffee or lunch; this will take a while. On a beefy computer expect about two hours. Once the pipeline has successfully finished you will see some similar messages on the command line prompt.
<pre style="background-color: #E0FFFF;">
<pre style="background-color: #E0FFFF;">
2021-11-25 00:55:08 INFO: Plotting oussid.s19_0._0137+331_3C48__bp.S_band.cont.I.iter0.psf.tt0
2021-11-25 00:55:08 INFO: Plotting oussid.s19_0._0137+331_3C48__bp.S_band.cont.I.iter0.psf.tt0
Line 85: Line 85:
</pre>
</pre>


In order to be able to continue calibration for polarization, i.e. the cross-hand correlations (RL/LR), on pre-calibrated visibilities, we need to perform some additional steps that remove the parallactic angle correction that was applied by the standard pipeline. To do so, start CASA and execute the following commands.
In order to be able to continue calibration for polarization, i.e. the cross-hand correlations (RL/LR), on pre-calibrated visibilities, we need to perform additional steps that remove the parallactic angle correction that was applied by the standard pipeline. To do so, start CASA and execute the following commands.


<source lang="python">
<source lang="python">
Line 125: Line 125:
</source>
</source>


This applies the flagging state before the final {{applycal}} stage of the pipeline, then reapplies the calibration to the corrected column with ''parang=False'', thus disabling the parallactic angle corrections. After that, we rerun target field flagging, and recompute the weights based on the new flags that were applied and split out the corrected column for the target spectral windows. Essentially, we repeated what pipeline tasks hifv_applycals, hifv_targetflag, and hifv_statwt did, but disabling application of parallactic angle corrections. This is the measurement set we will be using in the following to demonstrate polarization calibration.
This applies the flagging state before the final [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal] stage of the pipeline, then reapplies the calibration to the corrected column with ''parang=False'', disabling the parallactic angle corrections. After that, we rerun target field flagging, and recompute the weights based on the new flags that were applied and split out the corrected column for the target spectral windows. Essentially we repeated what pipeline tasks hifv_applycals, hifv_targetflag, and hifv_statwt did, but disabling application of parallactic angle corrections. This is the measurement set we will be using in the following to demonstrate polarization calibration.


== The Observation ==
== The Observation ==
Line 148: Line 148:
</pre>
</pre>


Before beginning our data reduction, we should inspect the pipeline calibration weblog for any obvious issues. You can download the weblog from [https://casa.nrao.edu/Data/VLA/Polarization/pipeline-20210121T202817.tgz https://casa.nrao.edu/Data/VLA/Polarization/pipeline-20210121T202817.tgz] or directly access it at [ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/ ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/].
Before beginning our data reduction, we should inspect the pipeline calibration weblog for any obvious issues. You can download the weblog from ['https://casa.nrao.edu/Data/VLA/Polarization/pipeline-20211207T212848.tgz '] or directly access it at [ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/ ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/].


Inside the weblog, you have access to the [ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/t2-1.html?sidebar=sidebar_TDRW0001_sb35624494_eb35628826_58395_23719237269_ms&subpage=t2-1_details.html overview page] and the [ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/t2-1.html?sidebar=sidebar_TDRW0001_sb35624494_eb35628826_58395_23719237269_ms&subpage=listobs.txt listobs] task output that provide some basic information about the data.   
Inside the weblog you have access to the [ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/t2-1.html?sidebar=sidebar_TDRW0001_sb35624494_eb35628826_58395_23719237269_ms&subpage=t2-1_details.html overview page] and the [ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/t2-1.html?sidebar=sidebar_TDRW0001_sb35624494_eb35628826_58395_23719237269_ms&subpage=listobs.txt listobs task output] that provide some basic information about the data.   


You will note that there are four sources observed. Here the various sources are introduced briefly, with more detail contained in the sections below in which they are used
You will note that there are four sources observed. Here the sources are introduced briefly, with more detail contained in the sections below in which they are used:
* 0137+331=3C48, which will serve as a calibrator for the visibility amplitudes, i.e., it is assumed to have precisely known flux density, the spectral bandpass, and the polarization position angle;
* 0137+331=3C48, which will serve as a calibrator for the visibility amplitudes, i.e., it is assumed to have precisely known flux density, the spectral bandpass, and the polarization position angle;
* J0259+0747, which will serve as a calibrator for the visibility phases and can be used to determine the instrumental polarization;
* J0259+0747, which will serve as a calibrator for the visibility phases and can be used to determine the instrumental polarization;
* J2355+4950, which can serve as a secondary instrumental polarization calibrator or to check residual instrumental polarization; and
* J2355+4950, which can serve as a secondary instrumental polarization calibrator or to check residual instrumental polarization, and;
* 3C75, which is the science target.
* 3C75, which is the science target.


Line 161: Line 161:
<pre style="background-color: #fffacd;">
<pre style="background-color: #fffacd;">
================================================================================
================================================================================
           MeasurementSet Name:  /lustre/aoc/sciops/akapinsk/casaguides/new-try/TDRW0001.sb35624494.eb35628826.58395.23719237269.ms      MS Version 2
           MeasurementSet Name:  /lustre/aoc/sciops/akapinsk/casaguides/TDRW0001.sb35624494.eb35628826.58395.23719237269.ms      MS Version 2
================================================================================
================================================================================
   Observer: Dr. Emmanuel Momjian    Project: uid://evla/pdb/35621723   
   Observer: Dr. Emmanuel Momjian    Project: uid://evla/pdb/35621723   
Line 282: Line 282:
</pre>
</pre>


Note that the antenna IDs (which are numbered sequentially up to the total number of antennas in the array (from 0 to 26 in this instance) do not correspond to the actual antenna names (ea01 to ea28), instead these numbers correspond to those painted on the antennas themselves. The antennas can be referenced using either convention; ''antenna='22' '' would correspond to ea23, whereas ''antenna='ea22' '' would correspond to ea22.  Note that the antenna numbers in the observer log correspond to the actual antenna names, i.e., the 'ea??' numbers given in {{listobs}}.
Note that the antenna IDs, which are numbered sequentially up to the total number of antennas in the array (from 0 to 26 in this instance), do not correspond to the actual antenna names (ea01 to ea28). Instead, these numbers correspond to those painted on the antennas themselves. The antennas can be referenced using either convention; ''antenna='22' '' would correspond to ea23, whereas ''antenna='ea22' '' would correspond to ea22.  Note that the antenna numbers in the observer log correspond to the actual antenna names, i.e., the 'ea??' numbers given in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.information.listobs.html listobs].


Both to get a sense of the array, as well as identify the location of the antenna that was picked by the pipeline for parallel hand calibration, have a look at the [ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/t2-1.html?sidebar=sidebar_TDRW0001_sb35624494_eb35628826_58395_23719237269_ms&subpage=t2-2-3.html antenna setup page]. Generally, for calibration purposes, you would prefer to select an antenna that is close to the center of the array (and that is not listed in the operator's log as having had problems!). As noted above, in a compact configuration there is a benefit to choose an outer antenna to increase the bias toward longer baselines.
Both to get a sense of the array, as well as identify the location of the antenna that was picked by the pipeline for parallel hand calibration, have a look at the [ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/t2-1.html?sidebar=sidebar_TDRW0001_sb35624494_eb35628826_58395_23719237269_ms&subpage=t2-2-3.html antenna setup page]. For calibration purposes, you would generally select an antenna that is close to the center of the array (and that is not listed in the operator's log as having had problems!). As noted above, in a compact configuration there is a benefit to choosing an outer antenna to increase the bias toward longer baselines.


At this point it is also a good idea to check the quality of the pipeline calibration. Go to the [ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/t1-4.html task overview page] and pay particular attention to [ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/t2-4m.html?sidebar=sidebar_stage14&ms=all&subpage=t2-4m_details.html hifv_finalcals] and [ftp://ftp.aoc.nrao.edu/staff/fschinze/casaguide_pol_6.1.2/pipeline-20210121T202817/html/t2-4m.html?sidebar=sidebar_stage18&ms=all&subpage=t2-4m_details.html hifv_plotsummary]. Try to see if you can recognize which reference antenna was picked. For more details on the pipeline output you can have a look at the [http://casaguides.nrao.edu/index.php/VLA_CASA_Pipeline VLA CASA Pipeline Guide]. Going forward we assume that the pipeline calibration is good and we can use it as a starting point for further calibration steps focusing on polarization calibration and imaging.
At this point it is also a good idea to check the quality of the pipeline calibration. Go to the [ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/t1-4.html task overview page] and pay particular attention to [ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/t2-4m.html?sidebar=sidebar_stage14&ms=all&subpage=t2-4m_details.html hifv_finalcals] and [ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/t2-4m.html?sidebar=sidebar_stage18&ms=all&subpage=t2-4m_details.html hifv_plotsummary]. Try to see if you can recognize which reference antenna was picked. For more details on the pipeline output you can have a look at the [https://science.nrao.edu/facilities/vla/data-processing/pipeline VLA CASA Pipeline Guide]. We assume that the pipeline calibration is good and can use it as a starting point for further calibration steps focusing on polarization calibration and imaging.


== Examining and Editing the Data ==
== Examining and Editing the Data ==


At this point we must start CASA.  If you have not used CASA before, some helpful tips are available on the [[Getting Started in CASA]] page.
At this point we must start CASA.  If you have not used CASA before, some helpful tips are available on the [https://casadocs.readthedocs.io/en/v6.2.1/notebooks/usingcasa.html Getting Started in CASA] page.


It is always a good idea to examine the data before jumping straight into calibration. From the observer's log there were no major issues noted, besides a potentially warm receiver on antenna ea05. Even though the pipeline did a good job of calibrating and flagging the data, it isn't perfect. From the pipeline weblog, looking at the final amplitude gain calibration vs time plots in hifv_finalcals, we can see that during the second half of the observation antennas ea03, ea12, and ea16 shows some gain instability; otherwise there are no issues identified at this point.   
It is always a good idea to examine the data before jumping straight into calibration. From the observer's log there were no major issues noted besides a potentially warm receiver on antenna ea05. Even though the pipeline did a good job of calibrating and flagging the data, it isn't perfect. From the pipeline weblog, looking at the final amplitude gain calibration vs time plots in hifv_finalcals, we can see that during the second half of the observation antennas ea03, ea12, and ea16 shows some gain instability; otherwise there are no issues identified at this point.   


Start by inspecting these three particular antennas using the CASA task {{plotms}}, plot frequency against amplitude and frequency against time for the parallel hands, iterate over field or scan, and note if you find something at odds.   
Start by inspecting these three particular antennas using the CASA task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms], plot frequency against amplitude and frequency against time for the parallel hands, iterate over field or scan, and note if you find something at odds.   


<source lang="python">
<source lang="python">
Line 306: Line 306:
* ''avgchannel='64' '': With this plot, we are mainly interested in the fields vs time. Averaging over all 64 channels in the spectral window makes the plotting faster.
* ''avgchannel='64' '': With this plot, we are mainly interested in the fields vs time. Averaging over all 64 channels in the spectral window makes the plotting faster.
* ''coloraxis='field' '': Color-code the plotting symbols by field name/number.
* ''coloraxis='field' '': Color-code the plotting symbols by field name/number.
The default x- and y-axis parameters are 'time' and 'amp', so the above call to {{plotms}} produces an amplitude vs time plot of the data for a selected subset of the data (if desired) and with data averaging (if desired). Many other values have been left to defaults, but it is possible to select them from within the plotms GUI.   
The default x- and y-axis parameters are 'time' and 'amp', so the above call to [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] produces an amplitude vs time plot of the data for a selected subset of the data (if desired) and with data averaging (if desired). Many other values have been left to defaults, but it is possible to select them from within the plotms GUI.   


Task {{plotms}} allows one to select and view the data in many ways. Figure 1 shows the result of running plotms with the field selection discussed above. You can quickly see that the first source observed, 3C48 (the primary flux density and polarization angle calibrator source), is the brightest source in this observation. The next brightest is the second source observed, J2355+4950, a compact symmetric object (CSO; radio galaxy) and the secondary instrumental polarization calibrator. The complex gain calibrator J0259+0747 (shown in orange) is around 1 Jy. The target scans on 3C75 are colored in green. The spread of amplitudes is primarily due to the presence of extended structure, thus every baseline sees a slightly different amplitude.  
Task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] allows one to select and view the data in many ways. Figure 1 shows the result of running plotms with the field selection discussed above. You can quickly see that the first source observed, 3C48 (the primary flux density, bandpass, and polarization angle calibrator source), is the brightest source in this observation. The next brightest is the second source observed, J2355+4950, a compact symmetric object (CSO; radio galaxy) and the secondary instrumental polarization calibrator. The complex gain calibrator J0259+0747 (shown in orange) is around 1 Jy. The target scans on 3C75 are colored in green. The spread of amplitudes is primarily due to the presence of extended structure, thus every baseline sees a slightly different amplitude.  


Across the top of the left panel of the GUI are a set of tabs labelled Plot, Flag, Tools, Annotate, and Options. By default, the Plot tab is visible. There are a number of tabs running down the side of the left hand panel: Data, Calibration, Axes, Page, Transform, Display, and Canvas; these allow you to make changes to the plotting selection without having to re-launch {{plotms}}. Even if it was started with ''xaxis=' ' '' (defaulting to 'time'), you can choose a different X-axis by selecting the Axes tab, then using the dropdown menu to switch (for example) to ''xaxis='Frequency' '' (to get something sensible when plotting with frequency, channel averaging must be turned off).
Across the top of the left panel of the GUI are a set of tabs labelled Plot, Flag, Tools, Annotate, and Options. By default, the Plot tab is visible. There are a number of tabs running down the side of the left hand panel: Data, Calibration, Axes, Page, Transform, Display, and Canvas; these allow you to make changes to the plotting selection without having to re-launch [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms]. Even if it was started with ''xaxis=' ' '' (defaulting to 'time'), you can choose a different X-axis by selecting the Axes tab, then using the dropdown menu to switch (for example) to ''xaxis='Frequency' '' (to get something sensible when plotting with frequency, channel averaging must be turned off).


You should spend several minutes displaying the data in various formats. You can save the version of the {{plotms}} plot as a graphics file by using the menu bar in the {{plotms}} GUI to select the ''Export...'' option under the Export menu.
You should spend several minutes displaying the data in various formats. You can save the version of the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] plot as a graphics file by using the menu bar in the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] GUI to select the ''Export...'' option under the Export menu.


Another example of using {{plotms}} for a quick look at your data, select the Data tab and specify ''field 2'' (the complex gain calibrator J0259+0747) to display data associated with the target, then select the Axes tab and change the X-axis to be ''UVdist'' (baseline length in meters). Remove the channel averaging (Data tab), and plot the data using the ''Plot'' button at the bottom of the {{plotms}} GUI. The important observation is that the amplitude distribution is relatively constant as a function of UV distance or baseline length (i.e., <math>\sqrt{u^2+v^2}</math> ) (see Figure 2A). A relatively constant visibility amplitude as a function of baseline length means that the source is very nearly a point source. (The Fourier transform of a point source, i.e. a delta function, is a constant function.) You can see occasional spikes in the calibrated amplitudes. This is most likely caused by radio frequency interference that correlates on certain baselines. We will get to those further in the guide.  
Another example of using [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] for a quick look at your data, select the Data tab and specify ''field 2'' (the complex gain calibrator J0259+0747) to display data associated with the target, then select the Axes tab and change the X-axis to be ''UVdist'' (baseline length in meters). Remove the channel averaging (Data tab), and plot the data using the ''Plot'' button at the bottom of the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] GUI. The important observation is that the amplitude distribution is relatively constant as a function of UV distance or baseline length (i.e., <math>\sqrt{u^2+v^2}</math>; see Figure 2A). A relatively constant visibility amplitude as a function of baseline length means that the source is very nearly a point source (the Fourier transform of a point source, i.e. a delta function, is a constant function). You can see occasional spikes in the calibrated amplitudes. This is most likely caused by radio frequency interference that correlates on certain baselines. We will get to those further in the guide.  


By contrast, if you make a similar plot for ''field 3'' (our target 3C 75), the result is a visibility function that falls rapidly with increasing baseline length. Figure 2B shows this example, including time averaging of '1e6' seconds (any large number that encompasses more than a full scan will do). Such a visibility function indicates a highly resolved source. The baseline length at which the visibility function falls to some fiducial value (e.g., 1/2 of its peak value) gives a rough estimate of the angular scale of the source (Angular scale [in radians] ~ 1/baseline [in wavelengths]). To plot baseline length in wavelengths rather than meters, select ''UVwave'' as the X-axis parameter.
By contrast, if you make a similar plot for ''field 3'' (our target 3C 75), the result is a visibility function that falls rapidly with increasing baseline length. Figure 2B shows this example, including time averaging of '1e6' seconds (any large number that encompasses more than a full scan will do, we want to fully average each scan). Such a visibility function indicates a highly resolved source. The baseline length at which the visibility function falls to some fiducial value (e.g., 1/2 of its peak value) gives a rough estimate of the angular scale of the source (Angular scale [in radians] ~ 1/baseline [in wavelengths]). To plot baseline length in wavelengths rather than meters, select ''UVwave'' as the X-axis parameter.


A final example is shown in Figure 2C. In this example, we have elected to show phase as a function of (frequency) channel for a single baseline (''antenna='ea01&ea21' '') on the bandpass calibrator, field 0. If you choose to iterate by baseline (e.g., ''antenna='ea01' '' and ''iteraxis='baseline' ''), you can see similar phase-frequency variations on all baselines. They center around zero phase, since we are looking at the calibrated visibilities, you are seeing, however, a butterfly shaped pattern with phase noise higher toward the channel edges. This pattern is due to a small mismatch in the delay measurement timing (also known as 'delay clunking') which is an internally generated effect and is typically averaged out over time.   
A final example is shown in Figure 2C. In this example, we have elected to show phase as a function of (frequency) channel for a single baseline (''antenna='ea01&ea21' '') on the bandpass calibrator, field 0, and non-averaged data. If you choose to iterate by baseline (e.g., ''antenna='ea01' '' and ''iteraxis='baseline' ''), you can see similar phase-frequency variations on all baselines. They center around zero phase, because we are looking at the calibrated visibilities; however, you are seeing a butterfly shaped pattern with phase noise higher toward the channel edges. This pattern is due to a small mismatch in the delay measurement timing (also known as 'delay clunking') which is an internally generated effect and is typically averaged out over time.   
{|
{|
| [[File:plotms-J0259+0747-Amp vs UVdist 6.1.2.jpeg|200px|left|thumb|Figure 2A: plotms view of amp vs. uvdist of J0259+0747, a point source]]
| [[File:Plotms-J0259+0747-Amp-vs-UVdist-CASA6.2.1.jpeg|200px|left|thumb|Figure 2A: plotms view of amp vs. uvdist of J0259+0747, a point source]]
| [[File:Plotms-3C75-Amp vs UVwave 6.1.2.jpeg|200px|center|thumb|Figure 2B: plotms view of amp vs. uvwave of 3C 75, a resolved source]]
| [[File:Plotms-3C75-Amp-vs-UVwave-CASA6.2.1.jpeg|200px|center|thumb|Figure 2B: plotms view of amp vs. uvwave of 3C 75, a resolved source]]
| [[File:Delays_ea01ea21_CASA6.1.2.jpeg|200px|right|thumb|Figure 2C: plotms view of phase vs. channel on one baselines, showing phase delay across the calibrated bandpass]]
| [[File:Plotms-3C48-delays-ea01ea21_CASA6.2.1.jpeg|200px|right|thumb|Figure 2C: plotms view of phase vs. channel on one baselines, showing phase delay across the calibrated bandpass]]
|}
|}


You can find similar plots in the CASA pipeline weblog under the task hifv_plotsummary. At this stage the pipeline has taken care of most of the calibration; there might be some remaining issues, though, that were not caught by the pipeline.  
You can find similar plots in the CASA pipeline weblog under the task hifv_plotsummary. At this stage the pipeline has taken care of most of the calibration. There might be some remaining issues, though, that were not caught by the pipeline.  


[[Image:plotms_3C75-datastream_CASA6.1.2.jpeg|200px|right|thumb|Figure 3: datastream view of MS]]
[[Image:Plotms-3C75guide-datastream-CASA6.2.1.jpeg|200px|right|thumb|Figure 3: datastream view of MS]]
One final useful plot we will make is a datastream plot of the antenna2 in a baseline for the data versus ea01. This shows, assuming that ea01 is in the entire observation, when various antennas drop out (see Figure 3).
One final useful plot we will make is a datastream plot of the antenna2 in a baseline for the data versus ea01. This shows, assuming that ea01 is in the entire observation, when various antennas drop out (see Figure 3).


Line 338: Line 338:
</source>
</source>


From this display you can immediately see that flagging performed by the pipeline is present. In the following we note on a couple issues that you might have found and will take care of those through additional flagging.
From this display you can immediately see that flagging performed by the pipeline is present.  
 
In the following we note on a couple issues that you might have found while inspecting data in this section. We will take care of those through additional flagging.


<pre style="background-color: #fffacd;">
<pre style="background-color: #fffacd;">
Issues that you might find:
Issues that you might find:
  - ea12, scan 17: amplitude spike at the end of the scan.
  - ea12, scan 17: amplitude spike at the end of the scan (can be spotted already in Figure 1)
  - Residual RFI
  - Residual RFI (see Figure 2A)
</pre>
</pre>


We can flag this time period, by invoking the casa task {{flagdata}}. It is also a good idea to save the original flags before performing any flagging by setting '''flagbackup=True'''.  
In the case of the amplitude spike, we can flag the affected time period by invoking the casa task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.flagging.flagdata.html flagdata]. It is a good idea to save the original flags before performing any flagging by setting '''flagbackup=True'''.  


<source lang="python">
<source lang="python">
Line 353: Line 355:
</source>
</source>


You can check the effect of this flagging by replotting Figure 2A. The spikes we saw before on some baselines should have disappeared. If you plot frequency against amplitude without averaging, however, you will still see some channels with interference that we will need to flag, especially on the instrumental polarization calibrators. Polarization calibration is very sensitive to interference, especially in the cross-hand correlations RL,LR. The pipeline does not (yet) do a good job at this, therefore we will need to cover some additional flagging steps in the next section.  
You can check the effect of this flagging by replotting Figure 2A. The spikes we saw before on some baselines should have disappeared. If you plot frequency against amplitude without averaging, however, you will still see some channels with interference that we will need to flag, especially on the instrumental polarization calibrators. Polarization calibration is very sensitive to interference, especially in the cross-hand correlations RL,LR. The pipeline does a good job at this, but there are still some RFI left; we will perform some additional flagging steps in the next section.  


=== Additional Flagging ===
=== Additional Flagging ===


At first, we try to get a good sense of additional flagging that might be needed by plotting frequency against amplitude for the RR,LL and RL,LR polarizations of our calibrators, fields 0 through 2. In particular we need to pay attention to RL,LR (see Figure 4A). We will perform additional flagging on the target field at a later stage.  
First we try to get a good sense of additional flagging that might be needed by plotting frequency against amplitude for the RR,LL and RL,LR polarizations of our calibrators (fields 0 through 2). You will notice some left over RFI on the bandpass calibrator in RR, LL. However, we also need to pay particular attention to RL, LR (see Figure 4A). Here we consider calibrators only; we will perform additional flagging on the target field at a later stage.  


<source lang="python">
<source lang="python">
Line 368: Line 370:


{|
{|
| [[File:Plotms-preflag-Amp vs Freq 6.1.2.jpeg|200px|left|thumb|Figure 4a: plotms view of calibrators freq vs. amp RL/LR before additional flagging]]
| [[File:Plotms-preflag-Amp-vs-Freq-CASA6.2.1.jpeg|200px|left|thumb|Figure 4a: plotms() view of calibrators' amplitudes (RL,LR) as a function of frequency before additional flagging]]
| [[File:Plotms-rflag-Amp vs Freq 6.1.2.jpeg|200px|right|thumb|Figure 4b: plotms view of calibrators freq vs. amp RL/LR after rflag]]
| [[File:Plotms-rflag-Amp-vs-Freq-CASA6.2.1.jpeg|200px|right|thumb|Figure 4b: plotms() view of calibrators' amplitudes (RL,LR) as a function of frequency after executing addition run of rflag]]
|}
|}


Since we are dealing with point sources, we do not have to worry about overflagging of shorter baselines, so we can run {{flagdata}} with ''mode='rflag' ''over the calibrator fields and cross-hand correlations to remove any residual RFI. For completeness, we also use ''mode='tfcrop' ''to reduce the amount of residual RFI in the parallel hands. This is not strictly needed at this point, since the polarization calibration is based on the cross-hand correlations.
Since we are dealing with point sources, we do not have to worry about overflagging of shorter baselines, so we can run [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.flagging.flagdata.html flagdata] with ''mode='rflag' ''over the calibrator fields and cross-hand correlations to remove any residual RFI. For completeness, we also use ''mode='tfcrop' ''to reduce the amount of residual RFI in the parallel hands. This is not strictly needed at this point, since the polarization calibration is based on the cross-hand correlations.


<source lang="python">
<source lang="python">
Line 400: Line 402:
== Polarization Calibration ==
== Polarization Calibration ==


<pre style="background-color: #fffacd;">
Polarization calibration is done in three steps:
Polarization calibration is done in three steps:
* First, we determine the instrumental delay between the two polarization outputs;
* First, we determine the instrumental delay between the two polarization outputs;
* Second, we solve for the instrumental polarization (the frequency-dependent leakage terms ('D-terms')), using either an unpolarized source or a source which has sufficiently good parallactic angle coverage;
 
* Second, we solve for the instrumental polarization (the frequency-dependent leakage terms, 'D-terms'), using either an unpolarized source or a source which has sufficiently good parallactic angle coverage;
 
* Third, we solve for the polarization position angle using a source with a known polarization position angle (we use 3C48 here).  
* Third, we solve for the polarization position angle using a source with a known polarization position angle (we use 3C48 here).  
</pre>
For information on polarization calibrators suitable for VLA observations, see the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol VLA Observing Guide on Polarimetry]. The [https://casadocs.readthedocs.io/en/v6.2.1/notebooks/synthesis_calibration.html#Polarization-Calibration CASA related documentation] also provides helpful information on polarization calibration steps and the different options that are available.


For information on polarization calibrators suitable for VLA observations, see the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol VLA Observing Guide on Polarimetry]. The [https://casa.nrao.edu/casadocs/casa-5.4.1/synthesis-calibration/instrumental-polarization-calibration CASA related documentation] also provides helpful information on polarization calibration steps and the different options that are available.  
Before solving for the calibration solutions, we first use [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy] to set the polarization model for our polarized position-angle calibrator. The pipeline only set the total intensity of the flux density calibrator source 3C48, which did not include any polarization information. This source is known to have a fairly stable linear fractional polarization (measured to be 2% in S-band around the time of the observations), a polarization position angle of -100 degrees at 3 GHz, and a rotation measure of -68 rad/m^2. Note that 3C48 had an outburst in 2017 and is expected to show a significant degree of variability at higher frequencies in the first instance, progressively affecting lower frequencies as time passes since the event. Since we have applied the pipeline calibration and not corrected for parallactic angle, we can continue polarization calibration using a split measurement set.  


Before solving for the calibration solutions, we first use {{setjy}} to set the polarization model for our polarized position-angle calibrator. The pipeline only set the total intensity of the flux density calibrator source 3C48, which did not include any polarization information. This source is known to have a fairly stable linear fractional polarization (measured to be 2% in S-band around the time of the observations), a polarization position angle of -100 degrees at 3 GHz, and a rotation measure of -68 rad/m^2. Note at higher frequencies, 3C48 has had an outburst in 2017 and thus is expected to show a significant degree of variability. Since we have applied the pipeline calibration and not corrected for parallactic angle, we can continue polarization calibration using a split measurement set.  
The [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy] task will calculate the values of Stokes Q and U (in the reference channel) for user inputs of the reference frequency, Stokes I, polarization fraction, polarization angle, and rotation measure. The setjy input parameters can be obtained from Perley & Butler (2017) for Stokes I information and Perley & Butler (2013) for polarization information. Other sources can also be consulted, such as archival observations of variable polarization calibrators available under the project codes TPOL0003 or TCAL0009.


The [https://casa.nrao.edu/casadocs/latest/global-task-list/task_setjy/about setjy] task will calculate the values of Stokes Q and U (in the reference channel) for user inputs of the reference frequency, Stokes I, polarization fraction, polarization angle, and rotation measure. The setjy input parameters can be obtained from Perley & Butler (2017) for Stokes I information and Perley & Butler (2013) for polarization information. Other sources can also be consulted, such as archival observations of variable polarization calibrators available under the project code TPOL0003 or TCAL0009. It is possible to capture a frequency variation in Q, U, and alpha terms by providing coefficients of polynomial expansion for polarization fraction, polarization angle, and spectral index as a function of frequency. At this time, it is left to the user to derive these coefficients, which can be accomplished by fitting a polynomial to observed values of the polarization fraction (here also called polarization index), polarization angle, and flux density (for the case of spectral index). Updated values of the broad band polarimetric information for the four calibration sources 3C48, 3C138, 3C147, and 3C286 (Of these sources, 3C48, 3C138, and 3C147 have been noticed to be variable) can be found at (https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/fdscale) and at (https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol). These coefficients are then passed to the setjy task as lists along with the reference frequency and the Stokes I flux density.
It is possible to capture a frequency variation in Q, U, and alpha terms by providing coefficients of polynomial expansion for polarization fraction, polarization angle, and spectral index as a function of frequency. At this time, it is left to the user to derive these coefficients, which can be accomplished by fitting a polynomial to observed values of the polarization fraction (here also called polarization index), polarization angle, and flux density (for the case of spectral index). Updated values of the broad band polarimetric information for the four calibration sources 3C48, 3C138, 3C147, and 3C286 can be found at (https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/fdscale) and at (https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol); of these sources, 3C48, 3C138, and 3C147 have been found to be variable. These coefficients are then passed to the setjy task as lists along with the reference frequency and the Stokes I flux density.


The calibrator used for this guide, 3C48, has a rotation measure and thus changes its Q and U with frequency. Therefore, for our purposes, it is not sufficient to use only the first Taylor term of the expansion. For deriving the setjy input parameters you can consult the [https://casa.nrao.edu/casadocs/latest/global-task-list/task_setjy/about setjy CASA documentation]. Currently setjy only supports unresolved polarized emission models assuming that the Stokes I,Q,U peak are co-located on the sky. This is not necessarily the case for more complicated objects or even for 3C48 in extended VLA configurations.
The calibrator used for this guide, 3C48, has a rotation measure and thus changes its Q and U with frequency. Therefore, for our purposes, it is not sufficient to use only the first Taylor term of the expansion. For deriving the setjy input parameters you can consult the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy CASA documentation]. Currently setjy only supports unresolved polarized emission models assuming that the Stokes I,Q,U peak are co-located on the sky. This is not necessarily the case for more complicated objects or even for 3C48 in extended VLA configurations.


As an example on how to derive the polarization parameters for the {{setjy}} call, you can perform the following next steps or jump right to the {{setjy}} call below.  
As an example on how to derive the polarization parameters for the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy] call, you can perform the following next steps or jump right to the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy] call below.  


=== Deriving the Polarization Properties of the Polarization Angle Calibrator ===
=== Deriving the Polarization Properties of the Polarization Angle Calibrator ===


At first we tabulate the frequency dependent Stokes I flux density, polarization fraction, and polarization angle in a textfile, we call 3C48.dat. The data is taken from [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol] and the corresponding Stokes I value is calculated from the Perley & Butler (2013) scale.  
First, we tabulate the frequency dependent Stokes I flux density, polarization fraction, and polarization angle in a textfile, which we will call 3C48.dat. The data is taken from [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/flux-density-scale-polarization-leakage-polarization-angle-tables] and the corresponding Stokes I value is calculated from the Perley & Butler (2017) scale.
 
Note: Due to a small change in the following python code users may notice small differences in results as compared to this guide. The principle of the method shown is still accurate and the results will be updated in the near future to fit the new code.  




<pre>
<pre>
# Frequency I    P.F.   P.A.
# 3C48
# (GHz)     (Jy)         (rad)
# Date 31Jan/01Feb 2019 polarization properties
1.05     20.38 0.003 0.43633
# Reference flux densities were used
1.45     15.80 0.005  2.44346
# Frequency       I    P.F.   P.A.
1.64     14.30 0.007 -0.08727
# (GHz)         (Jy)         (rad)
1.95     12.34 0.009 -2.61799
1.022 20.68 0.00293 0.07445
2.45     10.12 0.014 -2.09440
1.465 15.62 0.00457 -0.60282
2.95     8.58 0.020 -1.74533
1.865 12.88 0.00897 0.39760
3.25     7.86 0.025 -1.60570
2.565 9.82 0.01548 -1.97046
3.75     6.89 0.032 -1.46608
3.565 7.31 0.02911 -1.46542
4.50     5.81 0.038 -1.30900
4.885 5.48 0.04286 -1.24875
5.00     5.26 0.042 -1.25664
6.680 4.12 0.05356 -1.15533
6.50     4.09 0.052 -1.18682
8.435 3.34 0.05430 -1.10638
7.25     3.68 0.052 -1.16937
11.320 2.56 0.05727 -1.08602
8.10     3.30 0.053 -1.11701
14.065 2.14 0.06097 -1.09597
8.80     3.04 0.054 -1.08210
16.564 1.86 0.06296 -1.11891
12.8     2.10 0.060 -1.08210
19.064 1.67 0.06492 -1.18266
13.7     1.96 0.061 -1.08210
25.564 1.33 0.07153 -1.25369
14.6     1.84 0.064 -1.09956
32.064 1.11 0.06442 -1.32430
15.5     1.73 0.064 -1.11701
37.064 1.00 0.06686 -1.33697
18.1     1.48 0.069 -1.15192
42.064 0.92 0.05552 -1.46381
19.0     1.41 0.071 -1.16937
48.064 0.82 0.06773 -1.46412
22.4     1.20 0.077 -1.22173
23.3        1.15 0.078 -1.22173
36.5        0.74 0.074 -1.34390
43.5     0.62 0.075 -1.48353
</pre>
</pre>


Now to fit Stokes I, we execute in CASA the following commands. These could also be put into a textfile and run from inside the CASA prompt using {{execfile}}.
Now to fit Stokes I, we execute in CASA the following commands. These could also be put into a textfile and run from inside the CASA prompt using [https://casadocs.readthedocs.io/en/v6.2.1/api/casashell/execfile.html?highlight=execfile#casashell.execfile execfile].


<source lang="python">
<source lang="python">
Line 463: Line 470:
         return S*(f/3.0)**(alpha+beta*np.log10(f/3.0))
         return S*(f/3.0)**(alpha+beta*np.log10(f/3.0))


# Fit 1 -5 GHz data points
# Fit 1 - 5 GHz data points
popt, pcov = curve_fit(S, data[3:9,0], data[3:9,1])
popt, pcov = curve_fit(S, data[0:10,0], data[0:10,1])
print('I@3GHz', popt[0], ' Jy')
print('I@3GHz', popt[0], ' Jy')
print('alpha', popt[1])
print('alpha', popt[1])
Line 471: Line 478:
print(pcov)
print(pcov)


plt.plot(data[3:9,0], data[3:9,1], 'ro', label='data')
plt.plot(data[0:10,0], data[0:10,1], 'ro', label='data')
plt.plot(np.arange(1,5,0.1), S(np.arange(1,5,0.1), *popt), 'r-', label='fit')
plt.plot(np.arange(1,5,0.1), S(np.arange(1,5,0.1), *popt), 'r-', label='fit')


Line 485: Line 492:


<pre>
<pre>
I@3GHz 8.450106884747997 Jy
I@3GHz 8.461741965511111 Jy
alpha -0.9020768461856092
alpha -0.9042738492254004
beta -0.1236646276382762
beta -0.14600432138647343
Covariance
Covariance
[[ 6.70740271e-07 -2.01772800e-08 -1.22965356e-06]
[[ 3.18582527e-05  2.98095625e-06 -3.05312358e-06]
  [-2.01772800e-08 8.14211422e-08 3.31731932e-07]
  [ 2.98095625e-06 3.65971413e-06 7.73132558e-06]
  [-1.22965356e-06  3.31731932e-07 4.96444153e-06]]
  [-3.05312358e-06  7.73132558e-06 2.12210577e-05]]
</pre>
</pre>


Line 509: Line 516:
         return a+b*((f-3.0)/3.0)+c*((f-3.0)/3.0)**2+d*((f-3.0)/3.0)**3
         return a+b*((f-3.0)/3.0)+c*((f-3.0)/3.0)**2+d*((f-3.0)/3.0)**3


# Fit 1 -5 GHz data points
# Fit 1 - 5 GHz data points
popt, pcov = curve_fit(PF, data[3:9,0], data[3:9,2])
popt, pcov = curve_fit(PF, data[0:10,0], data[0:10,2])
print("Polfrac Polynomial: ", popt)
print("Polfrac Polynomial: ", popt)
print("Covariance")
print("Covariance")
print(pcov)
print(pcov)


plt.plot(data[3:9,0], data[3:9,2], 'ro', label='data')
plt.plot(data[0:10,0], data[0:10,2], 'ro', label='data')
plt.plot(np.arange(1,5,0.1), PF(np.arange(1,5,0.1), *popt), 'r-', label='fit')
plt.plot(np.arange(1,5,0.1), PF(np.arange(1,5,0.1), *popt), 'r-', label='fit')


Line 526: Line 533:


<pre>
<pre>
Polfrac Polynomial:  [ 0.02117795 0.04449939 0.00789694 -0.05895564]
Polfrac Polynomial:  [ 0.02142905 0.03918256 0.00234878 -0.02301251]
Covariance
Covariance
[[ 6.64660897e-08 -1.13146417e-07 -6.17380425e-07  1.11278900e-06]
[[ 1.21641658e-07  4.90635228e-08 -3.78036036e-07 -9.06935233e-08]
  [-1.13146417e-07 2.83125093e-06  3.07885385e-06 -1.72341073e-05]
  [ 4.90635228e-08 2.17869783e-06  1.50406983e-07 -5.93646707e-06]
  [-6.17380425e-07  3.07885385e-06 1.14521652e-05 -2.83997719e-05]
  [-3.78036036e-07  1.50406983e-07 2.10655624e-06 -4.88196071e-07]
  [ 1.11278900e-06 -1.72341073e-05 -2.83997719e-05 1.27165376e-04]]
  [-9.06935233e-08 -5.93646707e-06 -4.88196071e-07 1.86290160e-05]]
</pre>
</pre>


Line 542: Line 549:


def PA(f,a,b,c,d,e):
def PA(f,a,b,c,d,e):
         return a+b*((f-3.0)/3.0)+c*((f-3.0)/3.0)**2+d*((f-3.0)/3.0)**3+e**((f-3.0)/3.0)**4
         return a+b*((f-3.0)/3.0)+c*((f-3.0)/3.0)**2+d*((f-3.0)/3.0)**3+e*((f-3.0)/3.0)**4


# Fit 2 - 9 GHz data points
# Fit 2 - 9 GHz data points
popt, pcov = curve_fit(PA, data[5:11,0], data[5:11,3])
popt, pcov = curve_fit(PA, data[3:14,0], data[3:14,3])
print("Polangle Polynomial: ", popt)
print("Polangle Polynomial: ", popt)
print("Covariance")
print("Covariance")
print(pcov)
print(pcov)


plt.plot(data[3:11,0], data[3:11,3], 'ro', label='data')
plt.plot(data[3:14,0], data[3:14,3], 'ro', label='data')
plt.plot(np.arange(1,9,0.1), PA(np.arange(1,9,0.1), *popt), 'r-', label='fit')
plt.plot(np.arange(1,9,0.1), PA(np.arange(1,9,0.1), *popt), 'r-', label='fit')


Line 561: Line 568:


<pre>
<pre>
Polangle Polynomial:  [-2.71799143 1.33161446 -1.35326296 0.78461457 0.74089363]
Polangle Polynomial:  [-2.77397103 1.64922494 -1.51121195 0.47800055 0.97812597]
Covariance
Covariance
[[ 1.26427269e-04 -9.82170495e-04 1.59237062e-03 1.10065730e-03
[[ 0.00064289 -0.00019419 -0.00213386 0.0033356 -0.00153425]
  -2.01839906e-03]
  [-0.00019419 0.00999406 -0.00266842 -0.02268727  0.01468271]
  [-9.82170495e-04 2.01633469e-02 -3.70941069e-02 -5.34283393e-02
  [-0.00213386 -0.00266842 0.03013314 -0.05191129  0.02532522]
  7.86417890e-02]
  [ 0.0033356  -0.02268727 -0.05191129 0.18538412 -0.10450983]
  [ 1.59237062e-03 -3.70941069e-02 8.82904992e-02 -2.36326981e-02
  [-0.00153425 0.01468271  0.02532522 -0.10450983  0.06000995]]
  -2.87649289e-02]
  [ 1.10065730e-03 -5.34283393e-02 -2.36326981e-02 1.11093338e+00
  -1.16398860e+00]
  [-2.01839906e-03 7.86417890e-02 -2.87649289e-02 -1.16398860e+00
  1.25277633e+00]]
</pre>
</pre>


Line 583: Line 585:
reffreq = '3.0GHz'
reffreq = '3.0GHz'
# Stokes I flux density
# Stokes I flux density
I =        8.450107
I =        8.461742
# Spectral Index
# Spectral Index
alpha =    [-0.902, -0.1237]
alpha =    [-0.904, -0.1460]
# Polarization Fraction
# Polarization Fraction
polfrac = [ 0.02117795,  0.04449939,  0.00789694, -0.05895564]
polfrac = [0.02142905,  0.03918256,  0.00234878, -0.02301251]
# Polarization Angle
# Polarization Angle
polangle = [-2.71799143,  1.33161446, -1.35326296,  0.78461457,  0.74089363]
polangle = [-2.77397103,  1.64922494, -1.51121195,  0.47800055,  0.97812597]


setjy(vis='TDRW0001_calibrated.ms',
setjy(vis='TDRW0001_calibrated.ms',
Line 628: Line 630:
The Stokes V flux has been set to zero, corresponding to no circular polarization.
The Stokes V flux has been set to zero, corresponding to no circular polarization.


{{Setjy}} returns a Python dictionary (CASA record) that reports the Stokes I, Q, U and V terms. This is reported to the CASA command line window:
[https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html Setjy] returns a Python dictionary (CASA record) that reports the Stokes I, Q, U and V terms. This is reported to the CASA command line window (unless you used the execfile() method in which case results will be printed in the CASA log window only):
<pre>
<pre>
{'0': {'0': {'fluxd': array([9.98507906, 0.1342557 , 0.04260663, 0.        ])},
{'0': {'0': {'fluxd': array([9.98507906, 0.1342557 , 0.04260663, 0.        ])},
Line 641: Line 643:
  'format': "{field Id: {spw Id: {fluxd: [I,Q,U,V] in Jy}, 'fieldName':field name }}"}
  'format': "{field Id: {spw Id: {fluxd: [I,Q,U,V] in Jy}, 'fieldName':field name }}"}
</pre>
</pre>
Alternatively, you may capture this dictionary in a return variable, if you call {{setjy}} as '''myset=setjy(...)'''.
Alternatively, you may capture this dictionary in a return variable, if you call [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy] as '''myset=setjy(...)'''.


We can see the results in the model column in {{plotms}} (Figure 5A) showing the model source spectrum:
We can see the results in the model column in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] (Figure 5A) showing the model source spectrum:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 669: Line 671:


{|  
{|  
| [[Image:plotms_3C48-model-amp-RR_CASA6.1.2.jpeg|200px|thumb|left|Figure 5A: model RR amplitudes]]
| [[Image:Plotms-3C48-model-amp-RR-CASA6.2.1.jpeg|200px|thumb|left|Figure 5A: Model RR amplitudes of 3C48.]]
| [[Image:plotms_3C48-model-amp-RL_CASA6.1.2.jpeg|200px|thumb|center|Figure 5B: model RL amplitudes]]
| [[Image:Plotms-3C48-model-amp-RL-CASA6.2.1.jpeg|200px|thumb|center|Figure 5B: Model RL amplitudes of 3C48.]]
| [[Image:plotms_3C48-model-phase-RL_CASA6.1.2.jpeg|200px|thumb|right|Figure 5C: model RL phases]]
| [[Image:Plotms-3C48-model-phs-RL-CASA6.2.1.jpeg|200px|thumb|right|Figure 5C: Model RL phases of 3C48.]]
|}
|}


In order to obtain the correct amplitude scaling for instrumental polarization calibration, we need to also specify the Stokes I model that was used for the D-term calibrator(s). If we carried all tables, instead of splitting out the calibrated data from the pipeline, we wouldn't need to do this since the gain amplitudes provide the correct Stokes I scale for all the calibrators. The model values of the two D-term calibrators can be obtained from the pipeline weblog under the task hifv_fluxboot2 inside the CASA log.  
In order to obtain the correct amplitude scaling for instrumental polarization calibration, we need to also specify the Stokes I model that was used for the D-term calibrator(s). If we carried all tables, instead of splitting out the calibrated data from the pipeline, we wouldn't need to do this since the gain amplitudes provide the correct Stokes I scale for all the calibrators. The model values of the two D-term calibrators can be obtained from the pipeline weblog under the task hifv_fluxboot inside the CASA log (file stage12/casapy.log).  


<pre style="background-color: #fffacd;">
<pre style="background-color: #fffacd;">
Fitted spectrum for J2355+4950 with fitorder=2: Flux density = 1.76891 +/- 0.000620694 (freq=2.98457 GHz) spidx: a_1 (spectral index) =-0.599294 +/- 0.00265495 a_2=-0.203287 +/- 0.0644994 covariance matrix for the fit:  covar(0,0)=8.15918e-08 covar(0,1)=-1.14086e-07 covar(0,2)=-2.46839e-05 covar(1,0)=-1.14086e-07 covar(1,1)=2.47653e-05 covar(1,2)=0.000245534 covar(2,0)=-2.46839e-05 covar(2,1)=0.000245534 covar(2,2)=0.0146165
2021-12-07 23:11:53 INFO fluxscale:::: Fitted spectrum for J2355+4950 with fitorder=2: Flux density = 1.76871 +/- 0.000646424 (freq=2.98457 GHz) spidx: a_1 (spectral index) =-0.59957 +/- 0.00275983 a_2=-0.196834 +/- 0.0671363 covariance matrix for the fit:  covar(0,0)=8.14171e-08 covar(0,1)=-1.12868e-07 covar(0,2)=-2.46356e-05 covar(1,0)=-1.12868e-07 covar(1,1)=2.46143e-05 covar(1,2)=0.000242632 covar(2,0)=-2.46356e-05 covar(2,1)=0.000242632 covar(2,2)=0.0145659


Fitted spectrum for J0259+0747 with fitorder=2: Flux density = 0.970688 +/- 0.000678271 (freq=2.98457 GHz) spidx: a_1 (spectral index) =0.170883 +/- 0.00483357 a_2=-0.167167 +/- 0.127524 covariance matrix for the fit:  covar(0,0)=1.86864e-06 covar(0,1)=-2.48878e-06 covar(0,2)=-0.000583158 covar(1,0)=-2.48878e-06 covar(1,1)=0.000474074 covar(1,2)=-3.57109e-05 covar(2,0)=-0.000583158 covar(2,1)=-3.57109e-05 covar(2,2)=0.329984
2021-12-07 23:11:55 INFO fluxscale:::: Fitted spectrum for J0259+0747 with fitorder=2: Flux density = 0.970567 +/- 0.000712715 (freq=2.98457 GHz) spidx: a_1 (spectral index) =0.169924 +/- 0.00510264 a_2=-0.143228 +/- 0.134141 covariance matrix for the fit:  covar(0,0)=1.87165e-06 covar(0,1)=-2.56798e-06 covar(0,2)=-0.000583173 covar(1,0)=-2.56798e-06 covar(1,1)=0.000479142 covar(1,2)=-9.49244e-05 covar(2,0)=-0.000583173 covar(2,1)=-9.49244e-05 covar(2,2)=0.331129
</pre>
</pre>


This translates to the following {{setjy}} calls.
This translates to the following [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy] calls.


<source lang="python">
<source lang="python">
Line 698: Line 700:
       modimage="",
       modimage="",
       listmodels=False,
       listmodels=False,
       fluxdensity=[1.7689054559897328, 0, 0, 0],
       fluxdensity=[1.76871, 0, 0, 0],
       spix=[-0.5992942406258813, -0.20328749529071574],
       spix=[-0.59957, -0.196834],
       reffreq="2984571609.0079317Hz",
       reffreq="2984571609.0079317Hz",
       polindex=[],
       polindex=[],
Line 724: Line 726:
       modimage="",
       modimage="",
       listmodels=False,
       listmodels=False,
       fluxdensity=[0.9706883086751575, 0, 0, 0],
       fluxdensity=[0.970567, 0, 0, 0],
       spix=[0.17088309063387103, -0.16716711203636478],
       spix=[0.169924, -0.143228],
       reffreq='2984571609.0079317Hz',
       reffreq='2984571609.0079317Hz',
       polindex=[],
       polindex=[],
Line 740: Line 742:
==== Solving for the Cross-Hand delays ====
==== Solving for the Cross-Hand delays ====


Just as the pipeline did for the parallel-hand (RR,LL) delays before bandpass calibration, we solve for the cross-hand (RL, LR) delays due to the residual delay difference between the R and L on the reference antenna (''ea10'') used for the original delay calibration. In our case we simply use 3C48, which has a moderately polarized signal in the RL, LR correlations, and we set its polarized model above using {{setjy}}. In this CASA version and going forward there are two options to solve for the cross-hand delays, both of them will be illustrated here. The first option fits the cross-hand delay for the entire baseband (8 spectral windows in this example form a single baseband), which we call multiband delay. The second option solves the cross-hand delay independently per spectral window. Note that if a dataset contains multiple basebands and you wanted to solve for multiband delays, {{gaincal}} has to be executed for each baseband separately, selecting the appropriate spectral windows and appending the results to a single calibration table for later use.
Just as the pipeline did for the parallel-hand (RR,LL) delays before bandpass calibration, we solve for the cross-hand (RL,LR) delays because of the residual delay difference between the R and L on the reference antenna used for the original delay calibration (''ea10'' in this tutorial). In our case we simply use 3C48, which has a moderately polarized signal in the RL,LR correlations, and we set its polarized model above using [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy]. Starting with former version of CASA (6.1.2) there are two options to solve for the cross-hand delays, both of them will be illustrated here. The first option fits the cross-hand delay for the entire baseband (here 8 spectral windows form a single baseband), which we call multiband delay. The second option solves the cross-hand delay independently per spectral window. Note that if a dataset contains multiple basebands and you wanted to solve for multiband delays, [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.gaincal.html gaincal] has to be executed for each baseband separately, selecting the appropriate spectral windows and appending the results to a single calibration table for later use.


<source lang="python">
<source lang="python">
Line 782: Line 784:
</source>
</source>


[[Image:plotms_3c48-Kcross-delay_CASA6.1.2.jpeg|200px|thumb|right|Figure 6: single band cross-hand delay solutions]]
[[Image:Plotms-3C48-Kcross-delay-CASA6.2.1.jpeg|200px|thumb|right|Figure 6: Single band cross-hand delay solutions.]]
We can plot the single band solutions (see Figure 6):
We can plot the single band solutions (see Figure 6):
<source lang="python">
<source lang="python">
Line 789: Line 791:
</source>
</source>
You can also look at the solutions reported in the logger.
You can also look at the solutions reported in the logger.
<pre style="background-color: #fffacd;">
<pre style="background-color: #fffacd;">
For multiband delay there is one solution:
For multiband delay there is one solution:
Multi-band cross-hand delay=3.72174 nsec
Multi-band cross-hand delay=3.72994 nsec


For single band delay there are 8 solutions:
For single band delay there are 8 solutions:
Spw=0 Global cross-hand delay=5.72136 nsec
Spw=0 Global cross-hand delay=5.64778 nsec
Spw=1 Global cross-hand delay=1.53549 nsec
Spw=1 Global cross-hand delay=1.55875 nsec
Spw=2 Global cross-hand delay=-1.33461 nsec
Spw=2 Global cross-hand delay=-1.21461 nsec
Spw=3 Global cross-hand delay=0.511152 nsec
Spw=3 Global cross-hand delay=0.571198 nsec
Spw=4 Global cross-hand delay=4.33985 nsec
Spw=4 Global cross-hand delay=4.30379 nsec
Spw=5 Global cross-hand delay=1.27815 nsec
Spw=5 Global cross-hand delay=1.29297 nsec
Spw=6 Global cross-hand delay=3.76723 nsec
Spw=6 Global cross-hand delay=3.73613 nsec
Spw=7 Global cross-hand delay=3.08445 nsec
Spw=7 Global cross-hand delay=3.06041 nsec
 
 
</pre>
</pre>


Notice that the per spectral window solutions are very scattered. The mean delay is 2.36 ns, quite different from the multiband delay. This demonstrates the strength of fitting the cross-hand delay across multiple spectral windows, especially when using a calibrator with a significant frequency dependence, i.e. rotation measure and a polarization fraction of only a few percent. We will continue calibration using the single multiband delay that was derived at 3.72 ns.   
Notice that the per spectral window solutions are very scattered. The mean delay is 2.37 ns, quite different from the multiband delay. This demonstrates the strength of fitting the cross-hand delay across multiple spectral windows, especially when using a calibrator with a significant frequency dependence, i.e. rotation measure and a polarization fraction of only a few percent. We will continue calibration using the single multiband delay that was derived at 3.73 ns.   


Note that if we did not solve for this delay, it would be absorbed into the phases per channel of the following Df and Xf solutions. This would not cause us problems if we used an unpolarized D-term calibrator like J2355+4950, as we would not be solving for the Q+iU polarization. But if we were (e.g., using our gain calibrator J0259+0747 with parameter ''poltype='Df+QU' ''), then this step is essential.
Note that if we did not solve for this delay, it would be absorbed into the phases per channel of the following Df and Xf solutions. This would not cause us problems if we used an unpolarized D-term calibrator like J2355+4950, because we would not be solving for the Q+iU polarization. But if we were (e.g., using our gain calibrator J0259+0747 with parameter ''poltype='Df+QU' ''), then this step is essential.


==== Solving for the Leakage Terms ====
==== Solving for the Leakage Terms ====


The task {{polcal}} is used for polarization calibration. In this data set, we observed the unpolarized calibrator J2355+4950 to demonstrate solving for the instrumental polarization. Task {{polcal}} uses the Stokes I, Q, and U values in the model data (Q and U being zero for an unpolarized calibrator) to derive the leakage solutions. We also observed the polarized calibrator J0259+0747 (which has about 4.7% fractional polarization) that is our complex gain calibrator. The observations of J0259+0747 has a parallactic angle coverage of 31 degrees with 10 visits/slices, 3 of which were a bit longer to boost the signal-to-noise to at least 1000 per channel for each of the three passes. We will showcase solving for D-terms for both cases. The function calls are:
The task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.polcal.html polcal] is used for polarization calibration. In this data set, we observed the unpolarized calibrator J2355+4950 to demonstrate solving for the instrumental polarization. Task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.polcal.html polcal] uses the Stokes I, Q, and U values in the model data (Q and U being zero for an unpolarized calibrator) to derive the leakage solutions. We also observed the polarized calibrator J0259+0747 (which has about 4.7% fractional polarization) that is also our complex gain calibrator. The observations of J0259+0747 have a parallactic angle coverage of 31 degrees with 10 visits/slices, 3 of which were a bit longer to boost the signal-to-noise to at least 1000 per channel for each of the three passes. We will showcase solving for D-terms for both cases. The function calls are:


<source lang="python">
<source lang="python">
Line 848: Line 849:
</source>
</source>


* ''caltable '': {{polcal}} will create a new calibration table containing the leakage solutions, which we specify with the ''caltable'' parameter.
* ''caltable '': [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.polcal.html polcal] will create a new calibration table containing the leakage solutions, which we specify with the ''caltable'' parameter.
* ''field='' or ''intent='' : The unpolarized source J2355+4950 is used to solve for the leakage terms in the unpolarized case. For the polarized source J0259+0747 we set the intent leakage polarization.
* ''field='' or ''intent='' : The unpolarized source J2355+4950 is used to solve for the leakage terms in the unpolarized case. For the polarized source J0259+0747 we set the intent leakage polarization.
* ''spw='0~7' '': Select all spectral windows.
* ''spw='0~7' '': Select all spectral windows.
Line 858: Line 859:
In the case of Df+QU, the logger window will show the Q/U values it derived for the calibrator and the corresponding polarization fraction and angle that can be derived.
In the case of Df+QU, the logger window will show the Q/U values it derived for the calibrator and the corresponding polarization fraction and angle that can be derived.
<pre style="background-color: #fffacd;">
<pre style="background-color: #fffacd;">
Fractional polarization solution for J0259+0747 (spw = 0): : Q = 0.0222886, U = 0.0360251 (P = 0.0423626, X = 29.1276 deg)
Fractional polarization solution for J0259+0747 (spw = 0): : Q = 0.0214174, U = 0.0366555 (P = 0.0424539, X = 29.8514 deg)
Fractional polarization solution for J0259+0747 (spw = 1): : Q = 0.0114623, U = 0.0394172 (P = 0.04105, X = 36.8929 deg)
Fractional polarization solution for J0259+0747 (spw = 1): : Q = 0.0104099, U = 0.0393871 (P = 0.0407395, X = 37.5977 deg)
Fractional polarization solution for J0259+0747 (spw = 2): : Q = 0.0144047, U = 0.03993 (P = 0.0424488, X = 35.0816 deg)
Fractional polarization solution for J0259+0747 (spw = 2): : Q = 0.0143639, U = 0.0392768 (P = 0.041821, X = 34.956 deg)
Fractional polarization solution for J0259+0747 (spw = 3): : Q = 0.0105487, U = 0.0419017 (P = 0.0432091, X = 37.9347 deg)
Fractional polarization solution for J0259+0747 (spw = 3): : Q = 0.0110499, U = 0.0424822 (P = 0.0438958, X = 37.71 deg)
Fractional polarization solution for J0259+0747 (spw = 4): : Q = 0.00888572, U = 0.0403449 (P = 0.0413118, X = 38.7896 deg)
Fractional polarization solution for J0259+0747 (spw = 4): : Q = 0.00892886, U = 0.040305 (P = 0.0412822, X = 38.7544 deg)
Fractional polarization solution for J0259+0747 (spw = 5): : Q = 0.00803442, U = 0.0406955 (P = 0.041481, X = 39.4159 deg)
Fractional polarization solution for J0259+0747 (spw = 5): : Q = 0.00878222, U = 0.0408633 (P = 0.0417963, X = 38.9353 deg)
Fractional polarization solution for J0259+0747 (spw = 6): : Q = 0.00158955, U = 0.0432995 (P = 0.0433287, X = 43.9488 deg)
Fractional polarization solution for J0259+0747 (spw = 6): : Q = 0.00175604, U = 0.0429465 (P = 0.0429824, X = 43.8293 deg)
Fractional polarization solution for J0259+0747 (spw = 7): : Q = -0.00253994, U = 0.0481044 (P = 0.0481714, X = 46.5112 deg)
Fractional polarization solution for J0259+0747 (spw = 7): : Q = -0.00161836, U = 0.0480595 (P = 0.0480867, X = 45.9643 deg)


</pre>
</pre>
Line 871: Line 872:
From this you can see that J0259+0747 has a fractional polarization of 4.1&ndash;4.8% across the 1 GHz bandwidth with a small rotation measure causing a change in angle from 29 to 46 degrees over 1 GHz. In cases where the derived Q/U values seem random and the fractional polarization seems to be very small you might be able to derive better D-term solutions by using ''poltype='Df' ''.  
From this you can see that J0259+0747 has a fractional polarization of 4.1&ndash;4.8% across the 1 GHz bandwidth with a small rotation measure causing a change in angle from 29 to 46 degrees over 1 GHz. In cases where the derived Q/U values seem random and the fractional polarization seems to be very small you might be able to derive better D-term solutions by using ''poltype='Df' ''.  


After we run the two executions of {{polcal}}, you are strongly advised to examine the solutions with {{plotms}} to ensure that everything looks good and to compare the results using two different calibrators and poltype methods.
After we run the two executions of [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.polcal.html polcal], you are strongly advised to examine the solutions with [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] to ensure that everything looks good and to compare the results using two different calibrators and poltype methods.
{|
{|
  |[[Image:plotms_J0259-Damp-ea01_CASA6.1.2.jpeg|thumb|Figure 7a: J0259+0747 Df amp vs. freq for ea01]]
  |[[Image:Plotms-J0259-Damp-ea01-CASA6.2.1_Antennaea01@W06.jpeg|thumb|Figure 7a: J0259+0747 Df amplitude vs. frequency for antenna ea01.]]
  |[[Image:plotms_J2355-Damp-ea01_CASA6.1.2.jpeg|thumb|Figure 7b: J2355+4950 Df+QU amp vs. freq for ea01]]
  |[[Image:Plotms-J2355-Damp-ea01-CASA6.2.1_Antennaea01@W06.jpeg|thumb|Figure 7b: J2355+4950 Df+QU amplitude vs. frequency for antenna ea01.]]
  |[[Image:plotms_J0259-Dphase-ea01_CASA6.1.2.jpeg|thumb|Figure 7c: J0259+0747 Df phase vs. freq for ea01]]
  |[[Image:Plotms-J0259-Dphs-ea01-CASA6.2.1_Antennaea01@W06.jpeg|thumb|Figure 7c: J0259+0747 Df phase vs. frequency for antenna ea01.]]
  |[[Image:plotms_J2355-Dphase-ea01_CASA6.1.2.jpeg|thumb|Figure 7d: J2355+4950 Df+QU phase vs. freq for ea01]]
  |[[Image:Plotms-J2355-Dphs-ea01-CASA6.2.1_Antennaea01@W06.jpeg|thumb|Figure 7d: J2355+4950 Df+QU phase vs. frequency for antenna ea01.]]
|}
|}


<source lang="python">
<source lang="python">
# In CASA
# In CASA
plotms(vis=dtab_J0259,xaxis='freq',yaxis='amp',
      iteraxis='antenna',coloraxis='corr')
plotms(vis=dtab_J2355,xaxis='freq',yaxis='amp',  
plotms(vis=dtab_J2355,xaxis='freq',yaxis='amp',  
       iteraxis='antenna',coloraxis='corr')
       iteraxis='antenna',coloraxis='corr')


plotms(vis=dtab_J0259,xaxis='freq',yaxis='amp',  
plotms(vis=dtab_J0259,xaxis='chan',yaxis='phase',  
       iteraxis='antenna',coloraxis='corr')
       iteraxis='antenna',coloraxis='corr',plotrange=[-1,-1,-180,180])


plotms(vis=dtab_J2355,xaxis='chan',yaxis='phase',  
plotms(vis=dtab_J2355,xaxis='chan',yaxis='phase',  
       iteraxis='antenna',coloraxis='corr',plotrange=[-1,-1,-180,180])
       iteraxis='antenna',coloraxis='corr',plotrange=[-1,-1,-180,180])


plotms(vis=dtab_J0259,xaxis='chan',yaxis='phase',
      iteraxis='antenna',coloraxis='corr',plotrange=[-1,-1,-180,180])
</source>
</source>
This will produce plots similar to those shown in Figures 7A-D. You can cycle through the antennas by clicking the Next button. You should see leakages of between 5&ndash;15% in most cases. Both Df and Df+QU results should be comparable. However, we will be using the solutions from J0259+0747 to continue calibration and will use J2355+4950 to verify the polarization calibration.  
This will produce plots similar to those shown in Figures 7A-D. You can cycle through the antennas by clicking the Next button within [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms]. You should see leakages of between 5&ndash;17% in most cases. Both Df and Df+QU results should be comparable. However, we will be using the solutions from J0259+0747 to continue calibration and will use J2355+4950 to verify the polarization calibration.  


We can also display these in a single plot versus antenna index (see Figure 8):
We can also display these in a single plot versus antenna index (see Figure 8):
[[Image:plotms_J0259-DfQU_CASA6.1.2.jpeg|thumb|Figure 8: Df+QU solutions for J0259+0747 versus antenna index]]
[[Image:Plotms-J0259-DfQU-CASA6.2.1.jpeg|thumb|Figure 8: Df+QU solutions for J0259+0747 versus antenna index]]
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 902: Line 904:
</source>
</source>


In some cases there are outlier solutions above 0.25 that are most likely due to residual RFI. You can flag those from the Dterm table using {{flagdata}}. If everything went correctly, then this step is not necessary for this dataset.
In some cases there are outlier solutions above 0.25 that are most likely due to residual RFI. You can flag those from the Dterm table using [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.flagging.flagdata.html flagdata]. If everything went correctly, then this step should not be necessary for this dataset.
<source lang="python">
<source lang="python">
flagdata(vis=dtab_J2355, mode='clip', correlation='ABS_ALL', clipminmax=[0.0, 0.25], datacolumn='CPARAM', clipoutside=True, action='apply', flagbackup=False, savepars=False)
flagdata(vis=dtab_J2355, mode='clip', correlation='ABS_ALL', clipminmax=[0.0, 0.25], datacolumn='CPARAM', clipoutside=True, action='apply', flagbackup=False, savepars=False)
flagdata(vis=dtab_J0259, mode='clip', correlation='ABS_ALL', clipminmax=[0.0, 0.25], datacolumn='CPARAM', clipoutside=True, action='apply', flagbackup=False, savepars=False)
flagdata(vis=dtab_J0259, mode='clip', correlation='ABS_ALL', clipminmax=[0.0, 0.25], datacolumn='CPARAM', clipoutside=True, action='apply', flagbackup=False, savepars=False)
</source>
</source>
Line 910: Line 913:
==== Solving for the R-L polarization angle ====
==== Solving for the R-L polarization angle ====


Having calibrated for the instrumental polarization, the total polarization is now correct, but the R-L phase still needs to be calibrated in order to obtain an accurate polarization position angle.  We use the same task, {{polcal}}, but this time set parameter ''poltype='Xf', ''which specifies a frequency-dependent (''f'') position angle (''X'') calibration using the source 3C48, whose position angle is known, having set this earlier using {{setjy}}. Note that we must correct for the leakages before determining the R-L phase, which we do by adding the calibration table made in the previous step ('''dtab_J0259''') to the kcross table that is applied on-the-fly.
Having calibrated for the instrumental polarization, the total polarization is now correct, but the R-L phase still needs to be calibrated in order to obtain an accurate polarization position angle.  We use the same task, [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.polcal.html polcal], but this time set parameter ''poltype='Xf', ''which specifies a frequency-dependent (''f'') position angle (''X'') calibration using the source 3C48, the position angle of which is known, having set this earlier with [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.setjy.html setjy]. Note that we must correct for the leakages before determining the R-L phase, which we do by adding the calibration table made in the previous step ('''dtab_J0259''') to the kcross table that is applied on-the-fly.


<source lang="python">
<source lang="python">
Line 929: Line 932:
</source>
</source>


[[Image:plotms_3c48-Xf_CASA6.1.2.jpeg|thumb|Figure 9: Xf solutions versus frequency.]]
[[Image:Plotms-3C48-Xf-CASA6.2.1.jpeg|thumb|Figure 9: Xf solutions versus frequency.]]
Strictly speaking, there is no need to specify a reference antenna for ''poltype='Xf' ''(for circularly polarized receivers only) because the X solutions adjust the cross-hand phases for each antenna to match the given polarization angle of the model. However, for consistency/safety, it is recommended to always specify a refant when performing polarization calibration.
Strictly speaking, there is no need to specify a reference antenna for ''poltype='Xf' ''(for circularly polarized receivers only) because the X solutions adjust the cross-hand phases for each antenna to match the given polarization angle of the model. However, for consistency/safety, it is recommended to always specify refant when performing polarization calibration.


It is strongly suggested you check that the calibration worked properly by plotting up the newly-generated calibration table using {{plotms}} (see Figure 9):
It is strongly suggested you check that the calibration worked properly by plotting up the newly-generated calibration table using [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] (see Figure 9):
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 943: Line 946:
== Applying the Calibration ==
== Applying the Calibration ==


Now that we have derived all the calibration solutions, we need to apply them to the actual data using the task {{applycal}}. The measurement set DATA column contains the original split data. To apply the calibration we have derived, we specify the appropriate calibration tables which are then applied to the DATA column, with the results being written in the CORRECTED_DATA column. If the dataset does not already have a CORRECTED_DATA scratch column, then one will be created in the first {{applycal}} run.
Now that we have derived all the calibration solutions, we need to apply them to the actual data using the task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal]. The measurement set DATA column contains the original split data. To apply the calibration we have derived, we specify the appropriate calibration tables which are then applied to the DATA column, with the results being written in the CORRECTED_DATA column. If the dataset does not already have a CORRECTED_DATA scratch column, then one will be created in the first [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal] run.


<source lang="python">
<source lang="python">
Line 962: Line 965:
</source>
</source>


* ''gaintable'' : We provide a Python list of the calibration tables to be applied. This list must contain the cross-hand delays (kcross), the leakage calibration (dtab) (derived from J0259+0747), and the R-L phase corrections (xtab).
* ''gaintable'' : We provide a Python list of the calibration tables to be applied. This list must contain the cross-hand delays (kcross), the leakage calibration (dtab; here derived from J0259+0747), and the R-L phase corrections (xtab).
* ''calwt=[False] '': At the time of this writing, we are not yet using system calibration data to compute real (1/Jy<sup>2</sup>) weights, thus trying to calibrate them can produce nonsensical results. Experience has shown that calibrating the weights will lead to problems, especially in the self-calibration steps. You can specify ''calwt'' on a per-table basis, here is set all to ''False''.
* ''calwt=[False] '': At the time of this writing, we are not yet using system calibration data to compute real (1/Jy<sup>2</sup>) weights; trying to calibrate them can produce nonsensical results. Experience has shown that calibrating the weights will lead to problems, especially in the self-calibration steps. You can specify ''calwt'' on a per-table basis, here is set all to ''False''.
* ''parang '': If polarization calibration has been performed, set parameter ''parang=True''.
* ''parang '': If polarization calibration has been performed, set parameter ''parang=True''.


Line 973: Line 976:
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       coloraxis='corr',
       coloraxis='corr',
       plotfile='plotms_3c48-fld0-corrected-amp.jpeg')
       plotfile='Plotms-3C48-fld0-corrected-amp-CASA6.2.1.jpeg')


plotms(vis='TDRW0001_calibrated.ms',field='0',correlation='',
plotms(vis='TDRW0001_calibrated.ms',field='0',correlation='',
Line 979: Line 982:
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotfile='plotms_3c48-fld0-corrected-phase.jpeg')
       plotfile='Plotms-3C48-fld0-corrected-phase-CASA6.2.1.jpeg')


plotms(vis='TDRW0001_calibrated.ms',field='1',correlation='',
plotms(vis='TDRW0001_calibrated.ms',field='1',correlation='',
       timerange='',antenna='',avgtime='60',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       plotfile='plotms_J2355-fld1-corrected-amp.jpeg')
       plotfile='Plotms-J2355-fld1-corrected-amp-CASA6.2.1.jpeg')


plotms(vis='TDRW0001_calibrated.ms',field='1',correlation='RR,LL',
plotms(vis='TDRW0001_calibrated.ms',field='1',correlation='RR,LL',
Line 990: Line 993:
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotfile='plotms_J2355-fld1-corrected-phase.jpeg')
       plotfile='Plotms-J2355-fld1-corrected-phase-CASA6.2.1.jpeg')


plotms(vis='TDRW0001_calibrated.ms',field='2',correlation='',
plotms(vis='TDRW0001_calibrated.ms',field='2',correlation='',
       timerange='',antenna='',avgtime='60',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       plotfile='plotms_J0259-fld2-corrected-amp.jpeg')
       plotfile='Plotms-J0259-fld2-corrected-amp-CASA6.2.1.jpeg')


plotms(vis='TDRW0001_calibrated.ms',field='2',correlation='',
plotms(vis='TDRW0001_calibrated.ms',field='2',correlation='',
Line 1,001: Line 1,004:
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',avgbaseline=True,
       plotrange=[-1,-1,-180,180],coloraxis='corr',avgbaseline=True,
       plotfile='plotms_J0259-fld2-corrected-phase.jpeg')
       plotfile='Plotms-J0259-fld2-corrected-phase-CASA6.2.1.jpeg')
</source>
</source>
For 3C48 (figures 10A, 10B) we see the polarized signal in the cross-hands; there is some sign of bad data remaining in 3C48. Also, the RL phase plots of J0259+4950 (figure 10F) indicate that the Xf solutions, thus polarization angles, in the lowest two spectral windows are problematic. You can also estimate from the RL,LR amplitudes in J2355+4950 (figure 10E) what the level of residual instrumental polarization, which we expect to be around <0.5%. A more accurate evaluation of residual instrumental polarization fraction can be made imaging the secondary D-term calibrator per spectral window and calculating its residual polarization.  
For 3C48 (figures 10A, 10B) we see the polarized signal in the cross-hands; there is some sign of bad data remaining in 3C48. Also, the RL phase plots of J0259+4950 (figure 10F) indicate that the Xf solutions, thus polarization angles, in the lowest two spectral windows are problematic. You can also estimate from the RL,LR amplitudes in J2355+4950 (figure 10E) what the level of residual instrumental polarization, which we expect to be around <0.5%. A more accurate evaluation of residual instrumental polarization fraction can be made imaging the secondary D-term calibrator per spectral window and calculating its residual polarization.  


{|
{|
| [[Image:plotms_3c48-fld0-corrected-amp_6.1.2.jpeg|thumb|Figure 10A amp vs channel for 3C48 RR,RL,LR,LL]]
| [[Image:Plotms-3C48-fld0-corrected-amp-CASA6.2.1.jpeg|thumb|Figure 10A: amplitude vs channel for 3C48 RR,RL,LR,LL]]
| [[Image:plotms_3c48-fld0-corrected-phase_6.1.2.jpeg|thumb|Figure 10B: phase vs channel for 3C48 RR,RL,LR,LL]]
| [[Image:Plotms-3C48-fld0-corrected-phase-CASA6.2.1.jpeg|thumb|Figure 10B: phase vs channel for 3C48 RR,RL,LR,LL]]
| [[Image:plotms_J2355-fld1-corrected-amp_6.1.2.jpeg|thumb|Figure 10C: amp vs channel for J2355+4950 RR,LL,RL,LR]]
| [[Image:Plotms-J2355-fld1-corrected-amp-CASA6.2.1.jpeg|thumb|Figure 10C: amplitude vs channel for J2355+4950 RR,LL,RL,LR]]
| [[Image:plotms_J2355-fld1-corrected-phase_6.1.2.jpeg|thumb|Figure 10D: phase vs channel for J2355+4950 RR,LL]]
| [[Image:Plotms-J2355-fld1-corrected-phase-CASA6.2.1.jpeg|thumb|Figure 10D: phase vs channel for J2355+4950 RR,LL]]
| [[Image:plotms_J0259-fld1-corrected-amp_6.1.2.jpeg|thumb|Figure 10E: amp vs channel for J0259+4950 RR,LL,RL,LR]]
| [[Image:plotms_J0259-fld1-corrected-phase_6.1.2.jpeg|thumb|Figure 10F: phase vs channel for J0259+4950 RR,LL with baseline averaging]]
|}
|}


{|
| [[Image:Plotms-J0259-fld2-corrected-amp-CASA6.2.1.jpeg|thumb|Figure 10E: amplitude vs channel for J0259+4950 RR,LL,RL,LR]]
| [[Image:Plotms-J0259-fld2-corrected-phase-CASA6.2.1.jpeg|thumb|Figure 10F: phase vs channel for J0259+4950 RR,LL with baseline averaging]]
|}


Inspecting the data at this stage may well show up previously-unnoticed bad data. Plotting the '''corrected''' amplitude against UV distance or against time is a good way to find such issues. If you find bad data, you can remove them via interactive flagging in {{plotms}} or via manual flagging in {{flagdata}} once you have identified the offending antennas/baselines/channels/times. When you are happy that all data (particularly on your target source) look good, you may proceed. However, especially for the target, we will return to additional flagging at a later stage.  
 
Inspecting the data at this stage may well show up previously-unnoticed bad data. Plotting the '''corrected''' amplitude against UV distance or against time is a good way to find such issues. If you find bad data, you can remove them via interactive flagging in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] or via manual flagging in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.flagging.flagdata.html flagdata] once you have identified the offending antennas/baselines/channels/times. When you are happy that all data (particularly on your target source) look good, you may proceed. However, especially for the target, we will return to additional flagging at a later stage.  


Now that the calibration has been applied to the target data, we split off the science targets to create a new, calibrated measurement set containing the target field. This is not strictly necessary if you want to save disk space.  
Now that the calibration has been applied to the target data, we split off the science targets to create a new, calibrated measurement set containing the target field. This is not strictly necessary if you want to save disk space.  
Line 1,026: Line 1,032:


* ''outputvis '': We give the name of the new measurement set to be written, which will contain the calibrated data on the science target.
* ''outputvis '': We give the name of the new measurement set to be written, which will contain the calibrated data on the science target.
* ''datacolumn '': We use the CORRECTED_DATA column, containing the calibrated data which we just wrote using {{applycal}}.
* ''datacolumn '': We use the CORRECTED_DATA column, containing the calibrated data which we just wrote using [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal].
* ''field '': We wish to target field into a measurement set for imaging and joint deconvolution.
* ''field '': We wish to target field into a measurement set for imaging and joint deconvolution.


Prior to imaging, it is a good idea to run the {{statwt}} task to correct the data weights (<i>weight</i> and <i>sigma</i> columns) in the measurement set.  Running {{statwt}} will remove the effects of relative noise scatter that may have been introduced from flagging uneven bits in the visibility data between the channels and times. We will run this task here on the newly calibrated and split data set before moving on to imaging.
Prior to imaging, it is a good idea to run the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.manipulation.statwt.html statwt] task to correct the data weights (<i>weight</i> and <i>sigma</i> columns) in the measurement set.  Running [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.manipulation.statwt.html statwt] will remove the effects of relative noise scatter that may have been introduced from flagging uneven bits in the visibility data between the channels and times. We will run this task here on the newly calibrated and split data set before moving on to imaging.


<source lang="python">
<source lang="python">
Line 1,044: Line 1,050:
</math>
</math>


The <math>u</math> and <math>v</math> coordinates are the baselines measured in units of the observing wavelength, while the <math>l</math> and <math>m</math> coordinates are the direction cosines on the sky. For generality, the sky coordinates are written in terms of direction cosines; but for most VLA (and ALMA) observations they can be related simply to the right ascension (<math>l</math>) and declination (<math>m</math>). Also recall that this equation is valid only if the <math>w</math> coordinate of the baselines can be neglected; this assumption is almost always true at high frequencies and smaller VLA configurations. The <math>w</math> coordinate cannot be neglected at lower frequencies and larger configurations (e.g., 0.33 GHz, A-configuration observations). This expression also neglects other factors, such as the shape of the primary beam. For more information on imaging, see the [https://casa.nrao.edu/casadocs/latest/synthesis-imaging Synthesis Imaging] section of the CASA documentation.
The <math>u</math> and <math>v</math> coordinates are the baselines measured in units of the observing wavelength, while the <math>l</math> and <math>m</math> coordinates are the direction cosines on the sky. In general, the sky coordinates are written in terms of direction cosines; but for most VLA (and ALMA) observations, they can be related simply to the right ascension (<math>l</math>) and declination (<math>m</math>). Recall that this equation is valid only if the <math>w</math> coordinate of the baselines can be neglected; this assumption is almost always true at high frequencies and smaller VLA configurations. The <math>w</math> coordinate cannot be neglected at lower frequencies and larger configurations (e.g., 0.33 GHz, A-configuration observations). This expression also neglects other factors, such as the shape of the primary beam. For more information on imaging, see the [https://casadocs.readthedocs.io/en/v6.2.1/notebooks/synthesis_imaging.html section of the CASA documentation.]


[[Image:plotms_3c75-uvwave_6.1.2.jpeg|thumb|Figure 11: ''plotms'' plot showing Amplitude vs UV Distance in wavelengths for 3C75 at 3000 MHz]]
[[Image:Plotms-3C75-uvwave.jpeg|thumb|Figure 11: ''plotms'' plot showing Amplitude vs UV Distance in wavelengths for 3C75 at 3000 MHz]]
CASA has a task {{tclean}} which both Fourier transforms the data and deconvolves the resulting image. For the purposes of this tutorial, we will make a mosaic clean image in Stokes I only; polarimetric imaging will be addressed in an upcoming new CASAguide. We will use a multi-scale cleaning algorithm because the supernova remnant contains both diffuse, extended structure on large spatial scales and finer filamentary structure on smaller scales. This approach will do a better job of modeling the image than the classic clean delta function. For broader examples of many {{tclean}} options, please see the [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials#Imaging_VLA_Data_in_CASA Topical Guide for Imaging VLA Data].
CASA has a task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] which both Fourier transforms the data and deconvolves the resulting image. We will use a multi-scale cleaning algorithm because our target source, a complex radio galaxy, contains both diffuse, extended structures on large spatial scales as well as point-like components. This approach will do a better job of modeling the image than the classic clean delta function. For broader examples of many [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] options, please see the [https://casaguides.nrao.edu/index.php/Karl_G._Jansky_VLA_Tutorials#Imaging_VLA_Data_in_CASA Topical Guide for Imaging VLA Data].


== Multi-scale Clean ==
== Multi-scale Clean ==


It is important to have an idea of what values to use for the image pixel (cell) size and the overall size of the image. Setting the appropriate pixel size for imaging depends upon basic optics aspects of interferometry. Using {{plotms}} to look at the newly-calibrated, target-only data set:
It is important to have an idea of what values to use for the image pixel (cell) size and the overall size of the image. Setting the appropriate pixel size for imaging depends upon basic optics aspects of interferometry. Use [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] to look at the newly calibrated, target-only data set:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 1,058: Line 1,064:
       plotfile='plotms_3c75-uvwave.jpeg',avgspw=False,overwrite=True)
       plotfile='plotms_3c75-uvwave.jpeg',avgspw=False,overwrite=True)
</source>
</source>
You should obtain a plot similar to Figure 11 with the (calibrated) visibility amplitude as a function of <math>u</math>-<math>v</math> distance. You also see some outliers there which are primarily from residual amplitude errors of ea05, which had a warm receiver which we can isolate to particular time periods. We will be addressing this after the initial imaging. The maximum baseline is about 12,000 wavelengths, i.e., an angular scale of 17 arcseconds (<math>\lambda/D=1/12000</math>). The most effective cleaning occurs with 3&ndash;5 pixels across the synthesized beam. For example, a cell size of 3.4 arcseconds will give just about 5 pixels per beam.   
You should obtain a plot similar to Figure 11 with the (calibrated) visibility amplitude as a function of <math>u</math>-<math>v</math> distance. You will also see some outliers there which are primarily from residual amplitude errors of ea05, that had a warm receiver which we can isolate to particular time periods. We will be addressing this after the initial imaging. The maximum baseline is about 12,000 wavelengths, i.e., an angular scale of 17 arcseconds (<math>\lambda/D=1/12000</math>). The most effective cleaning occurs with 3&ndash;5 pixels across the synthesized beam. For example, a cell size of 3.4 arcseconds will give just about 5 pixels per beam.   


The binary black hole system is known to have a maximum extend of at least 8-9 arcminutes, corresponding to about 147 pixels for the chosen cell size. Thus, we need to choose an image size that covers most of the extent of the source. To be safe from bright, far out, sources we should at least cover the primary beam. Although CASA has the feature that its Fourier transform engine (FFTW) does ''not'' require a strict power of 2 for the number of linear pixels in a given image axis, it is somewhat more efficient if the number of pixels on a side is a composite number divisible by ''any pair'' of 2 and 3 and/or 5. Because {{tclean}} internally applies a padding of 1.2 (=3x2/5), choose 480 which is 2<sup>5</sup> &times; 3 &times; 5 (so 480 &times; 1.2 = 576 = 2<sup>6</sup> &times; 3<sup>2</sup>). We therefore set ''imsize=[480,480]'' and the source will fit comfortable within that image.
The 3C75 binary black hole system is known to have a maximum extent of at least 8-9 arcminutes, corresponding to about 147 pixels for the chosen cell size. Therefore, we need to choose an image size that covers most of the extent of the source. To aid deconvolution, especially when bright sources far from phase center are present, we should at the minimum image the size of the primary beam. Although CASA has the feature that its Fourier transform engine (FFTW) does ''not'' require a strict power of 2 for the number of linear pixels in a given image axis, it is somewhat more efficient if the number of pixels on a side is a composite number divisible by ''any pair'' of 2 and 3 and/or 5. Because [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] internally applies a padding of 1.2 (=3x2/5), choose 480 which is 2<sup>5</sup> &times; 3 &times; 5 (so 480 &times; 1.2 = 576 = 2<sup>6</sup> &times; 3<sup>2</sup>). We therefore set ''imsize=[480,480]'' and the source will fit comfortable within that image.


In this tutorial, we will run {{tclean}} interactively so that we can set and modify the mask:
In this tutorial, we will run [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] interactively so that we can set and modify the mask:
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 1,118: Line 1,124:
</source>
</source>


Task {{tclean}} is powerful with many inputs and a certain amount of experimentation likely is required.
[[Image:3C75-tclean-interactive-start-CASA6.2.1.png|thumb|Figure 12: Interactive clean at the beginning, having selected polygon region and ready to double-click inside to set the mask.]]
[[Image:3c75-tclean-interactive-start_CASA6.1.2.jpeg|thumb|Figure 12: Interactive clean at the beginning, having selected polygon region and ready to double-click inside to set the mask.]]
Task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] is powerful with many inputs and a certain amount of experimentation likely is required.
* ''vis='3C75.ms' '': this split MS contains the target field only.
* ''vis='3C75.ms' '': this split MS contains the target field only.
* ''imagename='3C75_initial' '': our output image cube will all start with this, e.g., 3C75_initial.image
* ''imagename='3C75_initial' '': our output image cubes will all start with this name root, e.g., 3C75_initial.image
* ''specmode='mfs' '': Use multi-frequency synthesis imaging. The fractional bandwidth of these data is non-zero (1000 MHz at a central frequency of 3.0 GHz). Recall that the <math>u</math> and <math>v</math> coordinates are defined as the baseline coordinates, measured in wavelengths. Thus, slight changes in the frequency from channel to channel result in slight changes in <math>u</math> and <math>v</math>. There is a concomitant improvement in <math>u</math>-<math>v</math> coverage if the visibility data from the multiple spectral channels are gridded separately onto the <math>u</math>-<math>v</math> plane, as opposed to treating all spectral channels as having the same frequency.
* ''specmode='mfs' '': Use multi-frequency synthesis imaging. The fractional bandwidth of these data is non-zero (1000 MHz at a central frequency of 3.0 GHz). Recall that the <math>u</math> and <math>v</math> coordinates are defined as the baseline coordinates, measured in wavelengths. Thus, slight changes in the frequency from channel to channel result in slight changes in <math>u</math> and <math>v</math>. There is a concomitant improvement in <math>u</math>-<math>v</math> coverage if the visibility data from the multiple spectral channels are gridded separately onto the <math>u</math>-<math>v</math> plane, as opposed to treating all spectral channels as having the same frequency.
* ''niter=20000,gain=0.1,threshold='0.0mJy' '': Recall that the gain is the amount by which a clean component is subtracted during the cleaning process. Parameters ''niter'' and ''threshold'' are (coupled) means of determining when to stop the cleaning process, with ''niter'' specifying to find and subtract that many clean components while ''threshold'' specifies a minimum flux density threshold a clean component can have before {{tclean}} stops (also see ''interactive'' below). Imaging is an iterative process, and to set the threshold and number of iterations, it is usually wise to clean interactively in the first instance, stopping when spurious emission from sidelobes (arising from gain errors) dominates the residual emission in the field. Here, we have set the threshold level to zero and let the {{tclean}} task define an appropriate threshold. The number of iterations should then be set high enough to reach the threshold found by tclean.
* ''niter=20000,gain=0.1,threshold='0.0mJy' '': Recall that the gain is the amount by which a clean component is subtracted during the cleaning process. Parameters ''niter'' and ''threshold'' are (coupled) means of determining when to stop the cleaning process, with ''niter'' specifying to find and subtract that many clean components while ''threshold'' specifies a minimum flux density threshold a clean component can have before [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] stops (also see ''interactive'' below). Imaging is an iterative process, and to set the threshold and number of iterations, it is usually wise to clean interactively in the first instance, stopping when spurious emission from sidelobes (arising from gain errors) dominates the residual emission in the field. Here, we have set the threshold level to zero and let the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] task define an appropriate threshold. The number of iterations should then be set high enough to reach the threshold found by tclean.
* ''gridder='standard' '': The standard tclean gridder is sufficient for our purposes, since we are not combining multiple pointings from a mosaic or try to perform widefield imaging in an extended configuration.
* ''gridder='standard' '': The standard tclean gridder is sufficient for our purposes, since we are not combining multiple pointings from a mosaic or try to perform widefield imaging in an extended configuration.
* ''interactive=True '': Very often, particularly when one is exploring how a source appears for the first time, it can be valuable to interact with the cleaning process. If True, ''interactive'' causes a {{viewer}} window to appear. One can then set clean regions, restricting where {{tclean}} searches for clean components, as well as monitor the cleaning process. A standard procedure is to set a large value for ''niter'', and stop the cleaning when it visually appears to be approaching the noise level. This procedure also allows one to change the cleaning region, in cases when low-level intensity becomes visible as the cleaning process proceeds.   
* ''interactive=True '': Very often, particularly when one is exploring how a source appears for the first time, it can be valuable to interact with the cleaning process. If True, ''interactive'' causes a [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.viewer.html viewer] window to appear. One can then set clean regions, restricting where [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] searches for clean components, as well as monitor the cleaning process. A standard procedure is to set a large value for ''niter'', and stop the cleaning when it visually appears to be approaching the noise level. This procedure also allows one to change the cleaning region, in cases when low-level intensity becomes visible as the cleaning process proceeds.   
* ''imsize=480,cell='3.4arcsec' '': See the discussion above regarding the setting of the image size and cell size. If only one value is specified, the same value is used in both directions.
* ''imsize=480,cell='3.4arcsec' '': See the discussion above regarding setting the image size and cell size. If only one value is specified for the parameter, the same value is used in both directions (declination and right ascension).
* ''stokes='IQUV' '': An image cube will be made containing total intensity I, and Stokes Q, U, and V.
* ''stokes='IQUV' '': [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] will output an image cube containing all: total intensity I, and Stokes Q, U, and V.
* ''deconvolver='multiscale', scales=[0, 6, 18], smallscalebias=0.9 '':  The settings for ''multiscale'' are in units of pixels, with 0 pixels equivalent to the traditional delta-function clean.  The scales here are chosen to provide delta functions and then two logarithmically scaled sizes to fit to the data. The first scale (6 pixels) is chosen to be comparable to the size of the synthesized beam. The ''smallscalebias'' attempts to balance the weight given to larger scales, which often have more flux density, and the smaller scales, which often are brighter. Considerable experimentation is likely to be necessary; one of the authors of this document found that it was useful to clean several rounds with this setting, change to ''multiscale=[]'' and remove much of the smaller scale structure, then return to this setting.
* ''deconvolver='multiscale', scales=[0, 6, 18], smallscalebias=0.9 '':  The settings for ''multiscale'' are in units of pixels, with 0 pixels equivalent to the traditional delta-function clean.  The scales here are chosen to provide delta functions and then two logarithmically scaled sizes to fit to the data. The first scale (6 pixels) is chosen to be comparable to the size of the synthesized beam. The ''smallscalebias'' attempts to balance the weight given to larger scales, which often have more flux density, and the smaller scales, which often are brighter. Considerable experimentation is likely to be necessary; one of the authors of this document found that it was useful to clean several rounds with this setting, change to ''multiscale=[]'' and remove much of the smaller scale structure, then return to this setting.
* ''weighting='briggs',robust=0.5 '': 3C75 has diffuse, extended emission that is, at least partially, resolved out by the interferometer even though we are in the most compact VLA configuration. A naturally-weighted image would show large-scale patchiness in the noise. In order to suppress this effect, Briggs weighting is used (intermediate between natural and uniform weighting), with a default robust factor of 0.5 (which corresponds to something between natural and uniform weighting).
* ''weighting='briggs',robust=0.5 '': 3C75 has diffuse, extended emission that is, at least partially, resolved out by the interferometer even though we are in the most compact VLA configuration. A naturally-weighted image would show large-scale patchiness in the noise. In order to suppress this effect, Briggs weighting is used (intermediate between natural and uniform weighting), with a default robust factor of 0.5 (which corresponds to something between natural and uniform weighting).
Line 1,133: Line 1,139:
* ''savemodel='modelcolumn' '': We recommend here the use of a physical MODEL_DATA scratch column. This will save some time, as it can be faster in the case of complicated gridding to read data from disk instead of doing all of the computations on-the-fly. However, this has the unfortunate side effect of increasing the size of the MS on disk.
* ''savemodel='modelcolumn' '': We recommend here the use of a physical MODEL_DATA scratch column. This will save some time, as it can be faster in the case of complicated gridding to read data from disk instead of doing all of the computations on-the-fly. However, this has the unfortunate side effect of increasing the size of the MS on disk.


[[Image:3c75-tclean-multiscale-300iters_CASA6.1.2.jpeg|thumb|Figure 13: After the first approximately 300 iterations of multi-scale mfs clean]]
[[Image:3C75-tclean-interactive-300iters-CASA6.2.1.png|thumb|Figure 13: After the first approximately 300 iterations of multi-scale mfs clean]]


As mentioned above, we can guide the clean process by allowing it to find clean components only within a user-specified region.  When {{tclean}} runs in interactive mode, a {{viewer}} window will pop up as shown in Figure 12. '''First, you'll want to navigate to the green box and select "All Polarizations" rather than use the default "This Polarization"'''; this way the cleaning we are about to do will apply to all of the polarizations rather than just the one we are currently viewing. To get a more detailed view of the central regions containing the emission, zoom in by first left clicking on the zoom button (leftmost button in third row) and tracing out a rectangle with the left mouse button and double-clicking inside the zoom box you just made. Play with the color scale to bring out the emission better by holding down the middle mouse button and moving it around. To create a clean box (a region within which components may be found), hold down the right mouse button and trace out a rectangle around the source, then double-click inside that rectangle to set it as a box. Note that the clean box must turn white for it to be registered; if the box is not white, it has not been set. Alternatively, you can trace out a more custom shape to better enclose the irregular outline of the supernova remnant. To do this, right-click on the closed polygonal icon then trace out a shape by right-clicking where you want the corners of that shape. Once you have come full circle, the shape will be traced out in green, with small squares at the corners. Double-click inside this region and the green outline will turn white. You have now set the clean region. If you have made a mistake with your clean box, click on the Erase button, trace out a rectangle around your erroneous region, and double-click inside that rectangle. You can also set multiple clean regions.
As mentioned above, we can guide the clean process by allowing it to find clean components only within a user-specified region.  When [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] runs in interactive mode, an [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.imview.html imview] window will pop up as shown in Figure 12. '''First, you'll want to navigate to the green box and select "All Polarizations" rather than use the default "This Polarization"'''; this way the cleaning we are about to do will apply to all of the polarizations rather than just the one we are currently viewing. Similarly, select "All channels". To get a more detailed view of the central regions containing the emission, zoom in by first left clicking on the zoom button (leftmost button in third row) and tracing out a rectangle with the left mouse button and double-clicking inside the zoom box you just made. Play with the color scale to bring out the emission better by holding down the middle mouse button and moving it around. To create a clean box (a region within which components may be found), hold down the right mouse button and trace out a rectangle around the source, then double-click inside that rectangle to set it as a box. Note that the clean box must turn white for it to be registered - if the box is not white, it has not been set. Alternatively, you can trace out a more custom shape to better enclose the irregular outline of the radio galaxy jets. To do this, right-click on the closed polygonal icon then trace out a shape by right-clicking where you want the corners of that shape. Once you have come full circle, the shape will be traced out in green, with small squares at the corners. Double-click inside this region and the green outline will turn white. You have now set the clean region. If you have made a mistake with your clean box, click on the Erase button, trace out a rectangle around your erroneous region, and double-click inside that rectangle. You can also set multiple clean regions.


At any stage in the cleaning, you can adjust the number of iterations that {{tclean}} will do before returning to the GUI. This is set to 1000 (see the iterations field in mid-upper left of panel), values from 500 to 1000 later on seem to work. Note that this will override the ''niter'' that was set when you started {{tclean}}. {{tclean}} will keep going until it reaches threshold or runs out of cycles (the cycles field to the right of the iterations).
At any stage in the cleaning, you can adjust the number of iterations that [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] will do before returning to the GUI (''cycleniter''). This is set to 1000 (see the iterations field in mid-upper left of panel), values from 500 to 1000 later on seem to work. Note that this will override the ''cycleniter'' value that you might had set before starting [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean]. [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] will keep going until it reaches threshold or runs out of cycles (the cycles field to the right of the iterations).


[[Image:3c75-tclean-residuals_CASA6.1.2.jpeg|thumb|Figure 14: Interactive residuals after about 13000 iterations of multi-scale mfs clean]]
[[Image:3C75-tclean-interactive-residuals-CASA6.2.1.png|thumb|Figure 14: Interactive residuals after about 13000 iterations of multi-scale mfs clean]]
When you are happy with the clean regions, press the green circular arrow button on the far right to continue deconvolution. After completing a cycle, a revised image will come up. As the brightest points are removed from the image (cleaned off), fainter emission may show up. You can adjust the clean boxes each cycle, to enclose all real emission. After many cycles, when only noise is left, you can hit the red-and-white stop-sign icon to stop cleaning. Figure 13 shows the interactive viewer panel later in the process, after cleaning about 500 iterations. We have used the polygon tool to add to the clean region, drawing around emission that shows up in the residual image outside of the original clean region. After about 13000 iterations (Figure 14) the residuals were looking good (similar noise level inside and outside of the clean region). As mentioned above, restarting {{tclean}} with different ''multiscale=[...]'' choices can help also. You see that there is a significant amount of residual structure, these are most likely due to calibration errors which we will try to correct for in the next section during self-calibration.
When you are happy with the clean regions, press the green circular arrow button on the far right to continue deconvolution. After completing a cycle, a revised image will come up. As the brightest points are removed from the image (cleaned off), fainter emission may show up. You can adjust the clean boxes each cycle, to enclose all real emission. After many cycles, when only noise is left, you can hit the red-and-white stop-sign icon to stop cleaning. Figure 13 shows the interactive viewer panel later in the process, after cleaning about 500 iterations. We have used the polygon tool to add to the clean region, drawing around emission that shows up in the residual image outside of the original clean region. After about 13000 iterations (Figure 14) the residuals were looking good (similar noise level inside and outside of the cleaned mask region). As mentioned before, restarting [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] with different ''multiscale=[...]'' choices can help also. You see that there is a significant amount of residual structure, these are most likely due to calibration errors which we will try to correct for in the next section during self-calibration.


Task {{tclean}} will make several output files, all named with the prefix given as ''imagename''.  These include:
Task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] will make several output files, all named with the prefix given as ''imagename''.  These include:
* ''.image'': final restored image(s) with the clean components convolved with a restoring beam and added to the remaining residuals at the end of the imaging process, one for each Taylor Term (.tt0 and .tt1)
* ''.image'': final restored image(s) with the clean components convolved with a restoring beam and added to the remaining residuals at the end of the imaging process, one for each Taylor Term (.tt0 and .tt1)
* ''.pb.tt0'': effective response of the telescope (the primary beam)
* ''.pb.tt0'': effective response of the telescope (the primary beam)
* ''.mask'': areas where {{tclean}} has been allowed to search for emission
* ''.mask'': areas where [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] has been allowed to search for emission
* ''.model'': sum of all the clean components, which also has been stored as the MODEL_DATA column in the measurement set, one for each Taylor Term (.tt0 and .tt1)
* ''.model'': sum of all the clean components, which also has been stored as the MODEL_DATA column in the measurement set, one for each Taylor Term (.tt0 and .tt1)
* ''.psf'': dirty beam, which is being deconvolved from the true sky brightness during the clean process, one for each Taylor Term (.tt0, .tt1, .tt2)
* ''.psf'': dirty beam, which is being deconvolved from the true sky brightness during the clean process, one for each Taylor Term (.tt0, .tt1, .tt2)
Line 1,152: Line 1,158:


{|
{|
  |[[Image:3c75-viewer-multiscale-initial_I_CASA6.1.2.jpeg|thumb|Figure 15A: Viewer panel of final restored Stokes I image (using HotMetal1 colormap and Scaling Power Cycles = -0.5)]]
  |[[Image:3c75-viewer-multiscale-initialI-CASA6.2.1.png|thumb|Figure 15A: Viewer panel of final restored Stokes I image (using HotMetal1 colormap and Scaling Power Cycles = -1)]]
  |[[Image:3c75-viewer-multiscale-initial_Q_CASA6.1.2.jpeg|thumb|Figure 15B: Viewer panel of final restored Stokes Q image (using HotMetal1 colormap and Scaling Power Cycles = -0.5)]]
  |[[Image:3c75-viewer-multiscale-initialQ-CASA6.2.1.png|thumb|Figure 15B: Viewer panel of final restored Stokes Q image (using HotMetal1 colormap and Scaling Power Cycles = -1)]]
  |[[Image:3c75-viewer-multiscale-initial_U_CASA6.1.2.jpeg|thumb|Figure 15C: Viewer panel of final restored Stokes U image (using HotMetal1 colormap and Scaling Power Cycles = -0.5)]]
  |[[Image:3c75-viewer-multiscale-initialU-CASA6.2.1.png|thumb|Figure 15C: Viewer panel of final restored Stokes U image (using HotMetal1 colormap and Scaling Power Cycles = -1)]]
  |[[Image:3c75-viewer-multiscale-initial_V_CASA6.1.2.jpeg|thumb|Figure 15D: Viewer panel of final restored Stokes V image (using HotMetal1 colormap and Scaling Power Cycles = -0.5)]]
  |[[Image:3c75-viewer-multiscale-initialV-CASA6.2.1.png|thumb|Figure 15D: Viewer panel of final restored Stokes V image (using HotMetal1 colormap and Scaling Power Cycles = -1)]]
|}
|}


After the imaging and deconvolution process has finished, you can use the {{viewer}} to look at your image.
After the imaging and deconvolution process has finished, you can use the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.imview.html imview] to look at your image.
<source lang="python">
<source lang="python">
# In CASA
# In CASA
viewer('3C75_initial.image.tt0')
imview('3C75_initial.image.tt0')
</source>
</source>
You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do). Also, using the wrench panel to change Display Options will be helpful here. Here we selected the Hot Metal 1 colormap and set the Scaling Power Cycles to -1 to better emphasize the faint emission and compare to the noise (Figures 15A - D). You can also use the Animators slider for Stokes to switch between the four different Stokes parameter images that were computed.  
You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do). Also, using the wrench panel to change Display Options will be helpful here. Here we selected the Hot Metal 1 colormap and set the Scaling Power Cycles to -1 to better emphasize the faint emission and compare to the noise (Figures 15A - D). You can also use the Animators slider for Stokes to switch between the four different Stokes parameter images that were computed.  


The {{tclean}} task naturally operates in a flat noise image, i.e., an image where the effective weighting across the field of view is set so that the noise is constant. This is so that the clean threshold has a uniform meaning for the stopping criterion and that the image fed into the minor cycles has uniform noise levels. This means, however, that the image does not take into account the primary beam fall-off in the edges. We could have set parameter ''pbcor=True'' in {{tclean}}, but it is useful to see the flat-noise image and residuals to evaluate the quality of the clean image. Therefore, we use {{impbcor}} to divide the ''.image'' by the ''.pb'' image to produce a primary beam corrected restored image:
The [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] task naturally operates in a flat noise image, i.e., an image where the effective weighting across the field of view is set so that the noise is constant. This is so that the clean threshold has a uniform meaning for the stopping criterion and that the image fed into the minor cycles has uniform noise levels. This means, however, that the image does not take into account the primary beam response fall-off in the edges. In principle, [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] produces primary beam response image, and if we had set parameter ''pbcor=True'' [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] would had saved a primary beam corrected restored image of our target. Since we used ''deconvolver='mtmfs' '' and ''nterms=2'', the calculation of the primary beam response requires special treatment. To perform wideband primary beam correction, we will use task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.widebandpbcor.html widebandpbcor]. In the future this task will be incorporated into [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean], but until then this separate task needs to be used.
<source lang="python">
<source lang="python">
# In CASA
# In CASA
impbcor(imagename='3C75_initial.image.tt0',pbimage='3C75_initial.pb.tt0',
widebandpbcor(vis='3C75.ms,'imagename='3C75_initial',nterms=2, action='pbcor'
        outfile='3C75_initial.pbcorimage')
              spwlist=[0,1,2,3,4,5,6,7], chanlist=[32,32,32,32,32,32,32,32], weightlist=[1,1,1,1,1,1,1,1])
</source>
</source>


You can open this in the {{viewer}} and see that it has indeed raised the noise (and signal) at the edges of the image.
The task will produce primary beam corrected images of our target (3C75_initial.pbcor.image.tt0, 3C75_initial.pbcor.image.tt1, 3C75_initial.pbcor.image.alpha, 3C75_initial.pbcor.image.alpha.error). You can open image 3C75_initial.pbcor.image.tt0 in the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.imview.html imview], and compare it to screenshots in Figure 15. You will see noise (and signal) at the edges of the image has indeed increased.


== Self-Calibration ==
== Self-Calibration ==


Before we get started with self-calibration, it might be good to check whether we need to perform some additional flagging on the target data. Since we have established an image model in the previous section, we can use it to look at the residuals by dividing out the model. We can make a similar plot to Figure 11 above, however, we will divide the image model that was created. Since we performed full-polarization imaging, we can also do the same to the cross-hand data RL,LR. Figures 16A & B shows example plots. You should also have a look at time plotted against amplitude and frequency against amplitude to see if there are any obvious times of interference.
Before we get started with self-calibration, it might be good to check whether we need to perform additional flagging on the target data. Since we have established an image model in the previous section, we can use it to look at the residuals by dividing out the model. We can make a similar plot to Figure 11 above, however, we will divide the image model that was created. Since we performed full-polarization imaging, we can also do the same to the cross-hand data RL,LR. Figures 16A & B shows example plots. You should also have a look at time plotted against amplitude and frequency against amplitude to see if there are any obvious times of interference.


<source lang="python">
<source lang="python">
Line 1,191: Line 1,197:


{|
{|
  | [[Image:plotms_3c75_uvdist_resid_RR_CASA6.1.2.png|thumb|Figure 16A: plotms plot showing Amplitude vs UV Distance residuals in wavelengths for 3C75 for RR correlations.]]
  | [[Image:plotms-3c75-uvdist-residRR-CASA6.2.1-corr.png|thumb|Figure 16A: plotms plot showing Amplitude vs UV Distance residuals in wavelengths for 3C75 for RR correlations.]]
  | [[Image:plotms_3c75_uvdist_resid_RL_CASA6.1.2.png|thumb|Figure 16B: plotms plot showing Amplitude vs UV Distance residuals in wavelengths for 3C75 for RL correlations.]]
  | [[Image:plotms-3c75-uvdist-residRL-CASA6.2.1-corr.png|thumb|Figure 16B: plotms plot showing Amplitude vs UV Distance residuals in wavelengths for 3C75 for RL correlations.]]
|}
|}


Line 1,213: Line 1,219:
</source>
</source>


This should have gotten rid of the worst remaining outliers, but will leave some residual weak RFI on certain baseline lengths. Since we are not trying to win any records on high dynamic range imaging this additional flagging should suffice for this dataset.   
This should have gotten rid of the worst remaining outliers, but will leave some residual weak RFI on certain baseline lengths. Since we are not trying to win any records on high dynamic range imaging, this additional flagging should suffice for our dataset.   


'''In addition to residual RFI, even after calibration using the amplitude calibrator and the phase calibrator, there are likely to be residual phase and/or amplitude errors in the data. Self-calibration uses an existing model, often constructed from imaging the data itself, provided that sufficient visibility data have been obtained.''' This is essentially always the case with data: the system of equations is wildly over-constrained for the number of unknowns.   
'''In addition to residual RFI, even after calibration using the amplitude calibrator and the phase calibrator, there are likely to be residual phase and/or amplitude errors in the data. Self-calibration uses an existing model, often constructed from imaging the data itself, provided that sufficient visibility data have been obtained.''' This is essentially always the case with data: the system of equations is wildly over-constrained for the number of unknowns.   


More specifically, the observed visibility data on the <math>i</math>-<math>j</math> baseline can be modeled as  
More specifically, the observed visibility data on the <math>i</math>-<math>j</math> baseline can be modeled as:


<math>
<math>
Line 1,223: Line 1,229:
</math>
</math>


where <math>G_i</math> is the complex gain for the <math>i^{\mathrm{th}}</math> antenna and <math>V_{ij}</math> is the true visibility.  For an array of <math>N</math> antennas, at any given instant, there are <math>N(N-1)/2</math> visibility data, but only <math>N</math> gain factors. For an array with a reasonable number of antennas, <math>N</math> >~ 8, solutions to this set of coupled equations converge quickly.
where <math>G_i</math> is the complex gain for the <math>i^{\mathrm{th}}</math> antenna and <math>V_{ij}</math> is the true visibility.  For an array of <math>N</math> antennas, at any given instant, there are <math>N(N-1)/2</math> visibility data, but only <math>N</math> gain factors. For an array with a reasonable number of antennas, <math>N</math> >~ 8, solutions to this set of coupled equations converge quickly. There is some discussion in the old CASA Reference Manual on [http://casa.nrao.edu/docs/cookbook/casa_cookbook006.html#sec355 self calibration] (see Section 5.11), but more detailed discussion can be found in lectures on [https://science.nrao.edu/facilities/alma/naasc-workshops/nrao-cd-stsci/cde_selfcal.pdf Self-calibration] given at NRAO community days.  


There is a small amount of discussion in the old CASA Reference Manual on
In self-calibrating data, it is useful to keep in mind the structure of a Measurement Set. There are three columns of interest for an MS: the DATA column, the MODEL column, and the CORRECTED_DATA column. In normal usage, as part of the initial split, the CORRECTED_DATA column is set equal to the DATA column. The self-calibration procedure is then:  
[http://casa.nrao.edu/docs/cookbook/casa_cookbook006.html#sec355 self calibration] (see Section 5.11), but we have lectures on [https://science.nrao.edu/facilities/alma/naasc-workshops/nrao-cd-stsci/cde_selfcal.pdf Self-calibration] given at NRAO community days. In self-calibrating data, it is useful to keep in mind the structure of a Measurement Set. There are three columns of interest for an MS: the DATA column, the MODEL column, and the CORRECTED_DATA column. In normal usage, as part of the initial split, the CORRECTED_DATA column is set equal to the DATA column. The self-calibration procedure is then:  


* Produce an image ({{tclean}}) using the CORRECTED_DATA column.
* Produce an image ([https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean]) using the CORRECTED_DATA column.
* Derive a series of gain corrections ({{gaincal}}) by comparing the DATA columns and the Fourier transform of the image, which is stored in the MODEL column. These corrections are stored in an external table.
* Derive a series of gain corrections ([https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.gaincal.html gaincal]) by comparing the DATA columns and the Fourier transform of the image, which is stored in the MODEL column. These corrections are stored in an external table.
** Optionally, we can also derive a bandpass correction&mdash;which is also referred to as bandpass self calibration&mdash;to correct for global amplitude errors.
** Optionally, we can also derive a bandpass correction&mdash;which is also referred to as bandpass self calibration&mdash;to correct for global amplitude errors.
* Apply these corrections ({{applycal}}) to the DATA column to form a new CORRECTED_DATA column ''overwriting'' the previous contents of CORRECTED_DATA.
* Apply these corrections ([https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal]) to the DATA column to form a new CORRECTED_DATA column ''overwriting'' the previous contents of CORRECTED_DATA.


The following example begins with the standard data set, 3C75.ms (resulting from the steps above). We have previously generated an IQUV multiscale image cube. We discard it for this step and generate a new Stokes I image, which we will use to generate a series of gain corrections (phase only self-calibration) that will be stored in 3C75.ScG0. With this solution, we then perform bandpass self-calibration to remove any amplitude slope that might be present and apply the derived phase and amplitude corrections to the data to form a set of self-calibrated data and a new image is then formed (3C75_selfcal.image). For the purpose of self-calibration, note that in the clean before the self-cal, it is important that we only use the Stokes I model so that any cleaned polarization do not affect the gaincal. We first use {{delmod}} on the MS to get rid of the previous polarized model and then run tclean to generate the Stokes I image. In principle, it is possible to use the previous image cube and extract the Stokes I model using the CASA toolkit and have {{tclean}} fill the model column appropriately. However, for simplicity we just re-image with {{tclean}} and selecting only Stokes I.
The following example begins with the standard data set, 3C75.ms (resulting from the steps above). We have previously generated an IQUV multiscale image cube. We discard it for this step and create a new Stokes I image, which we will use to generate a series of gain corrections (phase only self-calibration) that will be stored in 3C75.ScG0. With this solution, we then perform bandpass self-calibration to remove any amplitude slope that might be present. Next, we apply the derived phase and amplitude corrections to the data to form a set of self-calibrated data, and then re-image the dataset (3C75_selfcal.image). For the purpose of self-calibration, note that in the clean before the self-calibration, it is important that we only use the Stokes I model so that any cleaned polarization does not affect the gaincal. We first use [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.delmod.html?highlight=delmod# delmod] on the MS to get rid of the previous polarized model, and run [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] to generate Stokes I-only image. In principle, it is possible to use the previous image cube and extract the Stokes I model using the CASA toolkit and have [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] fill the model column appropriately. For simplicity, we just re-image with [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] selecting only Stokes I.
<source lang="python">
<source lang="python">
#In CASA
#In CASA
Line 1,277: Line 1,282:
       npixels=0,
       npixels=0,
       uvtaper=[],
       uvtaper=[],
       niter=20000,
       niter=3500,
       gain=0.1,
       gain=0.1,
       threshold=0.0,
       threshold=0.0,
       nsigma=0.0,
       nsigma=0.0,
       cycleniter=1000,
       cycleniter=750,
       cyclefactor=1.0,
       cyclefactor=1.0,
       restart=True,
       restart=True,
Line 1,290: Line 1,295:
       interactive=True)
       interactive=True)
</source>
</source>
This {{tclean}} call will only fill the model column with the Stokes I model and ignore the polarized structure. You should not clean very deeply at this point. You want to be sure to capture as much of the source total flux density as possible, but not include low-level questionable features or sub-structure (ripples) that might be due to calibration or clean artifacts.  
As discussed, this [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] call will ignore the polarized structure. You should not clean very deeply at this point. You want to be sure to capture as much of the source's total flux density as possible, but not include low level questionable features or sub-structures (ripples) that might be due to calibration or deconvolution artifacts. We modified the two parameters controlling [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean]'s minor and major cycles to the following values ''cycleniter=750'' and ''niter=3500'' to reflect this, but you may find that you don't even need 3500 iterations for this first [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] pass.


After you are happy with the image:
If you are happy with the new image, perform the following self-calibration steps:
<source lang="python">
<source lang="python">
#In CASA
#In CASA
Line 1,300: Line 1,305:


bandpass(vis='3C75.ms', caltable='3C75.ScB0', field='', solint='inf', refant='ea10', minsnr=3.0, spw='',
bandpass(vis='3C75.ms', caltable='3C75.ScB0', field='', solint='inf', refant='ea10', minsnr=3.0, spw='',
                 parang = False, gaintable=['3C75.ScG0'],
                 parang = False, gaintable=['3C75.ScG0'], interp=[])
                interp=[])


applycal(vis='3C75.ms', gaintable=['3C75.ScG0','3C75.ScB0'], spw='', applymode='calflagstrict', parang=False)
applycal(vis='3C75.ms', gaintable=['3C75.ScG0','3C75.ScB0'], spw='', applymode='calflagstrict', parang=False)
</source>
</source>
The ''CORRECTED_DATA'' column of the MS now contains the self-calibrated visibilities which will now be used by {{tclean}}. The  
The ''CORRECTED_DATA'' column of the MS now contains the self-calibrated visibilities which will be used by next execution of [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean]. The [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.gaincal.html gaincal] step will report a number of solutions with insufficient SNR. By default, with parameter ''applymode='calflag','' data with no good solutions will be flagged by [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal] which may or may not be a good thing. You can control the action of [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal] by changing the value of parameter ''applymode''. Setting ''applymode='calflagstrict' ''will be more stringent about flagging data points without valid calibration, while ''applymode='calonly'  '' will calibrate those with solutions while passing unchanged the data without solutions. You can see ahead of time what applycal will do by executing it with ''applymode='trial' ''which will do the reporting but nothing else. In our example we used '' applymode='calflagstrict' '', but you will notice that the reported flagged fraction has not changed much, only increasing by 0.5%. This is a good thing.
{{gaincal}} step will report a number of solutions with insufficient SNR. By default, with parameter ''applymode='calflag','' data with no good solutions will be flagged by {{applycal}}; in this case you will see it report the flagged fraction increasing to about 45%. This may or may not be a good thing. You can control the action of {{applycal}} in this regard by changing the value of parameter ''applymode''. The setting of ''applymode='calflagstrict' ''will be more stringent about flagging things without valid calibration, while ''applymode='calonly'  ''will calibrate those with solutions while passing through data without unchanged. You can see ahead of time what applycal will do by running with ''applymode='trial' ''which will do the reporting but nothing else.


Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently a better image. We do this by invoking the {{tclean}} command once again.  
Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently produce a better image. We do this by invoking the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] command once again.  


<source lang="python">
<source lang="python">
Line 1,351: Line 1,354:
       npixels=0,
       npixels=0,
       uvtaper=[],
       uvtaper=[],
       niter=20000,
       niter=3500,
       gain=0.1,
       gain=0.1,
       threshold=0.0,
       threshold=0.0,
       nsigma=0.0,
       nsigma=0.0,
       cycleniter=1000,
       cycleniter=750,
       cyclefactor=1.0,
       cyclefactor=1.0,
       restart=True,
       restart=True,
Line 1,365: Line 1,368:
</source>
</source>


Commonly, this self-cal procedure is applied multiple times. In Figures 17A & B you see a comparison of the Stokes I image before self-calibration and after two self-calibration steps.  
Commonly this self-calibration procedure is applied multiple times. In Figures 17A & B you can see a comparison of the shallow Stokes I image before self-calibration and after two self-calibration steps. The first self-calibration round was done as instructed in this section, while the second round was executed with ''solint='120s' '' and new solution tables were created (3C75.ScG1, 3C75.ScB1).
{|
{|
  | [[Image:3c75_initial_I_CASA6.1.2.jpeg|thumb|Figure 17A: Stokes I image before self-calibration.]]
  | [[Image:3c75-initial_I-CASA6.2.1.png|thumb|Figure 17A: Shallow Stokes I image before self-calibration.]]
  | [[Image:3c75_2selfcal_I_CASA6.1.2.jpeg|thumb|Figure 17B: Stokes I image after two rounds of self-calibration.]]
  | [[Image:3c75-selfcal_2-CASA6.2.1.png|thumb|Figure 17B: Stokes I image after two rounds of self-calibration.]]
|}  
|}  


The number of iterations is determined by a combination of the data quality, the number of antennas in the array, the structure of the source, the extent to which the original self-calibration assumptions are valid, and the user's patience. With reference to the original self-calibration equation above, if the observed visibility data cannot be modeled well by this equation, no amount of self-calibration will help. A not-uncommon limitation for moderately high dynamic range imaging is that there may be ''baseline-based'' factors that modify the true visibility. If the corruptions to the true visibility cannot be modeled as antenna-based, as they are above, self-calibration won't help.
The number of iterations is determined by a combination of the data quality, the number of antennas in the array, the structure of the source, the extent to which the original self-calibration assumptions are valid, and the user's patience. With reference to the original self-calibration equation above, if the observed visibility data cannot be modeled well by this equation, no amount of self-calibration will help. A not uncommon limitation for moderately high dynamic range imaging is that there may be ''baseline-based'' factors that modify the true visibility. If the corruptions to the true visibility cannot be modeled as antenna-based, as they are above, self-calibration won't help.


Self-calibration requires experimentation. Do not be afraid to dump an image, or even a set of gain corrections, change something and try again.  Having said that, here are several general comments or guidelines:
Self-calibration requires experimentation. Do not be afraid to remove an image, or even a set of gain corrections, change something and try again.  Having said that, here are several guidelines to consider:


* Bookkeeping is important! Suppose one conducts 9 iterations of self-calibration. Will it be possible to remember one month later (or maybe even one week later!) which set of gain corrections and images are which? In the example above, the descriptor'' 'selfcal1' ''is attached to various files to help keep straight which is what. Successive iterations of self-cal could then be'' 'selfcal2' '','' 'selfcal3' '', etc.
* Bookkeeping is important! Suppose one conducts 9 iterations of self-calibration. Will it be possible to remember one month later (or maybe even one week later!) which set of gain corrections and images are which? In the example above, the descriptor'' 'selfcal_1' ''is attached to various files to help keep straight what is what. Successive iterations of self-cal could then be'' 'selfcal_2' '','' 'selfcal_3' '', etc.


* Care is required in the setting of ''imagename''. If one has an image that already exists, CASA will continue cleaning it (if it can), which is almost certainly not what one wants during self-calibration. Rather, use a unique ''imagename'' for each pass of self-calibration.
* Care is required in setting ''imagename''. If one has an image that already exists, CASA will continue cleaning it (if it can), which is almost certainly not what one wants during self-calibration. Rather, use a unique ''imagename'' for each pass of self-calibration.


* A common metric for self-calibration is whether the image ''dynamic range'' (= max/rms) has improved. An improvement of 10% is quite acceptable.
* A common metric for self-calibration is whether the ''dynamic range'' (= peak flux density/rms) of the image has improved. An improvement of 10% is quite acceptable.


* Be careful when making images and setting clean regions or masks; self-calibration assumes that the model is perfect. If one cleans a noise bump, self-calibration will quite happily try to adjust the gains so that the CORRECTED_DATA describe a source at the location of the noise bump. It is far better to exclude some feature of a source or a weak source from initial cleaning and conduct another round of self-calibration than to create an artificial source. If a real source is excluded from initial cleaning, it will continue to be present in subsequent iterations of self-calibration; if it's not a real source, one probably isn't interested in it anyway.
* Be careful when making images and setting clean regions or masks; self-calibration assumes that the model is perfect. If one cleans a noise bump, self-calibration will quite happily try to adjust the gains so that the CORRECTED_DATA describe a source at the location of the noise bump. It is far better to exclude some features of a source, or a weak source, from initial cleaning and conduct another round of self-calibration than to create an artificial source. If a real source is excluded from initial cleaning, it will continue to be present in subsequent iterations of self-calibration; if it's not a real source, one probably isn't interested in it anyway.


* Start self-calibration with phase-only solutions (parameter ''calmode='p' ''in {{gaincal}}). As discussed in the [http://adsabs.harvard.edu/abs/1989ASPC....6..287P High Dynamic Range Imaging] lecture, a phase error of 20 deg is as bad as an amplitude error of 10%.
* Start self-calibration with phase-only solutions (parameter ''calmode='p' ''in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.gaincal.html gaincal]). As discussed in the [http://adsabs.harvard.edu/abs/1989ASPC....6..287P High Dynamic Range Imaging] lecture, a phase error of 20 deg is as bad as an amplitude error of 10%.


* In initial rounds of self-calibration, consider solution intervals longer than the nominal sampling time (parameter ''solint'' in {{gaincal}}) and/or lower signal-to-noise ratio thresholds (parameter ''minsnr'' in {{gaincal}}). Depending upon the frequency and configuration and fidelity of the model image, it can be quite reasonable to start with ''solint='30s' ''or'' solint='60s' ''and/or ''minsnr=3 (''or even lower). One might also want to consider specifying a uvrange, if, for example, the field has structure on large scales (small <math>u</math>-<math>v</math>) that is not well represented by the current image.
* In initial rounds of self-calibration, consider solution intervals longer than the nominal sampling time (parameter ''solint'' in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.gaincal.html gaincal]) and/or lower signal-to-noise ratio thresholds (parameter ''minsnr'' in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.gaincal.html gaincal]). Depending upon the frequency and configuration and fidelity of the model image, it can be quite reasonable to start with ''solint='30s' ''or'' solint='60s' ''and/or ''minsnr=3''. One may also want to consider specifying a uvrange, if, for example, the field has structure on large scales (small <math>u</math>-<math>v</math>) that is not well represented by the current image.


* The task {{applycal}} will flag data with no good calibration solutions. During the initial self-calibration steps, this flagging may be excessive. If so, one can restore the flags to the state right before running applycal by using the task '''[https://casa.nrao.edu/casadocs/latest/data-examination-and-editing/managing-flag-versions-flagmanager flagmanager]'''.
* The task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.applycal.html applycal] will flag data with no good calibration solutions. During the initial self-calibration steps, this flagging may be excessive. If so, one can restore the flags to the state right before running applycal by using the task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.flagging.flagmanager.html flagmanager].


* You can track the agreement between the DATA, CORRECTED_DATA, and MODEL in {{plotms}}. The options in Axes tab allows one to select which column is to be plotted. If the MODEL agrees well with the CORRECTED_DATA, one can use shorter solint and/or higher minsnr values.
* You can track the agreement between the DATA, CORRECTED_DATA, and MODEL in [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms]. The options in Axes tab allows one to select which column is to be plotted. If the MODEL agrees well with the CORRECTED_DATA, one can use shorter solint and/or higher minsnr values.


* You should consider examining the solutions from {{gaincal}} by using {{plotcal}} in order to assure that the corrections are sensible. Smoothly varying phases are good, jumps are usually not.  (However, because the phases are often plotted &plusmn;180 degrees, there can be apparent jumps if the phases are very near &#043;180 deg or &minus;180 deg.)
* You should consider examining the solutions from [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.calibration.gaincal.html gaincal] by using [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.plotms.html plotms] in order to assure that the corrections are sensible. Smoothly varying phases are good, jumps are usually not.  (However, because the phases are often plotted &plusmn;180 degrees, there can be apparent jumps if the phases are very near &#043;180 deg or &minus;180 deg.)


== Final Polarization Images ==
== Final Polarization Images ==


At this point, satisfied with the results of self-calibration, it might be a good idea to recalculate the visibility weights since some additional flagging was performed. After this, we get right to full-polarization imaging. We also suspect that there is a bright source outside of the masked field causing some imaging artifacts due to not being cleaned. We thus set the parameter ''pbmask'' value to 0.0 in order to disable masking of areas beyond the primary beam.
At this point, satisfied with the results of self-calibration, it might be a good idea to recalculate the visibility weights since some additional flagging was performed. After this, we get right to full-polarization imaging. We also suspect that there is a bright source outside of the masked field causing some imaging artifacts due to not being cleaned. We thus set the parameter ''pbmask'' value to 0.0 in order to disable masking of areas beyond the primary beam, and make the image larger to incorporate the bright source into our model in this tclean execution.


<source lang="python">
<source lang="python">
Line 1,406: Line 1,409:
       uvrange="",antenna="",scan="",observation="",intent="",
       uvrange="",antenna="",scan="",observation="",intent="",
       datacolumn="corrected",
       datacolumn="corrected",
       imagename="3C75_final",
       imagename="3C75_final_large",
       imsize=1024,
       imsize=800,
       cell="3.4arcsec",
       cell="3.4arcsec",
       phasecenter="",
       phasecenter="",
Line 1,455: Line 1,458:
</source>
</source>


The final restored Stokes I,Q,U, and V images are shown in Figures 18A&ndash;D. Note that there is a star like pattern in the residuals which are artifacts most likely due to the multi-scale multi-term multi-frequency synthesis. You can try on your own to improve upon the shown images by re-imaging and choosing a different set of multi-scale parameters that better match the scales found in the extended structure of 3C 75. Another issue to point out is looking at the Stokes V image. We do not expect a significant amount of Stokes V emission from this object, the emission you are seeing in Stokes V is most likely an effect of incorrectly solving for polarization leakages in the primary beam. In the above calibration we have only addressed leakage between the two polarization referring to the phase center. The extended beam itself, however, shows leakage which manifests itself spatially. Thus the extended polarized emission we see in the Stokes Q and U images is not corrected for beam polarization during imaging. This, in turn, contains errors leading to polarization and de-polarization effects and causes changes to the polarization angle which effect increases the further away once gets from the beam center. In addition, the two polarization beams do not sit ontop of each other but are slightly offset, introducing a polarization squint. For correct and accurate polarization imaging, these two effects have to be taken into account. Imaging algorithms to address beam polarization are currently under development and will be discussed in this guide when they become available to the general user.   
The final restored Stokes I,Q,U, and V images are shown in Figures 18A&ndash;D. Note that there is still a star like pattern in the residuals which are artifacts most likely due to the multi-scale multi-term multi-frequency synthesis. You can try on your own to improve upon the shown images by re-imaging and choosing a different set of multi-scale parameters that better match the scales found in the extended structure of 3C 75. Another issue to point out is looking at the Stokes V image. We do not expect a significant amount of Stokes V emission from this object, the emission you are seeing in Stokes V is most likely an effect of incorrectly solving for polarization leakages in the primary beam. In the above calibration we have only addressed leakage between the two polarization referring to the phase center. The extended beam itself, however, shows leakage which manifests itself spatially. The extended polarized emission we see in the Stokes Q and U images is not corrected for beam polarization during imaging. This, in turn, contains errors leading to polarization and de-polarization effects and causes changes to the polarization angle which effect increases the further away once gets from the beam center. Additionally, the two polarization beams do not sit on top of each other but are slightly offset, introducing a polarization squint. For correct and accurate polarization imaging, these two effects have to be taken into account. Imaging algorithms to address beam polarization are currently under development and will be discussed in this guide when they become available to the general user.   


{|
{|
  | [[Image:3c75_final_I_CASA6.1.2.jpeg|thumb|Figure 18A: Viewer panel of final restored Stokes I image.]]
  | [[Image:3C75-final-I-CASA6.2.1.png|thumb|Figure 18A: Viewer panel of final restored Stokes I image.]]
  | [[Image:3c75_final_Q_CASA6.1.2.jpeg|thumb|Figure 18B: Viewer panel of final restored Stokes Q image.]]
  | [[Image:3C75-final-Q-CASA6.2.1.png|thumb|Figure 18B: Viewer panel of final restored Stokes Q image.]]
  | [[Image:3c75_final_U_CASA6.1.2.jpeg|thumb|Figure 18C: Viewer panel of final restored Stokes U image.]]
  | [[Image:3C75-final-U-CASA6.2.1.png|thumb|Figure 18C: Viewer panel of final restored Stokes U image.]]
  | [[Image:3c75_final_V_CASA6.1.2.jpeg|thumb|Figure 18D: Viewer panel of final restored Stokes V image.]]
  | [[Image:3C75-final-V-CASA6.2.1.png|thumb|Figure 18D: Viewer panel of final restored Stokes V image.]]
|}
|}
Note, that these images are not yet primary beam corrected.


=== Spectral & Polarization Maps ===
=== Spectral & Polarization Maps ===


If you want to obtain a reasonable map of the in-band spectral index, like the one shown in Fig. 19A, we can compute it with the task {{widebandpbcor}}.
If you want to obtain a reasonable map of the in-band spectral index, like the one shown in Fig. 19A, we can compute it with the task [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.widebandpbcor.html widebandpbcor]. As demonstrated earlier, the task can also correct the images for the telescope's primary beam response corrected; this correction will make the images science ready. Parameter ''action='pbcor' '' will perform both actions (correct for the primary beam and calculate spectral index map) while parameter ''threshold'' sets minimum flux density above which the spectra index is calculated (this will allow us to mask all the noise regions).
 
<source lang="python">
<source lang="python">
widebandpbcor(vis='3C75.ms',imagename='3C75_final',nterms=2,action='calcalpha', threshold = '0.5mJy/beam')
# In CASA
widebandpbcor(vis='3C75.ms, 'imagename='3C75_final_large', nterms=2, threshold='1.0mJy/beam', action='pbcor'
              spwlist=[0,1,2,3,4,5,6,7], chanlist=[32,32,32,32,32,32,32,32], weightlist=[1,1,1,1,1,1,1,1])
</source>
</source>


For further study of polarization properties, you might want to convert the Stokes images into something more useful for scientific analysis. We will use the CASA to calculate polarization intensity (sqrt(Q^2 + U^2)/I) and polarization angle (0.5 arctan2 (U/Q)) maps from the final Stokes I,Q,U images. You can then look at those with the {{viewer}}. For example, Figure 19B shows the polarization intensity image. Since we haven't applied any mask the polarization angle image will also contain values for low S/N or noise values.   
 
For further study of polarization properties, you might want to convert the Stokes images into something more useful for scientific analysis. We will use CASA to calculate polarization intensity (sqrt(Q^2 + U^2)/I) and polarization angle (0.5 arctan2 (U/Q)) maps from the final Stokes I,Q,U images. You can then look at those with the [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.imview.html imview]. For example, Figure 19B shows the polarization intensity image. Since we haven't applied any mask the polarization angle image will also contain values for low S/N or noise values.   
<source lang="python">
<source lang="python">
# In CASA
# In CASA


# Obtain image for the polarization intensity
# Obtain image for the polarization intensity
immath(outfile='3C75_final.poli',mode='poli',imagename=['3C75_final.image.tt0'],sigma='0.0Jy/beam')
immath(outfile='3C75_final.poli',mode='poli',imagename=['3C75_final_large.image.tt0'],sigma='0.0Jy/beam')
# Obtain image for the polarization angle
# Obtain image for the polarization angle
immath(outfile='3C75_final.pola',mode='pola',imagename=['3C75_final.image.tt0'],sigma='0.0Jy/beam')
immath(outfile='3C75_final.pola',mode='pola',imagename=['3C75_final_large.image.tt0'],sigma='0.0Jy/beam')
</source>
</source>


{|
{|
  | [[Image:3c75_final_spix_CASA6.1.2.jpeg|thumb|Figure 19A: Computed spectral index map.]]
  | [[Image:3C75-final-spindx-CASA6.2.1.png|thumb|Figure 19A: Computed spectral index map.]]
  | [[Image:3c75_final_Ptot_CASA6.1.2.jpeg|thumb|Figure 19B: Computed polarization intensity image.]]
  | [[Image:3C75-final-Ptot-casa6.2.1.png|thumb|Figure 19B: Computed polarization intensity image.]]
  | [[Image:3c75_final_PAng_CASA6.1.2.jpeg|thumb|Figure 19C: Computed polarized angles as vectors ontop of the Stokes I raster image plane.]]
  | [[Image:3C75-final-PAng-CASA6.2.1.png|thumb|Figure 19C: Computed polarized angles (vectors) superposed on the Stokes I raster image plane.]]
|}
|}


If you want to visualize the polarization vectors ontop of the Stokes I image, we need to add a mask for the low noise values.  
Note that for calculations of the polarization intensity and angle images you may &mdash; but do not need to &mdash; use primary beam corrected images; your results will be the same. This is because the primary beam correction cancels out in the equations for these two polarization quantities. If you want to visualize the polarization vectors on top of the Stokes I image, we need to add a mask for the low noise values.  
 
<source lang="python">
<source lang="python">
# In CASA
# In CASA
!cp -rif '3C75_final.poli' polimg
!cp -r '3C75_final.poli' polimg


imsubimage(imagename='3C75_final.image.tt0',outfile='3C75_final.Q.image',stokes='Q')
imsubimage(imagename='3C75_final_large.image.tt0',outfile='3C75_final.Q.image',stokes='Q')
imsubimage(imagename='3C75_final.image.tt0',outfile='3C75_final.U.image',stokes='U')
imsubimage(imagename='3C75_final_large.image.tt0',outfile='3C75_final.U.image',stokes='U')


subimPI='polimg'
subimPI='polimg'
ia.open(subimPI)
ia.open(subimPI)
ia.calcmask(mask=subimPI+'>1e-4',name='mymask')
ia.calcmask(mask=subimPI+'>5e-4',name='mymask')
ia.done()
ia.done()


Line 1,512: Line 1,522:
immath(imagename=['3C75_final.Q.image', '3C75_final.U.image'], mode='pola', outfile='3C75_final.pola.masked')
immath(imagename=['3C75_final.Q.image', '3C75_final.U.image'], mode='pola', outfile='3C75_final.pola.masked')
</source>
</source>
These steps take the polarized intensity image calculated above (Figure 19B) and create a mask using a polarization fraction threshold of 1e-4 (0.01% linear polarization fraction). This mask is then applied to the Q and U images from the image cube that was generated above. Then a new polarization angle image is calculated from the Q & U image planes, applying the mask based on polarization fraction. Finally, we can load the Stokes I as raster image into the CASA {{viewer}} and add the polarization angle as vectors. Figure 19C shows the resulting image. One can clearly see that the linear polarization angle follows perpendicular to the extended structure. This indicates that the magnetic field lines are oriented along the extended structure, perpendicular to the linear polarization angles.
These steps take the polarized intensity image calculated above (Figure 19B) and create a mask using a polarization fraction threshold of 5e-4 (0.05% linear polarization fraction). This mask is then applied to the Q and U images from the image cube that was generated above. Then a new polarization angle image is calculated from the Q & U image planes, applying the mask based on polarization fraction. Finally, we can load the Stokes I as raster image into the CASA [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.visualization.imview.html imview] and add the polarization angle as vectors. Figure 19C shows the resulting image. One can clearly see that the linear polarization angle follows perpendicular to the extended structure. This indicates that the magnetic field lines are oriented along the extended structure, perpendicular to the linear polarization angles.


=== Rotation Measures ===
=== Rotation Measures ===


The plane of polarization of light is rotated by the magnetic fields present in the intervening plasma. The total rotation
The plane of polarization of light is rotated by the magnetic fields present in the intervening plasma. The total rotation to the plane of polarization of light between the source and the user is called Faraday Rotation. Prior to the wide bandwidth capabilities, these rotation measures were computed by fitting a line to the polarization position angle as a function of the square of the wavelength of measurement. The slope of the resulting fit was deemed to be the RM of the source while the intercept would give the true polarization position angle of the source. With the wide bandwidths, it is now possible to determine the rotation measure of the source using the naive fitting approach by making images per spectral window in IQUV and fitting the data (polarization position angle vs lambda^2) with a line.  
to the plane of polarization of light between the source and the user is called Faraday Rotation. Prior to the wide bandwidth
capabilities, these rotation measures were computed by fitting a line to the polarization position angle as a function of the
square of the wavelength of measurement. The slope of the resulting fit was deemed to be the RM of the source while
the intercept would give the true polarization position angle of the source. With the wide bandwidths, it is now possible to
determine the rotation measure of the source using the naive fitting approach by making images per spectral window in IQUV
and fitting the data (polarization position angle vs lambda^2) with a line.  


To produce an image cube with 8 channels, each image is using 128 MHz of bandwidth, we call {{tclean}} with the following parameters. Here we take advantage of the imaging mask we generated for the final image above, so we don't need to do an interactive clean.
To produce an image cube with 8 channels, each image is using 128 MHz of bandwidth, we call [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] with the following parameters. Here we take advantage of the imaging mask we generated for the final image above, so we don't need to do an interactive clean.
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 1,533: Line 1,537:
       datacolumn="corrected",
       datacolumn="corrected",
       imagename="3C75_chan8",
       imagename="3C75_chan8",
       imsize=1024,
       imsize=800,
       cell="3.4arcsec",
       cell="3.4arcsec",
       phasecenter="",
       phasecenter="",
Line 1,554: Line 1,558:
       pblimit=-0.0001,
       pblimit=-0.0001,
       pbmask=0.0,
       pbmask=0.0,
       mask='3C75_final.mask',
       mask='3C75_final_large.mask',
       normtype="flatnoise",
       normtype="flatnoise",
       deconvolver="multiscale",
       deconvolver="multiscale",
Line 1,582: Line 1,586:
</source>
</source>


Now we use the CASA toolkit to access data for four pixels in the image cube to visualize and fit the rotation measure. The script is also available here: [[File:Linregfit 6.1.2.py]].
Now we use the CASA toolkit to access data for four pixels in the image cube to visualize and fit the rotation measure.  
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 1,592: Line 1,596:


ia.open('3C75_chan8.image')
ia.open('3C75_chan8.image')
# number of channels/frequencies
nunr = 8


tt = ia.getchunk()
tt = ia.getchunk()
nu = np.linspace(2.551e9,3.319e9,num=8)
nu = np.linspace(2.551e9,3.319e9,num=nunr)
c = 2.99792458e8
c = 2.99792458e8


Q1 = tt[488,531,1,:8]
Q1 = tt[418,444,1,:nunr]
U1 = tt[488,531,2,:8]
U1 = tt[418,444,2,:nunr]
Q2 = tt[494,511,1,:8]
Q2 = tt[376,419,1,:nunr]
U2 = tt[494,511,2,:8]
U2 = tt[376,419,2,:nunr]
Q3 = tt[529,551,1,:8]
Q3 = tt[383,401,1,:nunr]
U3 = tt[529,551,2,:8]
U3 = tt[383,401,2,:nunr]
Q4 = tt[525,534,1,:8]
Q4 = tt[395,398,1,:nunr]
U4 = tt[525,534,2,:8]
U4 = tt[395,398,2,:nunr]


chi1 = 0.5*np.arctan2(U1,Q1)
chi1 = 0.5*np.arctan2(U1,Q1)
Line 1,659: Line 1,666:
plt.title('Overall Title')
plt.title('Overall Title')
plt.subplot(221)
plt.subplot(221)
plt.title('Point 1: (488,531)')
plt.title('Point 1: (418,444)')
plt.xlabel(r'$\lambda^{2}$')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi1$')
plt.ylabel(r'$\chi1$')
Line 1,667: Line 1,674:


plt.subplot(222)
plt.subplot(222)
plt.title('Point 2: (494,511)')
plt.title('Point 2: (376,419)')
plt.xlabel(r'$\lambda^{2}$')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi2$')
plt.ylabel(r'$\chi2$')
Line 1,675: Line 1,682:


plt.subplot(223)
plt.subplot(223)
plt.title('Point 3: (529,551)')
plt.title('Point 3: (383,401)')
plt.xlabel(r'$\lambda^{2}$')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi3$')
plt.ylabel(r'$\chi3$')
Line 1,683: Line 1,690:


plt.subplot(224)
plt.subplot(224)
plt.title('Point 4: (525,534)')
plt.title('Point 4: (395,398)')
plt.xlabel(r'$\lambda^{2}$')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi4$')
plt.ylabel(r'$\chi4$')
Line 1,696: Line 1,703:
</source>
</source>


The resulting plots are shown in Figure 20A. There exists a CASA task ''rmfit'' which does this basic
The resulting plots are shown in Figure 20A. Alternatively, there exists a CASA task ''rmfit'' which does this basic fitting for you while taking into account the n \pi ambiguity (refer to [http://adsabs.harvard.edu/full/1986A%26A...156..234L] for more info). The fits using ''rmfit'' for our case of 3C 75 by making images per spectral window is shown in Figure 20B. Here we set the maximum acceptable position angle error to 20 degrees. If larger, then no rotation measures are calculated.
fitting for you while taking into account the n \pi ambiguity (refer to [http://adsabs.harvard.edu/full/1986A%26A...156..234L] for more info).
The fits using ''rmfit'' for our case of 3C75 by making images per spectral window is shown in Figure 20B. In this case, we set the maximum acceptable position angle error to 20 degrees. If larger, then no rotation measures are calculated.
   
   
<source lang='python'>
<source lang='python'>
Line 1,706: Line 1,711:


{|
{|
  | [[Image:3C75_linred_att1 6.1.2.png|thumb|Figure 20A: Rotation measures extracted for 4 pixels from an 8 channel image cube of 3C75.]]
  | [[Image:3C75-rmscript-8chan4px-CASA6.2.1.png|thumb|Figure 20A: Rotation measures extracted for 4 pixels from an 8 channel image cube of 3C75.]]
  | [[Image:3C75_rmfit_8chan 6.1.2.png|thumb|Figure 20B: RMFIT rotation measure image generated from 8 channel image cube.]]
  | [[Image:3C75-rmfit-8chan-casaimg-CASA6.2.1.png|thumb|Figure 20B: RMFIT rotation measure image generated from 8 channel image cube.]]
|}
|}


The ''rmfit'' task has many more options; for example, you are able to provide a foreground rotation measure to subtract. For more information have a look at [https://casa.nrao.edu/casadocs/casa-5.4.0/global-task-list/task_rmfit/about].
The ''rmfit'' task has many more options; for example, you are able to provide a foreground rotation measure to subtract. For more information have a look at [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.analysis.rmfit.html].  
 
Now we can compare the rotation measures extracted for the 4 pixels from the 8 channel image cube with the values derived in the ''rmfit'' for the same pixels. In most cases the values are more or less comparable.
Now we can compare the rotation measures extracted for the 4 pixels from the 8 channel image cube with the values derived in the ''rmfit'' for the same pixels. In most cases the values are more or less comparable.


Line 1,720: Line 1,726:
|-
|-
| 1
| 1
| 29.70
| -37.31
| 26.84
| -31.30
|-
|-
| 2
| 2
| -65.91
| 35.48
| -59.73
| 26.50
|-
|-
| 3
| 3
| -14.4
| -49.65
| -12.53
| -45.14
|-
|-
| 4
| 4
| -42.83
| 17.60
| -34.46
| 17.12
|}
|}


As our source is rather bright, we can derive an IQUV image not just per spectral window but rather per channel. To achieve this you can change the above
As our source is rather bright, we can derive an IQUV image not just per averaged spectral window as we just did, but rather per channel. To achieve this you can change the above [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean] parameter width from 64 to 1, which will result in 512 channels spanning all 8 spectral windows. Note when imaging each channel, the edge channels are flagged which results in the PSF being blank for [C0:P0] [C0:P1] [C0:P2] [C0:P3] [C1:P0] and the first few images being blank. Don't forget to change the imagename parameter when re-running [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.imaging.tclean.html tclean]. Following the same steps as for the 8 channel image cube (you will need to adjust the script for number of channels), you would then obtain the results shown in Figure 21 where again the polarization position angle as a function of lambda square is shown together with the ''rmfit'' image. We can clearly see that the source exhibits complex structure beyond a simple linear fit we performed earlier. This suggests that deriving a single RM would be an oversimplification. We should ideally perform RM Synthesis (https://arxiv.org/pdf/astro-ph/0507349.pdf). At this point in time CASA does not have an RM synthesis task.
{{tclean}} parameter width from 64 to 1. Note when imaging each channel, the edge channels are flagged which results in the PSF being blank for [C0:P0] [C0:P1] [C0:P2] [C0:P3] [C1:P0] and the first few images also being blank. Also, don't forget to change the imagename parameter when re-running {{tclean}}. Following the same steps as for the 8 channel image cube, we obtain the results shown in Figure 21 where again the polarization position angle
as a function of lambda square is shown together with the ''rmfit'' image. We can clearly see that the source exhibits complex structure beyond a simple linear fit (like the one we performed earlier).
This suggests that deriving a single RM would be an oversimplification. We should ideally perform RM Synthesis (https://arxiv.org/pdf/astro-ph/0507349.pdf).
At this point in time CASA does not have an RM synthesis task.




{|
{|
  | [[Image:3C75_channelcube_linereg1 6.1.2.png|thumb|Figure 21A: Rotation measures extracted for 4 pixels from an 512 channel image cube of 3C75.]]
  | [[Image:3C75-rmscript-512chan4px-CASA6.2.1.png|thumb|Figure 21A: Rotation measures extracted for 4 pixels from an 512 channel image cube of 3C75.]]
  | [[Image:3C75_rmfit_512chan 6.1.2.png|thumb|Figure 21B: RMFIT rotation measure image generated from 512 channel image cube.]]
  | [[Image:3C75-rmscript-512chan4px-ylim-CASA6.2.1.png|thumb|Figure 21B: Rotation measures extracted for 4 pixels from an 512 channel image cube of 3C75 with enforced limits on y axis for points 1 and 2 to exclude outlier points from the view and see better the variation of the data.]]
| [[Image:3C75-rmfit-512chan-casaimg-CASA6.2.1.png|thumb|Figure 21C: RMFIT rotation measure image generated from 512 channel image cube.]]
|}
|}


<!-- == Image Analysis ==
<!-- == Image Analysis ==


The three most basic analyses are to determine the peak brightness, the flux density, and the image noise level.  These are useful measures of how well the imaging efforts are in approaching the thermal noise limit or in reproducing what is already known about a source. Additional discussion of image analysis and manipulation, including the combination of multiple images, mathematical operations on images, and much more can be found in the [https://casa.nrao.edu/casadocs/latest/image-analysis Image Analysis] section of the CASA documentation.
The three most basic analyses are to determine peak brightness, flux density, and noise level.  These are useful measures of how well the imaging efforts are in approaching the thermal noise limit or in reproducing what is already known about a source. Additional discussion of image analysis and manipulation, including the combination of multiple images, mathematical operations on images, and much more can be found in the [https://casa.nrao.edu/casadocs/latest/image-analysis Image Analysis] section of the CASA documentation.


The most straightforward statistic is the peak brightness, which is determined by {{imstat}}.
The most straightforward statistic is the peak brightness, which is determined by [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.information.imstat.html?highlight=imstat# imstat].
<source lang="python">
<source lang="python">
mystat = imstat(imagename='3C75_initial.image.tt0')
mystat = imstat(imagename='3C75_initial.image.tt0')
Line 1,758: Line 1,761:
This task returns a Python dictionary which we capture in the variable ''mystat''.
This task returns a Python dictionary which we capture in the variable ''mystat''.


The dictionary contains the values which you can extract for further use. For example, for a particular instance of
The dictionary contains the values which you can extract for further use. For example, for a particular instance of the previous clean, we found:
the previous clean, we found:
<pre style="background-color: #E0FFFF;">
<pre style="background-color: #E0FFFF;">
CASA <4>: mystat
CASA <4>: mystat
Line 1,791: Line 1,793:
and so the peak flux density is 0.150 Jy/beam.
and so the peak flux density is 0.150 Jy/beam.


The other two statistics require slightly more care. The flux density of a source is determined by integrating its brightness or intensity over some solid angle, i.e.,  
The other two statistics require slightly more care. The flux density of a source is determined by integrating its brightness or intensity over some solid angle, i.e.,  


<center>
<center>
Line 1,797: Line 1,799:
</center>
</center>


where <math>I</math> is the intensity (measured in units of Jy/beam), <math>\Omega</math> is the solid angle of the source (e.g., number of synthesized beams), and <math>S</math> is the flux density (measured in units of Jy). In general, if the noise is well-behaved in the image, when averaged over a reasonable solid angle, the noise contribution should approach 0 Jy. If that is the case, then the flux density of the source is also reported by {{imstat}}. However, there are many cases for which a noise contribution of 0 Jy may not be a safe assumption. If the source is in a complicated region (e.g., a star formation region, the Galactic center, near the edge of a galaxy), a better estimate of the source's flux density will be obtained by limiting carefully the solid angle over which the integration is performed.
where <math>I</math> is the intensity (measured in units of Jy/beam), <math>\Omega</math> is the solid angle of the source (e.g., number of synthesized beams), and <math>S</math> is the flux density (measured in units of Jy). In general, if the noise is well-behaved in the image, when averaged over a reasonable solid angle the noise contribution should approach 0 Jy. If that is the case, then the flux density of the source is also reported by [https://casadocs.readthedocs.io/en/v6.2.1/api/tt/casatasks.information.imstat.html?highlight=imstat# imstat]. However, there are many cases for which a noise contribution of 0 Jy may not be a safe assumption. If the source is in a complicated region (e.g., a star formation region, the Galactic center, near the edge of a galaxy), a better estimate of the source's flux density will be obtained by limiting carefully the solid angle over which the integration is performed.


Open '''[https://casa.nrao.edu/casadocs/latest/image-cube-visualization/viewer-basics viewer]''' and use it to display the corrected image (Figure 18). For this analysis, it is better to use the version of the viewer that is run from the OS command line rather than the CASA command line. You can open this from inside CASA using '!':
Open '''[https://casa.nrao.edu/casadocs/latest/image-cube-visualization/viewer-basics viewer]''' and use it to display the corrected image (Figure 18). For this analysis, it is better to use the version of the viewer that is run from the OS command line rather than the CASA command line. You can open this from inside CASA using '!':
<source lang="python">
<source lang="python">
# In CASA
# In CASA
Line 1,806: Line 1,808:
One can choose the function assigned to each mouse button; after zooming into the desired view, assign polygon region to a desired mouse button (e.g., left button) by selecting the polygon tool [[File:Polygon_btn.png]] to create the polygonal region with the desired mouse button.  
One can choose the function assigned to each mouse button; after zooming into the desired view, assign polygon region to a desired mouse button (e.g., left button) by selecting the polygon tool [[File:Polygon_btn.png]] to create the polygonal region with the desired mouse button.  


Using the mouse button just assigned to polygon region, outline the extended structures of 3C75. You start drawing vertices by clicking on points in the image in succession, when you draw the final vertex then you double-click to connect and close the region. When your mouse is inside the region, a bounding box will appear with the vertices shown as draggable solid squares. If you want to adjust the vertices you can do so.
Using the mouse button just assigned to polygon region, outline the extended structures of 3C75. You start drawing vertices by clicking on points in the image in succession, when you draw the final vertex then you double-click to connect and close the region. When your mouse is inside the region, a bounding box will appear with the vertices shown as draggable solid squares. If you want to adjust the vertices you can do so.


If you find you don't like your region you can dismiss it with with ESC key or using the remove region "X" button in lower right of the panel. You can also employ the region panel to save a region you have created for later use.
If you find you don't like your region you can dismiss it with with ESC key or using the remove region "X" button in lower right of the panel. You can also employ the region panel to save a region you have created for later use.


Double click inside of that region (using the same mouse button used to make the region), and the statistics will be reported. This will include the flux density value within the region selected.
Double click inside of that region (using the same mouse button used to make the region), and the statistics will be reported. This will include the flux density value within the region selected.
Line 1,822: Line 1,824:
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----     
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----     
</pre>
</pre>
In our example we find a total Flux density of 124 Jy. Note that the numbers you get may be slightly different depending on how deeply you cleaned. Also, this number is not the correct total flux density, to obtain the correct flux density you have to apply the primary beam correction similar to what we did above and repeat this analysis.
In our example we find a total Flux density of 124 Jy. Note that the numbers you get may be slightly different depending on how deeply you cleaned. Also, this number is not the correct total flux density. To obtain the correct flux density you have to apply the primary beam correction similar to what we did above and repeat this analysis.


By contrast, for the rms noise level ''exclude'' the source's emission to the extent possible as shown in Figure 27, as the source's emission will bias the estimated noise level high. Likewise, one should avoid the clean bowl around the source emission. One can repeat the procedure above, defining a polygonal region, then double clicking inside it to determine the statistics. For example, from the region selection shown to the right for off-source statistics:
By contrast, for the rms noise level ''exclude'' the source's emission to the extent possible as shown in Figure 27, as the source's emission will bias the estimated noise level high. Likewise, one should avoid the clean bowl around the source emission. One can repeat the procedure above, defining a polygonal region, then double clicking inside it to determine the statistics. For example, from the region selection shown to the right for off-source statistics:
<pre style="background-color: #E0FFFF;">
<pre style="background-color: #E0FFFF;">
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----                                                                                                                                                                                                                                                                           
---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----                                                                                                                                                                                                                                                                           
Line 1,842: Line 1,844:
The spectral index, defined as the slope of the radio spectrum between two different frequencies, <math>\log(S_{\nu_1}/S_{\nu_2})/\log(\nu_1/\nu_2)</math>, is a useful analytical tool that can convey information about the emission mechanism, the optical depth of the source or the underlying energy distribution of synchrotron-radiating electrons.  
The spectral index, defined as the slope of the radio spectrum between two different frequencies, <math>\log(S_{\nu_1}/S_{\nu_2})/\log(\nu_1/\nu_2)</math>, is a useful analytical tool that can convey information about the emission mechanism, the optical depth of the source or the underlying energy distribution of synchrotron-radiating electrons.  


Similar analysis can be performed on the polarization and spectral index maps, this will be left to the inclined user. -->
<!-- Similar analysis can be performed on the polarization and spectral index maps, this will be left to the user. -->


Questions about this tutorial? Please contact the [http://go.nrao.edu/obshelp NRAO Helpdesk].
Questions about this tutorial? Please contact the [http://go.nrao.edu/obshelp NRAO Helpdesk].
Line 1,848: Line 1,850:
[[Main Page | &#8629; '''CASAguides''']]
[[Main Page | &#8629; '''CASAguides''']]


<!--Edited by Frank Schinzel 2021-03-30-->
<!-- Edited for content & context by T. Perreault 2021-12-30 -->
<!-- Last edit by Anna D. Kapinska 2021-12-17 -->
<!-- Edited by Frank Schinzel 2021-03-30 -->

Latest revision as of 21:44, 6 December 2022


This CASA Guide is for version 6.2.1.7 of CASA that includes the VLA pipeline. If you are using a later version of CASA and this is the most recent available guide, then you should be able to use most, if not all, of this tutorial.

Overview

This CASA guide describes the calibration and imaging of a single-pointing continuum data set taken with the Karl G. Jansky Very Large Array (VLA) of the binary black hole system 3C 75 in Abell 400 cluster of galaxies. [1]. The data were taken as a demonstration for the VLA data reduction workshops under project code TDRW0001. To reduce the dataset size, the data was recorded with a single 1 GHz baseband centered at 3.0 GHz, resulting in 8x128 MHz wide spectral windows with 64 channels each. The observation was set up to allow for full polarization calibration. The calibration part of this tutorial can be performed on a laptop computer with sufficient storage. The imaging part, however, will require more computing power and memory. This CASA guide was also used as basis for a presentation on polarization calibration at the 7th VLA data reduction workshop: [2]

How to Use This CASA Guide

Here are a number of possible ways to run CASA, described in more detail in Getting Started in CASA. In brief, there are at least three different ways to run CASA:

  • Interactively examining task inputs. In this mode, one types default taskname to load the task (this will also set all the task parameters to default values), inp to examine the inputs, and go once those inputs have been set to your satisfaction. Allowed inputs are colored blue and bad inputs are colored red. The input parameters themselves are changed one by one, e.g., selectdata=True. Screenshots of the inputs to various tasks used in the data reduction are provided to illustrate which parameters need to be set. More detailed help can be obtained on any task by typing help taskname. Once a task is run, the set of inputs are stored and can be retrieved via tget taskname; subsequent runs will overwrite the previous tget file.
  • Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set.
  • Non-interactively via a script. A series of task function calls can be combined together into a script and run from within CASA via execfile('scriptname.py'). This and other CASA Tutorial Guides have been designed to be extracted into a script via the script extractor by using the method described at the Extracting scripts from these tutorials page. Should you decide to use the script generated by the script extractor for this CASA Guide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is, in fact, unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered).

If you are a relative novice or just new to CASA, it is strongly recommended to work through this tutorial by cutting and pasting the task function calls provided below after you have read all the associated explanations. Work at your own pace, look at the inputs to the tasks to see what other options exist, and read the help files. Later, when you are more comfortable, you might try to extract the script, modify it for your purposes, and begin to reduce other data.

Obtaining the Data

If starting from scratch, you can obtain the dataset from the NRAO archive and search for the Archive File ID: 'TDRW0001.sb35624494.eb35628826.58395.23719237269'. The uncalibrated visibilities have a size of 12.5 GB. Make sure to select to download the SDM-BDF dataset, if you want to start from the lowest level, because by default a .ms file will be provided by the archive.

For those who want to skip the step of obtaining a continuum Stokes I calibrated measurement set, we have created a starting dataset on which the polarization calibration steps and final imaging can be performed: ['https://casa.nrao.edu/Data/VLA/Polarization/TDRW0001_calibrated_CASA6.2.1.ms.tgz '] (size: 10 GB). It is recommended to use the command line tool wget to download the calibrated data or directly download through the browser. You will need to untar and unzip the file using the command: 'tar -xzvf TDRW0001_calibrated_CASA6.2.1.ms.tgz'. Then you can skip ahead to the section 'The Observation'.

Pipeline Calibration of Parallel Hands (RR/LL)

If you start with the uncalibrated visibilities obtained from the archive, you will need to first perform a standard continuum calibration of the parallel-hand (RR/LL) cross-correlation visibilities. In this guide we use the standard VLA pipeline that is packaged with the CASA release. You can find more information on the latest release of the VLA pipeline at: https://science.nrao.edu/facilities/vla/data-processing/pipeline.

In this example, we will not run the pipeline in its standard way but tweak it to force a certain reference antenna. The pipeline typically tries to pick a reference antenna at the center of the array; however, this dataset was observed in D array configuration with very short baselines. It is better to use one of the outer antennas for reference, which provides longer baselines and more stable phase solutions. To set the reference antenna, we specify the refantignore parameter in some of the pipeline tasks to exclude all but the reference antenna, and use a pipeline execution script ('casa_pipescript.py'). Take the script given below and paste it into a text file inside your working directory that also contains the dataset you downloaded from the NRAO archive and name it casa_pipescript.py. 

# casa_pipescript.py

__rethrow_casa_exceptions = True
context = h_init()
context.set_state('ProjectSummary', 'observatory', 'Karl G. Jansky Very Large Array')
context.set_state('ProjectSummary', 'telescope', 'EVLA')
context.set_state('ProjectStructure', 'recipe_name', 'hifv_cal')
try:
    hifv_importdata(vis=['TDRW0001.sb35624494.eb35628826.58395.23719237269'], session=['default'])
    hifv_hanning(pipelinemode="automatic")
    hifv_flagdata(hm_tbuff='1.5int', intents='*POINTING*,*FOCUS*,*ATMOSPHERE*,*SIDEBAND_RATIO*, *UNKNOWN*, *SYSTEM_CONFIGURATION*, *UNSPECIFIED#UNSPECIFIED*')
    hifv_vlasetjy(pipelinemode="automatic")
    hifv_priorcals(pipelinemode="automatic")
    hifv_testBPdcals(weakbp=False, refantignore='ea01,ea02,ea03,ea04,ea05,ea06,ea07,ea08,ea09,ea11,ea12,ea13,ea14,ea15,ea16,ea17,ea18,ea19,ea20,ea21,ea22,ea23,ea24,ea26,ea28')
    hifv_checkflag(pipelinemode="automatic")
    hifv_semiFinalBPdcals(weakbp=False, refantignore='ea01,ea02,ea03,ea04,ea05,ea06,ea07,ea08,ea09,ea11,ea12,ea13,ea14,ea15,ea16,ea17,ea18,ea19,ea20,ea21,ea22,ea23,ea24,ea26,ea28')
    hifv_checkflag(checkflagmode='semi')
    hifv_semiFinalBPdcals(weakbp=False, refantignore='ea01,ea02,ea03,ea04,ea05,ea06,ea07,ea08,ea09,ea11,ea12,ea13,ea14,ea15,ea16,ea17,ea18,ea19,ea20,ea21,ea22,ea23,ea24,ea26,ea28')
    hifv_solint(refantignore='ea01,ea02,ea03,ea04,ea05,ea06,ea07,ea08,ea09,ea11,ea12,ea13,ea14,ea15,ea16,ea17,ea18,ea19,ea20,ea21,ea22,ea23,ea24,ea26,ea28')
    hifv_fluxboot(fitorder=2, refantignore='ea01,ea02,ea03,ea04,ea05,ea06,ea07,ea08,ea09,ea11,ea12,ea13,ea14,ea15,ea16,ea17,ea18,ea19,ea20,ea21,ea22,ea23,ea24,ea26,ea28')
    hifv_finalcals(refantignore='ea01,ea02,ea03,ea04,ea05,ea06,ea07,ea08,ea09,ea11,ea12,ea13,ea14,ea15,ea16,ea17,ea18,ea19,ea20,ea21,ea22,ea23,ea24,ea26,ea28')
    hifv_applycals(pipelinemode="automatic")
    hifv_targetflag(intents='*CALIBRATE*,*TARGET*')
    hifv_statwt(pipelinemode="automatic")
    hifv_plotsummary(pipelinemode="automatic")
    hif_makeimlist(intent='PHASE,BANDPASS', specmode='cont')
    hif_makeimages(hm_masking='none')
finally:
    h_save()

Now that we have the script, we can execute the pipeline. Type on the command line the following. 

# On the command line, for your own installation of CASA 6.2.1-7
casa --pipeline --nogui -c casa_pipescript.py

# If using an NRAO computer, to select the right CASA version use instead
casa -r 6.2.1-7-pipeline-2021.2.0.128 --pipeline --nogui -c casa_pipescript.py

Now you can go and get a cup of coffee or lunch; this will take a while. On a beefy computer expect about two hours. Once the pipeline has successfully finished you will see some similar messages on the command line prompt. 

2021-11-25 00:55:08 INFO: Plotting oussid.s19_0._0137+331_3C48__bp.S_band.cont.I.iter0.psf.tt0
2021-11-25 00:55:08 INFO: Plotting oussid.s19_0._0137+331_3C48__bp.S_band.cont.I.iter0.pb.tt0
2021-11-25 00:55:09 INFO: Plotting oussid.s19_0._0137+331_3C48__bp.S_band.cont.I.iter0.residual.tt0
2021-11-25 00:55:09 INFO: Plotting oussid.s19_0._0137+331_3C48__bp.S_band.cont.I.iter1.image.tt0
2021-11-25 00:55:09 INFO: Plotting oussid.s19_0._0137+331_3C48__bp.S_band.cont.I.iter1.image.tt0

2021-11-25 00:55:20 INFO: Saving context to 'pipeline-20211124T225215.context'

In order to be able to continue calibration for polarization, i.e. the cross-hand correlations (RL/LR), on pre-calibrated visibilities, we need to perform additional steps that remove the parallactic angle correction that was applied by the standard pipeline. To do so, start CASA and execute the following commands. 

# In CASA
flagmanager(vis='TDRW0001.sb35624494.eb35628826.58395.23719237269.ms',mode='restore',versionname='applycal_5')

applycal(vis='TDRW0001.sb35624494.eb35628826.58395.23719237269.ms',
         antenna='*&*',
         gaintable=['TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_priorcals.s5_2.gc.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_priorcals.s5_3.opac.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_priorcals.s5_4.rq.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_priorcals.s5_6.ants.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_finalcals.s13_2.finaldelay.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_finalcals.s13_4.finalBPcal.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_finalcals.s13_5.averagephasegain.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_finalcals.s13_7.finalampgaincal.tbl',
         'TDRW0001.sb35624494.eb35628826.58395.23719237269.ms.hifv_finalcals.s13_8.finalphasegaincal.tbl'],
         gainfield=['', '', '', '', '', '', '', '', ''], interp=['', '', '', '',
         '', 'linear,linearflag', '', '', ''], spwmap=[[], [], [], [], [], [],
         [], [], []], calwt=[False, False, False, False, False, False, False,
         False, False], parang=False, applymode='calflagstrict', flagbackup=False)

flagdata(vis='TDRW0001.sb35624494.eb35628826.58395.23719237269.ms',
         mode='rflag', correlation='ABS_RR,LL', intent='*CALIBRATE*',
         datacolumn='corrected', ntime='scan', combinescans=False,
         extendflags=False, winsize=3, timedevscale=4.0, freqdevscale=4.0,
         action='apply', flagbackup=True, savepars=True)

flagdata(vis='TDRW0001.sb35624494.eb35628826.58395.23719237269.ms',
         mode='rflag', correlation='ABS_RR,LL', intent='*TARGET*',
         datacolumn='corrected', ntime='scan', combinescans=False,
         extendflags=False, winsize=3, timedevscale=4.0, freqdevscale=4.0,
         action='apply', flagbackup=True, savepars=True)

statwt(vis='TDRW0001.sb35624494.eb35628826.58395.23719237269.ms', minsamp=8,
       datacolumn='corrected')

split(vis='TDRW0001.sb35624494.eb35628826.58395.23719237269.ms',outputvis='TDRW0001_calibrated.ms',datacolumn='corrected',spw='2~9')

This applies the flagging state before the final applycal stage of the pipeline, then reapplies the calibration to the corrected column with parang=False, disabling the parallactic angle corrections. After that, we rerun target field flagging, and recompute the weights based on the new flags that were applied and split out the corrected column for the target spectral windows. Essentially we repeated what pipeline tasks hifv_applycals, hifv_targetflag, and hifv_statwt did, but disabling application of parallactic angle corrections. This is the measurement set we will be using in the following to demonstrate polarization calibration.

The Observation

Before starting the calibration process, we want to get some basic information about the data set. To examine the observing conditions during the observing run, and to find out any known problems with the data, download the observer log. Simply fill in the known observing date (in our case 2018-Oct-04) as both the Start and Stop date and click on the Show Logs button. The relevant log is labelled with the project code, TDRW0001, and can be downloaded as a PDF file. From this, we find the following: 

Information from observing log:
Antennas in the D-array may be shadowed at low elevations.  If shadowing
occurs, sensitivity will be affected.

NOTE!: The VLA is still recovering from a long power outage, and these data may
have unusual artifacts, missing antennas or IFs, ect., in them. NRAO staff will 
examine the data closely after observing to determine if they meet the criteria for 
a successful observation.

Antenna ea05: S-band receiver cooling after work performed, currently 65/177K,
              thus we expect lower sensitivity from this antenna.
Antenna ea12: C-band receiver warm for cold head replacement.

Winds at 5-7 m/s, API RMS phase around 4.5 deg, 10-20% sky cover, cumuliform and stratiform clouds.

Before beginning our data reduction, we should inspect the pipeline calibration weblog for any obvious issues. You can download the weblog from ['https://casa.nrao.edu/Data/VLA/Polarization/pipeline-20211207T212848.tgz '] or directly access it at ftp://ftp.aoc.nrao.edu/staff/akapinsk/pipeline-20211207T212848/html/.

Inside the weblog you have access to the overview page and the listobs task output that provide some basic information about the data.

You will note that there are four sources observed. Here the sources are introduced briefly, with more detail contained in the sections below in which they are used:

  • 0137+331=3C48, which will serve as a calibrator for the visibility amplitudes, i.e., it is assumed to have precisely known flux density, the spectral bandpass, and the polarization position angle;
  • J0259+0747, which will serve as a calibrator for the visibility phases and can be used to determine the instrumental polarization;
  • J2355+4950, which can serve as a secondary instrumental polarization calibrator or to check residual instrumental polarization, and;
  • 3C75, which is the science target.


================================================================================
           MeasurementSet Name:  /lustre/aoc/sciops/akapinsk/casaguides/TDRW0001.sb35624494.eb35628826.58395.23719237269.ms      MS Version 2
================================================================================
   Observer: Dr. Emmanuel Momjian     Project: uid://evla/pdb/35621723  
Observation: EVLA
Data records: 5752188       Total elapsed time = 10270 seconds
   Observed from   04-Oct-2018/05:41:35.0   to   04-Oct-2018/08:32:45.0 (UTC)

   ObservationID = 0         ArrayID = 0
  Date        Timerange (UTC)          Scan  FldId FieldName             nRows     SpwIds   Average Interval(s)    ScanIntent
  04-Oct-2018/05:41:35.0 - 05:42:31.0     1      0 0137+331=3C48            39312  [0,1]  [1, 1] [SYSTEM_CONFIGURATION#UNSPECIFIED]
              05:42:32.0 - 05:47:30.0     2      0 0137+331=3C48           209196  [0,1]  [1, 1] [SYSTEM_CONFIGURATION#UNSPECIFIED]
              05:47:35.0 - 05:48:30.0     3      0 0137+331=3C48            30888  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [SYSTEM_CONFIGURATION#UNSPECIFIED]
              05:48:35.0 - 05:49:00.0     4      0 0137+331=3C48            14040  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [SYSTEM_CONFIGURATION#UNSPECIFIED]
              05:49:05.0 - 05:53:25.0     5      0 0137+331=3C48           146016  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED,CALIBRATE_POL_ANGLE#UNSPECIFIED]
              05:53:30.0 - 05:57:55.0     6      1 J2355+4950              148824  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
              05:58:00.0 - 06:03:55.0     7      2 J0259+0747              199368  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              06:04:00.0 - 06:18:55.0     8      3 3C75                    502632  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              06:19:00.0 - 06:20:10.0     9      2 J0259+0747               39312  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              06:20:15.0 - 06:35:05.0    10      3 3C75                    499824  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              06:35:10.0 - 06:36:20.0    11      2 J0259+0747               39312  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              06:36:25.0 - 06:51:20.0    12      3 3C75                    502632  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              06:51:25.0 - 06:52:30.0    13      2 J0259+0747               36504  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              06:52:35.0 - 07:07:30.0    14      3 3C75                    502632  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              07:07:35.0 - 07:08:45.0    15      2 J0259+0747               39312  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              07:08:50.0 - 07:23:40.0    16      3 3C75                    499824  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              07:23:45.0 - 07:26:25.0    17      2 J0259+0747               89856  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              07:26:30.0 - 07:41:25.0    18      3 3C75                    502632  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              07:41:30.0 - 07:42:40.0    19      2 J0259+0747               39312  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              07:42:45.0 - 07:57:35.0    20      3 3C75                    499824  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              07:57:40.0 - 07:58:50.0    21      2 J0259+0747               39312  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              07:58:55.0 - 08:13:50.0    22      3 3C75                    502632  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              08:13:55.0 - 08:15:05.0    23      2 J0259+0747               39312  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
              08:15:10.0 - 08:30:00.0    24      3 3C75                    499824  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [OBSERVE_TARGET#UNSPECIFIED]
              08:30:05.0 - 08:32:45.0    25      2 J0259+0747               89856  [2,3,4,5,6,7,8,9]  [5, 5, 5, 5, 5, 5, 5, 5] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED,CALIBRATE_POL_LEAKAGE#UNSPECIFIED]
           (nRows = Total number of rows per scan) 
Fields: 4
  ID   Code Name                RA               Decl           Epoch   SrcId      nRows
  0    NONE 0137+331=3C48       01:37:41.299431 +33.09.35.13299 J2000   0         439452
  1    NONE J2355+4950          23:55:09.458169 +49.50.08.34001 J2000   1         148824
  2    NONE J0259+0747          02:59:27.076633 +07.47.39.64322 J2000   2         651456
  3    NONE 3C75                02:57:42.630000 +06.01.04.80000 J2000   3        4512456
Spectral Windows:  (10 unique spectral windows and 1 unique polarization setups)
  SpwID  Name          #Chans   Frame   Ch0(MHz)  ChanWid(kHz)  TotBW(kHz) CtrFreq(MHz) BBC Num  Corrs          
  0      EVLA_C#A0C0#0     64   TOPO    4832.000      2000.000    128000.0   4895.0000       12  RR  RL  LR  LL
  1      EVLA_C#B0D0#1     64   TOPO    4960.000      2000.000    128000.0   5023.0000       15  RR  RL  LR  LL
  2      EVLA_S#A0C0#2     64   TOPO    2488.000      2000.000    128000.0   2551.0000       12  RR  RL  LR  LL
  3      EVLA_S#A0C0#3     64   TOPO    2616.000      2000.000    128000.0   2679.0000       12  RR  RL  LR  LL
  4      EVLA_S#A0C0#4     64   TOPO    2744.000      2000.000    128000.0   2807.0000       12  RR  RL  LR  LL
  5      EVLA_S#A0C0#5     64   TOPO    2872.000      2000.000    128000.0   2935.0000       12  RR  RL  LR  LL
  6      EVLA_S#A0C0#6     64   TOPO    3000.000      2000.000    128000.0   3063.0000       12  RR  RL  LR  LL
  7      EVLA_S#A0C0#7     64   TOPO    3128.000      2000.000    128000.0   3191.0000       12  RR  RL  LR  LL
  8      EVLA_S#A0C0#8     64   TOPO    3256.000      2000.000    128000.0   3319.0000       12  RR  RL  LR  LL
  9      EVLA_S#A0C0#9     64   TOPO    3384.000      2000.000    128000.0   3447.0000       12  RR  RL  LR  LL
Sources: 34
  ID   Name                SpwId RestFreq(MHz)  SysVel(km/s) 
  0    0137+331=3C48       0     -              -            
  0    0137+331=3C48       1     -              -            
  0    0137+331=3C48       2     -              -            
  0    0137+331=3C48       3     -              -            
  0    0137+331=3C48       4     -              -            
  0    0137+331=3C48       5     -              -            
  0    0137+331=3C48       6     -              -            
  0    0137+331=3C48       7     -              -            
  0    0137+331=3C48       8     -              -            
  0    0137+331=3C48       9     -              -            
  1    J2355+4950          2     -              -            
  1    J2355+4950          3     -              -            
  1    J2355+4950          4     -              -            
  1    J2355+4950          5     -              -            
  1    J2355+4950          6     -              -            
  1    J2355+4950          7     -              -            
  1    J2355+4950          8     -              -            
  1    J2355+4950          9     -              -            
  2    J0259+0747          2     -              -            
  2    J0259+0747          3     -              -            
  2    J0259+0747          4     -              -            
  2    J0259+0747          5     -              -            
  2    J0259+0747          6     -              -            
  2    J0259+0747          7     -              -            
  2    J0259+0747          8     -              -            
  2    J0259+0747          9     -              -            
  3    3C75                2     -              -            
  3    3C75                3     -              -            
  3    3C75                4     -              -            
  3    3C75                5     -              -            
  3    3C75                6     -              -            
  3    3C75                7     -              -            
  3    3C75                8     -              -            
  3    3C75                9     -              -            
Antennas: 27:
  ID   Name  Station   Diam.    Long.         Lat.                Offset from array center (m)                ITRF Geocentric coordinates (m)        
                                                                     East         North     Elevation               x               y               z
  0    ea01  W06       25.0 m   -107.37.15.6  +33.53.56.4       -275.8278     -166.7360       -2.0595 -1601447.195400 -5041992.497600  3554739.694800
  1    ea02  W04       25.0 m   -107.37.10.8  +33.53.59.1       -152.8711      -83.7955       -2.4675 -1601315.900500 -5041985.306670  3554808.309400
  2    ea03  W07       25.0 m   -107.37.18.4  +33.53.54.8       -349.9804     -216.7527       -1.7877 -1601526.383100 -5041996.851000  3554698.331400
  3    ea04  N04       25.0 m   -107.37.06.5  +33.54.06.1        -42.6260      132.8521       -3.5428 -1601173.981600 -5041902.657800  3554987.528200
  4    ea05  E05       25.0 m   -107.36.58.4  +33.53.58.8        164.9709      -92.7908       -2.5361 -1601014.465100 -5042086.235700  3554800.804900
  5    ea06  N06       25.0 m   -107.37.06.9  +33.54.10.3        -54.0745      263.8800       -4.2325 -1601162.598500 -5041828.990800  3555095.895300
  6    ea07  E04       25.0 m   -107.37.00.8  +33.53.59.7        102.8035      -63.7671       -2.6299 -1601068.794800 -5042051.918100  3554824.842700
  7    ea08  E01       25.0 m   -107.37.05.7  +33.53.59.2        -23.8867      -81.1272       -2.5808 -1601192.486700 -5042022.840700  3554810.460900
  8    ea09  N05       25.0 m   -107.37.06.7  +33.54.08.0        -47.8569      192.6072       -3.8789 -1601168.794400 -5041869.042300  3555036.937000
  9    ea10  E08       25.0 m   -107.36.48.9  +33.53.55.1        407.8379     -206.0064       -3.2255 -1600801.917500 -5042219.370600  3554706.449200
  10   ea11  N07       25.0 m   -107.37.07.2  +33.54.12.9        -61.1072      344.2424       -4.6414 -1601155.630600 -5041783.816000  3555162.366400
  11   ea12  E07       25.0 m   -107.36.52.4  +33.53.56.5        318.0401     -164.1704       -2.6834 -1600880.582300 -5042170.386600  3554741.476400
  12   ea13  W02       25.0 m   -107.37.07.5  +33.54.00.9        -67.9810      -26.5266       -2.7142 -1601225.261900 -5041980.363990  3554855.705700
  13   ea14  E09       25.0 m   -107.36.45.1  +33.53.53.6        506.0539     -251.8836       -3.5735 -1600715.958300 -5042273.202200  3554668.175800
  14   ea15  N03       25.0 m   -107.37.06.3  +33.54.04.8        -39.1086       93.0234       -3.3585 -1601177.399560 -5041925.041300  3554954.573300
  15   ea16  E02       25.0 m   -107.37.04.4  +33.54.01.1          9.8042      -20.4562       -2.7822 -1601150.083300 -5042000.626900  3554860.706200
  16   ea17  N09       25.0 m   -107.37.07.8  +33.54.19.0        -77.4340      530.6515       -5.5829 -1601139.481300 -5041679.026500  3555316.554900
  17   ea18  W09       25.0 m   -107.37.25.2  +33.53.51.0       -521.9447     -332.7673       -1.2061 -1601710.016800 -5042006.914600  3554602.360000
  18   ea19  W05       25.0 m   -107.37.13.0  +33.53.57.8       -210.1007     -122.3814       -2.2582 -1601377.012800 -5041988.659800  3554776.399200
  19   ea20  N02       25.0 m   -107.37.06.2  +33.54.03.5        -35.6257       53.1906       -3.1311 -1601180.861780 -5041947.450400  3554921.638900
  20   ea21  N01       25.0 m   -107.37.06.0  +33.54.01.8        -30.8742       -1.4746       -2.8653 -1601185.628465 -5041978.158516  3554876.414800
  21   ea22  W03       25.0 m   -107.37.08.9  +33.54.00.1       -105.3218      -51.7280       -2.6013 -1601265.134100 -5041982.547450  3554834.851200
  22   ea23  E06       25.0 m   -107.36.55.6  +33.53.57.7        236.9085     -126.3395       -2.4685 -1600951.579800 -5042125.894100  3554772.996600
  23   ea24  W08       25.0 m   -107.37.21.6  +33.53.53.0       -432.1080     -272.1502       -1.5080 -1601614.082500 -5042001.654800  3554652.505900
  24   ea25  N08       25.0 m   -107.37.07.5  +33.54.15.8        -68.9105      433.1823       -5.0689 -1601147.943900 -5041733.832200  3555235.945600
  25   ea26  E03       25.0 m   -107.37.02.8  +33.54.00.5         50.6698      -39.4668       -2.7317 -1601114.356200 -5042023.141200  3554844.955400
  26   ea28  W01       25.0 m   -107.37.05.9  +33.54.00.5        -27.3603      -41.2944       -2.7520 -1601189.030040 -5042000.479400  3554843.427200

Note that the antenna IDs, which are numbered sequentially up to the total number of antennas in the array (from 0 to 26 in this instance), do not correspond to the actual antenna names (ea01 to ea28). Instead, these numbers correspond to those painted on the antennas themselves. The antennas can be referenced using either convention; antenna='22' would correspond to ea23, whereas antenna='ea22' would correspond to ea22. Note that the antenna numbers in the observer log correspond to the actual antenna names, i.e., the 'ea??' numbers given in listobs.

Both to get a sense of the array, as well as identify the location of the antenna that was picked by the pipeline for parallel hand calibration, have a look at the antenna setup page. For calibration purposes, you would generally select an antenna that is close to the center of the array (and that is not listed in the operator's log as having had problems!). As noted above, in a compact configuration there is a benefit to choosing an outer antenna to increase the bias toward longer baselines.

At this point it is also a good idea to check the quality of the pipeline calibration. Go to the task overview page and pay particular attention to hifv_finalcals and hifv_plotsummary. Try to see if you can recognize which reference antenna was picked. For more details on the pipeline output you can have a look at the VLA CASA Pipeline Guide. We assume that the pipeline calibration is good and can use it as a starting point for further calibration steps focusing on polarization calibration and imaging.

Examining and Editing the Data

At this point we must start CASA. If you have not used CASA before, some helpful tips are available on the Getting Started in CASA page.

It is always a good idea to examine the data before jumping straight into calibration. From the observer's log there were no major issues noted besides a potentially warm receiver on antenna ea05. Even though the pipeline did a good job of calibrating and flagging the data, it isn't perfect. From the pipeline weblog, looking at the final amplitude gain calibration vs time plots in hifv_finalcals, we can see that during the second half of the observation antennas ea03, ea12, and ea16 shows some gain instability; otherwise there are no issues identified at this point.

Start by inspecting these three particular antennas using the CASA task plotms, plot frequency against amplitude and frequency against time for the parallel hands, iterate over field or scan, and note if you find something at odds. 

# In CASA
plotms(vis='TDRW0001_calibrated.ms', selectdata=True, correlation='RR,LL', averagedata=True, avgchannel='64', coloraxis='field', plotfile='colorbyfield.jpeg')
Figure 1: Overview of the observation: amplitude vs time, color-coded by field.
  • selectdata=True : One can choose to plot only selected subsets of the data.
  • correlation='RR,LL' : Plot only the left- and right-handed polarization products. The cross-terms ('RL' and 'LR') will be close to zero for non-polarized sources.
  • averagedata=True: One can choose to average data points before plotting them.
  • avgchannel='64' : With this plot, we are mainly interested in the fields vs time. Averaging over all 64 channels in the spectral window makes the plotting faster.
  • coloraxis='field' : Color-code the plotting symbols by field name/number.

The default x- and y-axis parameters are 'time' and 'amp', so the above call to plotms produces an amplitude vs time plot of the data for a selected subset of the data (if desired) and with data averaging (if desired). Many other values have been left to defaults, but it is possible to select them from within the plotms GUI.

Task plotms allows one to select and view the data in many ways. Figure 1 shows the result of running plotms with the field selection discussed above. You can quickly see that the first source observed, 3C48 (the primary flux density, bandpass, and polarization angle calibrator source), is the brightest source in this observation. The next brightest is the second source observed, J2355+4950, a compact symmetric object (CSO; radio galaxy) and the secondary instrumental polarization calibrator. The complex gain calibrator J0259+0747 (shown in orange) is around 1 Jy. The target scans on 3C75 are colored in green. The spread of amplitudes is primarily due to the presence of extended structure, thus every baseline sees a slightly different amplitude.

Across the top of the left panel of the GUI are a set of tabs labelled Plot, Flag, Tools, Annotate, and Options. By default, the Plot tab is visible. There are a number of tabs running down the side of the left hand panel: Data, Calibration, Axes, Page, Transform, Display, and Canvas; these allow you to make changes to the plotting selection without having to re-launch plotms. Even if it was started with xaxis=' ' (defaulting to 'time'), you can choose a different X-axis by selecting the Axes tab, then using the dropdown menu to switch (for example) to xaxis='Frequency' (to get something sensible when plotting with frequency, channel averaging must be turned off).

You should spend several minutes displaying the data in various formats. You can save the version of the plotms plot as a graphics file by using the menu bar in the plotms GUI to select the Export... option under the Export menu.

Another example of using plotms for a quick look at your data, select the Data tab and specify field 2 (the complex gain calibrator J0259+0747) to display data associated with the target, then select the Axes tab and change the X-axis to be UVdist (baseline length in meters). Remove the channel averaging (Data tab), and plot the data using the Plot button at the bottom of the plotms GUI. The important observation is that the amplitude distribution is relatively constant as a function of UV distance or baseline length (i.e., [math]\displaystyle{ \sqrt{u^2+v^2} }[/math]; see Figure 2A). A relatively constant visibility amplitude as a function of baseline length means that the source is very nearly a point source (the Fourier transform of a point source, i.e. a delta function, is a constant function). You can see occasional spikes in the calibrated amplitudes. This is most likely caused by radio frequency interference that correlates on certain baselines. We will get to those further in the guide.

By contrast, if you make a similar plot for field 3 (our target 3C 75), the result is a visibility function that falls rapidly with increasing baseline length. Figure 2B shows this example, including time averaging of '1e6' seconds (any large number that encompasses more than a full scan will do, we want to fully average each scan). Such a visibility function indicates a highly resolved source. The baseline length at which the visibility function falls to some fiducial value (e.g., 1/2 of its peak value) gives a rough estimate of the angular scale of the source (Angular scale [in radians] ~ 1/baseline [in wavelengths]). To plot baseline length in wavelengths rather than meters, select UVwave as the X-axis parameter.

A final example is shown in Figure 2C. In this example, we have elected to show phase as a function of (frequency) channel for a single baseline (antenna='ea01&ea21' ) on the bandpass calibrator, field 0, and non-averaged data. If you choose to iterate by baseline (e.g., antenna='ea01' and iteraxis='baseline' ), you can see similar phase-frequency variations on all baselines. They center around zero phase, because we are looking at the calibrated visibilities; however, you are seeing a butterfly shaped pattern with phase noise higher toward the channel edges. This pattern is due to a small mismatch in the delay measurement timing (also known as 'delay clunking') which is an internally generated effect and is typically averaged out over time.

Figure 2A: plotms view of amp vs. uvdist of J0259+0747, a point source
Figure 2B: plotms view of amp vs. uvwave of 3C 75, a resolved source
Figure 2C: plotms view of phase vs. channel on one baselines, showing phase delay across the calibrated bandpass

You can find similar plots in the CASA pipeline weblog under the task hifv_plotsummary. At this stage the pipeline has taken care of most of the calibration. There might be some remaining issues, though, that were not caught by the pipeline.

Figure 3: datastream view of MS

One final useful plot we will make is a datastream plot of the antenna2 in a baseline for the data versus ea01. This shows, assuming that ea01 is in the entire observation, when various antennas drop out (see Figure 3). 

# In CASA
plotms(vis='TDRW0001_calibrated.ms',field='',correlation='RR,LL',
       timerange='',antenna='ea01',spw='0:31',
       xaxis='time',yaxis='antenna2',
       plotrange=[-1,-1,0,26],coloraxis='field')

From this display you can immediately see that flagging performed by the pipeline is present.

In the following we note on a couple issues that you might have found while inspecting data in this section. We will take care of those through additional flagging. 

Issues that you might find:
 - ea12, scan 17: amplitude spike at the end of the scan (can be spotted already in Figure 1)
 - Residual RFI (see Figure 2A)

In the case of the amplitude spike, we can flag the affected time period by invoking the casa task flagdata. It is a good idea to save the original flags before performing any flagging by setting flagbackup=True. 

# In CASA
flagdata(vis='TDRW0001_calibrated.ms', flagbackup=True, mode='manual', antenna='ea12',scan='17',timerange='07:25:57~07:26:18')

You can check the effect of this flagging by replotting Figure 2A. The spikes we saw before on some baselines should have disappeared. If you plot frequency against amplitude without averaging, however, you will still see some channels with interference that we will need to flag, especially on the instrumental polarization calibrators. Polarization calibration is very sensitive to interference, especially in the cross-hand correlations RL,LR. The pipeline does a good job at this, but there are still some RFI left; we will perform some additional flagging steps in the next section.

Additional Flagging

First we try to get a good sense of additional flagging that might be needed by plotting frequency against amplitude for the RR,LL and RL,LR polarizations of our calibrators (fields 0 through 2). You will notice some left over RFI on the bandpass calibrator in RR, LL. However, we also need to pay particular attention to RL, LR (see Figure 4A). Here we consider calibrators only; we will perform additional flagging on the target field at a later stage. 

# In CASA
# for parallel hands
plotms(vis='TDRW0001_calibrated.ms',xaxis='frequency',yaxis='amplitude',field='0~2',correlation='RR,LL')
# for cross-hands
plotms(vis='TDRW0001_calibrated.ms',xaxis='frequency',yaxis='amplitude',field='0~2',correlation='RL,LR')
Figure 4a: plotms() view of calibrators' amplitudes (RL,LR) as a function of frequency before additional flagging
Figure 4b: plotms() view of calibrators' amplitudes (RL,LR) as a function of frequency after executing addition run of rflag

Since we are dealing with point sources, we do not have to worry about overflagging of shorter baselines, so we can run flagdata with mode='rflag' over the calibrator fields and cross-hand correlations to remove any residual RFI. For completeness, we also use mode='tfcrop' to reduce the amount of residual RFI in the parallel hands. This is not strictly needed at this point, since the polarization calibration is based on the cross-hand correlations. 

# In CASA

# for the parallel hands
flagdata(vis='TDRW0001_calibrated.ms',
	 mode='tfcrop',
	 field='0~2',
	 correlation='',
	 freqfit='line',
	 extendflags=False,
	 flagbackup=False)

# for the cross-hands
flagdata(vis='TDRW0001_calibrated.ms',
	 mode='rflag',
	 datacolumn='data',
	 field='0~2',
	 correlation='RL,LR',
	 extendflags=True,
	 flagbackup=False)

As you can see in Figure 4B, this additional flagging step took care of most of the obvious residual RFI. We are now ready to move on to calibrate the visibilities for linear polarization.

Polarization Calibration

Polarization calibration is done in three steps:

* First, we determine the instrumental delay between the two polarization outputs;

* Second, we solve for the instrumental polarization (the frequency-dependent leakage terms, 'D-terms'), using either an unpolarized source or a source which has sufficiently good parallactic angle coverage;

* Third, we solve for the polarization position angle using a source with a known polarization position angle (we use 3C48 here).

For information on polarization calibrators suitable for VLA observations, see the VLA Observing Guide on Polarimetry. The CASA related documentation also provides helpful information on polarization calibration steps and the different options that are available.

Before solving for the calibration solutions, we first use setjy to set the polarization model for our polarized position-angle calibrator. The pipeline only set the total intensity of the flux density calibrator source 3C48, which did not include any polarization information. This source is known to have a fairly stable linear fractional polarization (measured to be 2% in S-band around the time of the observations), a polarization position angle of -100 degrees at 3 GHz, and a rotation measure of -68 rad/m^2. Note that 3C48 had an outburst in 2017 and is expected to show a significant degree of variability at higher frequencies in the first instance, progressively affecting lower frequencies as time passes since the event. Since we have applied the pipeline calibration and not corrected for parallactic angle, we can continue polarization calibration using a split measurement set.

The setjy task will calculate the values of Stokes Q and U (in the reference channel) for user inputs of the reference frequency, Stokes I, polarization fraction, polarization angle, and rotation measure. The setjy input parameters can be obtained from Perley & Butler (2017) for Stokes I information and Perley & Butler (2013) for polarization information. Other sources can also be consulted, such as archival observations of variable polarization calibrators available under the project codes TPOL0003 or TCAL0009.

It is possible to capture a frequency variation in Q, U, and alpha terms by providing coefficients of polynomial expansion for polarization fraction, polarization angle, and spectral index as a function of frequency. At this time, it is left to the user to derive these coefficients, which can be accomplished by fitting a polynomial to observed values of the polarization fraction (here also called polarization index), polarization angle, and flux density (for the case of spectral index). Updated values of the broad band polarimetric information for the four calibration sources 3C48, 3C138, 3C147, and 3C286 can be found at (https://science.nrao.edu/facilities/vla/docs/manuals/oss/performance/fdscale) and at (https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/modes/pol); of these sources, 3C48, 3C138, and 3C147 have been found to be variable. These coefficients are then passed to the setjy task as lists along with the reference frequency and the Stokes I flux density.

The calibrator used for this guide, 3C48, has a rotation measure and thus changes its Q and U with frequency. Therefore, for our purposes, it is not sufficient to use only the first Taylor term of the expansion. For deriving the setjy input parameters you can consult the setjy CASA documentation. Currently setjy only supports unresolved polarized emission models assuming that the Stokes I,Q,U peak are co-located on the sky. This is not necessarily the case for more complicated objects or even for 3C48 in extended VLA configurations.

As an example on how to derive the polarization parameters for the setjy call, you can perform the following next steps or jump right to the setjy call below.

Deriving the Polarization Properties of the Polarization Angle Calibrator

First, we tabulate the frequency dependent Stokes I flux density, polarization fraction, and polarization angle in a textfile, which we will call 3C48.dat. The data is taken from [3] and the corresponding Stokes I value is calculated from the Perley & Butler (2017) scale.

Note: Due to a small change in the following python code users may notice small differences in results as compared to this guide. The principle of the method shown is still accurate and the results will be updated in the near future to fit the new code. 


# 3C48
# Date 31Jan/01Feb 2019 polarization properties
# Reference flux densities were used
# Frequency       I     P.F.  	  P.A.
# (GHz)         (Jy)         	 (rad)
1.022		20.68	0.00293	 0.07445
1.465		15.62	0.00457	-0.60282
1.865		12.88	0.00897	 0.39760
2.565		9.82	0.01548	-1.97046
3.565		7.31	0.02911	-1.46542
4.885		5.48	0.04286	-1.24875
6.680		4.12	0.05356	-1.15533
8.435		3.34	0.05430	-1.10638
11.320		2.56	0.05727	-1.08602
14.065		2.14	0.06097	-1.09597
16.564		1.86	0.06296	-1.11891
19.064		1.67	0.06492	-1.18266
25.564		1.33	0.07153	-1.25369
32.064		1.11	0.06442	-1.32430
37.064		1.00	0.06686	-1.33697
42.064		0.92	0.05552	-1.46381
48.064		0.82	0.06773	-1.46412

Now to fit Stokes I, we execute in CASA the following commands. These could also be put into a textfile and run from inside the CASA prompt using execfile. 

# In CASA

import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt

data = np.loadtxt('3C48.dat')

def S(f,S,alpha,beta):
        return S*(f/3.0)**(alpha+beta*np.log10(f/3.0))

# Fit 1 - 5 GHz data points
popt, pcov = curve_fit(S, data[0:10,0], data[0:10,1])
print('I@3GHz', popt[0], ' Jy')
print('alpha', popt[1])
print('beta', popt[2])
print( 'Covariance')
print(pcov)

plt.plot(data[0:10,0], data[0:10,1], 'ro', label='data')
plt.plot(np.arange(1,5,0.1), S(np.arange(1,5,0.1), *popt), 'r-', label='fit')

plt.title('3C48')
plt.legend()
plt.xlabel('Frequency (GHz)')
plt.ylabel('Flux Density (Jy)')
plt.show()

This will generate a plot for visual inspection, as well as the following text output. 

I@3GHz 8.461741965511111  Jy
alpha -0.9042738492254004
beta -0.14600432138647343
Covariance
[[ 3.18582527e-05  2.98095625e-06 -3.05312358e-06]
 [ 2.98095625e-06  3.65971413e-06  7.73132558e-06]
 [-3.05312358e-06  7.73132558e-06  2.12210577e-05]]

This provides the coefficients for Stokes I flux density at 3 GHz, the spectral index (alpha), and curvature (beta). It also provides the covariance matrix for the fit.

We repeat the same for the polarization fraction. 

# In CASA

import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt

data = np.loadtxt('3C48.dat')

def PF(f,a,b,c,d):
        return a+b*((f-3.0)/3.0)+c*((f-3.0)/3.0)**2+d*((f-3.0)/3.0)**3

# Fit 1 - 5 GHz data points
popt, pcov = curve_fit(PF, data[0:10,0], data[0:10,2])
print("Polfrac Polynomial: ", popt)
print("Covariance")
print(pcov)

plt.plot(data[0:10,0], data[0:10,2], 'ro', label='data')
plt.plot(np.arange(1,5,0.1), PF(np.arange(1,5,0.1), *popt), 'r-', label='fit')

plt.title('3C48')
plt.legend()
plt.xlabel('Frequency (GHz)')
plt.ylabel('Lin. Pol. Fraction')
plt.show()

Polfrac Polynomial:  [ 0.02142905  0.03918256  0.00234878 -0.02301251]
Covariance
[[ 1.21641658e-07  4.90635228e-08 -3.78036036e-07 -9.06935233e-08]
 [ 4.90635228e-08  2.17869783e-06  1.50406983e-07 -5.93646707e-06]
 [-3.78036036e-07  1.50406983e-07  2.10655624e-06 -4.88196071e-07]
 [-9.06935233e-08 -5.93646707e-06 -4.88196071e-07  1.86290160e-05]]

import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt

data = np.loadtxt('3C48.dat')

def PA(f,a,b,c,d,e):
        return a+b*((f-3.0)/3.0)+c*((f-3.0)/3.0)**2+d*((f-3.0)/3.0)**3+e*((f-3.0)/3.0)**4

# Fit 2 - 9 GHz data points
popt, pcov = curve_fit(PA, data[3:14,0], data[3:14,3])
print("Polangle Polynomial: ", popt)
print("Covariance")
print(pcov)

plt.plot(data[3:14,0], data[3:14,3], 'ro', label='data')
plt.plot(np.arange(1,9,0.1), PA(np.arange(1,9,0.1), *popt), 'r-', label='fit')

plt.title('3C48')
plt.legend()
plt.xlabel('Frequency (GHz)')
plt.ylabel('Lin. Pol. Angle (rad)')
plt.show()

Polangle Polynomial:  [-2.77397103  1.64922494 -1.51121195  0.47800055  0.97812597]
Covariance
[[ 0.00064289 -0.00019419 -0.00213386  0.0033356  -0.00153425]
 [-0.00019419  0.00999406 -0.00266842 -0.02268727  0.01468271]
 [-0.00213386 -0.00266842  0.03013314 -0.05191129  0.02532522]
 [ 0.0033356  -0.02268727 -0.05191129  0.18538412 -0.10450983]
 [-0.00153425  0.01468271  0.02532522 -0.10450983  0.06000995]]

Setting the Polarization Calibrator Models

# In CASA

# Reference Frequency for fit values
reffreq = '3.0GHz'
# Stokes I flux density
I =        8.461742
# Spectral Index
alpha =    [-0.904, -0.1460]
# Polarization Fraction
polfrac =  [0.02142905,  0.03918256,  0.00234878, -0.02301251]
# Polarization Angle
polangle = [-2.77397103,  1.64922494, -1.51121195,  0.47800055,  0.97812597]

setjy(vis='TDRW0001_calibrated.ms',
      field='0137+331=3C48',
      spw='',
      selectdata=False,
      timerange="",
      scan="",
      intent="",
      observation="",
      scalebychan=True,
      standard="manual",
      model="",
      modimage="",
      listmodels=False,
      fluxdensity=[I,0,0,0],
      spix=alpha,
      reffreq=reffreq,
      polindex=polfrac,
      polangle=polangle,
      rotmeas=0,
      fluxdict={},
      useephemdir=False,
      interpolation="nearest",
      usescratch=True,
      ismms=False,
)
  • field='0137+331=3C48' : if the flux density calibrator is not specified then all sources will be assumed to have the input model parameters.
  • standard='manual' : the user will supply the flux density, spectral index, and polarization parameters rather than giving a model (the CASA models currently do not include polarization).
  • fluxdensity=[I,0,0,0] : you may provide values of Q and U rather than having setjy calculate them.However, if you set Q and U as input using the fluxdensity parameter, then the first value given in polindex or polangle will be ignored.
  • spix=[-0.90366565, -0.14262821] : set the spectral index using the value above. This will apply to all non-zero Spokes parameters. In this example, we only use the first two coefficients of the Taylor expansion.
  • reffreq='3.0GHz' : The reference frequency for the input Stokes values.
  • polindex=[0.021429,0.0391826,0.00234878,-0.0230125 : The coefficients of polynomial expansion for the polarization index as a function of frequency.
  • polangle=[1.4215,1.36672,-2.12678,3.48384,-2.71914] : The coefficients of polynomial expansion for the polarization angle as a function of frequency.
  • scalebychan=True: This allows setjy to compute unique values per channel, rather than applying the reference frequency values to the entire spectral window.
  • usescratch=True: DO create/use the MODEL_DATA column explicitly. (usescratch=False saves disk space by not filling the model column)

The Stokes V flux has been set to zero, corresponding to no circular polarization.

Setjy returns a Python dictionary (CASA record) that reports the Stokes I, Q, U and V terms. This is reported to the CASA command line window (unless you used the execfile() method in which case results will be printed in the CASA log window only): 

{'0': {'0': {'fluxd': array([9.98507906, 0.1342557 , 0.04260663, 0.        ])},
  '1': {'fluxd': array([9.55165555, 0.13473475, 0.06659856, 0.        ])},
  '2': {'fluxd': array([9.15413885, 0.1320863 , 0.09023629, 0.        ])},
  '3': {'fluxd': array([8.78817831, 0.12654546, 0.11291257, 0.        ])},
  '4': {'fluxd': array([8.450107  , 0.11848356, 0.13411515, 0.        ])},
  '5': {'fluxd': array([8.13681135, 0.10834506, 0.15345622, 0.        ])},
  '6': {'fluxd': array([7.84562971, 0.09658875, 0.17067456, 0.        ])},
  '7': {'fluxd': array([7.57427253, 0.08364073, 0.18561939, 0.        ])},
  'fieldName': '0137+331=3C48'},
 'format': "{field Id: {spw Id: {fluxd: [I,Q,U,V] in Jy}, 'fieldName':field name }}"}

Alternatively, you may capture this dictionary in a return variable, if you call setjy as myset=setjy(...).

We can see the results in the model column in plotms (Figure 5A) showing the model source spectrum: 

# In CASA
plotms(vis='TDRW0001_calibrated.ms',field='0',correlation='RR',
       timerange='',antenna='ea01&ea02',
       xaxis='frequency',yaxis='amp',ydatacolumn='model')

We can see this translates to the spectrum in QU (Figure 5B): 

# In CASA
plotms(vis='TDRW0001_calibrated.ms',field='0',correlation='RL',
       timerange='',antenna='ea01&ea02',
       xaxis='frequency',yaxis='amp',ydatacolumn='model')

Finally, our R-L phase difference is constant at 66 degrees (twice the polarization angle) as desired (Figure 5C): 

# In CASA
plotms(vis='TDRW0001_calibrated.ms',field='0',correlation='RL',
       timerange='',antenna='ea01&ea02',
       xaxis='frequency',yaxis='phase',ydatacolumn='model')
Figure 5A: Model RR amplitudes of 3C48.
Figure 5B: Model RL amplitudes of 3C48.
Figure 5C: Model RL phases of 3C48.

In order to obtain the correct amplitude scaling for instrumental polarization calibration, we need to also specify the Stokes I model that was used for the D-term calibrator(s). If we carried all tables, instead of splitting out the calibrated data from the pipeline, we wouldn't need to do this since the gain amplitudes provide the correct Stokes I scale for all the calibrators. The model values of the two D-term calibrators can be obtained from the pipeline weblog under the task hifv_fluxboot inside the CASA log (file stage12/casapy.log). 

2021-12-07 23:11:53	INFO	fluxscale::::	 Fitted spectrum for J2355+4950 with fitorder=2: Flux density = 1.76871 +/- 0.000646424 (freq=2.98457 GHz) spidx: a_1 (spectral index) =-0.59957 +/- 0.00275983 a_2=-0.196834 +/- 0.0671363 covariance matrix for the fit:  covar(0,0)=8.14171e-08 covar(0,1)=-1.12868e-07 covar(0,2)=-2.46356e-05 covar(1,0)=-1.12868e-07 covar(1,1)=2.46143e-05 covar(1,2)=0.000242632 covar(2,0)=-2.46356e-05 covar(2,1)=0.000242632 covar(2,2)=0.0145659

2021-12-07 23:11:55	INFO	fluxscale::::	 Fitted spectrum for J0259+0747 with fitorder=2: Flux density = 0.970567 +/- 0.000712715 (freq=2.98457 GHz) spidx: a_1 (spectral index) =0.169924 +/- 0.00510264 a_2=-0.143228 +/- 0.134141 covariance matrix for the fit:  covar(0,0)=1.87165e-06 covar(0,1)=-2.56798e-06 covar(0,2)=-0.000583173 covar(1,0)=-2.56798e-06 covar(1,1)=0.000479142 covar(1,2)=-9.49244e-05 covar(2,0)=-0.000583173 covar(2,1)=-9.49244e-05 covar(2,2)=0.331129

This translates to the following setjy calls. 

setjy(vis='TDRW0001_calibrated.ms',
      field='J2355+4950',
      spw='',
      selectdata=False,
      timerange="",
      scan="",
      intent="",
      observation="",
      scalebychan=True,
      standard="manual",
      model="",
      modimage="",
      listmodels=False,
      fluxdensity=[1.76871, 0, 0, 0],
      spix=[-0.59957, -0.196834],
      reffreq="2984571609.0079317Hz",
      polindex=[],
      polangle=[],
      rotmeas=0,
      fluxdict={},
      useephemdir=False,
      interpolation="nearest",
      usescratch=True,
      ismms=False,
)

setjy(vis='TDRW0001_calibrated.ms',
      field='J0259+0747',
      spw='',
      selectdata=False,
      timerange="",
      scan="",
      intent="",
      observation="",
      scalebychan=True,
      standard="manual",
      model="",
      modimage="",
      listmodels=False,
      fluxdensity=[0.970567, 0, 0, 0],
      spix=[0.169924, -0.143228],
      reffreq='2984571609.0079317Hz',
      polindex=[],
      polangle=[],
      rotmeas=0,
      fluxdict={},
      useephemdir=False,
      interpolation="nearest",
      usescratch=True,
      ismms=False,
)

Solving for the Cross-Hand delays

Just as the pipeline did for the parallel-hand (RR,LL) delays before bandpass calibration, we solve for the cross-hand (RL,LR) delays because of the residual delay difference between the R and L on the reference antenna used for the original delay calibration (ea10 in this tutorial). In our case we simply use 3C48, which has a moderately polarized signal in the RL,LR correlations, and we set its polarized model above using setjy. Starting with former version of CASA (6.1.2) there are two options to solve for the cross-hand delays, both of them will be illustrated here. The first option fits the cross-hand delay for the entire baseband (here 8 spectral windows form a single baseband), which we call multiband delay. The second option solves the cross-hand delay independently per spectral window. Note that if a dataset contains multiple basebands and you wanted to solve for multiband delays, gaincal has to be executed for each baseband separately, selecting the appropriate spectral windows and appending the results to a single calibration table for later use. 

# In CASA

# Solve using Multiband Delay
kcross_mbd = "TDRW0001_calibrated.Kcross_mbd" 
gaincal(vis='TDRW0001_calibrated.ms',
    caltable=kcross_mbd,
    field='0137+331=3C48',
    spw='0~7:5~58',
    refant='ea10',
    gaintype="KCROSS",
    solint="inf",
    combine="scan,spw",
    calmode="ap",
    append=False,
    gaintable=[''],
    gainfield=[''],
    interp=[''],
    spwmap=[[]],
    parang=True)

# Solve using Single Band Delay
kcross_sbd = "TDRW0001_calibrated.Kcross_sbd"
gaincal(vis='TDRW0001_calibrated.ms',
    caltable=kcross_sbd,
    field='0137+331=3C48',
    spw='0~7:5~58',
    refant='ea10',
    gaintype="KCROSS",
    solint="inf",
    combine="scan",
    calmode="ap",
    append=False,
    gaintable=[''],
    gainfield=[''],
    interp=[''],
    spwmap=[[]],
    parang=True)
Figure 6: Single band cross-hand delay solutions.

We can plot the single band solutions (see Figure 6): 

# In CASA
plotms(vis=kcross_sbd,xaxis='frequency',yaxis='delay',antenna='ea10',coloraxis='corr')

You can also look at the solutions reported in the logger. 

For multiband delay there is one solution:
Multi-band cross-hand delay=3.72994 nsec

For single band delay there are 8 solutions:
Spw=0 Global cross-hand delay=5.64778 nsec
Spw=1 Global cross-hand delay=1.55875 nsec
Spw=2 Global cross-hand delay=-1.21461 nsec
Spw=3 Global cross-hand delay=0.571198 nsec
Spw=4 Global cross-hand delay=4.30379 nsec
Spw=5 Global cross-hand delay=1.29297 nsec
Spw=6 Global cross-hand delay=3.73613 nsec
Spw=7 Global cross-hand delay=3.06041 nsec

Notice that the per spectral window solutions are very scattered. The mean delay is 2.37 ns, quite different from the multiband delay. This demonstrates the strength of fitting the cross-hand delay across multiple spectral windows, especially when using a calibrator with a significant frequency dependence, i.e. rotation measure and a polarization fraction of only a few percent. We will continue calibration using the single multiband delay that was derived at 3.73 ns.

Note that if we did not solve for this delay, it would be absorbed into the phases per channel of the following Df and Xf solutions. This would not cause us problems if we used an unpolarized D-term calibrator like J2355+4950, because we would not be solving for the Q+iU polarization. But if we were (e.g., using our gain calibrator J0259+0747 with parameter poltype='Df+QU' ), then this step is essential.

Solving for the Leakage Terms

The task polcal is used for polarization calibration. In this data set, we observed the unpolarized calibrator J2355+4950 to demonstrate solving for the instrumental polarization. Task polcal uses the Stokes I, Q, and U values in the model data (Q and U being zero for an unpolarized calibrator) to derive the leakage solutions. We also observed the polarized calibrator J0259+0747 (which has about 4.7% fractional polarization) that is also our complex gain calibrator. The observations of J0259+0747 have a parallactic angle coverage of 31 degrees with 10 visits/slices, 3 of which were a bit longer to boost the signal-to-noise to at least 1000 per channel for each of the three passes. We will showcase solving for D-terms for both cases. The function calls are: 

# In CASA

# J2355+4950 / Df
dtab_J2355 = 'TDRW0001_calibrated.Df' 
polcal(vis='TDRW0001_calibrated.ms',
       caltable=dtab_J2355,
       field='J2355+4950',
       spw='0~7',
       refant='ea10',
       poltype='Df',
       solint='inf,2MHz',
       combine='scan',
       gaintable=[kcross_mbd],
       gainfield=[''],
       spwmap=[[0,0,0,0,0,0,0,0]], 
       append=False)

# J0259+0747 / Df+QU
dtab_J0259 = 'TDRW0001_calibrated.DfQU' 
polcal(vis='TDRW0001_calibrated.ms',
       caltable=dtab_J0259,
       intent='CALIBRATE_POL_LEAKAGE#UNSPECIFIED',
       spw='0~7',
       refant='ea10',
       poltype='Df+QU',
       solint='inf,2MHz',
       combine='scan',
       gaintable=[kcross_mbd],
       gainfield=[''],
       spwmap=[[0,0,0,0,0,0,0,0]], 
       append=False)
  • caltable : polcal will create a new calibration table containing the leakage solutions, which we specify with the caltable parameter.
  • field= or intent= : The unpolarized source J2355+4950 is used to solve for the leakage terms in the unpolarized case. For the polarized source J0259+0747 we set the intent leakage polarization.
  • spw='0~7' : Select all spectral windows.
  • poltype='Df' or poltype='Df+QU' : Solve for the leakages (D) on a per-channel basis (f), assuming zero source polarization, +QU will also solve for the calibrator polarization Q,U per spectral window.
  • solint='inf,2MHz', combine='scan' : One solution over the entire run, per spectral channel of 2 MHz
  • gaintable=['kcross_mbd']: The previous Kcross multiband delay is applied
  • spwmap=0,0,0,0,0,0,0,0: This applies a spectral window map, where the first spw solution in the kcross_mbd table is mapped to all other spectral windows. Note there is only one value listed inside the kcross calibration table which is for the lowest spectral window that was used when solving using the multiband delay option (i.e. combine='spw' ).

In the case of Df+QU, the logger window will show the Q/U values it derived for the calibrator and the corresponding polarization fraction and angle that can be derived. 

Fractional polarization solution for J0259+0747 (spw = 0): : Q = 0.0214174, U = 0.0366555  (P = 0.0424539, X = 29.8514 deg)
Fractional polarization solution for J0259+0747 (spw = 1): : Q = 0.0104099, U = 0.0393871  (P = 0.0407395, X = 37.5977 deg)
Fractional polarization solution for J0259+0747 (spw = 2): : Q = 0.0143639, U = 0.0392768  (P = 0.041821, X = 34.956 deg)
Fractional polarization solution for J0259+0747 (spw = 3): : Q = 0.0110499, U = 0.0424822  (P = 0.0438958, X = 37.71 deg)
Fractional polarization solution for J0259+0747 (spw = 4): : Q = 0.00892886, U = 0.040305  (P = 0.0412822, X = 38.7544 deg)
Fractional polarization solution for J0259+0747 (spw = 5): : Q = 0.00878222, U = 0.0408633  (P = 0.0417963, X = 38.9353 deg)
Fractional polarization solution for J0259+0747 (spw = 6): : Q = 0.00175604, U = 0.0429465  (P = 0.0429824, X = 43.8293 deg)
Fractional polarization solution for J0259+0747 (spw = 7): : Q = -0.00161836, U = 0.0480595  (P = 0.0480867, X = 45.9643 deg)

From this you can see that J0259+0747 has a fractional polarization of 4.1–4.8% across the 1 GHz bandwidth with a small rotation measure causing a change in angle from 29 to 46 degrees over 1 GHz. In cases where the derived Q/U values seem random and the fractional polarization seems to be very small you might be able to derive better D-term solutions by using poltype='Df' .

After we run the two executions of polcal, you are strongly advised to examine the solutions with plotms to ensure that everything looks good and to compare the results using two different calibrators and poltype methods.

Figure 7a: J0259+0747 Df amplitude vs. frequency for antenna ea01.
Figure 7b: J2355+4950 Df+QU amplitude vs. frequency for antenna ea01.
Figure 7c: J0259+0747 Df phase vs. frequency for antenna ea01.
Figure 7d: J2355+4950 Df+QU phase vs. frequency for antenna ea01.
# In CASA
plotms(vis=dtab_J0259,xaxis='freq',yaxis='amp', 
       iteraxis='antenna',coloraxis='corr')

plotms(vis=dtab_J2355,xaxis='freq',yaxis='amp', 
       iteraxis='antenna',coloraxis='corr')

plotms(vis=dtab_J0259,xaxis='chan',yaxis='phase', 
       iteraxis='antenna',coloraxis='corr',plotrange=[-1,-1,-180,180])

plotms(vis=dtab_J2355,xaxis='chan',yaxis='phase', 
       iteraxis='antenna',coloraxis='corr',plotrange=[-1,-1,-180,180])

This will produce plots similar to those shown in Figures 7A-D. You can cycle through the antennas by clicking the Next button within plotms. You should see leakages of between 5–17% in most cases. Both Df and Df+QU results should be comparable. However, we will be using the solutions from J0259+0747 to continue calibration and will use J2355+4950 to verify the polarization calibration.

We can also display these in a single plot versus antenna index (see Figure 8):

Figure 8: Df+QU solutions for J0259+0747 versus antenna index
# In CASA
plotms(vis=dtab_J0259,xaxis='antenna1',yaxis='amp',coloraxis='corr')

In some cases there are outlier solutions above 0.25 that are most likely due to residual RFI. You can flag those from the Dterm table using flagdata. If everything went correctly, then this step should not be necessary for this dataset. 

flagdata(vis=dtab_J2355, mode='clip', correlation='ABS_ALL', clipminmax=[0.0, 0.25], datacolumn='CPARAM', clipoutside=True, action='apply', flagbackup=False, savepars=False)

flagdata(vis=dtab_J0259, mode='clip', correlation='ABS_ALL', clipminmax=[0.0, 0.25], datacolumn='CPARAM', clipoutside=True, action='apply', flagbackup=False, savepars=False)

Solving for the R-L polarization angle

Having calibrated for the instrumental polarization, the total polarization is now correct, but the R-L phase still needs to be calibrated in order to obtain an accurate polarization position angle. We use the same task, polcal, but this time set parameter poltype='Xf', which specifies a frequency-dependent (f) position angle (X) calibration using the source 3C48, the position angle of which is known, having set this earlier with setjy. Note that we must correct for the leakages before determining the R-L phase, which we do by adding the calibration table made in the previous step (dtab_J0259) to the kcross table that is applied on-the-fly. 

# In CASA
xtab = "TDRW0001_calibrated.Xf"
polcal(vis='TDRW0001_calibrated.ms',
       caltable=xtab,
       spw='0~7',
       field='0137+331=3C48',
       solint='inf,2MHz',
       combine='scan',
       poltype='Xf',
       refant = 'ea10',
       gaintable=[kcross_mbd,dtab_J0259],
       gainfield=['',''],
       spwmap=[[0,0,0,0,0,0,0,0],[]],
       append=False)
Figure 9: Xf solutions versus frequency.

Strictly speaking, there is no need to specify a reference antenna for poltype='Xf' (for circularly polarized receivers only) because the X solutions adjust the cross-hand phases for each antenna to match the given polarization angle of the model. However, for consistency/safety, it is recommended to always specify refant when performing polarization calibration.

It is strongly suggested you check that the calibration worked properly by plotting up the newly-generated calibration table using plotms (see Figure 9): 

# In CASA
plotms(vis=xtab,xaxis='frequency',yaxis='phase',coloraxis='spw')

Because the Xf term captures the residual R-L phase on the reference antenna over the array, there is only one value for all antennas. Also, as we took out the RL delays using the Kcross solution, these Xf variations do not show a significant slope in phase. And since we were using a single multiband delay, the phases connect from one spectral window to another; had we used the single band delays, we would see phase jumps from one to another spectral window.

At this point, you have all the necessary polarization calibration tables.

Applying the Calibration

Now that we have derived all the calibration solutions, we need to apply them to the actual data using the task applycal. The measurement set DATA column contains the original split data. To apply the calibration we have derived, we specify the appropriate calibration tables which are then applied to the DATA column, with the results being written in the CORRECTED_DATA column. If the dataset does not already have a CORRECTED_DATA scratch column, then one will be created in the first applycal run. 

# In CASA
applycal(vis = 'TDRW0001_calibrated.ms',
         field='',
         gainfield=['', '', ''], 
         flagbackup=True,
         interp=['', '', ''],
         gaintable=[kcross_mbd,dtab_J0259,xtab],
         spw='0~7', 
         calwt=[False, False, False], 
         applymode='calflagstrict', 
         antenna='*&*', 
         spwmap=[[0,0,0,0,0,0,0,0],[],[]], 
         parang=True)
  • gaintable : We provide a Python list of the calibration tables to be applied. This list must contain the cross-hand delays (kcross), the leakage calibration (dtab; here derived from J0259+0747), and the R-L phase corrections (xtab).
  • calwt=[False] : At the time of this writing, we are not yet using system calibration data to compute real (1/Jy2) weights; trying to calibrate them can produce nonsensical results. Experience has shown that calibrating the weights will lead to problems, especially in the self-calibration steps. You can specify calwt on a per-table basis, here is set all to False.
  • parang : If polarization calibration has been performed, set parameter parang=True.

We should now have fully-calibrated visibilities in the CORRECTED_DATA column of the measurement set, and it is worthwhile pausing to inspect them to ensure that the calibration did what we expected it to. We make some standard plots (see Figures 10A-10F): 

# In CASA
plotms(vis='TDRW0001_calibrated.ms',field='0',correlation='',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       coloraxis='corr',
       plotfile='Plotms-3C48-fld0-corrected-amp-CASA6.2.1.jpeg')

plotms(vis='TDRW0001_calibrated.ms',field='0',correlation='',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotfile='Plotms-3C48-fld0-corrected-phase-CASA6.2.1.jpeg')

plotms(vis='TDRW0001_calibrated.ms',field='1',correlation='',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       plotfile='Plotms-J2355-fld1-corrected-amp-CASA6.2.1.jpeg')

plotms(vis='TDRW0001_calibrated.ms',field='1',correlation='RR,LL',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',
       plotfile='Plotms-J2355-fld1-corrected-phase-CASA6.2.1.jpeg')

plotms(vis='TDRW0001_calibrated.ms',field='2',correlation='',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='amp',ydatacolumn='corrected',
       plotfile='Plotms-J0259-fld2-corrected-amp-CASA6.2.1.jpeg')

plotms(vis='TDRW0001_calibrated.ms',field='2',correlation='',
       timerange='',antenna='',avgtime='60',
       xaxis='frequency',yaxis='phase',ydatacolumn='corrected',
       plotrange=[-1,-1,-180,180],coloraxis='corr',avgbaseline=True,
       plotfile='Plotms-J0259-fld2-corrected-phase-CASA6.2.1.jpeg')

For 3C48 (figures 10A, 10B) we see the polarized signal in the cross-hands; there is some sign of bad data remaining in 3C48. Also, the RL phase plots of J0259+4950 (figure 10F) indicate that the Xf solutions, thus polarization angles, in the lowest two spectral windows are problematic. You can also estimate from the RL,LR amplitudes in J2355+4950 (figure 10E) what the level of residual instrumental polarization, which we expect to be around <0.5%. A more accurate evaluation of residual instrumental polarization fraction can be made imaging the secondary D-term calibrator per spectral window and calculating its residual polarization.

Figure 10A: amplitude vs channel for 3C48 RR,RL,LR,LL
Figure 10B: phase vs channel for 3C48 RR,RL,LR,LL
Figure 10C: amplitude vs channel for J2355+4950 RR,LL,RL,LR
Figure 10D: phase vs channel for J2355+4950 RR,LL
Figure 10E: amplitude vs channel for J0259+4950 RR,LL,RL,LR
Figure 10F: phase vs channel for J0259+4950 RR,LL with baseline averaging


Inspecting the data at this stage may well show up previously-unnoticed bad data. Plotting the corrected amplitude against UV distance or against time is a good way to find such issues. If you find bad data, you can remove them via interactive flagging in plotms or via manual flagging in flagdata once you have identified the offending antennas/baselines/channels/times. When you are happy that all data (particularly on your target source) look good, you may proceed. However, especially for the target, we will return to additional flagging at a later stage.

Now that the calibration has been applied to the target data, we split off the science targets to create a new, calibrated measurement set containing the target field. This is not strictly necessary if you want to save disk space. 

# In CASA
split(vis='TDRW0001_calibrated.ms',outputvis='3C75.ms',
      datacolumn='corrected',field='3')
  • outputvis : We give the name of the new measurement set to be written, which will contain the calibrated data on the science target.
  • datacolumn : We use the CORRECTED_DATA column, containing the calibrated data which we just wrote using applycal.
  • field : We wish to target field into a measurement set for imaging and joint deconvolution.

Prior to imaging, it is a good idea to run the statwt task to correct the data weights (weight and sigma columns) in the measurement set. Running statwt will remove the effects of relative noise scatter that may have been introduced from flagging uneven bits in the visibility data between the channels and times. We will run this task here on the newly calibrated and split data set before moving on to imaging. 

# In CASA
statwt(vis='3C75.ms',datacolumn='data',minsamp=8)

Imaging

Now that we have split off the target data into a separate measurement set with all the calibration applied, it's time to make an image. Recall that the visibility data and the sky brightness distribution (a.k.a. image) are Fourier transform pairs.

[math]\displaystyle{ I(l,m) = \int V(u,v) e^{[2\pi i(ul + vm)]} dudv }[/math]

The [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] coordinates are the baselines measured in units of the observing wavelength, while the [math]\displaystyle{ l }[/math] and [math]\displaystyle{ m }[/math] coordinates are the direction cosines on the sky. In general, the sky coordinates are written in terms of direction cosines; but for most VLA (and ALMA) observations, they can be related simply to the right ascension ([math]\displaystyle{ l }[/math]) and declination ([math]\displaystyle{ m }[/math]). Recall that this equation is valid only if the [math]\displaystyle{ w }[/math] coordinate of the baselines can be neglected; this assumption is almost always true at high frequencies and smaller VLA configurations. The [math]\displaystyle{ w }[/math] coordinate cannot be neglected at lower frequencies and larger configurations (e.g., 0.33 GHz, A-configuration observations). This expression also neglects other factors, such as the shape of the primary beam. For more information on imaging, see the section of the CASA documentation.

Figure 11: plotms plot showing Amplitude vs UV Distance in wavelengths for 3C75 at 3000 MHz

CASA has a task tclean which both Fourier transforms the data and deconvolves the resulting image. We will use a multi-scale cleaning algorithm because our target source, a complex radio galaxy, contains both diffuse, extended structures on large spatial scales as well as point-like components. This approach will do a better job of modeling the image than the classic clean delta function. For broader examples of many tclean options, please see the Topical Guide for Imaging VLA Data.

Multi-scale Clean

It is important to have an idea of what values to use for the image pixel (cell) size and the overall size of the image. Setting the appropriate pixel size for imaging depends upon basic optics aspects of interferometry. Use plotms to look at the newly calibrated, target-only data set: 

# In CASA
plotms(vis='3C75.ms',xaxis='uvwave',yaxis='amp',
       ydatacolumn='data', field='0',avgtime='30',correlation='RR',
       plotfile='plotms_3c75-uvwave.jpeg',avgspw=False,overwrite=True)

You should obtain a plot similar to Figure 11 with the (calibrated) visibility amplitude as a function of [math]\displaystyle{ u }[/math]-[math]\displaystyle{ v }[/math] distance. You will also see some outliers there which are primarily from residual amplitude errors of ea05, that had a warm receiver which we can isolate to particular time periods. We will be addressing this after the initial imaging. The maximum baseline is about 12,000 wavelengths, i.e., an angular scale of 17 arcseconds ([math]\displaystyle{ \lambda/D=1/12000 }[/math]). The most effective cleaning occurs with 3–5 pixels across the synthesized beam. For example, a cell size of 3.4 arcseconds will give just about 5 pixels per beam.

The 3C75 binary black hole system is known to have a maximum extent of at least 8-9 arcminutes, corresponding to about 147 pixels for the chosen cell size. Therefore, we need to choose an image size that covers most of the extent of the source. To aid deconvolution, especially when bright sources far from phase center are present, we should at the minimum image the size of the primary beam. Although CASA has the feature that its Fourier transform engine (FFTW) does not require a strict power of 2 for the number of linear pixels in a given image axis, it is somewhat more efficient if the number of pixels on a side is a composite number divisible by any pair of 2 and 3 and/or 5. Because tclean internally applies a padding of 1.2 (=3x2/5), choose 480 which is 25 × 3 × 5 (so 480 × 1.2 = 576 = 26 × 32). We therefore set imsize=[480,480] and the source will fit comfortable within that image.

In this tutorial, we will run tclean interactively so that we can set and modify the mask: 

# In CASA
tclean(vis='3C75.ms',
       field="3C75",
       spw="",timerange="",
       uvrange="",antenna="",scan="",observation="",intent="",
       datacolumn="data",
       imagename="3C75_initial",
       imsize=480,
       cell="3.4arcsec",
       phasecenter="",
       stokes="IQUV",
       projection="SIN",
       specmode="mfs",
       reffreq="3.0GHz",
       nchan=-1,
       start="",
       width="",
       outframe="LSRK",
       veltype="radio",
       restfreq=[],
       interpolation="linear",
       gridder="standard",
       mosweight=True,
       cfcache="",
       computepastep=360.0,
       rotatepastep=360.0,
       pblimit=0.0001,
       normtype="flatnoise",
       deconvolver="mtmfs",
       scales=[0, 6, 18],
       nterms=2,
       smallscalebias=0.6,
       restoration=True,
       restoringbeam=[],
       pbcor=False,
       outlierfile="",
       weighting="briggs",
       robust=0.5,
       npixels=0,
       uvtaper=[],
       niter=20000,
       gain=0.1,
       threshold=0.0,
       nsigma=0.0,
       cycleniter=1000,
       cyclefactor=1.0,
       restart=True,
       savemodel="modelcolumn",
       calcres=True,
       calcpsf=True,
       parallel=False,
       interactive=True)
Figure 12: Interactive clean at the beginning, having selected polygon region and ready to double-click inside to set the mask.

Task tclean is powerful with many inputs and a certain amount of experimentation likely is required.

  • vis='3C75.ms' : this split MS contains the target field only.
  • imagename='3C75_initial' : our output image cubes will all start with this name root, e.g., 3C75_initial.image
  • specmode='mfs' : Use multi-frequency synthesis imaging. The fractional bandwidth of these data is non-zero (1000 MHz at a central frequency of 3.0 GHz). Recall that the [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math] coordinates are defined as the baseline coordinates, measured in wavelengths. Thus, slight changes in the frequency from channel to channel result in slight changes in [math]\displaystyle{ u }[/math] and [math]\displaystyle{ v }[/math]. There is a concomitant improvement in [math]\displaystyle{ u }[/math]-[math]\displaystyle{ v }[/math] coverage if the visibility data from the multiple spectral channels are gridded separately onto the [math]\displaystyle{ u }[/math]-[math]\displaystyle{ v }[/math] plane, as opposed to treating all spectral channels as having the same frequency.
  • niter=20000,gain=0.1,threshold='0.0mJy' : Recall that the gain is the amount by which a clean component is subtracted during the cleaning process. Parameters niter and threshold are (coupled) means of determining when to stop the cleaning process, with niter specifying to find and subtract that many clean components while threshold specifies a minimum flux density threshold a clean component can have before tclean stops (also see interactive below). Imaging is an iterative process, and to set the threshold and number of iterations, it is usually wise to clean interactively in the first instance, stopping when spurious emission from sidelobes (arising from gain errors) dominates the residual emission in the field. Here, we have set the threshold level to zero and let the tclean task define an appropriate threshold. The number of iterations should then be set high enough to reach the threshold found by tclean.
  • gridder='standard' : The standard tclean gridder is sufficient for our purposes, since we are not combining multiple pointings from a mosaic or try to perform widefield imaging in an extended configuration.
  • interactive=True : Very often, particularly when one is exploring how a source appears for the first time, it can be valuable to interact with the cleaning process. If True, interactive causes a viewer window to appear. One can then set clean regions, restricting where tclean searches for clean components, as well as monitor the cleaning process. A standard procedure is to set a large value for niter, and stop the cleaning when it visually appears to be approaching the noise level. This procedure also allows one to change the cleaning region, in cases when low-level intensity becomes visible as the cleaning process proceeds.
  • imsize=480,cell='3.4arcsec' : See the discussion above regarding setting the image size and cell size. If only one value is specified for the parameter, the same value is used in both directions (declination and right ascension).
  • stokes='IQUV' : tclean will output an image cube containing all: total intensity I, and Stokes Q, U, and V.
  • deconvolver='multiscale', scales=[0, 6, 18], smallscalebias=0.9 : The settings for multiscale are in units of pixels, with 0 pixels equivalent to the traditional delta-function clean. The scales here are chosen to provide delta functions and then two logarithmically scaled sizes to fit to the data. The first scale (6 pixels) is chosen to be comparable to the size of the synthesized beam. The smallscalebias attempts to balance the weight given to larger scales, which often have more flux density, and the smaller scales, which often are brighter. Considerable experimentation is likely to be necessary; one of the authors of this document found that it was useful to clean several rounds with this setting, change to multiscale=[] and remove much of the smaller scale structure, then return to this setting.
  • weighting='briggs',robust=0.5 : 3C75 has diffuse, extended emission that is, at least partially, resolved out by the interferometer even though we are in the most compact VLA configuration. A naturally-weighted image would show large-scale patchiness in the noise. In order to suppress this effect, Briggs weighting is used (intermediate between natural and uniform weighting), with a default robust factor of 0.5 (which corresponds to something between natural and uniform weighting).
  • pbcor=False : by default pbcor=False and a flat-noise image is produced. We can do the primary beam correction later (see below).
  • savemodel='modelcolumn' : We recommend here the use of a physical MODEL_DATA scratch column. This will save some time, as it can be faster in the case of complicated gridding to read data from disk instead of doing all of the computations on-the-fly. However, this has the unfortunate side effect of increasing the size of the MS on disk.
Figure 13: After the first approximately 300 iterations of multi-scale mfs clean

As mentioned above, we can guide the clean process by allowing it to find clean components only within a user-specified region. When tclean runs in interactive mode, an imview window will pop up as shown in Figure 12. First, you'll want to navigate to the green box and select "All Polarizations" rather than use the default "This Polarization"; this way the cleaning we are about to do will apply to all of the polarizations rather than just the one we are currently viewing. Similarly, select "All channels". To get a more detailed view of the central regions containing the emission, zoom in by first left clicking on the zoom button (leftmost button in third row) and tracing out a rectangle with the left mouse button and double-clicking inside the zoom box you just made. Play with the color scale to bring out the emission better by holding down the middle mouse button and moving it around. To create a clean box (a region within which components may be found), hold down the right mouse button and trace out a rectangle around the source, then double-click inside that rectangle to set it as a box. Note that the clean box must turn white for it to be registered - if the box is not white, it has not been set. Alternatively, you can trace out a more custom shape to better enclose the irregular outline of the radio galaxy jets. To do this, right-click on the closed polygonal icon then trace out a shape by right-clicking where you want the corners of that shape. Once you have come full circle, the shape will be traced out in green, with small squares at the corners. Double-click inside this region and the green outline will turn white. You have now set the clean region. If you have made a mistake with your clean box, click on the Erase button, trace out a rectangle around your erroneous region, and double-click inside that rectangle. You can also set multiple clean regions.

At any stage in the cleaning, you can adjust the number of iterations that tclean will do before returning to the GUI (cycleniter). This is set to 1000 (see the iterations field in mid-upper left of panel), values from 500 to 1000 later on seem to work. Note that this will override the cycleniter value that you might had set before starting tclean. tclean will keep going until it reaches threshold or runs out of cycles (the cycles field to the right of the iterations).

Figure 14: Interactive residuals after about 13000 iterations of multi-scale mfs clean

When you are happy with the clean regions, press the green circular arrow button on the far right to continue deconvolution. After completing a cycle, a revised image will come up. As the brightest points are removed from the image (cleaned off), fainter emission may show up. You can adjust the clean boxes each cycle, to enclose all real emission. After many cycles, when only noise is left, you can hit the red-and-white stop-sign icon to stop cleaning. Figure 13 shows the interactive viewer panel later in the process, after cleaning about 500 iterations. We have used the polygon tool to add to the clean region, drawing around emission that shows up in the residual image outside of the original clean region. After about 13000 iterations (Figure 14) the residuals were looking good (similar noise level inside and outside of the cleaned mask region). As mentioned before, restarting tclean with different multiscale=[...] choices can help also. You see that there is a significant amount of residual structure, these are most likely due to calibration errors which we will try to correct for in the next section during self-calibration.

Task tclean will make several output files, all named with the prefix given as imagename. These include:

  • .image: final restored image(s) with the clean components convolved with a restoring beam and added to the remaining residuals at the end of the imaging process, one for each Taylor Term (.tt0 and .tt1)
  • .pb.tt0: effective response of the telescope (the primary beam)
  • .mask: areas where tclean has been allowed to search for emission
  • .model: sum of all the clean components, which also has been stored as the MODEL_DATA column in the measurement set, one for each Taylor Term (.tt0 and .tt1)
  • .psf: dirty beam, which is being deconvolved from the true sky brightness during the clean process, one for each Taylor Term (.tt0, .tt1, .tt2)
  • .residual: what is left at the end of the deconvolution process; this is useful to diagnose whether or not to clean more deeply, one for each Taylor Term (.tt0, .tt1)
  • .sumwt: a single pixel image containing sum of weights per plane, one for each Taylor Term (.tt0, .tt1, .tt2)
Figure 15A: Viewer panel of final restored Stokes I image (using HotMetal1 colormap and Scaling Power Cycles = -1)
Figure 15B: Viewer panel of final restored Stokes Q image (using HotMetal1 colormap and Scaling Power Cycles = -1)
Figure 15C: Viewer panel of final restored Stokes U image (using HotMetal1 colormap and Scaling Power Cycles = -1)
Figure 15D: Viewer panel of final restored Stokes V image (using HotMetal1 colormap and Scaling Power Cycles = -1)

After the imaging and deconvolution process has finished, you can use the imview to look at your image. 

# In CASA
imview('3C75_initial.image.tt0')

You can adjust the color scale and zoom in to a selected region by assigning mouse buttons to the icons immediately above the image (hover over the icons to get a description of what they do). Also, using the wrench panel to change Display Options will be helpful here. Here we selected the Hot Metal 1 colormap and set the Scaling Power Cycles to -1 to better emphasize the faint emission and compare to the noise (Figures 15A - D). You can also use the Animators slider for Stokes to switch between the four different Stokes parameter images that were computed.

The tclean task naturally operates in a flat noise image, i.e., an image where the effective weighting across the field of view is set so that the noise is constant. This is so that the clean threshold has a uniform meaning for the stopping criterion and that the image fed into the minor cycles has uniform noise levels. This means, however, that the image does not take into account the primary beam response fall-off in the edges. In principle, tclean produces primary beam response image, and if we had set parameter pbcor=True tclean would had saved a primary beam corrected restored image of our target. Since we used deconvolver='mtmfs' and nterms=2, the calculation of the primary beam response requires special treatment. To perform wideband primary beam correction, we will use task widebandpbcor. In the future this task will be incorporated into tclean, but until then this separate task needs to be used. 

# In CASA
widebandpbcor(vis='3C75.ms,'imagename='3C75_initial',nterms=2, action='pbcor'
              spwlist=[0,1,2,3,4,5,6,7], chanlist=[32,32,32,32,32,32,32,32], weightlist=[1,1,1,1,1,1,1,1])

The task will produce primary beam corrected images of our target (3C75_initial.pbcor.image.tt0, 3C75_initial.pbcor.image.tt1, 3C75_initial.pbcor.image.alpha, 3C75_initial.pbcor.image.alpha.error). You can open image 3C75_initial.pbcor.image.tt0 in the imview, and compare it to screenshots in Figure 15. You will see noise (and signal) at the edges of the image has indeed increased.

Self-Calibration

Before we get started with self-calibration, it might be good to check whether we need to perform additional flagging on the target data. Since we have established an image model in the previous section, we can use it to look at the residuals by dividing out the model. We can make a similar plot to Figure 11 above, however, we will divide the image model that was created. Since we performed full-polarization imaging, we can also do the same to the cross-hand data RL,LR. Figures 16A & B shows example plots. You should also have a look at time plotted against amplitude and frequency against amplitude to see if there are any obvious times of interference. 

# In CASA
plotms(vis='3C75.ms',xaxis='uvdist',yaxis='amp',plotrange=[0,0,0,20],
       ydatacolumn='data/model_vector', field='3C75',avgtime='30',correlation='RR',
       plotfile='plotms_3c75-uvdist_resid_RR.png',avgspw=False,overwrite=True)

# If you made a mistake above and didn't clean the polarization as well, then this plot will be empty.
plotms(vis='3C75.ms',xaxis='uvdist',yaxis='amp',plotrange=[0,0,0,20],
       ydatacolumn='data/model_vector', field='3C75',avgtime='30',correlation='RL',
       plotfile='plotms_3c75-uvdist_resid_RL.png',avgspw=False,overwrite=True)
Figure 16A: plotms plot showing Amplitude vs UV Distance residuals in wavelengths for 3C75 for RR correlations.
Figure 16B: plotms plot showing Amplitude vs UV Distance residuals in wavelengths for 3C75 for RL correlations.

Since we are seeing a significant amount of weak residual interference, we will take a few steps to reduce these. There seem to be spikes at scan boundaries, we will use the mode quack to remove the first unflagged integrations from the beginning and end of each target scan. 

# In CASA

# quack
cmd = ["mode='quack' quackmode='beg' quackincrement=True quackinterval=5.0",
       "mode='quack' quackmode='endb' quackincrement=False quackinterval=5.0"]
flagdata(vis='3C75.ms',mode='list',inpfile=cmd,flagbackup=False)
# tfcrop
flagdata(vis='3C75.ms',mode='tfcrop',correlation='ABS_RR,LL',freqfit='line',extendflags=False,flagbackup=False,datacolumn='residual_data',flagdimension='freq',ntime='scan')
flagdata(vis='3C75.ms',mode='tfcrop',correlation='ABS_RL,LR',freqfit='line',extendflags=False,flagbackup=False,datacolumn='residual_data',flagdimension='freq',ntime='scan')
# rflag
flagdata(vis='3C75.ms',mode='rflag',correlation='RR,LL',extendflags=False,flagbackup=False,datacolumn='residual_data',ntime='scan')
flagdata(vis='3C75.ms',mode='rflag',correlation='RL,LR',extendflags=False,flagbackup=False,datacolumn='residual_data',ntime='scan')
# extend flags
flagdata(vis='3C75.ms',mode='extend',flagbackup=False)

This should have gotten rid of the worst remaining outliers, but will leave some residual weak RFI on certain baseline lengths. Since we are not trying to win any records on high dynamic range imaging, this additional flagging should suffice for our dataset.

In addition to residual RFI, even after calibration using the amplitude calibrator and the phase calibrator, there are likely to be residual phase and/or amplitude errors in the data. Self-calibration uses an existing model, often constructed from imaging the data itself, provided that sufficient visibility data have been obtained. This is essentially always the case with data: the system of equations is wildly over-constrained for the number of unknowns.

More specifically, the observed visibility data on the [math]\displaystyle{ i }[/math]-[math]\displaystyle{ j }[/math] baseline can be modeled as:

[math]\displaystyle{ V'_{ij} = G_i G^*_j V_{ij} }[/math]

where [math]\displaystyle{ G_i }[/math] is the complex gain for the [math]\displaystyle{ i^{\mathrm{th}} }[/math] antenna and [math]\displaystyle{ V_{ij} }[/math] is the true visibility. For an array of [math]\displaystyle{ N }[/math] antennas, at any given instant, there are [math]\displaystyle{ N(N-1)/2 }[/math] visibility data, but only [math]\displaystyle{ N }[/math] gain factors. For an array with a reasonable number of antennas, [math]\displaystyle{ N }[/math] >~ 8, solutions to this set of coupled equations converge quickly. There is some discussion in the old CASA Reference Manual on self calibration (see Section 5.11), but more detailed discussion can be found in lectures on Self-calibration given at NRAO community days.

In self-calibrating data, it is useful to keep in mind the structure of a Measurement Set. There are three columns of interest for an MS: the DATA column, the MODEL column, and the CORRECTED_DATA column. In normal usage, as part of the initial split, the CORRECTED_DATA column is set equal to the DATA column. The self-calibration procedure is then:

  • Produce an image (tclean) using the CORRECTED_DATA column.
  • Derive a series of gain corrections (gaincal) by comparing the DATA columns and the Fourier transform of the image, which is stored in the MODEL column. These corrections are stored in an external table.
    • Optionally, we can also derive a bandpass correction—which is also referred to as bandpass self calibration—to correct for global amplitude errors.
  • Apply these corrections (applycal) to the DATA column to form a new CORRECTED_DATA column overwriting the previous contents of CORRECTED_DATA.

The following example begins with the standard data set, 3C75.ms (resulting from the steps above). We have previously generated an IQUV multiscale image cube. We discard it for this step and create a new Stokes I image, which we will use to generate a series of gain corrections (phase only self-calibration) that will be stored in 3C75.ScG0. With this solution, we then perform bandpass self-calibration to remove any amplitude slope that might be present. Next, we apply the derived phase and amplitude corrections to the data to form a set of self-calibrated data, and then re-image the dataset (3C75_selfcal.image). For the purpose of self-calibration, note that in the clean before the self-calibration, it is important that we only use the Stokes I model so that any cleaned polarization does not affect the gaincal. We first use delmod on the MS to get rid of the previous polarized model, and run tclean to generate Stokes I-only image. In principle, it is possible to use the previous image cube and extract the Stokes I model using the CASA toolkit and have tclean fill the model column appropriately. For simplicity, we just re-image with tclean selecting only Stokes I. 

#In CASA
delmod('3C75.ms')

tclean(vis='3C75.ms',
       field="3C75",
       spw="",timerange="",
       uvrange="",antenna="",scan="",observation="",intent="",
       datacolumn="data",
       imagename="3C75_initial_I",
       imsize=480,
       cell="3.4arcsec",
       phasecenter="",
       stokes="I",
       projection="SIN",
       specmode="mfs",
       reffreq="3.0GHz",
       nchan=-1,
       start="",
       width="",
       outframe="LSRK",
       veltype="radio",
       restfreq=[],
       interpolation="linear",
       gridder="standard",
       mosweight=True,
       cfcache="",
       computepastep=360.0,
       rotatepastep=360.0,
       pblimit=0.0001,
       normtype="flatnoise",
       deconvolver="mtmfs",
       scales=[0, 6, 18],
       nterms=2,
       smallscalebias=0.6,
       restoration=True,
       restoringbeam=[],
       pbcor=False,
       outlierfile="",
       weighting="briggs",
       robust=0.5,
       npixels=0,
       uvtaper=[],
       niter=3500,
       gain=0.1,
       threshold=0.0,
       nsigma=0.0,
       cycleniter=750,
       cyclefactor=1.0,
       restart=True,
       savemodel="modelcolumn",
       calcres=True,
       calcpsf=True,
       parallel=False,
       interactive=True)

As discussed, this tclean call will ignore the polarized structure. You should not clean very deeply at this point. You want to be sure to capture as much of the source's total flux density as possible, but not include low level questionable features or sub-structures (ripples) that might be due to calibration or deconvolution artifacts. We modified the two parameters controlling tclean's minor and major cycles to the following values cycleniter=750 and niter=3500 to reflect this, but you may find that you don't even need 3500 iterations for this first tclean pass.

If you are happy with the new image, perform the following self-calibration steps: 

#In CASA
# In CASA
gaincal(vis='3C75.ms', caltable='3C75.ScG0', field='', solint='inf', refant='ea10', 
           spw='',minsnr=3.0, gaintype='G', parang=False, calmode='p')

bandpass(vis='3C75.ms', caltable='3C75.ScB0', field='', solint='inf', refant='ea10', minsnr=3.0, spw='',
                parang = False, gaintable=['3C75.ScG0'], interp=[])

applycal(vis='3C75.ms', gaintable=['3C75.ScG0','3C75.ScB0'], spw='', applymode='calflagstrict', parang=False)

The CORRECTED_DATA column of the MS now contains the self-calibrated visibilities which will be used by next execution of tclean. The gaincal step will report a number of solutions with insufficient SNR. By default, with parameter applymode='calflag', data with no good solutions will be flagged by applycal which may or may not be a good thing. You can control the action of applycal by changing the value of parameter applymode. Setting applymode='calflagstrict' will be more stringent about flagging data points without valid calibration, while applymode='calonly' will calibrate those with solutions while passing unchanged the data without solutions. You can see ahead of time what applycal will do by executing it with applymode='trial' which will do the reporting but nothing else. In our example we used applymode='calflagstrict' , but you will notice that the reported flagged fraction has not changed much, only increasing by 0.5%. This is a good thing.

Having applied these gain and bandpass solutions, we will once again image the target measurement set which we now expect to have better gain solutions and consequently produce a better image. We do this by invoking the tclean command once again. 

#In CASA
tclean(vis='3C75.ms',
       field="3C75",
       spw="",timerange="",
       uvrange="",antenna="",scan="",observation="",intent="",
       datacolumn="corrected",
       imagename="3C75_selfcal_1",
       imsize=480,
       cell="3.4arcsec",
       phasecenter="",
       stokes="I",
       projection="SIN",
       specmode="mfs",
       reffreq="3.0GHz",
       nchan=-1,
       start="",
       width="",
       outframe="LSRK",
       veltype="radio",
       restfreq=[],
       interpolation="linear",
       gridder="standard",
       mosweight=True,
       cfcache="",
       computepastep=360.0,
       rotatepastep=360.0,
       pblimit=0.0001,
       normtype="flatnoise",
       deconvolver="mtmfs",
       scales=[0, 6, 18],
       nterms=2,
       smallscalebias=0.6,
       restoration=True,
       restoringbeam=[],
       pbcor=False,
       outlierfile="",
       weighting="briggs",
       robust=0.5,
       npixels=0,
       uvtaper=[],
       niter=3500,
       gain=0.1,
       threshold=0.0,
       nsigma=0.0,
       cycleniter=750,
       cyclefactor=1.0,
       restart=True,
       savemodel="modelcolumn",
       calcres=True,
       calcpsf=True,
       parallel=False,
       interactive=True)

Commonly this self-calibration procedure is applied multiple times. In Figures 17A & B you can see a comparison of the shallow Stokes I image before self-calibration and after two self-calibration steps. The first self-calibration round was done as instructed in this section, while the second round was executed with solint='120s' and new solution tables were created (3C75.ScG1, 3C75.ScB1).

Figure 17A: Shallow Stokes I image before self-calibration.
Figure 17B: Stokes I image after two rounds of self-calibration.

The number of iterations is determined by a combination of the data quality, the number of antennas in the array, the structure of the source, the extent to which the original self-calibration assumptions are valid, and the user's patience. With reference to the original self-calibration equation above, if the observed visibility data cannot be modeled well by this equation, no amount of self-calibration will help. A not uncommon limitation for moderately high dynamic range imaging is that there may be baseline-based factors that modify the true visibility. If the corruptions to the true visibility cannot be modeled as antenna-based, as they are above, self-calibration won't help.

Self-calibration requires experimentation. Do not be afraid to remove an image, or even a set of gain corrections, change something and try again. Having said that, here are several guidelines to consider:

  • Bookkeeping is important! Suppose one conducts 9 iterations of self-calibration. Will it be possible to remember one month later (or maybe even one week later!) which set of gain corrections and images are which? In the example above, the descriptor 'selfcal_1' is attached to various files to help keep straight what is what. Successive iterations of self-cal could then be 'selfcal_2' , 'selfcal_3' , etc.
  • Care is required in setting imagename. If one has an image that already exists, CASA will continue cleaning it (if it can), which is almost certainly not what one wants during self-calibration. Rather, use a unique imagename for each pass of self-calibration.
  • A common metric for self-calibration is whether the dynamic range (= peak flux density/rms) of the image has improved. An improvement of 10% is quite acceptable.
  • Be careful when making images and setting clean regions or masks; self-calibration assumes that the model is perfect. If one cleans a noise bump, self-calibration will quite happily try to adjust the gains so that the CORRECTED_DATA describe a source at the location of the noise bump. It is far better to exclude some features of a source, or a weak source, from initial cleaning and conduct another round of self-calibration than to create an artificial source. If a real source is excluded from initial cleaning, it will continue to be present in subsequent iterations of self-calibration; if it's not a real source, one probably isn't interested in it anyway.
  • Start self-calibration with phase-only solutions (parameter calmode='p' in gaincal). As discussed in the High Dynamic Range Imaging lecture, a phase error of 20 deg is as bad as an amplitude error of 10%.
  • In initial rounds of self-calibration, consider solution intervals longer than the nominal sampling time (parameter solint in gaincal) and/or lower signal-to-noise ratio thresholds (parameter minsnr in gaincal). Depending upon the frequency and configuration and fidelity of the model image, it can be quite reasonable to start with solint='30s' or solint='60s' and/or minsnr=3. One may also want to consider specifying a uvrange, if, for example, the field has structure on large scales (small [math]\displaystyle{ u }[/math]-[math]\displaystyle{ v }[/math]) that is not well represented by the current image.
  • The task applycal will flag data with no good calibration solutions. During the initial self-calibration steps, this flagging may be excessive. If so, one can restore the flags to the state right before running applycal by using the task flagmanager.
  • You can track the agreement between the DATA, CORRECTED_DATA, and MODEL in plotms. The options in Axes tab allows one to select which column is to be plotted. If the MODEL agrees well with the CORRECTED_DATA, one can use shorter solint and/or higher minsnr values.
  • You should consider examining the solutions from gaincal by using plotms in order to assure that the corrections are sensible. Smoothly varying phases are good, jumps are usually not. (However, because the phases are often plotted ±180 degrees, there can be apparent jumps if the phases are very near +180 deg or −180 deg.)

Final Polarization Images

At this point, satisfied with the results of self-calibration, it might be a good idea to recalculate the visibility weights since some additional flagging was performed. After this, we get right to full-polarization imaging. We also suspect that there is a bright source outside of the masked field causing some imaging artifacts due to not being cleaned. We thus set the parameter pbmask value to 0.0 in order to disable masking of areas beyond the primary beam, and make the image larger to incorporate the bright source into our model in this tclean execution. 

# In CASA
statwt(vis='3C75.ms', minsamp=8, datacolumn='corrected', flagbackup=True)

tclean(vis='3C75.ms',
       field="3C75",
       spw="",timerange="",
       uvrange="",antenna="",scan="",observation="",intent="",
       datacolumn="corrected",
       imagename="3C75_final_large",
       imsize=800,
       cell="3.4arcsec",
       phasecenter="",
       stokes="IQUV",
       projection="SIN",
       specmode="mfs",
       reffreq="3.0GHz",
       nchan=-1,
       start="",
       width="",
       outframe="LSRK",
       veltype="radio",
       restfreq=[],
       interpolation="linear",
       gridder="standard",
       mosweight=True,
       cfcache="",
       computepastep=360.0,
       rotatepastep=360.0,
       pblimit=-0.0001,
       pbmask=0.0,
       normtype="flatnoise",
       deconvolver="mtmfs",
       scales=[0, 6, 18],
       nterms=2,
       smallscalebias=0.6,
       restoration=True,
       restoringbeam=[],
       pbcor=False,
       outlierfile="",
       weighting="briggs",
       robust=0.5,
       npixels=0,
       uvtaper=[],
       niter=20000,
       gain=0.1,
       threshold=0.0,
       nsigma=0.0,
       cycleniter=1000,
       cyclefactor=1.0,
       restart=True,
       savemodel="modelcolumn",
       calcres=True,
       calcpsf=True,
       parallel=False,
       interactive=True)

The final restored Stokes I,Q,U, and V images are shown in Figures 18A–D. Note that there is still a star like pattern in the residuals which are artifacts most likely due to the multi-scale multi-term multi-frequency synthesis. You can try on your own to improve upon the shown images by re-imaging and choosing a different set of multi-scale parameters that better match the scales found in the extended structure of 3C 75. Another issue to point out is looking at the Stokes V image. We do not expect a significant amount of Stokes V emission from this object, the emission you are seeing in Stokes V is most likely an effect of incorrectly solving for polarization leakages in the primary beam. In the above calibration we have only addressed leakage between the two polarization referring to the phase center. The extended beam itself, however, shows leakage which manifests itself spatially. The extended polarized emission we see in the Stokes Q and U images is not corrected for beam polarization during imaging. This, in turn, contains errors leading to polarization and de-polarization effects and causes changes to the polarization angle which effect increases the further away once gets from the beam center. Additionally, the two polarization beams do not sit on top of each other but are slightly offset, introducing a polarization squint. For correct and accurate polarization imaging, these two effects have to be taken into account. Imaging algorithms to address beam polarization are currently under development and will be discussed in this guide when they become available to the general user.

Figure 18A: Viewer panel of final restored Stokes I image.
Figure 18B: Viewer panel of final restored Stokes Q image.
Figure 18C: Viewer panel of final restored Stokes U image.
Figure 18D: Viewer panel of final restored Stokes V image.

Note, that these images are not yet primary beam corrected.

Spectral & Polarization Maps

If you want to obtain a reasonable map of the in-band spectral index, like the one shown in Fig. 19A, we can compute it with the task widebandpbcor. As demonstrated earlier, the task can also correct the images for the telescope's primary beam response corrected; this correction will make the images science ready. Parameter action='pbcor' will perform both actions (correct for the primary beam and calculate spectral index map) while parameter threshold sets minimum flux density above which the spectra index is calculated (this will allow us to mask all the noise regions). 

# In CASA
widebandpbcor(vis='3C75.ms, 'imagename='3C75_final_large', nterms=2, threshold='1.0mJy/beam', action='pbcor'
              spwlist=[0,1,2,3,4,5,6,7], chanlist=[32,32,32,32,32,32,32,32], weightlist=[1,1,1,1,1,1,1,1])


For further study of polarization properties, you might want to convert the Stokes images into something more useful for scientific analysis. We will use CASA to calculate polarization intensity (sqrt(Q^2 + U^2)/I) and polarization angle (0.5 arctan2 (U/Q)) maps from the final Stokes I,Q,U images. You can then look at those with the imview. For example, Figure 19B shows the polarization intensity image. Since we haven't applied any mask the polarization angle image will also contain values for low S/N or noise values. 

# In CASA

# Obtain image for the polarization intensity
immath(outfile='3C75_final.poli',mode='poli',imagename=['3C75_final_large.image.tt0'],sigma='0.0Jy/beam')
# Obtain image for the polarization angle
immath(outfile='3C75_final.pola',mode='pola',imagename=['3C75_final_large.image.tt0'],sigma='0.0Jy/beam')
Figure 19A: Computed spectral index map.
Figure 19B: Computed polarization intensity image.
Figure 19C: Computed polarized angles (vectors) superposed on the Stokes I raster image plane.

Note that for calculations of the polarization intensity and angle images you may — but do not need to — use primary beam corrected images; your results will be the same. This is because the primary beam correction cancels out in the equations for these two polarization quantities. If you want to visualize the polarization vectors on top of the Stokes I image, we need to add a mask for the low noise values. 

# In CASA
!cp -r '3C75_final.poli' polimg

imsubimage(imagename='3C75_final_large.image.tt0',outfile='3C75_final.Q.image',stokes='Q')
imsubimage(imagename='3C75_final_large.image.tt0',outfile='3C75_final.U.image',stokes='U')

subimPI='polimg'
ia.open(subimPI)
ia.calcmask(mask=subimPI+'>5e-4',name='mymask')
ia.done()

ia.open('3C75_final.Q.image')
ia.maskhandler('copy',['polimg:mymask','polithreshmask'])
ia.maskhandler('set','polithreshmask')
ia.done()

ia.open('3C75_final.U.image')
ia.maskhandler('copy',['polimg:mymask','polithreshmask'])
ia.maskhandler('set','polithreshmask')
ia.done()

immath(imagename=['3C75_final.Q.image', '3C75_final.U.image'], mode='pola', outfile='3C75_final.pola.masked')

These steps take the polarized intensity image calculated above (Figure 19B) and create a mask using a polarization fraction threshold of 5e-4 (0.05% linear polarization fraction). This mask is then applied to the Q and U images from the image cube that was generated above. Then a new polarization angle image is calculated from the Q & U image planes, applying the mask based on polarization fraction. Finally, we can load the Stokes I as raster image into the CASA imview and add the polarization angle as vectors. Figure 19C shows the resulting image. One can clearly see that the linear polarization angle follows perpendicular to the extended structure. This indicates that the magnetic field lines are oriented along the extended structure, perpendicular to the linear polarization angles.

Rotation Measures

The plane of polarization of light is rotated by the magnetic fields present in the intervening plasma. The total rotation to the plane of polarization of light between the source and the user is called Faraday Rotation. Prior to the wide bandwidth capabilities, these rotation measures were computed by fitting a line to the polarization position angle as a function of the square of the wavelength of measurement. The slope of the resulting fit was deemed to be the RM of the source while the intercept would give the true polarization position angle of the source. With the wide bandwidths, it is now possible to determine the rotation measure of the source using the naive fitting approach by making images per spectral window in IQUV and fitting the data (polarization position angle vs lambda^2) with a line.

To produce an image cube with 8 channels, each image is using 128 MHz of bandwidth, we call tclean with the following parameters. Here we take advantage of the imaging mask we generated for the final image above, so we don't need to do an interactive clean. 

# In CASA
tclean(vis='3C75.ms',
       field="3C75",
       spw="",timerange="",
       uvrange="",antenna="",scan="",observation="",intent="",
       datacolumn="corrected",
       imagename="3C75_chan8",
       imsize=800,
       cell="3.4arcsec",
       phasecenter="",
       stokes="IQUV",
       projection="SIN",
       specmode="cube",
       reffreq="",
       nchan=-1,
       start="",
       width=64,
       outframe="LSRK",
       veltype="radio",
       restfreq=[],
       interpolation="linear",
       gridder="standard",
       mosweight=True,
       cfcache="",
       computepastep=360.0,
       rotatepastep=360.0,
       pblimit=-0.0001,
       pbmask=0.0,
       mask='3C75_final_large.mask',
       normtype="flatnoise",
       deconvolver="multiscale",
       scales=[0, 6, 18],
       nterms=1,
       smallscalebias=0.6,
       restoration=True,
       restoringbeam=[],
       pbcor=False,
       outlierfile="",
       weighting="briggs",
       robust=0.5,
       npixels=0,
       uvtaper=[],
       niter=20000,
       gain=0.1,
       threshold=0.0,
       nsigma=0.0,
       cycleniter=1000,
       cyclefactor=1.0,
       restart=True,
       savemodel="none",
       calcres=True,
       calcpsf=True,
       parallel=False,
       interactive=False)

Now we use the CASA toolkit to access data for four pixels in the image cube to visualize and fit the rotation measure. 

# In CASA
import matplotlib
# to display
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt
import numpy as np

ia.open('3C75_chan8.image')

# number of channels/frequencies
nunr = 8 

tt = ia.getchunk()
nu = np.linspace(2.551e9,3.319e9,num=nunr)
c = 2.99792458e8

Q1 = tt[418,444,1,:nunr]
U1 = tt[418,444,2,:nunr]
Q2 = tt[376,419,1,:nunr]
U2 = tt[376,419,2,:nunr]
Q3 = tt[383,401,1,:nunr]
U3 = tt[383,401,2,:nunr]
Q4 = tt[395,398,1,:nunr]
U4 = tt[395,398,2,:nunr]

chi1 = 0.5*np.arctan2(U1,Q1)
chi2 = 0.5*np.arctan2(U2,Q2)
chi3 = 0.5*np.arctan2(U3,Q3)
chi4 = 0.5*np.arctan2(U4,Q4)

#locate the values that are nan and delete these indices from nu
indx1=np.argwhere(chi1==0)
indx2=np.argwhere(chi2==0)
indx3=np.argwhere(chi3==0)
indx4=np.argwhere(chi4==0)

nu1=np.delete(nu,indx1)
lam1 = c/nu1
lamsq1 = lam1*lam1

nu2=np.delete(nu,indx2)
lam2 = c/nu2
lamsq2 = lam2*lam2

nu3=np.delete(nu,indx3)
lam3 = c/nu3
lamsq3 = lam3*lam3

nu4=np.delete(nu,indx4)
lam4 = c/nu4
lamsq4 = lam4*lam4

#drop the zero values
chi1=np.delete(chi1,indx1)
chi2=np.delete(chi2,indx2)
chi3=np.delete(chi3,indx3)
chi4=np.delete(chi4,indx4)

fit1 = np.polyfit(lamsq1,chi1,1)
fit_fn1 = np.poly1d(fit1)
slope1 = fit1[0]
intercept1 = fit1[1]
fit2 = np.polyfit(lamsq2,chi2,1)
fit_fn2 = np.poly1d(fit2)
slope2 = fit2[0]
intercept2 = fit2[1]
fit3 = np.polyfit(lamsq3,chi3,1)
fit_fn3 = np.poly1d(fit3)
slope3 = fit3[0]
intercept3 = fit3[1]
fit4 = np.polyfit(lamsq4,chi4,1)
fit_fn4 = np.poly1d(fit4)
slope4 = fit4[0]
intercept4 = fit4[1]

plt.figure(1)
plt.title('Overall Title')
plt.subplot(221)
plt.title('Point 1: (418,444)')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi1$')
plt.scatter(lamsq1,chi1,color='r')
plt.plot(lamsq1,fit_fn1(lamsq1),'r--',label='$\chi$ = {:.2f}$\lambda^2$ + {:.2f}'.format(slope1,intercept1))
plt.legend(loc=2)

plt.subplot(222)
plt.title('Point 2: (376,419)')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi2$')
plt.scatter(lamsq2,chi2,color='b')
plt.plot(lamsq2,fit_fn2(lamsq2),'b--',label='$\chi$ = {:.2f}$\lambda^2$ + {:.2f}'.format(slope2,intercept2))
plt.legend(loc=1)

plt.subplot(223)
plt.title('Point 3: (383,401)')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi3$')
plt.scatter(lamsq3,chi3,color='g')
plt.plot(lamsq3,fit_fn3(lamsq3),'g--',label='$\chi$ = {:.2f}$\lambda^2$ + {:.2f}'.format(slope3,intercept3))
plt.legend(loc=3)

plt.subplot(224)
plt.title('Point 4: (395,398)')
plt.xlabel(r'$\lambda^{2}$')
plt.ylabel(r'$\chi4$')
plt.scatter(lamsq4,chi4,color='m')
plt.plot(lamsq4,fit_fn4(lamsq4),'m--',label='$\chi$ = {:.2f}$\lambda^2$ + {:.2f}'.format(slope4,intercept4))
plt.legend(loc=1)
plt.tight_layout()

plt.show()

ia.close()

The resulting plots are shown in Figure 20A. Alternatively, there exists a CASA task rmfit which does this basic fitting for you while taking into account the n \pi ambiguity (refer to [4] for more info). The fits using rmfit for our case of 3C 75 by making images per spectral window is shown in Figure 20B. Here we set the maximum acceptable position angle error to 20 degrees. If larger, then no rotation measures are calculated. 

# In CASA
rmfit('3C75_chan8.image',rm='3C75_chan8_rm.image',rmerr='3C75_chan8_rm.image.err',maxpaerr=0.35)
Figure 20A: Rotation measures extracted for 4 pixels from an 8 channel image cube of 3C75.
Figure 20B: RMFIT rotation measure image generated from 8 channel image cube.

The rmfit task has many more options; for example, you are able to provide a foreground rotation measure to subtract. For more information have a look at [5].

Now we can compare the rotation measures extracted for the 4 pixels from the 8 channel image cube with the values derived in the rmfit for the same pixels. In most cases the values are more or less comparable.

Point RM Lin. Fit. RM RMFIT
1 -37.31 -31.30
2 35.48 26.50
3 -49.65 -45.14
4 17.60 17.12

As our source is rather bright, we can derive an IQUV image not just per averaged spectral window as we just did, but rather per channel. To achieve this you can change the above tclean parameter width from 64 to 1, which will result in 512 channels spanning all 8 spectral windows. Note when imaging each channel, the edge channels are flagged which results in the PSF being blank for [C0:P0] [C0:P1] [C0:P2] [C0:P3] [C1:P0] and the first few images being blank. Don't forget to change the imagename parameter when re-running tclean. Following the same steps as for the 8 channel image cube (you will need to adjust the script for number of channels), you would then obtain the results shown in Figure 21 where again the polarization position angle as a function of lambda square is shown together with the rmfit image. We can clearly see that the source exhibits complex structure beyond a simple linear fit we performed earlier. This suggests that deriving a single RM would be an oversimplification. We should ideally perform RM Synthesis (https://arxiv.org/pdf/astro-ph/0507349.pdf). At this point in time CASA does not have an RM synthesis task.


Figure 21A: Rotation measures extracted for 4 pixels from an 512 channel image cube of 3C75.
Figure 21B: Rotation measures extracted for 4 pixels from an 512 channel image cube of 3C75 with enforced limits on y axis for points 1 and 2 to exclude outlier points from the view and see better the variation of the data.
Figure 21C: RMFIT rotation measure image generated from 512 channel image cube.


Questions about this tutorial? Please contact the NRAO Helpdesk.

Last checked on CASA Version 6.2.1

↵ CASAguides


Retrieved from "https://casaguides.nrao.edu/index.php?title=CASA_Guides:Polarization_Calibration_based_on_CASA_pipeline_standard_reduction:_The_radio_galaxy_3C75-CASA6.2.1&oldid=33618"