Antennae Band7 - Calibration for CASA 3.3: Difference between revisions

From CASA Guides
Jump to navigationJump to search
Line 27: Line 27:


==Initial Inspection==
==Initial Inspection==
We will eventually concatenate the ten datasets used here into two large datasets,  one for the northern mosaic and another for the southern mosaic.  However, we will keep them separate for now until imaging.  We start by defining an array "basename_all" that includes the names of the ten files, including datasets. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python.  Remember that if you log out of CASA, you will have to re-issue the array "basename_all".   
We will eventually concatenate the ten datasets used here into two large datasets,  one for the northern mosaic and another for the southern mosaic.  However, we will keep them separate for now until imaging.  We start by defining an array "basename_all" that includes the names of the ten measurement sets. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python.  Remember that if you log out of CASA, you will have to re-issue the array "basename_all".   


<source lang="python">
<source lang="python">
Line 50: Line 50:
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu41.html#UserMansu42.html logger].  You will have to scroll up to see the individual output for each of the datasets.   
The output will be sent to the CASA [http://casa.nrao.edu/docs/userman/UserMansu41.html#UserMansu42.html logger].  You will have to scroll up to see the individual output for each of the datasets.   


Here is an example of the original output for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms (northern mosaic).
Here is an example of the (abridged) output from {{listobs}} for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms (northern mosaic).


<pre style="background-color: #fffacd;">
<pre style="background-color: #fffacd;">
Line 111: Line 111:
This output shows that three fields were observed: 3c279, Titan, and the different pointing of the Antennae mosaic (NGC 4038 in this case).  Field 0 (3c279) will serve as the gain calibrator and bandpass calibrator; field 1 (Titan) will serve as the flux calibrator; and source id 2 (NGC 4038) is, of course, the science target.  Note that the source id 2 is composed of a number of field ids, each corresponding to a different pointing in the mosaic. In particular, the observations are characterized by 23 pointings for the Northern mosaic (NGC 4038), and 29 pointings for the Southern mosaic (NGC 4039).
This output shows that three fields were observed: 3c279, Titan, and the different pointing of the Antennae mosaic (NGC 4038 in this case).  Field 0 (3c279) will serve as the gain calibrator and bandpass calibrator; field 1 (Titan) will serve as the flux calibrator; and source id 2 (NGC 4038) is, of course, the science target.  Note that the source id 2 is composed of a number of field ids, each corresponding to a different pointing in the mosaic. In particular, the observations are characterized by 23 pointings for the Northern mosaic (NGC 4038), and 29 pointings for the Southern mosaic (NGC 4039).


Note that in the original data there are more than two SpwIDs even though the observations were set up to have two spectral windows. The first one (spw 1) is centered on the CO(3-2) emission line in the Antennae, in the Lower Side Band (LSB). In the original datasets there is also one additional spectral window (spw 3) in the Upper Side Band (USB). However, in order to make the data transfer as fast as possible, note that the data we provided only contains the spw 1. The spectral resolution and total bandwidth of these two spectral windows are 488.28 MHz and 1.875 GHz (3840 channels), respectively, corresponding to 0.426 km/s and 1634 km/s. In the original datasets there are also two spectral windows (spw 5 and 7) fully covering spw 1 and 3 but in TDM mode (2 GHz bandwidth, 128 channels), which are used to derive the Tsys calibration tables. We provided in Antennae_Band7_UnCalibratedMSandTablesForReduction.tar.gz the Tsys calibration tables to apply.
Note that in the data there are more than two spectral windows IDs even though the observations were set up to have two spectral windows. The first one, spectral window 1, is centered on the CO(3-2) emission line in the Antennae, in the Lower Side Band (LSB). There is also one additional spectral window, id = 3, in the Upper Side Band (USB). The spectral resolution and total bandwidth of these two spectral windows are 488.28 MHz and 1.875 GHz (3840 channels), respectively, corresponding to 0.426 km/s and 1634 km/s. In the original datasets there are also two spectral windows (spw 5 and 7) fully covering spw 1 and 3 but in TDM mode (2 GHz bandwidth, 128 channels), which are used to derive the Tsys calibration tables. We provided in Antennae_Band7_UnCalibratedMSandTablesForReduction.tar.gz the Tsys calibration tables to apply (asdm+'.tsys.cal.fdm'), so there is no need to use these spectral windows.
Also, we removed spectral windows 2, 4, 6 and 8, which contained channel averages of spectral windows 1, 3, 5 and 7, respectively, as these are not useful for the offline data reduction. Spectral window 0 contained the WVR data, whose solutions are provided in Antennae_Band7_UnCalibratedMSandTablesForReduction.tar.gz.   
Also, we find spectral windows 2, 4, 6 and 8, which contains channel averages of spectral windows 1, 3, 5 and 7, respectively, but these are not useful for the offline data reduction. Spectral window 0 contained the WVR data, but these are provided in the form of a calibration table (asdm+'.wvr.cal') in Antennae_Band7_UnCalibratedMSandTablesForReduction.tar.gz.   


The final column of the listobs output in the logger (not shown above) gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. Also these intents will be used in the future for pipeline processing.  
The final column of the listobs output in the logger (not shown above) gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. Also these intents will be used in the future for pipeline processing.  

Revision as of 00:49, 26 July 2011


Overview

This portion of the AntennaeBand7 CASA Guide will cover the calibration of the raw visibility data. To skip to the imaging portion of the guide, see: Antennae Band7 - Imaging.

If you haven't already downloaded the raw data, you may do that now by clicking on the region closest to your location and downloading the file named 'Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz':

North America

Europe

East Asia

Once the download has finished, unpack the file:

# In a terminal outside CASA
tar -xvzf Antennae_Band7_UnCalibratedMSandTablesForReduction.tgz

cd Antennae_Band7_UnCalibratedMSandTablesForReduction

# Start CASA
casapy

The data have already been converted to CASA Measurement Set (MS) format using the CASA task importasdm. Accompanying the data are some basic calibration tables you will need for the following reduction, specifically, the Tsys and WVR calibration tables.

Initial Inspection

We will eventually concatenate the ten datasets used here into two large datasets, one for the northern mosaic and another for the southern mosaic. However, we will keep them separate for now until imaging. We start by defining an array "basename_all" that includes the names of the ten measurement sets. This will simplify the following steps by allowing us to loop through the files using a simple for-loop in python. Remember that if you log out of CASA, you will have to re-issue the array "basename_all".

# In CASA

basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9",
     "uid___A002_X2181fb_X49","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7",
     "uid___A002_X207fe4_X4d7","uid___A002_X215db8_X18","uid___A002_X215db8_X1d5",
     "uid___A002_X215db8_X392"]

The usual first step is then to get some basic information about the data. We do this using the task listobs, which will output a detailed summary of each dataset supplied.

# In CASA

for name in basename_all:
        listobs(vis=name+'.ms')

Note that after cutting and pasting a for-loop you often have to press return twice to execute. The output will be sent to the CASA logger. You will have to scroll up to see the individual output for each of the datasets.

Here is an example of the (abridged) output from listobs for the first dataset in the list, uid___A002_X1ff7b0_Xb.ms (northern mosaic).

================================================================================
         MeasurementSet Name:/Users/despada/Desktop/Imaging/Antennae/Datasets/band7/uid___A002_X1ff7b0_Xb.ms      MS Version 2
================================================================================
   Observer: Unknown     Project: T.B.D.  
Observation: ALMA(11 antennas)
Data records: 181357       Total integration time = 4931.71 seconds
   Observed from   28-May-2011/01:25:27.6   to   28-May-2011/02:47:39.3 (UTC)
Fields: 26
  ID   Code Name         RA            Decl           Epoch   SrcId 
  0    none 3c279        12:56:11.1666 -05.47.21.5247 J2000   0     
  1    none Titan        12:42:43.9481 -01.43.38.3190 J2000   1     
  2    none NGC4038 - A* 12:01:53.1701 -18.52.37.9200 J2000   2     
  3    none NGC4038 - A* 12:01:51.9030 -18.51.49.9437 J2000   2     
  4    none NGC4038 - A* 12:01:52.4309 -18.51.49.9437 J2000   2     
  5    none NGC4038 - A* 12:01:52.9587 -18.51.49.9437 J2000   2     
  6    none NGC4038 - A* 12:01:53.4866 -18.51.49.9436 J2000   2     
  7    none NGC4038 - A* 12:01:54.0144 -18.51.49.9436 J2000   2     
  8    none NGC4038 - A* 12:01:52.1669 -18.51.56.4319 J2000   2     
  9    none NGC4038 - A* 12:01:52.6948 -18.51.56.4318 J2000   2     
  10   none NGC4038 - A* 12:01:53.2226 -18.51.56.4318 J2000   2     
  11   none NGC4038 - A* 12:01:53.7505 -18.51.56.4318 J2000   2     
  12   none NGC4038 - A* 12:01:51.9030 -18.52.02.9201 J2000   2     
  13   none NGC4038 - A* 12:01:52.4309 -18.52.02.9200 J2000   2     
  14   none NGC4038 - A* 12:01:52.9587 -18.52.02.9200 J2000   2     
  15   none NGC4038 - A* 12:01:53.4866 -18.52.02.9200 J2000   2     
  16   none NGC4038 - A* 12:01:54.0144 -18.52.02.9199 J2000   2     
  17   none NGC4038 - A* 12:01:52.1669 -18.52.09.4082 J2000   2     
  18   none NGC4038 - A* 12:01:52.6948 -18.52.09.4082 J2000   2     
  19   none NGC4038 - A* 12:01:53.2226 -18.52.09.4082 J2000   2     
  20   none NGC4038 - A* 12:01:53.7505 -18.52.09.4081 J2000   2     
  21   none NGC4038 - A* 12:01:51.9030 -18.52.15.8964 J2000   2     
  22   none NGC4038 - A* 12:01:52.4309 -18.52.15.8964 J2000   2     
  23   none NGC4038 - A* 12:01:52.9587 -18.52.15.8963 J2000   2     
  24   none NGC4038 - A* 12:01:53.4866 -18.52.15.8963 J2000   2     
  25   none NGC4038 - A* 12:01:54.0144 -18.52.15.8963 J2000   2     
   (nVis = Total number of time/baseline visibilities per field) 
Spectral Windows:  (9 unique spectral windows and 2 unique polarization setups)
  SpwID  #Chans Frame Ch1(MHz)    ChanWid(kHz)TotBW(kHz)  Ref(MHz)    Corrs   
  0           4 TOPO  184550      1500000     7500000     183300      I   
  1        3840 TOPO  344845.586  488.28125   1875000     344908.33   XX  YY  
  2           1 TOPO  343908.086  1875000     1875000     344908.33   XX  YY  
  3        3840 TOPO  356845.586  488.28125   1875000     344908.33   XX  YY  
  4           1 TOPO  343908.086  1875000     1875000     344908.33   XX  YY  
  5         128 TOPO  344900.518  15625       2000000     344908.33   XX  YY  
  6           1 TOPO  343892.705  1796875     1796875     344908.33   XX  YY  
  7         128 TOPO  356900.518  15625       2000000     344908.33   XX  YY  
  8           1 TOPO  343892.705  1796875     1796875     344908.33   XX  YY  

Antennas: 11 'name'='station' 
   ID=   0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', 
   ID=   4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', 
   ID=  8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'
================================================================================

This output shows that three fields were observed: 3c279, Titan, and the different pointing of the Antennae mosaic (NGC 4038 in this case). Field 0 (3c279) will serve as the gain calibrator and bandpass calibrator; field 1 (Titan) will serve as the flux calibrator; and source id 2 (NGC 4038) is, of course, the science target. Note that the source id 2 is composed of a number of field ids, each corresponding to a different pointing in the mosaic. In particular, the observations are characterized by 23 pointings for the Northern mosaic (NGC 4038), and 29 pointings for the Southern mosaic (NGC 4039).

Note that in the data there are more than two spectral windows IDs even though the observations were set up to have two spectral windows. The first one, spectral window 1, is centered on the CO(3-2) emission line in the Antennae, in the Lower Side Band (LSB). There is also one additional spectral window, id = 3, in the Upper Side Band (USB). The spectral resolution and total bandwidth of these two spectral windows are 488.28 MHz and 1.875 GHz (3840 channels), respectively, corresponding to 0.426 km/s and 1634 km/s. In the original datasets there are also two spectral windows (spw 5 and 7) fully covering spw 1 and 3 but in TDM mode (2 GHz bandwidth, 128 channels), which are used to derive the Tsys calibration tables. We provided in Antennae_Band7_UnCalibratedMSandTablesForReduction.tar.gz the Tsys calibration tables to apply (asdm+'.tsys.cal.fdm'), so there is no need to use these spectral windows. Also, we find spectral windows 2, 4, 6 and 8, which contains channel averages of spectral windows 1, 3, 5 and 7, respectively, but these are not useful for the offline data reduction. Spectral window 0 contained the WVR data, but these are provided in the form of a calibration table (asdm+'.wvr.cal') in Antennae_Band7_UnCalibratedMSandTablesForReduction.tar.gz.

The final column of the listobs output in the logger (not shown above) gives the scan intent. This information is used later to flag the pointing scans and the hot and ambient load calibration scans, using scan intent as a selection option. Also these intents will be used in the future for pipeline processing.

A total of 11 antennas were used for the dataset listed above. Note that numbering in python always begins with "0", so the antennas have IDs 0-10. To see what the antenna configuration looked like at the time of this observation, we will use the task plotants (Figure 1).

Fig. 1. Position of antennas in dataset uid_A002_X1ff7b0_Xb, as obtained using task plotants
# In CASA
plotants(vis=basename[0]+'.ms', figfile=basename[0]+'_plotants.png')

This will plot the antenna configuration on your screen as well as save it under the specified filename for future reference. You can check using plotants for the other datasets that the configuration stays relatively constant during the course of the observations. This corresponds to a configuration similar to extended configuration.

Data edition

General flagging

The first editing we will do is some a priori flagging with flagdata to flag shadowed data, and flagautocorr to flag the autocorrelation data contained in ALMA data, as we are only interested in the cross-correlation data. Remember that you first need to redefine the "basename_all" array if you logged out of CASA prior to starting this subsection.

# In CASA

basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X3a",
                          "uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X3b9","uid___A002_X207fe4_X4d7",
                          "uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392",
                          "uid___A002_X2181fb_X49"]

Now we will loop over the datasets, running the two flagging commands:

# In CASA
for asdm in basename_all:
	flagdata(vis=asdm+'.ms', mode='manualflag', flagbackup = F, mode = 'shadow')
	flagautocorr(vis=asdm+'.ms')

In the flagdata task we chose:

  • vis = asdm+'.ms' : each measurement set
  • flagbackup = F: Do not automatically backup any flag file, as we will save it together using flagmanager after we finish the initial flagging
  • mode='manualflag': flag only selected data

There are a number of scans in the data that were used for pointing and atmospheric (i.e. Tsys) calibration, and are no longer needed. We can flag them using flagdata by selecting them on the 'intent' keyword:

# In CASA

for asdm in basename_all:
        flagdata(vis=asdm+'.ms', mode='manualflag', flagbackup = F, intent='*POINTING*')
        flagdata(vis=name+'.ms', mode='manualflag', flagbackup = F, intent='*ATMOSPHERE*')

Next we store the current flagging information using flagmanager for each dataset:

# In CASA
for name in basename:
        flagmanager(vis = name+'.ms', mode = 'save', versionname = 'Apriori')

Note that it is also possible to set flagdata with flagbackup=T so that it stores automatically the flags at each of the flagging stages without the need of flagmanager.

Apply WVR and Tsys calibration tables

First, remember to re-define the "basename_all" array if you have logged out of CASA at any point up until now during the data edition process.

# In CASA
basename_all=["uid___A002_X1ff7b0_Xb","uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X3a",
                          "uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X3b9","uid___A002_X207fe4_X4d7",
                          "uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392",
                           "uid___A002_X2181fb_X49"]

Tsys tables

ALMA does not automatically scale the data by system temperature (Tsys) as would be done at other radio telescopes. Tsys measurements correct for the atmospheric opacity (to first-order), both as a function of time and as a function of frequency, and put weights in the subsequent imaging. The Tsys tables for these datasets were provided with the downloadable data. The FDM Tsys table has been obtained using that of the TDM mode spectral window (2 GHz, 128 channels per baseband, XX and YY polarizations), which we interpolated to have tsys data for all the FDM channels (3840 channels per baseband, XX and YY polarizations, over 1.875 GHz).

We inspect the Tsys tables for spw=1 with the task plotcal, for example for the first dataset:

Fig. 2. Example of Tsys plot for uid_A002_X1ff7b0_Xb (northern mosaic)
#In CASA

asdm=basename_all[0]

plotcal(caltable=asdm+'.tsys.cal.fdm', 
             xaxis="freq",yaxis="amp",
             timerange='<1e8',spw='1',
             overplot=False,plotrange=[0, 0, 0, 0],plotsymbol=".")

In Figure 2 we show the Tsys for all the antennas. The Tsys solutions look reliable, except for one antenna that shows large values of Tsys (DV04, 2 polarizations per baseband). We will flag that antenna. Also note that there is an absorption line at about 345.2 GHz that makes Tsys larger, as seen in all of the other antennas.

Apply Tsys and WVR tables

We will now apply the Tsys and the WVR calibration tables to the data with the task applycal. We do this for each field separately so that the appropriate calibration data are applied to the right fields. The "field" parameter specifies the field to which we will apply the calibration, and the "gainfield" parameter specifies the field from which we wish to take the calibration solutions from the gaintable. In the call to applycal, we will specify interpolation="nearest". Now loop through the datasets as described above, applying the Tsys and WVR calibration tables to all the datasets and sources:

We will apply the calibration tables with applycal:

# In CASA
for asdm in basename_all:
	for fieldname in ['Titan','3c279','NGC4038*','Antenna*']:
		applycal(vis=asdm+".ms", spw='1,3', flagbackup=F, 
                                field=fieldname, gainfield=fieldname,
                                interp=['nearest','nearest'], 
                                 gaintable=[asdm+".tsys.cal.fdm",asdm+'.wvr.cal'])

Now you can use plotms to show some of the before-and-after effects of calibration. Just re-run the previous plotms command, repeated below, or if the window is still open, check 'force reload'. This will display the uncorrected phases across the band. To display the corrected phases, you will need to select the 'corrected' in the 'Data Column' field of the 'Axes' tab of plotms and re-plot.

# In CASA
plotms(vis=basename[0]+'.ms', 
              field='3c279'
              xaxis='channel', yaxis='phase',
              selectdata=T, spw='1', correlation='XX', 
              avgtime='1e8',avgscan=T,
              coloraxis='baseline' )

Additional individual flagging

Continuum plot

Next we will do some additional inspection with plotms. First we will plot amplitude versus time (see Figure 3), averaging over all channels of spectral window 1 (spw = 1, where the CO(3-2) line is) and colorizing by field, for example for the first dataset. Scans on Titan are colored red, the bandpass and phase calibrator 3c279 is colored black, and the different pointings of the Antennae mosaic in different colors. Figure 4 shows the phase versus time plot, with the same color.

Check carefully that the amplitudes and phases vary smoothly with time. The Tsys corrected data should have approximately constant amplitudes and the WVR corrected data should usually have lower phase scatter. Note that the amplitudes in Figure 3 are decreasing, as a result of the decreasing elevation of the source. The Tsys corrected data (choose 'corrected' in Axes > Data Column in plotms) show constant amplitudes.

Fig. 3. Amplitude vs. time, for the baseline DV02&DV07 of dataset Uid_A002_X1ff7b0_Xb, averaged over channel. DATA column (uncorrected)
Fig. 4. Phase vs. time, for the baseline DV02&DV07 of dataset Uid_A002_X1ff7b0_Xb, averaged over channel. DATA column (uncorrected)

These are the plotms instances to produce the continuum plots:

# In CASA

asdm=basename_all[0]

plotms(vis=asdm+'.ms', 
            xaxis='time', yaxis='amp', 
            selectdata=True, spw='1', correlation='XX',antenna='*&*',
            avgchannel='3840', avgscan=T, 
            iteraxis='baseline',coloraxis='field')

plotms(vis=asdm+'.ms', 
            xaxis='time', yaxis='phase', 
            selectdata=True, spw='1', correlation='XX',antenna='*&*',
            avgchannel='3840', avgscan=T, 
            iteraxis='baseline',coloraxis='field')

where:

  • xaxis='time', yaxis='X'  : a plot of X (amplitude or phase) versus time.
  • avgchannel='3840' : average over all the channels in the spectral window.
  • spw='1', correlation='XX',antenna='*&*': Select only the spectral window 1, polarization XX and cross-correlation data.
  • iteraxis='baseline',coloraxis='field': Iterate over baseline, and colorize by different fields.

Select correlation='YY' to inspect the other polarization.

Spectral plot

Second we plot the amplitude and phase versus frequency for the two correlations, XX and YY. Figure 5 and 6 show an example of spectral plot for the bandpass and phase calibrator 3c279, for both correlations. Again, since this source is a quasar, the amplitudes should be constant (Tsys corrected) and phases varying smoothly.

These are the task instances to obtain these plots:

Fig. 5. Amplitude vs. Frequency for spw=1, baseline DV02&DV07 of dataset Uid_A002_X1ff7b0_Xb, averaged over time
Fig. 6. Phase vs. Frequency for spw=1, baseline DV02&DV07 of dataset Uid_A002_X1ff7b0_Xb, averaged over time
# In CASA
plotms(vis = asdm+'.split.ms',
             xaxis = 'frequency',yaxis = 'phase',
             field='3c279'
             avgtime = '1e8',avgscan = T,
             selectdata=True, antenna = '*&*',
             iteraxis='baseline')

plotms(vis = asdm+'.split.ms',
             xaxis = 'frequency',yaxis = 'amp',
             field='3c279'
             avgtime = '1e8',avgscan = T,
             selectdata=True, antenna = '*&*',
             iteraxis='baseline')

where:

  • xaxis = 'frequency',yaxis = 'X': plot X (amplitude or phase) versus frequency
  • field = '3c279': plot only our bandpass and phase calibrator
  • avgtime = '1e8',avgscan = T: average all scans and integrations
  • antenna = '*&*: plot only cross-correlation data
  • iteraxis='baseline': iterate for each baseline

There are no large phase delays are found in any of the datasets (i.e., usually less than one wrap over the bandpass), so bandpass calibration should remove this effect properly.

You can plot other sources as well. By selecting any pointing of Antennae, you should be able to see clearly the (still uncalibrated) CO(3-2) line. The flux calibrator also present some emission line in this spectral window.

Individual Flagging

First we use flagdata to remove the edge channels from both sides of the bandpass:

# In CASA
flagdata(flagbackup = F,vis = asdm+'.split.ms',spw = '1:0~7,1:3831~3839')

Continue to inspect the data with plotms, plotting different axes and colorizing by the different parameters. Don't forget to average the data if possible to speed the plotting process. The time ranges to insert in flagdata can be obtained using plotms Tools Hover/Display. Instead of using the following flagdata commands, you can also flag by hand in plotms. To do this, select your bad data by clicking on the 'Mark Regions" button, then on 'Flag".

File:AntennaeDataInspection-Band7.txt contains the different problems that have been identified for all the datasets. We indicated how to flag the bad data in different instances of the flagdata command. For example, for the first dataset:

asdm="uid___A002_X1ff7b0_Xb"
#0-3: 'DV02'='A015', 'DV04'='J505', 'DV06'='T704', 'DV07'='A004', 
#4-7: 'DV08'='A072', 'DV09'='A008', 'DV10'='A009', 'DV11'='A016', 
#8-10: 'PM01'='T702', 'PM02'='A017', 'PM03'='J504'
#Quality: good
#wvr: cal table created succesfully.
#Antennas: 11
#Tsys: DV04 Tsys huge in comparison with the other antennas (AIV-4694)
 flagdata(flagbackup = F,vis = asdm+'.ms',mode='manualflag',antenna='DV04')
#El. From 80 to 60, calibrator increasingly separated.
#Spectrum phase: DV02 10deg peak to peak noise in one of the corr.
#Continuum phase: *&DV09, sudden change of ~  200 deg. at 2:03:20, but it can be calibrated.
#Continuum phase: *&PM03, corr=YY, spw=3, need to be flagged.
 flagdata(flagbackup = F,vis = asdm+'.ms',mode='manualflag',antenna='PM03',correlation='YY',spw='3')
#Continuum phase: Phase jump ~ 30 deg in DV02&DV04 and DV04&DV07. 
#wvr corrected makes it better, so there is no need to flag.
flagmanager(vis =asdm+'.ms',mode = 'save',versionname = 'FlagFinal')

Split the Tsys/WVR calibrated data

We split out the CORRECTED_DATA column with the task split. This will get rid of the extraneous spectral windows, including the channel averaged spectral windows and spw 0, which is the one that contained the WVR data. We give the resulting datasets the extension ".split.ms". Since split will not overwrite existing files, we start by removing any previous versions of the measurement sets created in this step before running the split command again.

# In CASA
for asdm in basename_all:
	os.system('rm -rf '+asdm+'.split.ms')
	split(vis=directory+asdm+'.ms', outputvis=asdm+'.split.ms', 
                 datacolumn='corrected', spw='1')

The WVR and Tsys tables are now applied in the DATA column of the new measurement sets (i.e. asdm+'.split.ms').

Bandpass and gain calibration

Next we plot the phase as a function of time and frequency for the bandpass calibrator, 3c279. For the first plot, Figure 7, we use avgscan=T and avgtime='1e8' to average in time over all scans and integrations, and we specify coloraxis='baseline' to colorize by baseline. For the second, Figure 8, we use spw='0:40~3800' and avgchannel='3840' to average over the central channels of the first spectral window. For both plots we will iterate on antenna (interaxis='antenna'). Use the green arrows of the plotms GUI to view the plots for different antennas.

Fig. 7. Phase vs. time for the phase calibrator, 3c279. Averaged over channel. Only baselines with antenna DV02, and corr='XX'
Fig. 8. Phase vs. frequency for the phase calibrator, 3c279. Averaged over time, and corr='XX'
# In CASA

plotms(vis= asdm+'.split.ms', 
            xaxis='freq', yaxis='phase', 
            selectdata=True, field='3c279', corr='XX', antenna='*&*',
            avgtime='1e8', avgscan=T, 
            coloraxis='baseline', iteraxis='antenna')
# In CASA

plotms(vis= asdm+'.split.ms', 
           xaxis='time', yaxis='phase', 
           selectdata=True, field='3c279', spw='0:40~3800', antenna='*&*',corr='XX',
           avgchannel='3840',  avgscan=T, 
           coloraxis='baseline', iteraxis='antenna')

Figure 7 shows that the phase variation as function of time, although smoothly varying, will need to be calibrated in order to obtain proper bandpass solutions. In Figure 8 we see that phase variations in dataset uid_A002_X1ff7b0_Xb are small, typically ~ 30 degrees.

We issue gaincal on 3c279 to determine phase(-only) gain solutions. We use solint='int' for the solution interval, which means that one gain solution will be determined for every integration time to prevent de-correlation of the signal. Once phase is corrected, we can determine the bandpass solutions with bandpass. We apply the phase calibration table on-the-fly with the parameter "gaintable". Bandpass response can vary from day to day, therefore we calculate bandpass solutions for each day.

for asdm in basename_all:

#  os.system('rm -rf '+asdm+'.g1.cal,'+asdm+'.b1.cal')
  os.system('rm -rf '+asdm+'.bpphase.gcal,'+asdm+'.bandpass.bcal')

  gaincal(vis = asdm+'.split.ms',
               selectdata=T,field = '3c279',spw = '0:40~3800',
               #caltable = asdm+'.g1.cal',
               caltable = asdm+'.bpphase.gcal',
               solint = 'int',refant = 'DV06',calmode='p')

  bandpass(vis = asdm+'.split.ms',
               field = '3c279',
               #gaintable = [asdm+'.g1.cal'],
               caltable = asdm+'.bpphase.gcal',
               #caltable = asdm+'.b1.cal',
               caltable = asdm+'.bandpass.bcal',
               bandtype='B',
               solint = 'inf',combine = 'scan', solnorm=T,refant = 'DV06',
               minblperant=3,minsnr=2,fillgaps=1)

where:

  • gaintable = 'asdm+'.g1.cal', caltable = asdm+'.b1.cal': Gain calibration table, and bandpass calibration table
  • solint='int' or 'inf': The former is to consider integration by integration. The latter, combined with the default combine='scan', sets the solution interval to the entire observation
  • refant = 'DV06': Set the reference antenna to DV06
  • calmode='p': Gain cal calibration only phase
  • minblperant=3: Minimum number of baselines required per antenna for each solve
  • minsnr=2: Minimum SNR for solutions
  • bandtype='B': Channel by channel solution for each specified spw
  • fillgaps=1: Interpolate channel gaps 1 channel wide
  • solnorm=T: Normalize the bandpass amplitudes and phases of the corrections to unity

Do not worry about the message "Insufficient unflagged antennas" when running the bandpass task, which relates to the flagged edge channels.

Next we check the time variations of the phase solutions with plotcal. Note that this table will serve as the phase calibration table to apply to the science source. We also plot the bandpass solutions with plotcal, and we see that the solutions seem reasonable, with amplitudes close to 1 (Figure 9), and phases that varies smoothly over the spectral window.

Fig. 9. Bandpass amplitude solutions
  plotcal(caltable = asdm+'.g1.cal',
              xaxis = 'time',yaxis = 'phase',
              iteration = 'antenna',subplot = 211,plotrange=[0,0,-180,180])

  plotcal(caltable = asdm+'.b1.cal', 
              xaxis = 'freq',yaxis = 'amp',
              plotrange = [0,0,0.8,1.2])
  plotcal(caltable = asdm+'.b1.cal', 
              xaxis = 'freq',yaxis = 'phase',
              plotrange = [0,0,-100,100])

Flux Calibration

We set the flux for our flux calibrator, Titan, using the task setjy, by applying the Butler-JPL-Horizons 2010 model. First, we check the flux calibrator data. We plot the spectrum and find a bright CO emission (Figure 10).

Fig. 10. Amplitude vs. channel plot for the flux calibrator, Titan (uid___A002_X1ff7b0_Xb dataset). Averaged over time, corr='XX', and colorized by baseline.
# in CASA
for asdm in basename_all:
  flagdata(vis=asdm+'.split.ms',flagbackup=F, 
         field=['Titan'],
         spw=['0:1100~1700'])
  flagmanager(vis =asdm+'.split.ms',mode = 'save',versionname = 'FlagFlux')


Second, we do a new gain calibration applying the bandpass calibration solutions on-the-fly.

We solve for amplitude and phase simultaneously and determine average solutions per scan. Finally, we will bootstrap the flux density of the secondary calibrator from that of Titan using the task fluxscale.

# in CASA
for asdm in basename_all:
#  os.system('rm -rf '+asdm+'.g2.cal,'+asdm+'.f1.cal')
  os.system('rm -rf '+asdm+'.amp.gcal,'+asdm+'.flux.cal')
  setjy(vis = asdm+'.split.ms',field = 'Titan',
           standard = 'Butler-JPL-Horizons 2010')

  gaincal(vis=asdm+'.split.ms',
                gaintable=asdm+'.bandpass.bcal', caltable=asdm+'.intphase.gcal',
                calmode='p',
                field='Titan,3c279',
                spw='0~1:40~3800',
                refant='DV06', solint='int',minsnr=2.0,minblperant=4)

  gaincal(vis=asdm+'.split.ms',
                gaintable=asdm+'.bandpass.bcal', caltable=asdm+'.scanphase.gcal',
                calmode='p',
                field='Titan,3c279',
                spw='0~1:40~3800',
                refant='DV06', solint='inf',minsnr=2.0,minblperant=4)

  gaincal(vis = asdm+'.split.ms',
 #              gaintable = asdm+'.b1.cal',caltable = asdm+'.g2.cal',
              gaintable =[asdm+'.bandpass.bcal',asdm+'.intphase.gcal'],caltable = asdm+'.amp.cal',
               calmode='ap'
               field = 'Titan, 3c279',
               refant = 'DV06',solint = 'int')

  fluxscale(vis = asdm+'.split.ms',
 #                caltable = asdm+'.g2.cal',fluxtable = asdm+'.f1.cal',
                 caltable = asdm+'.amp.gcal',fluxtable = asdm+'.flux.cal',
                 reference = 'Titan',
                 transfer = '3c279')

The flux of Titan at these frequencies is about 2.9 Jy. For example, for dataset uid___A002_X1ff7b0_Xb.f1.cal:

  #2011-07-13 07:31:04 INFO setjy	       Titan  spwid=  0  [I=2.846, Q=0, U=0, V=0] Jy, (JPL-Butler Solar System Object)

The new flux table asdm+'.flux.gcal' replaces the previous asdm+'.amp.gcal table in future application of the calibration to the data, i.e. the new flux table contains both asdm+'.amp.gcal and the newly acquired flux scaling. Unlike the gain calibration steps, this is not an incremental table.

  • gaintable = asdm+'.bandpass.bcal': We apply the bandpass calibration on-the-fly
  • caltable = 'asdm+'.amp.cal: the output gain calibration table
  • calmode = 'ap': To solve for amplitude and phase

We find that the flux of 3c279 is 10.45 Jy, by averaging the fluxes obtained from the ten available datasets. This flux agree within 10% with the most recent 0.850 mm measurements from the SMA calibrator list [1] : (01 Jul 2011, SMA 9.75 ± 0.49).

Applying the calibrations

Now we will use applycal to apply the bandpass and gaincal tables that we generated in the previous sections. First, we will apply the solutions to the bandpass and secondary calibrator (in this case, it is the same, 3c279). Second to the flux calibrator from the bandpass solutions and its own phase and flux solutions. Finally we will apply the bandpass and phase/amplitude solutions to the science target.

from the secondary calibrator to the science target and the secondary calibrator itself.

#In CASA

for asdm in basename_all: 
  applycal(vis=asdm+'.split.ms',field='3c279',
        gaintable=[asdm+'bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],
        interp=['nearest','nearest','nearest'],
        gainfield=['3c279','3c279','3c279'],flagbackup=T)

for asdm in basename_all: 
  applycal(vis=asdm+'.split.ms',field='Titan',
        gaintable=[asdm+'bandpass.bcal',asdm+'.intphase.gcal',asdm+'.flux.cal'],
        interp=['nearest','nearest','nearest'],
         gainfield=['3c279','titan','titan'],flagbackup=T)

for asdm in basename_all: 
  applycal(vis=asdm+'.split.ms',field=['Ant*','NGC*']
        interp=['nearest','linear','linear'],
        gaintable=[asdm+'bandpass.bcal',asdm+'.scanphase.gcal',asdm+'.flux.cal'],
        gainfield=['3c279','3c279','3c279'],flagbackup=T)


Checking the calibration

We plot the corrected amplitudes and phases of 3c279 as a function of time and frequency, to check that the phases are close to zero and the amplitudes are constant.

# In CASA
asdm=basename_all[0]
plotms(vis = asdm+'.split.ms', xaxis='time', yaxis='amp',
	ydatacolumn='corrected', selectdata=True, field='3c279',
	averagedata=True, avgchannel='3840', avgtime='',
	avgscan=False, avgbaseline=F, coloraxis='spw')

plotms(vis = asdm+'.split.ms', xaxis='time', yaxis='pha',
	ydatacolumn='corrected', selectdata=True, field='3c279',
	averagedata=True, avgchannel='3840', avgtime='',
	avgscan=False, avgbaseline=F, coloraxis='spw')

Next we can use plotms to examine the corrected amplitude and phase of Antennae as a function of frequency, just by changing the field to field='NGC*','Antennae*'.

Split source data and smoothing

We split the spectral window 0 for Antennae galaxies,where CO(3-2) line is. We separate between Northern and Southern mosaic datasets. We smooth the velocity to a width of 23 channels, corresponding to ~10 km/s.

basename_all_North=["uid___A002_X1ff7b0_Xb","uid___A002_X207fe4_X3a","uid___A002_X207fe4_X3b9","uid___A002_X2181fb_X49"]

for asdm in basename_all_North:
    os.system('rm -rf Antennae-'+asdm+'.cal.ms')
    split(vis = asdm+'.split.ms',outputvis = 'Antennae-'+asdm+'.cal.ms',
             field = ['NGC*'],spw='0',width=23)
    listobs(asdm+'.split.ms')

basename_all_South=["uid___A002_X1ff7b0_X1c8","uid___A002_X207fe4_X1f7","uid___A002_X207fe4_X4d7",
                           "uid___A002_X215db8_X18","uid___A002_X215db8_X1d5","uid___A002_X215db8_X392"]

for asdm in basename_all_South:
    os.system('rm -rf Antennae-'+asdm+'.cal.ms')
    split(vis = asdm+'.split.ms',outputvis = 'Antennae-'+asdm+'.cal.ms',
             field = ['Antennae*'],spw='0',width=23)
    listobs(asdm+'.split.ms')

Concatenating datasets for northern and southern mosaics

Once each individual dataset is calibrated, we finally concatenate the data sets corresponding to one either north and south mosaics into one big measurement set. We define an array "comvis" that contains the names of the measurement sets we wish to concatenate, and then we run the task concat.

# In CASA
comvis_South=[]
comvis_North=[]
for asdm in basename_all_South:
	comvis_South.append('Antennae-'+asdm+'.cal.ms')
for asdm in basename_all_North:
	comvis_North.append('Antennae-'+asdm+'.cal.ms')        
os.system('rm -rf Antennae_South.cal.ms,Antennae_North.cal.ms')
concat(vis=comvis_South,concatvis='Antennae_South.cal.ms',timesort=T)
concat(vis=comvis_South,concatvis='Antennae_North.cal.ms',timesort=T)


Daniel Espada 12:00 UT, 21 July 2011