EVLA 6-cm Wideband Tutorial SN2010FZ (Caltech): Difference between revisions
No edit summary |
|||
| Line 40: | Line 40: | ||
mkdir evla_6-cm_wideband | mkdir evla_6-cm_wideband | ||
cd evla_6-cm_wideband | cd evla_6-cm_wideband | ||
tar -xvf /data/casa/evla/tar/ | tar -xvf /data/casa/evla/tar/SN2010FZ_10s.tar | ||
</source> | </source> | ||
Note that you will need to use either <tt>/scr</tt> or <tt>/scr2</tt>, depending on your node assignment. This will take a minute, but once it's complete, you will have a directory called <tt> | Note that you will need to use either <tt>/scr</tt> or <tt>/scr2</tt>, depending on your node assignment. This will take a minute, but once it's complete, you will have a directory called <tt>SN2010FZ_10s.ms</tt> which is the filled data, with online flags applied (which delete known bad data), and time-averaged to 10 seconds. (The data were taken in D-configuration [max baselines 1km], so one can safely average to 3s or even 10s to reduce dataset size.) This is equivalent to what you would download from the archive if you request time-averaging and online flag application. | ||
=== On your own === | === On your own === | ||
| Line 58: | Line 58: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
listobs(' | listobs('SN2010FZ_10s.ms') | ||
</source> | </source> | ||
| Line 68: | Line 68: | ||
============================================================================== | ============================================================================== | ||
MeasurementSet Name: /scr2/casa/evla_6-cm_wideband/ | MeasurementSet Name: /scr2/casa/evla_6-cm_wideband/SN2010FZ_10s.ms MS Version 2 | ||
============================================================================== | ============================================================================== | ||
Observer: Dr. Alicia M. Soderberg Project: T.B.D. | Observer: Dr. Alicia M. Soderberg Project: T.B.D. | ||
| Line 244: | Line 244: | ||
== Flagging the MS == | == Flagging the MS == | ||
The online flags, which are a record of known bad data produced by the EVLA online system, have already been applied by the archive as it generated the MS. However, it's good to have a sense of what was deleted in this process. A record of the flags is also stored in a separate table in the MS, called <tt>FLAG_CMD</tt>. (In fact, the information for this table is actually a subdirectory within the MS; you can see this by listing the contents of <tt> | The online flags, which are a record of known bad data produced by the EVLA online system, have already been applied by the archive as it generated the MS. However, it's good to have a sense of what was deleted in this process. A record of the flags is also stored in a separate table in the MS, called <tt>FLAG_CMD</tt>. (In fact, the information for this table is actually a subdirectory within the MS; you can see this by listing the contents of <tt>SN2010FZ_10s.ms</tt>.) | ||
[[Image:plotSN2010FZ_flagcmd.png|200px|thumb|right|online flags plotted from flagcmd]] | [[Image:plotSN2010FZ_flagcmd.png|200px|thumb|right|online flags plotted from flagcmd]] | ||
| Line 250: | Line 250: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
flagcmd(vis=' | flagcmd(vis='SN2010FZ_10s.ms',flagmode='table',optype='list') | ||
</source> | </source> | ||
These will go to the logger. | These will go to the logger. | ||
| Line 274: | Line 274: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotants(' | plotants('SN2010FZ_10s.ms') | ||
</source> | </source> | ||
| Line 300: | Line 300: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='',spw='2~17:31~31', \ | ||
antenna='ea01&ea02',correlation='RR,LL',xaxis='time',yaxis='amp') | antenna='ea01&ea02',correlation='RR,LL',xaxis='time',yaxis='amp') | ||
</source> | </source> | ||
| Line 310: | Line 310: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17:31~31', \ | ||
antenna='ea01',correlation='RR,LL',xaxis='antenna2',yaxis='amp') | antenna='ea01',correlation='RR,LL',xaxis='antenna2',yaxis='amp') | ||
</source> | </source> | ||
| Line 320: | Line 320: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17', \ | ||
antenna='ea02;!ea13',correlation='RR,LL',xaxis='frequency',yaxis='amp') | antenna='ea02;!ea13',correlation='RR,LL',xaxis='frequency',yaxis='amp') | ||
</source> | </source> | ||
| Line 329: | Line 329: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17',antenna='ea02;!ea13', \ | ||
correlation='RR,LL',xaxis='frequency',yaxis='amp',iteraxis='baseline') | correlation='RR,LL',xaxis='frequency',yaxis='amp',iteraxis='baseline') | ||
</source> | </source> | ||
| Line 335: | Line 335: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17',antenna='ea02;!ea13', \ | ||
correlation='RR,LL',xaxis='frequency',yaxis='phase',iteraxis='baseline') | correlation='RR,LL',xaxis='frequency',yaxis='phase',iteraxis='baseline') | ||
</source> | </source> | ||
| Line 350: | Line 350: | ||
'antenna="ea25" spw="10~17"', | 'antenna="ea25" spw="10~17"', | ||
'antenna="ea20" spw="2~9"'] | 'antenna="ea20" spw="2~9"'] | ||
flagcmd(vis=' | flagcmd(vis='SN2010FZ_10s.ms',flagmode='cmd',command=flaglist,optype='apply',flagbackup=False) | ||
</source> | </source> | ||
These commands will be carried out as well as being added to the FLAG_CMD table (marked as applied). | These commands will be carried out as well as being added to the FLAG_CMD table (marked as applied). | ||
| Line 357: | Line 357: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17',antenna='ea02', \ | ||
correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44') | correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44') | ||
</source> | </source> | ||
| Line 365: | Line 365: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='0',spw='2~17',antenna='ea02', \ | ||
correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44') | correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44') | ||
</source> | </source> | ||
| Line 380: | Line 380: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
plotms(vis=' | plotms(vis='SN2010FZ_10s.ms',field='0',spw='2~17:4~59',antenna='', \ | ||
correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44') | correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44') | ||
</source> | </source> | ||
| Line 388: | Line 388: | ||
<source lang="python"> | <source lang="python"> | ||
# In CASA | # In CASA | ||
split(vis=' | split(vis='SN2010FZ_10s.ms',outputvis='SN2010FZ_flagged10s.ms',datacolumn='data',spw='2~9,12~17',scan='7~44') | ||
</source> | </source> | ||
Revision as of 00:16, 7 January 2012
Overview
This article describes the calibration and imaging of a single-pointing 6-cm EVLA wideband continuum dataset of the galaxy NGC2967 (UGC5180) which was the location of the supernova candidate SN2010FZ. No supernova was detected in this observation, but the galactic continuum emission from this face-on spiral is nicely imaged. The data were taken in with 1024 MHz of bandwidth in each of two widely spaced basebands (each comprised of 8 128-MHz spectral windows), spanning 4.5 to 7.5 GHz. We will use wideband imaging techniques in this tutorial.
This is a more advanced tutorial, and if you are a relative novice (and particularly for EVLA continuum calibration and imaging), it is strongly recommended that you start with the EVLA Continuum Tutorial 3C391 (at least read it through) before tackling this dataset. We will not include basic information on CASA processing in this tutorial.
From the [MainPage] of this guide you can find helpful information:
- What is CASA?
- Getting Started in CASA
- CASA Reference Manuals
- Hints, Tips, & Tricks
- AIPS-to-CASA Cheat Sheet
In this tutorial we will be invoking the tasks as function calls. You can cut and paste these to your casapy session. We also recommend that you copy all the commands you use, with any relevant commentary, to a text file. This is good practice when tackling large datasets. If you wish, you can use the Script Extractor to create a file with the tutorial commands, which can subsequently be edited as desired.
Occasionally we will be setting Python variables (e.g. as lists for flags) outside the function call so make sure you set those before running the task command. Note that when you call a CASA task as a function the task parameters you do not set in the function call (assuming there is at least one) will be set to their defaults, and will not use values you set in previous calls or outside the call. See Getting_Started_in_CASA#Task_Execution for more on calling tasks and setting parameters in the scripting interface.
NOTE: If you find that the figures on the right margin of the browser window overlap the text too much and make reading difficult, go ahead and widen the browser window.
CASA Versions
This tutorial was written for the CASA Version 3.2.1 (release r15198 26 May 2011). It has also been verified to work in the release Version 3.3.0 (r16617, 12 Oct 2011).
Obtaining the Data
The method you should use to get your data depends on whether you are at the Caltech CASA Radio Analysis Workshop or are doing the tutorial on your own.
At the Caltech CASA Radio Analysis Workshop
At the workshop, we will be using data which has already been placed on a drive which is accessible with the cluster machines we'll be using. The data directory is /data/casa/, and either evla or alma, depending on the tutorial. Note that many CASA operations are very i/o intensive, so you will place your working data on a disk local to your login node. While this is the fastest setup, there is limited space on these local disks. Therefore, we suggest that you check how much free space remains (using "df -k ."), and delete any data from a previous tutorial before starting a new one, if needed.
While you should read through the first few sections, the Measurement Set (MS) you will be starting with has already had the first few steps performed, to save both time and disk space. To copy this your local directory, do the following:
cd /scr[2]/casa/
mkdir evla_6-cm_wideband
cd evla_6-cm_wideband
tar -xvf /data/casa/evla/tar/SN2010FZ_10s.tar
Note that you will need to use either /scr or /scr2, depending on your node assignment. This will take a minute, but once it's complete, you will have a directory called SN2010FZ_10s.ms which is the filled data, with online flags applied (which delete known bad data), and time-averaged to 10 seconds. (The data were taken in D-configuration [max baselines 1km], so one can safely average to 3s or even 10s to reduce dataset size.) This is equivalent to what you would download from the archive if you request time-averaging and online flag application.
On your own
The data for this tutorial were taken with the EVLA under program AS1015 as the scheduling block (SB) AS1015_sb1658169_1.55388.89474846065, and was run on 2010-07-11 from 21:28 to 22:28 UT (size 37.74GB). This can be downloaded from the NRAO Science Data Archive, with online flags applied (check box "Apply flags generated during observing") and time-averaging set to 10 seconds. (The data were taken in D-configuration [max baselines 1km], so one can safely average to 3s or even 10s to reduce dataset size.) This will create a file equivalent to what is used at the workshop. (Note that these data will not become public, and will therefore not be downloadable, until 25 Aug 2012.)
Starting CASA
To start CASA, type:
casapy
This will run a script to initialize CASA, setting paths appropriately. It will also start writing to a file called ipython.log, which will contain a record of all the text you enter at the CASA prompt.
A logger window will also appear; note that you can rescale this window or change the font size as desired (the latter is under "View"). The messages which are printed to the logger are also saved to a file called casalog.py, and any previous version of casalog.py which may have been present is moved to a backup version with a date stamp. Note that this does not happen for any previous versions of ipython.log, so if you wish to save one, be sure to rename it before restarting CASA.
Examining the MS
We use listobs to summarize our MS:
# In CASA
listobs('SN2010FZ_10s.ms')
In the logger you should see:
#####################################
Begin Task: listobs #####
==============================================================================
MeasurementSet Name: /scr2/casa/evla_6-cm_wideband/SN2010FZ_10s.ms MS Version 2
==============================================================================
Observer: Dr. Alicia M. Soderberg Project: T.B.D.
Observation: EVLA
Data records: 1570154 Total integration time = 3359 seconds
Observed from 11-Jul-2010/21:30:44.0 to 11-Jul-2010/22:26:43.0 (UTC)
ObservationID = 0 ArrayID = 0
Date Timerange (UTC) Scan FldId FieldName nRows Int(s) SpwIds ScanIntent
11-Jul-2010/21:30:44.0 - 21:30:52.0 2 0 J0925+0019 362 6.71 [0, 1] CALIBRATE_PHASE#UNSPECIFIED
21:31:02.0 - 21:32:22.0 3 0 J0925+0019 5061 9.65 [0, 1] CALIBRATE_PHASE#UNSPECIFIED
21:32:32.0 - 21:33:52.0 4 0 J0925+0019 6277 9.91 [0, 1] CALIBRATE_PHASE#UNSPECIFIED
21:34:02.0 - 21:34:52.0 5 0 J0925+0019 4148 9.96 [0, 1] CALIBRATE_PHASE#UNSPECIFIED
21:35:02.0 - 21:35:51.5 6 0 J0925+0019 32640 9.44 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_PHASE#UNSPECIFIED
21:36:01.0 - 21:38:20.5 7 0 J0925+0019 81600 9.93 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_PHASE#UNSPECIFIED
21:38:44.0 - 21:39:51.0 9 1 SN2010FZ 43520 9.11 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:40:01.0 - 21:41:20.5 10 1 SN2010FZ 48960 9.89 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:41:30.0 - 21:42:50.0 11 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:43:00.0 - 21:44:20.0 12 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:44:30.0 - 21:45:50.0 13 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:46:00.0 - 21:47:19.5 14 1 SN2010FZ 48960 9.89 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:47:29.0 - 21:47:48.5 15 1 SN2010FZ 16320 9.67 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:48:12.0 - 21:49:18.5 16 0 J0925+0019 43520 8.92 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_PHASE#UNSPECIFIED
21:49:42.0 - 21:50:49.0 17 1 SN2010FZ 43520 9.11 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:50:59.0 - 21:52:19.0 18 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:52:29.0 - 21:53:48.5 19 1 SN2010FZ 48960 9.89 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:53:58.0 - 21:55:18.0 20 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:55:28.0 - 21:56:48.0 21 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:56:58.0 - 21:58:18.0 22 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:58:28.0 - 21:58:47.0 23 1 SN2010FZ 16320 9.33 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
21:59:11.0 - 22:00:17.0 24 0 J0925+0019 43520 8.93 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_PHASE#UNSPECIFIED
22:00:40.0 - 22:01:47.0 25 1 SN2010FZ 43520 9.12 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:01:57.0 - 22:03:17.0 26 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:03:27.0 - 22:04:47.0 27 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:04:57.0 - 22:06:16.5 28 1 SN2010FZ 48960 9.89 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:06:26.0 - 22:07:46.0 29 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:07:56.0 - 22:09:16.0 30 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:09:26.0 - 22:09:45.0 31 1 SN2010FZ 16320 9.33 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:10:08.5 - 22:11:15.0 32 0 J0925+0019 43520 8.94 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_PHASE#UNSPECIFIED
22:11:38.5 - 22:12:45.5 33 1 SN2010FZ 43520 9.11 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:12:55.0 - 22:14:15.0 34 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:14:25.0 - 22:15:45.0 35 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:15:55.0 - 22:17:15.0 36 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:17:25.0 - 22:18:44.5 37 1 SN2010FZ 48960 9.89 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:18:54.0 - 22:20:14.0 38 1 SN2010FZ 48960 10 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:20:24.0 - 22:20:43.5 39 1 SN2010FZ 16320 9.6 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]OBSERVE_TARGET#UNSPECIFIED
22:21:06.5 - 22:22:13.5 40 0 J0925+0019 42690 8.96 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_PHASE#UNSPECIFIED
22:25:13.0 - 22:25:13.0 42 2 3C286 860 2.64 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_AMPLI#UNSPECIFIED
22:25:23.0 - 22:26:43.0 43 2 3C286 47396 9.62 [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_AMPLI#UNSPECIFIED
(nVis = Total number of time/baseline visibilities per scan)
elds: 3
ID Code Name RA Decl Epoch SrcId nVis
0 D J0925+0019 09:25:07.81503 +00.19.13.9334 J2000 0 303338
1 NONE SN2010FZ 09:42:04.77000 +00.19.51.0000 J2000 1 1218560
2 K 3C286 13:31:08.28798 +30.30.32.9589 J2000 2 48256
(nVis = Total number of time/baseline visibilities per field)
ectral Windows: (18 unique spectral windows and 1 unique polarization setups)
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz) TotBW(kHz) Corrs
0 64 TOPO 7686 2000 128000 RR RL LR LL
1 64 TOPO 7836 2000 128000 RR RL LR LL
2 64 TOPO 4488 2000 128000 RR RL LR LL
3 64 TOPO 4616 2000 128000 RR RL LR LL
4 64 TOPO 4744 2000 128000 RR RL LR LL
5 64 TOPO 4872 2000 128000 RR RL LR LL
6 64 TOPO 5000 2000 128000 RR RL LR LL
7 64 TOPO 5128 2000 128000 RR RL LR LL
8 64 TOPO 5256 2000 128000 RR RL LR LL
9 64 TOPO 5384 2000 128000 RR RL LR LL
10 64 TOPO 6488 2000 128000 RR RL LR LL
11 64 TOPO 6616 2000 128000 RR RL LR LL
12 64 TOPO 6744 2000 128000 RR RL LR LL
13 64 TOPO 6872 2000 128000 RR RL LR LL
14 64 TOPO 7000 2000 128000 RR RL LR LL
15 64 TOPO 7128 2000 128000 RR RL LR LL
16 64 TOPO 7256 2000 128000 RR RL LR LL
17 64 TOPO 7384 2000 128000 RR RL LR LL
urces: 50
ID Name SpwId RestFreq(MHz) SysVel(km/s)
0 J0925+0019 0 - -
0 J0925+0019 1 - -
0 J0925+0019 2 - -
0 J0925+0019 3 - -
0 J0925+0019 4 - -
0 J0925+0019 5 - -
0 J0925+0019 6 - -
0 J0925+0019 7 - -
0 J0925+0019 8 - -
0 J0925+0019 9 - -
0 J0925+0019 10 - -
0 J0925+0019 11 - -
0 J0925+0019 12 - -
0 J0925+0019 13 - -
0 J0925+0019 14 - -
0 J0925+0019 15 - -
0 J0925+0019 16 - -
0 J0925+0019 17 - -
1 SN2010FZ 2 - -
1 SN2010FZ 3 - -
1 SN2010FZ 4 - -
1 SN2010FZ 5 - -
1 SN2010FZ 6 - -
1 SN2010FZ 7 - -
1 SN2010FZ 8 - -
1 SN2010FZ 9 - -
1 SN2010FZ 10 - -
1 SN2010FZ 11 - -
1 SN2010FZ 12 - -
1 SN2010FZ 13 - -
1 SN2010FZ 14 - -
1 SN2010FZ 15 - -
1 SN2010FZ 16 - -
1 SN2010FZ 17 - -
2 3C286 2 - -
2 3C286 3 - -
2 3C286 4 - -
2 3C286 5 - -
2 3C286 6 - -
2 3C286 7 - -
2 3C286 8 - -
2 3C286 9 - -
2 3C286 10 - -
2 3C286 11 - -
2 3C286 12 - -
2 3C286 13 - -
2 3C286 14 - -
2 3C286 15 - -
2 3C286 16 - -
2 3C286 17 - -
Antennas: 27:
ID Name Station Diam. Long. Lat.
0 ea01 W09 25.0 m -107.37.25.2 +33.53.51.0
1 ea02 E02 25.0 m -107.37.04.4 +33.54.01.1
2 ea03 E09 25.0 m -107.36.45.1 +33.53.53.6
3 ea04 W01 25.0 m -107.37.05.9 +33.54.00.5
4 ea05 W08 25.0 m -107.37.21.6 +33.53.53.0
5 ea06 N06 25.0 m -107.37.06.9 +33.54.10.3
6 ea08 N01 25.0 m -107.37.06.0 +33.54.01.8
7 ea09 E06 25.0 m -107.36.55.6 +33.53.57.7
8 ea10 N03 25.0 m -107.37.06.3 +33.54.04.8
9 ea11 E04 25.0 m -107.37.00.8 +33.53.59.7
10 ea12 E08 25.0 m -107.36.48.9 +33.53.55.1
11 ea13 N07 25.0 m -107.37.07.2 +33.54.12.9
12 ea14 E05 25.0 m -107.36.58.4 +33.53.58.8
13 ea15 W06 25.0 m -107.37.15.6 +33.53.56.4
14 ea16 W02 25.0 m -107.37.07.5 +33.54.00.9
15 ea17 W07 25.0 m -107.37.18.4 +33.53.54.8
16 ea18 N09 25.0 m -107.37.07.8 +33.54.19.0
17 ea19 W04 25.0 m -107.37.10.8 +33.53.59.1
18 ea20 N05 25.0 m -107.37.06.7 +33.54.08.0
19 ea21 E01 25.0 m -107.37.05.7 +33.53.59.2
20 ea22 N04 25.0 m -107.37.06.5 +33.54.06.1
21 ea23 E07 25.0 m -107.36.52.4 +33.53.56.5
22 ea24 W05 25.0 m -107.37.13.0 +33.53.57.8
23 ea25 N02 25.0 m -107.37.06.2 +33.54.03.5
24 ea26 W03 25.0 m -107.37.08.9 +33.54.00.1
25 ea27 E03 25.0 m -107.37.02.8 +33.54.00.5
26 ea28 N08 25.0 m -107.37.07.5 +33.54.15.8
##### End Task: listobs #####
##########################################
This task displays a lot of information about the MS. We can see that the observation was performed with the EVLA, for a total integration of 3359 seconds (1 hour). The number of data records (1570154) is equal to the number of baselines (N_antenna * [N_antenna - 1] / 2) X the number of integrations (observing time / time-average binning) X the number of spectral windows. For this observation, this is roughly 351 X 360 X 16 = 2021760. Note that this is high by ~30%; this is because we have already flagged bad data, and there are some scans which only have two (rather than 16) spectral windows present. Extra exercise: examine the MS using browsetable to see what a data record looks like (equivalent to a row, as displayed by this task).
The most useful parts of the listobs output are the scan, field, and spectral window listings.
From the spectral window information, we can see that there were a total of 18 (0 through 17) spectral windows in this dataset. The first two of these (0 and 1) were only used to help set up the correlator.
Looking at the scan listing, we can see that the first four scans which are present used only these spectral windows. These are referred to as "dummy scans". We will not be using these, since they often contain bad data.
The C-band data of interest is contained in scans 6-44 and spectral windows 2 to 17. Careful examination shows that scan 8 is missing but from the time ranges that the data has been merged into scan 7. This sort of correlator back-end data-capture issue was occasionally seen during 2010. Hopefully, it will not affect our data, but we should keep an eye out for problems with scan 7.
The field listing shows three sources: J0925+0019 (also referred to by its field ID, 0), the phase calibration source, SN2010FZ (1), our science target, and 3C286 (2), which will provide our flux standard.
Flagging the MS
The online flags, which are a record of known bad data produced by the EVLA online system, have already been applied by the archive as it generated the MS. However, it's good to have a sense of what was deleted in this process. A record of the flags is also stored in a separate table in the MS, called FLAG_CMD. (In fact, the information for this table is actually a subdirectory within the MS; you can see this by listing the contents of SN2010FZ_10s.ms.)
You can examine the commands stored in the FLAG_CMD table using flagcmd.
# In CASA
flagcmd(vis='SN2010FZ_10s.ms',flagmode='table',optype='list')
These will go to the logger.
You can also plot the commands stored in the FLAG_CMD table:
# In CASA
myrows = range(285)
flagcmd(vis='SN2010FZ_filled.ms',flagmode='table',optype='plot',flagrows=myrows)
This will bring up a matplotlib plotter. You can have it plot to a PNG file instead:
# In CASA
flagcmd(vis='SN2010FZ_filled.ms',flagmode='table',optype='plot',flagrows=myrows,outfile='plotSN2010FZ_flagcmd.png')
The flags as plotted in the figure to the above right look normal. They are color-coded by REASON, and you see OFF_SOURCE flags between scans, and the occasional SUBREFLECTOR flag also between scans (most likely after band changes when the subreflector rotates to pick up the new feed on the ring, some are slower than others). What you watch for here are long blocks of unexpected flags like FOCUS, which might be false alarms and cause you to flag too much data. In that case, look at the data itself in plotms (see below for examples) to decide whether or not to apply all flags.
To plot up the antenna positions in the array:
# In CASA
plotants('SN2010FZ_10s.ms')
NOTE: if after this point (or any other) you get "table locks", which may occur erroneously and are sometimes triggered by plotting tasks, use clearstat to clear them:
# In CASA
clearstat
Now we examine the MS looking for bad data to flag. We will use plotms to bring up an interactive GUI that will display 2-D Y vs.X style line plots. NOTE: We do not recommend using the editing/flagging features of plotms. It is very easy to mess up your data this way. Also, to improve speed we will be restricting the scope of plotting so most box/flag operations would not get rid of all the bad data.
We will instead use plotms to identify bad data and then use flagcmd to flag it. This will also allow full scripting of the flagging, which is ultimately the best way to keep track of what's been deleted. Given the large dataset sizes now being generated, reproducibility is extremely important. Imaging spending a day flagging your data, then a disk error corrupts the MS. It's imperative that you have an automated way to regenerate your work. This is also why we encourage you to keep a running file with all the commands you use on a dataset.
NOTE: If you need an introduction to plotms, see:
- Data flagging with plotms
- Averaging data in plotms
- What's the difference between Antenna1 and Antenna2? Axis definitions in plotms
WARNING: The Flag 
button on the plotms GUI is close to other buttons you will be using, in particular the one that gets rid of boxes you have drawn. Be careful and don't hit the Flag button by mistake!
As we found above, the useful spectral windows are 2-17. To get an idea of the data layout, plot a single baseline (ea01&ea02) and channel (31, for all spectral windows) versus time:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='',spw='2~17:31~31', \
antenna='ea01&ea02',correlation='RR,LL',xaxis='time',yaxis='amp')
Here, we can see the alternating phase calibration and science target scans, as well as the (brighter) flux calibrator at the end of the observation. Feel free to play with ways to view, or color the data: for example, go to the "Display" left-hand tab, and choose "Colorize by: Field". You can also change the size of the plotted points, if they are too small to see easily, by setting "Unflagged Points Symbol" to "Custom" and increasing the number of pixels under "Style".
Look for bad antennas by picking the last field and plotting baselines versus antenna ea01:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17:31~31', \
antenna='ea01',correlation='RR,LL',xaxis='antenna2',yaxis='amp')
You should be able to see that antenna 11 (= ea13) is bad (very low amplitude, it has no C-band receiver!) and that some of the spectral windows on 15 and 23 (ea17, ea25) are also on the low side. Boxing with the Mark Regions 
tool and using the Locate 
tool will show in the logger that spw 10-17 are suspect for these antennas.
Now look at the bandpass for ea02 - it is in the inner core and a prospective reference antenna. Exclude ea13 using negation (represented by "!") in the selection:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17', \
antenna='ea02;!ea13',correlation='RR,LL',xaxis='frequency',yaxis='amp')
There is clearly less data for spw 11, and use of Locate shows spw 11 data only for ea02,ea03,04,08,09,11,12. We will later delete this incomplete spw. Note also the very strong RFI spike at 6614 MHz (spw 10 ch 63) with clear ringing contaminating both spw 10 and 11. There is also a tremendous roll-off in spw 10. We will drop these spectral window when we process the data.
We can also step through the baselines to our antenna using iteraxis - use the Next Iteration 
button to step through:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17',antenna='ea02;!ea13', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',iteraxis='baseline')
This will make it easier to isolate the bad antennas. Now plot the phases, iterating through baselines to ea02:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17',antenna='ea02;!ea13', \
correlation='RR,LL',xaxis='frequency',yaxis='phase',iteraxis='baseline')
You see the slopes due to residual delays. Mostly a turn or less over a 128MHz subband, but there are some outliers. Step through to ea20. You see that there is a very large delay in RR (via locate) for the first baseband (spw 0~7). We will delete this (will also delete LL so there are no orphan polarization products). Note ea17 and ea25 baselines drop close to zero in the middle of upper baseband (e.g. plot 'ea17&ea25') so we will delete these.
To carry out flagging, we again use flagcmd in the mode where it takes a list of command strings:
# In CASA
flaglist = ['antenna="ea13"',
'antenna="ea17" spw="10~17"',
'antenna="ea25" spw="10~17"',
'antenna="ea20" spw="2~9"']
flagcmd(vis='SN2010FZ_10s.ms',flagmode='cmd',command=flaglist,optype='apply',flagbackup=False)
These commands will be carried out as well as being added to the FLAG_CMD table (marked as applied).
Plot the data again now having flagged:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='2',spw='2~17',antenna='ea02', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44')
Now our phase calibrator - it is weaker, and we now start to see the RFI:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='0',spw='2~17',antenna='ea02', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44')
Use the Zoom feature, Mark rectangles and use Locate to identify the frequency/channel of RFI. In particular, we note in our analysis:
- 6614MHz (spw 10 ch 63) super strong
- 6772-6778MHz (spw 12 ch 14-17)
- 7260-7264MHz (spw 16 ch 2-4)
- 7314-7340MHz (spw 16 ch 29-42)
- 7402-7418MHz (spw 17 ch 9-17)
- 7458-7466MHz (spw 17 ch 37-41)
- 7488MHz (spw 17 ch 52)
If you plot all antennas and avoid the band edges you see spw 16,17 are pretty wiped out:
# In CASA
plotms(vis='SN2010FZ_10s.ms',field='0',spw='2~17:4~59',antenna='', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',scan='7~44')
For now we will not flag these channels but note them to mask out when creating continuum calibration tables and images.
Finally, split off the good scans and spw, this will allow us to work on the data without having to start completely over if we mess something up badly as well as letting us do simpler selection:
# In CASA
split(vis='SN2010FZ_10s.ms',outputvis='SN2010FZ_flagged10s.ms',datacolumn='data',spw='2~9,12~17',scan='7~44')
You now have a MS called SN2010FZ_flagged10s.ms in your working area. This should be 2.8GB in size.
Calibration
Summarize the split flagged MS:
# In CASA
listobs('SN2010FZ_flagged10s.ms')
In the logger we see:
================================================================================
MeasurementSet Name: /home/sandrock2/smyers/casa/tutorials/SN2010FZ/SN2010FZ_flagged10s.ms MS Version 2
================================================================================
Observer: Dr. Alicia M. Soderberg Project: T.B.D.
Observation: EVLA
Data records: 1374548 Total integration time = 3042 seconds
Observed from 11-Jul-2010/21:36:01.0 to 11-Jul-2010/22:26:43.0 (UTC)
ObservationID = 0 ArrayID = 0
Date Timerange (UTC) Scan FldId FieldName nVis Int(s) SpwIds ScanIntent
11-Jul-2010/21:36:01.0 - 21:38:20.5 7 0 J0925+0019 73710 9.93 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]CALIBRATE_PHASE#UNSPECIFIED
21:38:44.0 - 21:39:51.0 9 1 SN2010FZ 39312 9.11 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:40:01.0 - 21:41:20.5 10 1 SN2010FZ 44226 9.89 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:41:30.0 - 21:42:50.0 11 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:43:00.0 - 21:44:20.0 12 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:44:30.0 - 21:45:50.0 13 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:46:00.0 - 21:47:19.5 14 1 SN2010FZ 44226 9.89 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:47:29.0 - 21:47:48.5 15 1 SN2010FZ 14742 9.67 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:48:12.0 - 21:49:18.5 16 0 J0925+0019 39312 8.92 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]CALIBRATE_PHASE#UNSPECIFIED
21:49:42.0 - 21:50:49.0 17 1 SN2010FZ 39312 9.11 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:50:59.0 - 21:52:19.0 18 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:52:29.0 - 21:53:48.5 19 1 SN2010FZ 44226 9.89 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:53:58.0 - 21:55:18.0 20 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:55:28.0 - 21:56:48.0 21 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:56:58.0 - 21:58:18.0 22 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:58:28.0 - 21:58:47.0 23 1 SN2010FZ 14742 9.33 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
21:59:11.0 - 22:00:17.0 24 0 J0925+0019 39312 8.93 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]CALIBRATE_PHASE#UNSPECIFIED
22:00:40.0 - 22:01:47.0 25 1 SN2010FZ 39312 9.12 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:01:57.0 - 22:03:17.0 26 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:03:27.0 - 22:04:47.0 27 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:04:57.0 - 22:06:16.5 28 1 SN2010FZ 44226 9.89 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:06:26.0 - 22:07:46.0 29 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:07:56.0 - 22:09:16.0 30 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:09:26.0 - 22:09:45.0 31 1 SN2010FZ 14742 9.33 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:10:08.5 - 22:11:15.0 32 0 J0925+0019 39312 8.94 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]CALIBRATE_PHASE#UNSPECIFIED
22:11:38.5 - 22:12:45.5 33 1 SN2010FZ 39312 9.11 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:12:55.0 - 22:14:15.0 34 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:14:25.0 - 22:15:45.0 35 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:15:55.0 - 22:17:15.0 36 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:17:25.0 - 22:18:44.5 37 1 SN2010FZ 44226 9.89 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:18:54.0 - 22:20:14.0 38 1 SN2010FZ 44226 10 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:20:24.0 - 22:20:43.5 39 1 SN2010FZ 14742 9.6 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]OBSERVE_TARGET#UNSPECIFIED
22:21:06.5 - 22:22:13.5 40 0 J0925+0019 38584 8.96 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]CALIBRATE_PHASE#UNSPECIFIED
22:25:13.0 - 22:25:13.0 42 2 3C286 770 2.65 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_AMPLI#UNSPECIFIED
22:25:23.0 - 22:26:43.0 43 2 3C286 42812 9.62 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_AMPLI#UNSPECIFIED
(nVis = Total number of time/baseline visibilities per scan)
Fields: 3
ID Code Name RA Decl Epoch SrcId nVis
0 D J0925+0019 09:25:07.8150 +00.19.13.9334 J2000 0 230230
1 NONE SN2010FZ 09:42:04.7700 +00.19.51.0000 J2000 1 1100736
2 K 3C286 13:31:08.2880 +30.30.32.9589 J2000 2 43582
(nVis = Total number of time/baseline visibilities per field)
Spectral Windows: (14 unique spectral windows and 1 unique polarization setups)
SpwID #Chans Frame Ch1(MHz) ChanWid(kHz)TotBW(kHz) Ref(MHz) Corrs
0 64 TOPO 4488 2000 128000 4488 RR RL LR LL
1 64 TOPO 4616 2000 128000 4616 RR RL LR LL
2 64 TOPO 4744 2000 128000 4744 RR RL LR LL
3 64 TOPO 4872 2000 128000 4872 RR RL LR LL
4 64 TOPO 5000 2000 128000 5000 RR RL LR LL
5 64 TOPO 5128 2000 128000 5128 RR RL LR LL
6 64 TOPO 5256 2000 128000 5256 RR RL LR LL
7 64 TOPO 5384 2000 128000 5384 RR RL LR LL
8 64 TOPO 6744 2000 128000 6744 RR RL LR LL
9 64 TOPO 6872 2000 128000 6872 RR RL LR LL
10 64 TOPO 7000 2000 128000 7000 RR RL LR LL
11 64 TOPO 7128 2000 128000 7128 RR RL LR LL
12 64 TOPO 7256 2000 128000 7256 RR RL LR LL
13 64 TOPO 7384 2000 128000 7384 RR RL LR LL
Sources: 42
ID Name SpwId RestFreq(MHz) SysVel(km/s)
0 J0925+0019 0 - -
0 J0925+0019 1 - -
0 J0925+0019 2 - -
0 J0925+0019 3 - -
0 J0925+0019 4 - -
0 J0925+0019 5 - -
0 J0925+0019 6 - -
0 J0925+0019 7 - -
0 J0925+0019 8 - -
0 J0925+0019 9 - -
0 J0925+0019 10 - -
0 J0925+0019 11 - -
0 J0925+0019 12 - -
0 J0925+0019 13 - -
1 SN2010FZ 0 - -
1 SN2010FZ 1 - -
1 SN2010FZ 2 - -
1 SN2010FZ 3 - -
1 SN2010FZ 4 - -
1 SN2010FZ 5 - -
1 SN2010FZ 6 - -
1 SN2010FZ 7 - -
1 SN2010FZ 8 - -
1 SN2010FZ 9 - -
1 SN2010FZ 10 - -
1 SN2010FZ 11 - -
1 SN2010FZ 12 - -
1 SN2010FZ 13 - -
2 3C286 0 - -
2 3C286 1 - -
2 3C286 2 - -
2 3C286 3 - -
2 3C286 4 - -
2 3C286 5 - -
2 3C286 6 - -
2 3C286 7 - -
2 3C286 8 - -
2 3C286 9 - -
2 3C286 10 - -
2 3C286 11 - -
2 3C286 12 - -
2 3C286 13 - -
Antennas: 27:
ID Name Station Diam. Long. Lat.
0 ea01 W09 25.0 m -107.37.25.2 +33.53.51.0
1 ea02 E02 25.0 m -107.37.04.4 +33.54.01.1
2 ea03 E09 25.0 m -107.36.45.1 +33.53.53.6
3 ea04 W01 25.0 m -107.37.05.9 +33.54.00.5
4 ea05 W08 25.0 m -107.37.21.6 +33.53.53.0
5 ea06 N06 25.0 m -107.37.06.9 +33.54.10.3
6 ea08 N01 25.0 m -107.37.06.0 +33.54.01.8
7 ea09 E06 25.0 m -107.36.55.6 +33.53.57.7
8 ea10 N03 25.0 m -107.37.06.3 +33.54.04.8
9 ea11 E04 25.0 m -107.37.00.8 +33.53.59.7
10 ea12 E08 25.0 m -107.36.48.9 +33.53.55.1
11 ea13 N07 25.0 m -107.37.07.2 +33.54.12.9
12 ea14 E05 25.0 m -107.36.58.4 +33.53.58.8
13 ea15 W06 25.0 m -107.37.15.6 +33.53.56.4
14 ea16 W02 25.0 m -107.37.07.5 +33.54.00.9
15 ea17 W07 25.0 m -107.37.18.4 +33.53.54.8
16 ea18 N09 25.0 m -107.37.07.8 +33.54.19.0
17 ea19 W04 25.0 m -107.37.10.8 +33.53.59.1
18 ea20 N05 25.0 m -107.37.06.7 +33.54.08.0
19 ea21 E01 25.0 m -107.37.05.7 +33.53.59.2
20 ea22 N04 25.0 m -107.37.06.5 +33.54.06.1
21 ea23 E07 25.0 m -107.36.52.4 +33.53.56.5
22 ea24 W05 25.0 m -107.37.13.0 +33.53.57.8
23 ea25 N02 25.0 m -107.37.06.2 +33.54.03.5
24 ea26 W03 25.0 m -107.37.08.9 +33.54.00.1
25 ea27 E03 25.0 m -107.37.02.8 +33.54.00.5
26 ea28 N08 25.0 m -107.37.07.5 +33.54.15.8
Note that the spws are re-numbered to 0-13.
Prepare the MS for calibration by adding scratch columns. This will take a few minutes.
# In CASA
clearcal('SN2010FZ_flagged10s.ms')
Setting the flux density scale
Before calibrating, we put a model for flux calibration source 3C286 into the MS (in the MODEL_DATA column we just created). Have it set the flux on a per-channel basis. NOTE: This uses the AOC Unix location for casapy models (yours may be different):
# In CASA
setjy(vis='SN2010FZ_flagged10s.ms',field='2',spw='',scalebychan=True, \
modimage='/usr/lib64/casapy/data/nrao/VLA/CalModels/3C286_C.im')
It reports to logger that its about 7.7Jy at lower end to 5.7Jy at upper frequency limit.
Calibrating delays and bandpass
First, we do a phase-only calibration solution on a narrow range of channels in each spw on the bandpass/flux calibrator 3c286 to flatten them before solving for the bandpass. Note where we saw RFI in the higher spw and avoid those channels. The range 23~28 should work. Pick a refant near center - ea02 is a reasonable bet:
# In CASA
gaincal(vis='SN2010FZ_flagged10s.ms',caltable='calSN2010FZ.G0',field='2',spw='0~13:23~28', \
gaintype='G',refant='ea02',calmode='p',solint='int',minsnr=3)
Plot the phase solutions (using full phase range instead of autorange)
# In CASA
plotcal(caltable='calSN2010FZ.G0',xaxis='time',yaxis='phase',iteration='antenna', \
plotrange=[-1,-1,-180,180])
Step through the antenna-based solutions. They look good (and fairly flat over the scans). NOTE: If you want to make single-page multipanel plots (like those shown to the right), particularly for hardcopy (where it only shows the first page), you can do:
# In CASA
plotcal(caltable='calSN2010FZ.G0',xaxis='time',yaxis='phase', \
antenna='0~10,12~15',subplot=531,iteration='antenna', \
plotrange=[-1,-1,-180,180],showgui=False,fontsize=6.0, \
figfile='plotSN2010FZ_plotcal_G0p1.png')
plotcal(caltable='calSN2010FZ.G0',xaxis='time',yaxis='phase', \
antenna='16~26',subplot=531,iteration='antenna', \
plotrange=[-1,-1,-180,180],showgui=False,fontsize=6.0, \
figfile='plotSN2010FZ_plotcal_G0p2.png')
We can now solve for the residual antenna-based delays that we saw in phase vs. frequency. This uses the gaintype='K' option in gaincal (this is not documented in earlier releases, but is available in Version 3.2.1 and later). Note that this currently does not do a "global fringe-fitting" solution for delays, but instead does a baseline-based delay solution to all baselines to the refant, treating these as antenna-based delays. In most cases with high-enough S/N to get baseline-based delay solutions this will suffice. We avoid the beginning of spw 0 due to the extreme roll-off (with loss of S/N) at the starting edge.
# In CASA
gaincal(vis='SN2010FZ_flagged10s.ms',caltable='calSN2010FZ.K0',gaintable='calSN2010FZ.G0', \
field='2',spw='0:8~59,1~13:4~59',gaintype='K', \
refant='ea02',combine='scan',solint='inf',minsnr=3)
We pre-apply our initial phase table, and produce a new K-type caltable for input to bandpass calibration. The delays found are sent to the terminal. They are up to a few nanoseconds.
Now solve for the bandpass using the previous tables:
# In CASA
bandpass(vis='SN2010FZ_flagged10s.ms',caltable='calSN2010FZ.B0', \
gaintable=['calSN2010FZ.G0','calSN2010FZ.K0'], \
field='2',refant='ea02',solnorm=False, \
bandtype='B', combine='scan', solint='inf', gaincurve=True)
WARNING: You must set solnorm=False here or later on you will find some offsets between spw due to how amplitude scaling adjusts weights internally during solving.
This is the first amplitude-scaling calibration that we do, and thus we have turned on the application of gain-elevation curves (setting gaincurve=True). If we were at higher frequency we would set the opacity here also. We will set these in every amplitude solve and application from now on.
Now plot this, in amplitude then phase:
# In CASA
plotcal(caltable='calSN2010FZ.B0',xaxis='freq',yaxis='amp',iteration='antenna')
#
plotcal(caltable='calSN2010FZ.B0',xaxis='freq',yaxis='phase',iteration='antenna', \
plotrange=[-1,-1,-180,180])
In the bandpass phase you no longer see the residual antenna delays (just residual spw phase offsets from the delay solution registration) but there are some band edge effects. Note that some antennas have a little strange bandpasses at upper end of lower baseband in spw 5,6,7 (e.g. ea14,ea16,ea17,ea25). To plot amp and phase for a single antenna versus frequency (see plots at right):
# In CASA
plotcal(caltable='calSN2010FZ.B0',xaxis='freq',yaxis='amp', \
antenna='ea14',subplot=211,figfile='')
plotcal(caltable='calSN2010FZ.B0',xaxis='freq',yaxis='phase', \
antenna='ea14',subplot=212,plotrange=[-1,-1,-180,180], \
figfile='plotSN2010FZ_plotcal_B0ea14.png')
Because our flux density calibrator 3C286 is bright enough, we were able to use this as the bandpass calibrator. Since setjy put the correct spectrum for 3C286 into the MODEL_DATA column, our bandpass will reflect the true bandpass of the instrument. However, if for your observation you were unable to use a source of known spectrum as the bandpass calibrator, then you would need to follow this bandpass with a second one on a source of known spectrum in order to take out the spurious bandpass slope introduced by the (unknown) intrinsic spectral shape of your calibrator. Running {{bandpass} with bandtype='BPOLY' and degamp=1 should suffice to take out a slope albeit on a per-antenna basis rather than over all the array. This should work as long as you have enough S/N on your flux calibrator to solve for two polynomial orders (might be hard if you are using very narrow bands at high frequency).
Final phase and amplitude calibration
Now calibrate phases for real with wider bandwidth. First the flux calibrator again, with a per-integration solution time:
# In CASA
gaincal(vis='SN2010FZ_flagged10s.ms',caltable='calSN2010FZ.G1', \
gaintable=['calSN2010FZ.K0','calSN2010FZ.B0'], \
field='2',refant='ea02',solnorm=F, spw='0~13:4~59', \
solint='int',gaintype='G',calmode='p')
Next our phase calibrator, appending these solutions to previous table. Exclude RFI channels here, and obtain one solution per scan:
# In CASA
gaincal(vis='SN2010FZ_flagged10s.ms',caltable='calSN2010FZ.G1',
gaintable=['calSN2010FZ.K0','calSN2010FZ.B0'], \
field='0',refant='ea02',solnorm=F, \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
solint='inf',gaintype='G',calmode='p',append=True)
The phases look reasonably connected:
# In CASA
plotcal(caltable='calSN2010FZ.G1',xaxis='time',yaxis='phase',iteration='antenna', \
plotrange=[-1,-1,-180,180])
NOTE: If there were significant phase variations then you would also do short-timescale phase solutions on the gain calibrator (field 0), as we did for 3C286, that you would apply only to that calibrator in order to get the amplitude solutions correct. You would still apply the per-scan phases to the target. Because we have good phase stability we will only do per-scan phase solutions on J0925+0019 and use that in both the amplitude solutions and to correct the target phases.
Now solve for amplitudes on a per scan interval. Do these separately using gainfield so phases don't get transferred across fields. Uses linear interpolation of the previously determined phases by default. Pre-apply the gaincurve also:
# In CASA
gaincal(vis='SN2010FZ_flagged10s.ms', caltable='calSN2010FZ.G2', \
gaintable=['calSN2010FZ.K0','calSN2010FZ.B0','calSN2010FZ.G1'], \
gainfield=['2','2','2'], field='2',refant='ea02',solnorm=F,
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
solint='inf',gaintype='G',calmode='a',gaincurve=True)
#
gaincal(vis='SN2010FZ_flagged10s.ms', caltable='calSN2010FZ.G2', \
gaintable=['calSN2010FZ.K0','calSN2010FZ.B0','calSN2010FZ.G1'],\
gainfield=['2','2','0'], field='0',refant='ea02',solnorm=F, \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
solint='inf',gaintype='G',calmode='a',gaincurve=True,append=True)
Since the flux on the gain calibrator is not scaled to its correct flux (but to 1.0Jy) use fluxscale to transfer the amplitude gains from 3c286:
# In CASA
fluxscale(vis='SN2010FZ_flagged10s.ms',caltable='calSN2010FZ.G2', \
fluxtable='calSN2010FZ.F2',reference='2',transfer='0')
The logger output gives:
Found reference field(s): 3C286 Found transfer field(s): J0925+0019 Flux density for J0925+0019 in SpW=0 is: 0.976065 +/- 0.000880351 (SNR = 1108.72, nAnt= 25) Flux density for J0925+0019 in SpW=1 is: 0.97776 +/- 0.000811257 (SNR = 1205.24, nAnt= 25) Flux density for J0925+0019 in SpW=2 is: 0.979643 +/- 0.00069687 (SNR = 1405.78, nAnt= 25) Flux density for J0925+0019 in SpW=3 is: 0.980233 +/- 0.000838793 (SNR = 1168.62, nAnt= 25) Flux density for J0925+0019 in SpW=4 is: 0.981888 +/- 0.000689379 (SNR = 1424.31, nAnt= 25) Flux density for J0925+0019 in SpW=5 is: 0.982243 +/- 0.000813425 (SNR = 1207.54, nAnt= 21) Flux density for J0925+0019 in SpW=6 is: 0.981002 +/- 0.000894034 (SNR = 1097.27, nAnt= 21) Flux density for J0925+0019 in SpW=7 is: 0.98006 +/- 0.000672249 (SNR = 1457.88, nAnt= 21) Flux density for J0925+0019 in SpW=8 is: 0.960763 +/- 0.000715631 (SNR = 1342.54, nAnt= 24) Flux density for J0925+0019 in SpW=9 is: 0.957227 +/- 0.000781501 (SNR = 1224.86, nAnt= 24) Flux density for J0925+0019 in SpW=10 is: 0.955039 +/- 0.000774128 (SNR = 1233.7, nAnt= 24) Flux density for J0925+0019 in SpW=11 is: 0.953599 +/- 0.000944027 (SNR = 1010.14, nAnt= 24) Flux density for J0925+0019 in SpW=12 is: 0.950831 +/- 0.000933345 (SNR = 1018.73, nAnt= 24) Flux density for J0925+0019 in SpW=13 is: 0.946425 +/- 0.00101842 (SNR = 929.306, nAnt= 24)
You may see slightly different numbers on your machine.
As it so happens, the derived flux for J0925+0019 is about 1 Jy (you can plot up the raw amplitudes for fields 0,2 and convince yourself this is indeed true and not a bug). The spectrum rises a bit to peak in spw 5 then falls again.
Plot these solutions:
# In CASA
plotcal(caltable='calSN2010FZ.F2',xaxis='time',yaxis='amp',iteration='antenna')
The gains on 3C286 are about 1 (the bandpass solution on 3C286 has absorbed the calibration from counts to Jy) but fluxscale has adjusted the per-spw scale on J0925+0019 to get its correct spectrum rather than the assumed 1 Jy flat spectrum.
Applying the Calibration and Final Editing
Next we actually apply all our accumulated calibration tables. We apply these to the calibration fields individually using the appropriate gainfields and interpolation for each:
- For 3C286 (field 2) we did short-timescale phase solutions and a single scan amplitude, so use "linear" and "nearest" interpolation respectively.
- For the nearby gain calibrator (field 0) we did only scan-based phase and amplitude solutions so we use "nearest" interpolation
- For the target source we use field 0 to calibrate field 1, so use "linear" interpolation. This takes a few minutes.
# In CASA
applycal(vis='SN2010FZ_flagged10s.ms',field='2', \
gaintable=['calSN2010FZ.K0','calSN2010FZ.B0','calSN2010FZ.G1','calSN2010FZ.F2'], \
gainfield=['','','2','2'],interp=['nearest','nearest','linear','nearest'], \
parang=False,calwt=F,gaincurve=T)
#
applycal(vis='SN2010FZ_flagged10s.ms',field='0', \
gaintable=['calSN2010FZ.K0','calSN2010FZ.B0','calSN2010FZ.G1','calSN2010FZ.F2'], \
gainfield=['','','0','0'], interp=['nearest','nearest','nearest','nearest'], \
parang=False,calwt=F,gaincurve=T)
#
applycal(vis='SN2010FZ_flagged10s.ms',field='1', \
gaintable=['calSN2010FZ.K0','calSN2010FZ.B0','calSN2010FZ.G1','calSN2010FZ.F2'], \
gainfield=['','','0','0'], interp=['nearest','nearest','linear','linear'], \
parang=False,calwt=F,gaincurve=T)
We can examine the corrected data on 3c286 using our RFI mask from above and avoiding band edges
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='2', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
See figure above right. There is clearly discrepant data visible spw 5 and 6, in particular for baseline ea17&ea25 (use the
Mark Regions tool on some of it and then use the Locate tool), which gives a really strange response. You can plot just this baseline to be sure:
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='2', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
antenna='ea17&ea25', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
You can exclude this through antenna negation:
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='2', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
antenna='!ea17&ea25', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
Then use Locate for the other bad points, which seem to indicate spw 5,6,7 for ea14,ea16,ea17,ea25.
Exclude these and replot:
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='2', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
antenna='!ea14;!ea16;!ea17;!ea25', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
This now looks clean except for the RFI in the upper subbands.
Do flagging based on these:
# In CASA
flaglist = ['antenna="ea14,ea16,ea17,ea25" spw="5~7"']
flagcmd(vis='SN2010FZ_flagged10s.ms',flagmode='cmd',command=flaglist, \
optype='apply',flagbackup=False)
Now replot the corrected data (you may have to force reload if you plotted same thing right before this):
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='2', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
Looks good. If we were more diligent we would go back and recalibrate, but this looks good enough for now.
Plot the phase:
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='2', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',xaxis='frequency',yaxis='phase',ydatacolumn='corrected')
Note the characteristic "bowtie" pattern of the phases about the sub-band centers. Here we can see the effect of the EVLA "delay clunking", where the delay steps through discrete values such that the phase goes from -11deg to +11deg across the sub-band as the delay changes due to geometry. This is D-configuration so the delays change slowly, it will change faster in wider configurations. As of Q3 2011 we have not enabled the corrections for this in the EVLA system so you will always have this remaining delay error in your data. In principle you could solve for delays on short timescales and take this out, in practice this in not possible for your weaker target source in any event.
Now lets plot the corrected data amplitude for the phase calibrator (field 0):
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='0', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
You can see the roll-off pumping up noise at the baseband edges (about 8-16 channels worth). Also, we can see some RFI we missed:
- <6804MHz spw 8 below ch 30 lots of bad stuff (alot from ea18,ea22 but others too)
- 7168MHz spw 11 ch 20
- pretty much all of spw 12,13
The ch 20 ones are all harmonics of the notorious 128MHz tone. NOTE: You can get the frequency of a RFI feature by looking at
the logger report from using the Locate tool.
We will not flag these, but exclude them in imaging (so that more advanced students can try flagging these in detail or using auto-flagging). A good channel selection string for imaging might be:
spw = '0:16~59,1~6:4~59,7:4~54,8:30~59,9~10:4~59,11:4~19;21~59'
Without further flagging, it may be best to drop spw 12-13 for imaging (we will do so from now on).
Plot again (including this selection for spw 0-11):
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='0', \
spw='0:16~59,1~6:4~59,7:4~54,8:30~59,9~10:4~59,11:4~19;21~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
Looks better.
Now plot amplitudes for the corrected data averaged over baseline to see the source spectrum
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='0', \
spw='0:16~59,1~6:4~59,7:4~54,8:30~59,9~10:4~59,11:4~19;21~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',avgbaseline=True,avgtime='60000s',
xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
The last two sub-bands spw 12-13 give reasonable values, with only a tiny offset from spw 8-11. There are also strange amplitude excursions, particularly in the low end of the first baseband. These must be coming from one or more scans. You can iterate over scan to see the strange amplitudes (mostly in first scan):
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='0', \
spw='0:16~59,1~6:4~59,7:4~54,8:30~59,9~10:4~59,11:4~19;21~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',avgbaseline=True,avgtime='600s',iteraxis='scan',
xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
If we were interested in imaging the calibrator we would try to flag or fix this, but it shouldn't affect the calibration transfer to our target very much (we did not see anomalous solutions in that first scan).
We can also plot the corrected phase - looks fairly good
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='0', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',xaxis='frequency',yaxis='phase',ydatacolumn='corrected')
We can average over baseline and each scan:
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='0', \
spw='0:10~59,1~7:4~59,8:4~13;18~59,9~11:4~59,12:4~13;18~29;31~33;46~51;53~59,13:4~8;15~36;42~59', \
correlation='RR,LL',avgbaseline=True,avgtime='600s',
xaxis='frequency',yaxis='phase',ydatacolumn='corrected')
In this case we can see the residual effect of the EVLA "delay clunking" described above. It is reduced due to the averaging that we applied, but it is still there.
You can look at the target source field='1' but there are lots of data so you will need to do alot of averaging. For example:
# In CASA
plotms(vis='SN2010FZ_flagged10s.ms',field='1',avgtime='300s', \
spw='0:16~59,1~6:4~59,7:4~54,8:30~59,9~10:4~59,11:4~19;21~59', \
correlation='RR,LL',xaxis='frequency',yaxis='amp',ydatacolumn='corrected')
Alas, the upper baseband still has lots of low level RFI.
Now split off the data for calibrators and target, to avoid later issues that can corrupt the MSes. These will take a bit of time...field 1 took 18min on my workstation.
# In CASA
split(vis='SN2010FZ_flagged10s.ms',outputvis='SN2010FZ_split10s.ms',datacolumn='corrected',field='1')
#
split(vis='SN2010FZ_flagged10s.ms',outputvis='SN2010FZ_3c28610s.ms',datacolumn='corrected',field='2')
#
split(vis='SN2010FZ_flagged10s.ms',outputvis='SN2010FZ_J092510s.ms',datacolumn='corrected',field='0')
Imaging
This is D-config data at C-band, look at the Obs Status Summary: [1] Synthesized beam should be 12" at 6GHz with primary beam FOV 7.5arcmin (450"). Our data spans 4.5-7.5GHz : beam 9.6 at 7.5GHz and FOV 10' at 4.5GHz A cellsize of 3" should work, with an imsize > 200 to cover 2xFWHM FOV An imsize of 400+ will put the main beam inside the inner quarter safely The Briggs robust (0.5) weighting is somewhere between uniform and natural and will give reasonable resolution but still see some larger scale structure.
Due to the numerology of FFTW (which clean uses underneath for FFTs) optimal sizes, imsize should be composite number with two and only two prime factors chosen from 2,3,5. Taking into account the x1.2 padding that clean uses internally to the imsize you give it (and 1.2=2*3/5), we choose 640 or 1280 as our imsize (640=2^7*5). Other reasonable sets would be 405,1215,etc. (405=3^4*5) or 432,648,1296 (these are 2^n*3^m*5). In practice, if you give it non-optimal values for imsize, you may find that the transforms take a bit longer, which is noticeable if you are doing interactive clean.
WARNING: By default, a single-field nterms=1 clean does NOT use Cotton-Schwab (CS) clean to break into major cycles going back to data residuals, it just does cleaning in a bunch of minor cycles in the image plane. This can give much poorer imaging quality in cases with poor psf (snapshots) or in the case of complex emission structure (like ours) - clean tends to diverge in this case. You should explicitly set imagermode='csclean' in your call to clean. Also, in our case the psf is very good using mfs, so by default it will not take many major cycle breaks. We use the cyclefactor parameter to control this, which sets the break threshold to be cyclefactor times the max psf sidelobe level (outside the main peak). We start at cyclefactor=1.5 in a single spw, and ratchet it up to 4.5 when we clean all the spw. This seems to work ok. Rule of thumb is if it is gobbling up many hundreds of clean iterations in the minor cycles early on, increase cyclefactor. Conversely, if your psf is poor but you source structure is simple, you can reduce cyclefactor (e.g. below 1) to stop it from taking lots of extra major cycles.
For more information on using clean, in particular on using the interactive GUI, see EVLA_Continuum_Tutorial_3C391#Imaging.
NOTE: If you are pressed for time, then you might want to jump ahead to EVLA_6cmWideband_Tutorial_SN2010FZ#Cleaning_the_lower_baseband_using_two_MFS_Taylor_terms and while it is cleaning you can read the other Imaging descriptions.
Cleaning a single spectral window
Let us start by interactively cleaning one of the lower baseband spw (spw 5 in this example). NOTE: this first time will take a few minutes at start to create scratch columns in the MS in case we want to do self-calibration later.
# In CASA
clean(vis='SN2010FZ_split10s.ms',spw='5:4~59', \
imagename='imgSN2010FZ10s_spw5_clean640', \
mode='mfs',nterms=1,niter=10000,gain=0.1,threshold='0.0mJy', \
psfmode='clark',imsize=[640,640],cell=['3.0arcsec'],stokes='I', \
imagermode='csclean', cyclefactor=1.5, \
weighting='briggs',robust=0.5,calready=True,interactive=True)
Start carefully by boxing the bright source and setting iterations to 10 at first Gradually add more boxes and increase the number of iterations Since this is not much more than a snapshot you see the six-fold sidelobe pattern of the extended emission in the center of the map. This decreases as you clean out this emission. Stop cleaning when the residuals look like noise (and you cannot clearly see sources). To stop click the red X button.
The top figure to the right shows a zoom in on the end state of the clean, where we have marked a number of boxes and cleaned them out.
Note that there are some strange sidelobe patterns in lower left, possibly from a source outside the image area. We can make a bigger image starting from our current model:
# In CASA
clean(vis='SN2010FZ_split10s.ms',spw='5:4~59', \
imagename='imgSN2010FZ10s_spw5_clean1280', \
mode='mfs',nterms=1,niter=10000,gain=0.1,threshold='0.0mJy', \
psfmode='clark',imsize=[1280,1280],cell=['3.0arcsec'],stokes='I', \
imagermode='csclean', cyclefactor=1.5, \
modelimage='imgSN2010FZ10s_spw5_clean640.model', \
weighting='briggs',robust=0.5,calready=True,interactive=True)
Sure enough, there is a bright source near the lower left (see middle panel at right). Box it, clean it a bit, and look again. There is a second source in the mid-left (track it down by its sidelobes). Box this one, clean it a bit, and when satisfied stop.
You can use the CASA viewer to display the images that clean creates. If you need more guidance on using the viewer, see the [CASAViewerDemo] video. For now just bring up your restored image directly:
# In CASA
viewer('imgSN2010FZ10s_spw5_clean1280.image')
The restored image is shown in bottom panel to the right. I have chosen the Grayscale1 instead of default color map as I prefer grayscale to false color for assessing image quality. Check the rms of the residuals using the imstat task:
# In CASA
mystat = imstat('imgSN2010FZ10s_spw5_clean1280.residual')
print 'Residual standard deviation = '+str(mystat['sigma'][0])
I got 31.8uJy for mine.
Cleaning the lower baseband
Now, image the entire lower baseband (spw 0-7). Follow same iterative procedure as before, and get the best residuals you can. Because of the bandwidth and frequency synthesis, the sidelobe pattern is different than before and it is much easier to see fainter emission. Be careful cleaning sources that lie near or on sidelobe splotches. Clean the central emission region way down first to reduce the sidelobe level before adding components in the sidelobe areas.
# In CASA
clean(vis='SN2010FZ_split10s.ms',spw='0:16~59,1~6:4~59,7:4~54', \
imagename='imgSN2010FZ10s_spw0to7_clean1280', \
mode='mfs',nterms=1,niter=10000,gain=0.1,threshold='0.0mJy', \
psfmode='clark',imsize=[1280,1280],cell=['3.0arcsec'],stokes='I', \
imagermode='csclean', cyclefactor=1.5, \
weighting='briggs',robust=0.5,calready=True,interactive=True)
#
mystat = imstat('imgSN2010FZ10s_spw0to7_clean1280.residual')
print 'Residual standard deviation = '+str(mystat['sigma'][0])
I got 11.3uJy (and there is clearly structure left in the residual). To the right is a zoom-in on the center of the restored image.
Cleaning the lower baseband using two MFS Taylor terms
The mfs nterms=2 option creates two "Taylor Term" images - an average intensity image (with suffix .image.tt0) and a spectral slope image (with suffix .image.tt1) which is intensity x alpha (where alpha is spectral index). For convenience there is a spectral index image (with suffix .image.alpha). These Taylor expansions are with respect to the "Reference Frequency" of the image (by default the center frequency of the spw selected, but can be specified using the reffreq parameter in clean). The convention for spectral index alpha is that
[math]\displaystyle{ S \propto \nu^\alpha }[/math]
so negative spectral indexes indicate a "steep" spectrum (falling with frequency).
Lets try using multi-frequency synthesis with nterms=2 on the lower baseband. The dirty beam will have lower sidelobes so we turn up cyclefactor for csclean a bit:
# In CASA
clean(vis='SN2010FZ_split10s.ms',spw='0:16~59,1~6:4~59,7:4~54', \
imagename='imgSN2010FZ10s_spw0to7_mfs2_clean1280', \
mode='mfs',nterms=2,niter=10000,gain=0.1,threshold='0.0mJy', \
psfmode='clark',imsize=[1280,1280],cell=['3.0arcsec'],stokes='I', \
imagermode='csclean', cyclefactor=4.5, \
weighting='briggs',robust=0.5,calready=True,interactive=True)
#
mystat = imstat('imgSN2010FZ10s_spw0to7_mfs2_clean1280.residual.tt0')
print 'Residual standard deviation = '+str(mystat['sigma'][0])
I got 10.5uJy (somewhat better looking than the nterms=1). The top screenshot to the right shows an intermediate but early stage of cleaning where we are looking at the central emission and cleaning it out slowly.
You can use the viewer to load the average intensity
# In CASA
viewer('imgSN2010FZ10s_spw0to7_mfs2_clean1280.image.tt0')
and then use the Open Data panel to load the spectral index imgSN2010FZ10s_spw0to7_mfs2_clean1280.image.alpha which can then be blinked (optionally plotted side-by-side using the Panel Display Options panel to set 2 panels in the x direction).
Note there is alot of noise in alpha in the low-intensity regions, and thus filtering the alpha image based on the values in the tt0 image is desirable. You can use the immath task to make this filtered alpha image explicitly, using a Lattice Expression Language (LEL) expression:
# In CASA
immath(imagename=['imgSN2010FZ10s_spw0to7_mfs2_clean1280.image.alpha',
'imgSN2010FZ10s_spw0to7_mfs2_clean1280.image.tt0'],
mode='evalexpr',
expr='IM0[IM1>5.0E-5]',
outfile='imgSN2010FZ10s_spw0to7_mfs2_clean1280.image.alpha.filtered')
This will use 50uJy (or 5 x the sigma we found) as the cutoff. You can then view or manipulate the filtered alpha image as normal.
We can also use LEL to filter the alpha image on the intensity on-the-fly when we load this raster in the Open Data panel by specifying a LEL string in the LEL box instead of selecting the image from the directory listing. The LEL string:
'imgSN2010FZ10s_spw0to7_mfs2_clean1280.image.alpha'['imgSN2010FZ10s_spw0to7_mfs2_clean1280.image.tt0'>5.0E-05]
will replicate what we did above. The middle figure to the right shows the Open Data panel with our LEL string in it. Just click the Raster button to load this.
The lower panel to the right shows the intensity and LEL-filtered alpha images side-by-side in the viewer, zoomed in on the galaxy emission. Mousing over the alpha shows spectral indexes ranging from -1 to +1 in the center, with the brightest emission with alpha -0.7 in the knots in the disk.
Cleaning using both basebands combined
For the ultimate image, use the "clean" part of the upper baseband in addition to the lower (use spw 0-11). We will use mfs with nterms=2 (if you try nterms=1 on this wide bandwidth you will get much poorer residuals). Because of the added work and extra data involved, this will take much longer than our other runs of clean. Therefore, we will get a head start by doing a non-interactive clean using the mask left from the previous clean (spw 0-7). We will insert a clean threshold to limit runaway cleaning too far beneath the noise level. This took 40min on my workstation.
# In CASA
clean(vis='SN2010FZ_split10s.ms', \
spw='0:16~59,1~6:4~59,7:4~54,8:30~59,9~10:4~59,11:4~19;21~59', \
imagename='imgSN2010FZ10s_spw0to11_mfs2_clean1280', \
mode='mfs',nterms=2,niter=3000,gain=0.1,threshold='0.002mJy', \
psfmode='clark',imsize=[1280,1280],cell=['3.0arcsec'],stokes='I', \
imagermode='csclean', cyclefactor=4.5, \
mask=['imgSN2010FZ10s_spw0to7_mfs2_clean1280.mask'], \
weighting='briggs',robust=0.5,calready=True,interactive=False)
#
mystat = imstat('imgSN2010FZ10s_spw0to11_mfs2_clean1280.residual.tt0')
print 'Residual standard deviation = '+str(mystat['sigma'][0])
I got 8.6uJy for this effort (noticeably better than the lower baseband only results).
Let us see if there is more to clean. Bring this up in interactive mode:
# In CASA
clean(vis='SN2010FZ_split10s.ms', \
spw='0:16~59,1~6:4~59,7:4~54,8:30~59,9~10:4~59,11:4~19;21~59', \
imagename='imgSN2010FZ10s_spw0to11_mfs2_clean1280', \
mode='mfs',nterms=2,niter=3000,gain=0.1,threshold='0.001mJy', \
psfmode='clark',imsize=[1280,1280],cell=['3.0arcsec'],stokes='I', \
imagermode='csclean', cyclefactor=4.5, \
weighting='briggs',robust=0.5,calready=True,interactive=True)
I found a few more sources revealed in the outer parts of the image, and also more emission around the galaxy disk in the center. So I drew new boxes, extended the box in the center, and did about 1000 new iterations. At the end, what was left was dominated by the error patterns from mis-calibration. Only self-calibration will get rid of these. Stop cleaning for now. See the figure to the right for the interactive display panel showing final residuals and mask (changing the colormap to Greyscale 1).
Check the residual levels:
# In CASA
mystat = imstat('imgSN2010FZ10s_spw0to11_mfs2_clean1280.residual.tt0')
sigma = mystat['sigma'][0]
print 'Residual standard deviation = '+str(mystat['sigma'][0])
Now I get 8.3uJy.
Analyzing the image and comparing with the Optical/Infrared
Lets see how close we got to expected noise and dynamic range.
# In CASA
mystat = imstat('imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0')
peak = mystat['max'][0]
print 'Image max flux = '+str(mystat['max'][0])
#
mystat = imstat('imgSN2010FZ10s_spw0to11_mfs2_clean1280.model.tt0')
total = mystat['sum'][0]
print 'Model total flux = '+str(mystat['sum'][0])
#
snr = peak/sigma
print 'SN2010FZ peak S/N = '+str(snr)
#
snr = total/sigma
print 'SN2010FZ total S/N = '+str(snr)
The output gives:
Residual standard deviation = 8.33040657806e-06 Image max flux = 0.00995613634586 Model total flux = 0.0329372803525 SN2010FZ peak S/N = 1195.15611304 SN2010FZ total S/N = 3953.86228077
What do we expect? If we do listobs on this MS we see the scans:
Date Timerange (UTC) Scan FldId FieldName nVis Int(s)
11-Jul-2010/21:38:44.0 - 21:39:51.0 9 0 SN2010FZ 39312 9.11
21:40:01.0 - 21:41:20.5 10 0 SN2010FZ 44226 9.89
21:41:30.0 - 21:42:50.0 11 0 SN2010FZ 44226 10
21:43:00.0 - 21:44:20.0 12 0 SN2010FZ 44226 10
21:44:30.0 - 21:45:50.0 13 0 SN2010FZ 44226 10
21:46:00.0 - 21:47:19.5 14 0 SN2010FZ 44226 9.89
21:47:29.0 - 21:47:48.5 15 0 SN2010FZ 14742 9.67
21:49:42.0 - 21:50:49.0 17 0 SN2010FZ 39312 9.11
21:50:59.0 - 21:52:19.0 18 0 SN2010FZ 44226 10
21:52:29.0 - 21:53:48.5 19 0 SN2010FZ 44226 9.89
21:53:58.0 - 21:55:18.0 20 0 SN2010FZ 44226 10
21:55:28.0 - 21:56:48.0 21 0 SN2010FZ 44226 10
21:56:58.0 - 21:58:18.0 22 0 SN2010FZ 44226 10
21:58:28.0 - 21:58:47.0 23 0 SN2010FZ 14742 9.33
22:00:40.0 - 22:01:47.0 25 0 SN2010FZ 39312 9.12
22:01:57.0 - 22:03:17.0 26 0 SN2010FZ 44226 10
22:03:27.0 - 22:04:47.0 27 0 SN2010FZ 44226 10
22:04:57.0 - 22:06:16.5 28 0 SN2010FZ 44226 9.89
22:06:26.0 - 22:07:46.0 29 0 SN2010FZ 44226 10
22:07:56.0 - 22:09:16.0 30 0 SN2010FZ 44226 10
22:09:26.0 - 22:09:45.0 31 0 SN2010FZ 14742 9.33
22:11:38.5 - 22:12:45.5 33 0 SN2010FZ 39312 9.11
22:12:55.0 - 22:14:15.0 34 0 SN2010FZ 44226 10
22:14:25.0 - 22:15:45.0 35 0 SN2010FZ 44226 10
22:15:55.0 - 22:17:15.0 36 0 SN2010FZ 44226 10
22:17:25.0 - 22:18:44.5 37 0 SN2010FZ 44226 9.89
22:18:54.0 - 22:20:14.0 38 0 SN2010FZ 44226 10
22:20:24.0 - 22:20:43.5 39 0 SN2010FZ 14742 9.6
(nVis = Total number of time/baseline visibilities per scan)
(listing columns truncated) and we estimate about 37min on target. We had about 25 antennas on average, and our spw selection picked out 610 channels (2MHz each) for a total of 1220MHz bandwidth. If we plug this into the "EVLA exposure calculator" then we find that we expect a rms thermal noise level of 7.6uJy so we are close!
Look at this in the viewer:
# In CASA
viewer('imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0')
Zoom in on the center (see figure to the right).
In the previous section we demonstrated how to process and display the spectral index image. You can do the same for this final image. Here, we will do some rough analysis on the spectral index to determine an intensity-weighted mean spectral index over the core region. The .image.tt1 from our mfs is an intensity times alpha image. See the figure to the right. Lets gate the Taylor-term images on intensity as before:
# In CASA
immath(imagename=['imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt1',
'imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0'],
mode='evalexpr',
expr='IM0[IM1>5.0E-5]',
outfile='imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt1.filtered')
#
immath(imagename=['imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0'],
mode='evalexpr',
expr='IM0[IM0>5.0E-5]',
outfile='imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0.filtered')
We can identify a box containing the central emission (see figure of tt1 in viewer) and note the corners. (We could also use the region tools from the viewer, but that is for another exercise.) Let us compute the intensity-weighted spectral index over this box by averaging these masked images using imstat and computing the ratio:
# In CASA
mystat = imstat('imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt1.filtered',
box='503,533,756,762')
avgtt0alpha = mystat['mean'][0]
#
mystat = imstat('imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0.filtered',
box='503,533,756,762')
avgtt0 = mystat['mean'][0]
avgalpha = avgtt0alpha/avgtt0
print 'SN2010FZ I-weighted Alpha = '+str(avgalpha)
We get
SN2010FZ I-weighted Alpha = -1.54687681754
The emission in this source is on the steep side. At this point we do not know how reliable this is or what we expect (though our calibrators come out with correct spectral indexes if we image them the same way). But this illustrates a way to extract spectral information from our wideband mfs images.
As a final comparison, we turn to the Sloan Digital Sky Survey (SDSS) and a cutout image of our galaxy:
from their RC3 album (courtesy D.Hogg,M.Blanton,SDSS collaboration - see #Credits). This looks like a nice nearby face-on spiral galaxy. How does our 6cm continuum emission line up with the optical?
Here is the EVLA 6cm image side by side with a i-band image from the Sloan Digital Sky Survey (SDSS) registered to our image:
You can also find this image and load it into your viewer, and blink against our 6cm image.
We can also plot one as a raster and the other overlaid as contours. You can load the SDSS image from the viewer Load Data panel and fiddle with contours. Once you know contour levels, you can also use the imview task to load a raster and contour image:
# In CASA
imview(raster={ 'file' : 'imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0'},
contour = { 'file' : 'NGC_2967_UGC_5180_IRAS_09394+0033-i.fits',
'levels' : [0.2, 0.5, 1, 1.5, 3],
'base' : 0.0,
'unit' : 1.0 } )
The figure below shows the SDSS contours overlaid on our 6cm image (after fiddling with the colormap shift/slope for the EVLA raster image).
Likewise, we can plot the SDSS image as a raster and overlay EVLA 6cm contours:
# In CASA
imview(raster={ 'file' : 'NGC_2967_UGC_5180_IRAS_09394+0033-i.fits',
'scaling' : -2.0,
'range' : [0,10] },
contour = { 'file' : 'imgSN2010FZ10s_spw0to11_mfs2_clean1280.image.tt0',
'levels' : [0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56],
'base' : 0.0,
'unit' : 0.001 },
zoom = { 'blc' : [397,300],
'trc' : [1567,1231] } )
This is shown in the figure below. Is the compact 6cm emission in upper left associated with a spiral arm?
What to do next: some exercises for the user
Here are a number of things you can try after completing this tutorial:
- Use self-calibration to improve the data and re-clean to make a better image.
- Use multi-scale clean by adding non-zero scales to the multiscale parameter.
- Image the calibrators. What sort of dynamic range can you get on them? Is self-calibration needed (and if so what dynamic range do you get when you use it)?
- Try the testautoflag task (in 3.3.0 and later) to automatically flag RFI from the upper sideband.
Credits
The EVLA data was taken by A. Soderberg et al. as part of project AS1015. See NRAO eNews 3.8 (1-Sep-2010) for more on this result.
The Expanded Very Large Array (EVLA) is a partnership of the United States, Canada, and Mexico. The EVLA is funded in the United States by the National Science Foundation, in Canada by the National Research Council, and in Mexico by the Comisión Nacional de Investigación Científica y Tecnológica (CONICyT).
The National Radio Astronomy Observatory is a facility of the National Science Foundation operated under cooperative agreement by Associated Universities, Inc.
SDSS image courtesy David Hogg & Michael Blanton, private communication. Data comes from SDSS DR7, see Abazajian et. al 2009.
Funding for the SDSS and SDSS-II has been provided by the Alfred P. Sloan Foundation, the Participating Institutions, the National Science Foundation, the U.S. Department of Energy, the National Aeronautics and Space Administration, the Japanese Monbukagakusho, the Max Planck Society, and the Higher Education Funding Council for England. The SDSS Web Site is [2].
The SDSS is managed by the Astrophysical Research Consortium for the Participating Institutions. The Participating Institutions are the American Museum of Natural History, Astrophysical Institute Potsdam, University of Basel, University of Cambridge, Case Western Reserve University, University of Chicago, Drexel University, Fermilab, the Institute for Advanced Study, the Japan Participation Group, Johns Hopkins University, the Joint Institute for Nuclear Astrophysics, the Kavli Institute for Particle Astrophysics and Cosmology, the Korean Scientist Group, the Chinese Academy of Sciences (LAMOST), Los Alamos National Laboratory, the Max-Planck-Institute for Astronomy (MPIA), the Max-Planck-Institute for Astrophysics (MPA), New Mexico State University, Ohio State University, University of Pittsburgh, University of Portsmouth, Princeton University, the United States Naval Observatory, and the University of Washington.
