HI 21cm (1.4 GHz) spectral line data reduction: LEDA 44055
Overview
This tutorial describes the data reduction of the HI spectral line observed of the nearby (4.8 Mpc), gas-rich dwarf galaxy LEDA44055 (Figure 1: grab HST image). For its gas-rich nature, LEDA44055 is quiescent and is an Hα non-detection. Observations by the HST show a weak blue plume structure, and further inspection of GALEX archival images show some faint emission in the optical body, which taken together suggest that LEDA44055 may be in a "post-starburst" phase.
In this 1.4 GHz observation, a 16 MHz wide subband (spectral window, abbreviated as spw in CASA) using 4096 channels was observed, each channel providing ~ 1700 km/s of velocity coverage that results in a channel width of 3.906 kHz per channel. The second IF subband is used to acquire 1-2 GHz continuum imaging of the field; if LEDA44055 is in post-starbust phase, then it may display significant synchotron emission.
The TDEM0025 observation at the VLA was done during C-configuration and spanned 2 hours on the instrument from 31 July 2017 at 12:39 UT to 1 August 2017 at 14:39 UT. Information about the observation can be found on the corresponding VLA Observing Log and the continuation log (the observing log was split at the monthly boundary). This observing log is a record of events that transpired during the observation of the project, including weather conditions and any loss of antenna(s) and/or components that could affect the outcome of the observation.
This observation of LEDA44055 was taken as part of the Observing for University Classes program. The project code for this VLA observation is TDEM0025. This paper by Cannon et. al is the result of the observation.
How to use this CASA guide
Please use CASA 5.4.0 for this tutorial (typing casa -ls in a linux window shows the available versions and the current version; to explicitly change the current version type, e.g., casa -r 5.4.0-68)
There are at least three different ways to interact with CASA, described in more detail in Getting Started in CASA. In this guide we provide the pseudo-interactive method for every step and the interactive method for some of the steps. Since it is possible to use both methods, take care not to run the same task with identical parameters twice using both methods.
- Interactively examining task inputs. In this mode, one types taskname to load the task, inp to examine the inputs (see Figure 2), and go once those inputs have been set to your satisfaction. Allowed inputs are shown in blue and bad inputs are colored red. The input parameters themselves are changed one by one, e.g., selectdata=True. Summaries of the inputs to various tasks used in the data reduction below are provided, to illustrate which parameters need to be set. More detailed help can be obtained on any task by typing help taskname. Once a task is run, the set of inputs are stored and can be retrieved via tget taskname; subsequent runs will overwrite the previous tget file. To reset a task to its default settings type, default taskname.
# Interactive CASA <default|tget> taskname inp parameter1 = value1 parameter2 = value2 (etc) inp (Always double check the input parameters before running the task.) go
- Pseudo-interactively via task function calls. In this case, all of the desired inputs to a task are provided at once on the CASA command line. This tutorial is made up of such calls, which were developed by looking at the inputs for each task and deciding what needed to be changed from default values. For task function calls, only parameters that you want to be different from their defaults need to be set.
# Pseudo-interactive CASA
taskname('input parameters')
- Non-interactively via a script. A series of task function calls can be combined together into a script, and run from within CASA via execfile('scriptname.py'). This and other CASA Tutorial Guides have been designed to be extracted into a script via the script extractor by using the method described within the Extracting scripts from these tutorials page. Should you use the script generated by the script extractor for this CASA Guide, be aware that it will require some small amount of interaction related to the plotting, occasionally suggesting that you close the graphics window and hitting return in the terminal to proceed. It is in fact unnecessary to close the graphics windows (it is suggested that you do so purely to keep your desktop uncluttered).
Step 1: Obtain the Dataset
From the NRAO Data Archive enter TDEM0025 in the Project Code field and select the data set TDEM0025.sb34039638.eb34043648.57965.965492013886. (This should be the only dataset for the project). The dataset on the archive is around 85 GB in size.
You will download the dataset as an SDM file, either as a .tar file or as an uncompressed file. Under the Jansky VLA datasets options, check the "SDM-BDF dataset (all files)" button and, if you want the dataset downloaded as a .tar file, check the "Create tar file" box.
Once you have your dataset, copy it into a directory where you can launch CASA to begin the data reduction steps below. If you downloaded the dataset as a .tar file, you need to perform the following extra step to extract the dataset before beginning the data reduction steps.
#In a terminal outside of CASA: tar -xf TDEM0025.sb34039638.eb34043648.57965.965492013886.tar
Step 2: Import the Dataset into CASA
In earlier versions of CASA, you would import your data using the CASA command importevla. With CASA 5.4.0 and higher this task has been deprecated and, while it is still functional, there will be no further support for this task and you should instead use the CASA task importasdm to import your dataset into CASA. In order to make importasdm duplicate the task importevla, several parameters will need to be set from their default values.
# Interactive CASA default importasdm inp asdm='TDEM0025.sb34039638.eb34043648.57965.965492013886' vis='TDEM0025.ms' ocorr_mode='co' savecmds=True outfile='TDEM0025_onlineflags.txt' applyflags=True inp go
# Psuedo-interactive CASA
importasdm(asdm='TDEM0025.sb34039638.eb34043648.57965.965492013886',vis='TDEM0025.ms',ocorr_mode='co',savecmds=True,outfile='TDEM0025_onlineflags.txt',applyflags=True)
Where:
inp # lists the inputs available for this task
adsm='SDM-BDF File ID' # this is the filename of the SDM-BDF to use
vis='filename.ms' # this is the name of the output measurement set created (.ms)
ocorr_mode='co' # the VLA is a cross-correlator
savecmds=True # write the online flagging commands to an output file
outfile='filename.txt' # name of the file containing the online flags
applyflags=True # apply the online flags during creation of the MS
go # executes the task with the given inputs
Next we will use flagcmd to look at the table of online flags. This plot will show a graphical view of the online flags, which are antenna and/or time based flags.
From this plot (see Figure 1), we can see that ea22 has a subreflector error during the beginning of the observation. We will flag this in a later step.
# Interactive CASA default flagcmd vis='TDEM0025.ms' inpmode='table' useapplied=True action='plot' savepars=True plotfile='flagcmd-table.png' inp go
flagcmd(vis='TDEM0025.ms', inpmode='table', useapplied=True, action='plot', savepars=True, plotfile='flagcmd-table.png')
Step 3: Flag Antenna Shadowing, Zeros, and Very Bright Values
Next we will use flagdata to flag any antennas that may have been shadowed during the observation. This is a necessary step when observing in D- or C-configuration. Once we set mode='shadow' more parameters become available to edit specific to antenna shadowing, such as tolerance (the amount of shadow allowed (in meters)) and addantenna (file name or dictionary with additional antenna names, positions, and diameters). We will leave those parameters as the default settings of tolerance=0.0 (very conservative) and addantenna (no file or dictionary). Note, if an observation was taken in A- or B-configuration, this step is unnecessary.
# Interactive CASA default flagdata inp vis='TDEM0025.ms' mode='shadow' inp go
# Psuedo-interactive CASA
flagdata(vis='TDEM0025.ms', mode='shadow')
The correlator is known to generate a small number of zeros in the data. We will use flagdata to remove those zeros and to clip the very bright values. Setting mode='clip' in flagdata will reveal new parameters specific to this mode. We will leave most of the parameters as the default settings, however we will set two in particular: correlation='ABS_ALL' will take the absolute value of RR and LL and clip the very bright values and clipzeros=True will clip the zero-value data generated by the correlator.
# Interactive CASA default flagdata inp vis='TDEM0025.ms' mode='clip' correlation='ABS_ALL' clipzeros=True inp go
# Psuedo-interactive CASA
flagdata(vis='TDEM0025.ms', mode='clip', correlation='ABS_ALL', clipzeros=True)
More details regarding the use of flagdata can be found under the importasdm task.
Step 4: Initial Inspection of the Dataset
The next step is to inspect the contents of the MS using listobs. The task listobs provides almost all relevant observational parameters such as correlator setup (frequencies, bandwidths, channel number and widths, polarization products), sources, scans, scan intents, and antenna locations. Setting verbose=True will display all of the contents of the raw data and setting listfile='listobs.txt' will create a text file you can refer to later.
# Interactive CASA default listobs inp vis='TDEM0025.ms' verbose=True listfile='listobs.txt' inp go
# Psuedo-interactive CASA
listobs(vis='TDEM0025.ms', verbose=True, listfile='listobs.txt')
Below is a copy/paste of a portion of the listobs output:
Observer: Dr. John M. Cannon Project: uid://evla/pdb/34039543
Observation: EVLA
Data records: 7480512 Total elapsed time = 7176 seconds
Observed from 31-Jul-2017/23:10:27.0 to 01-Aug-2017/01:10:03.0 (UTC)
ObservationID = 0 ArrayID = 0
Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent
31-Jul-2017/23:10:27.0 - 23:11:24.0 1 0 1331+305=3C286 60021 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [SYSTEM_CONFIGURATION#UNSPECIFIED]
23:11:27.0 - 23:15:54.0 2 0 1331+305=3C286 281151 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED]
23:15:57.0 - 23:20:21.0 3 0 1331+305=3C286 277992 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_BANDPASS#UNSPECIFIED,CALIBRATE_FLUX#UNSPECIFIED]
23:20:24.0 - 23:22:21.0 4 1 J1330+2509 123201 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
23:22:24.0 - 23:24:21.0 5 1 J1330+2509 123201 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
23:24:24.0 - 23:30:45.0 6 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
23:30:48.0 - 23:37:09.0 7 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
23:37:12.0 - 23:43:30.0 8 2 LEDA44055 398034 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
23:43:33.0 - 23:45:30.0 9 1 J1330+2509 123201 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
23:45:33.0 - 23:51:54.0 10 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
23:51:57.0 - 23:58:15.0 11 2 LEDA44055 398034 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
23:58:18.0 - 00:04:39.0 12 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
01-Aug-2017/00:04:42.0 - 00:06:39.0 13 1 J1330+2509 123201 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
00:06:42.0 - 00:13:03.0 14 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
00:13:06.0 - 00:19:24.0 15 2 LEDA44055 398034 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
00:19:27.0 - 00:25:48.0 16 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
00:25:51.0 - 00:27:48.0 17 1 J1330+2509 123201 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
00:27:51.0 - 00:34:09.0 18 2 LEDA44055 398034 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
00:34:12.0 - 00:40:33.0 19 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
00:40:36.0 - 00:46:57.0 20 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
00:47:00.0 - 00:48:57.0 21 1 J1330+2509 123201 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
00:49:00.0 - 00:55:18.0 22 2 LEDA44055 398034 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
00:55:21.0 - 01:01:42.0 23 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
01:01:45.0 - 01:08:06.0 24 2 LEDA44055 401193 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [OBSERVE_TARGET#UNSPECIFIED]
01:08:09.0 - 01:10:03.0 25 1 J1330+2509 120042 [0,1,2,3,4,5,6,7,8] [3, 3, 3, 3, 3, 3, 3, 3, 3] [CALIBRATE_AMPLI#UNSPECIFIED,CALIBRATE_PHASE#UNSPECIFIED]
(nRows = Total number of rows per scan)
Fields: 3
ID Code Name RA Decl Epoch SrcId nRows
0 NONE 1331+305=3C286 13:31:08.287984 +30.30.32.95886 J2000 0 619164
1 NONE J1330+2509 13:30:37.689201 +25.09.10.97800 J2000 1 859248
2 NONE LEDA44055 12:55:41.000000 +19.12.33.00000 J2000 2 6002100
Spectral Windows: (9 unique spectral windows and 2 unique polarization setups)
SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) BBC Num Corrs
0 EVLA_L#A0C0#0 4096 TOPO 1410.330 3.906 16000.0 1418.3276 12 RR LL
1 EVLA_L#B0D0#1 128 TOPO 988.000 1000.000 128000.0 1051.5000 15 RR RL LR LL
2 EVLA_L#B0D0#2 128 TOPO 1116.000 1000.000 128000.0 1179.5000 15 RR RL LR LL
3 EVLA_L#B0D0#3 128 TOPO 1244.000 1000.000 128000.0 1307.5000 15 RR RL LR LL
4 EVLA_L#B0D0#4 128 TOPO 1372.000 1000.000 128000.0 1435.5000 15 RR RL LR LL
5 EVLA_L#B0D0#5 128 TOPO 1500.000 1000.000 128000.0 1563.5000 15 RR RL LR LL
6 EVLA_L#B0D0#6 128 TOPO 1628.000 1000.000 128000.0 1691.5000 15 RR RL LR LL
7 EVLA_L#B0D0#7 128 TOPO 1756.000 1000.000 128000.0 1819.5000 15 RR RL LR LL
8 EVLA_L#B0D0#8 128 TOPO 1884.000 1000.000 128000.0 1947.5000 15 RR RL LR LL
Sources: 27
ID Name SpwId RestFreq(MHz) SysVel(km/s)
0 1331+305=3C286 0 1420.405752 419
0 1331+305=3C286 1 1420.405752 419
0 1331+305=3C286 2 1420.405752 419
0 1331+305=3C286 3 1420.405752 419
0 1331+305=3C286 4 1420.405752 419
0 1331+305=3C286 5 1420.405752 419
0 1331+305=3C286 6 1420.405752 419
0 1331+305=3C286 7 1420.405752 419
0 1331+305=3C286 8 1420.405752 419
1 J1330+2509 0 1420.405752 419
1 J1330+2509 1 1420.405752 419
1 J1330+2509 2 1420.405752 419
1 J1330+2509 3 1420.405752 419
1 J1330+2509 4 1420.405752 419
1 J1330+2509 5 1420.405752 419
1 J1330+2509 6 1420.405752 419
1 J1330+2509 7 1420.405752 419
1 J1330+2509 8 1420.405752 419
2 LEDA44055 0 1420.405752 419
2 LEDA44055 1 1420.405752 419
2 LEDA44055 2 1420.405752 419
2 LEDA44055 3 1420.405752 419
2 LEDA44055 4 1420.405752 419
2 LEDA44055 5 1420.405752 419
2 LEDA44055 6 1420.405752 419
2 LEDA44055 7 1420.405752 419
2 LEDA44055 8 1420.405752 419
Antennas: 27:
ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m)
East North Elevation x y z
0 ea01 W12 25.0 m -107.37.37.4 +33.53.44.2 -835.3760 -544.2316 0.5650 -1602044.902600 -5042025.803400 3554427.822700
1 ea02 W04 25.0 m -107.37.10.8 +33.53.59.1 -152.8711 -83.7955 -2.4675 -1601315.900500 -5041985.306670 3554808.309400
2 ea03 W10 25.0 m -107.37.28.9 +33.53.48.9 -619.2934 -398.4403 -0.5229 -1601814.060900 -5042012.886450 3554548.229820
From the listobs output, note the field ID for each of the sources and scan intent(s) (or source type). This information will be used for future calibration tasks and when splitting the data.
Field ID Source Scan Intent 0 3C286 flux density scale and bandpass calibrator 1 J1330+2509 complex gain (amplitude and phase) calibrator 2 LEDA44055 observe target
Since the goal of this tutorial is to find the HI (1.420405752 GHz rest frequency) spectral line in LEDA 44055, note the spw ID containing the spectral line setup. From this particular instrument configuration, the spectral line setup is spw ID 0, while spw ID's 1-8 are continuum. Make note of the spectral line setup for spw 0. This information will be used later.
spw 0 4096 channels TOPO frame indicates Barycentric Optical Doppler setting 3.906 kHz channel width indicates the resolution 16 MHz bandwidth indicates the full width of the spw 1418.3276 MHz center frequency of the spw
Step 5: Split Out the HI Line
Before we begin calibration or flagging of misbehaving antennas or RFI, we will run split to create a new smaller MS containing only the HI line (spw 0).
# Interactive CASA default split inp vis='TDEM0025.ms' outputvis='TDEM0025_HI.ms' spw=0 datacolumn='data' inp go
# Psuedo-interactive CASA
split(vis='TDEM0025.ms', outputvis='TDEM0025_HI.ms', spw=0, datacolumn='data')
Now if we run listobs on this new MS, we can see the output shows everything but the continuum spw's 1-8.
# Psuedo-interactive CASA
listobs(vis='TDEM0025_HI.ms', verbose=True, listfile='listobs_HI.txt')
Step 6: Initial Flux Density Scaling
Using setjy, we will list available models and set the model for the flux calibrator. By setting listmodels=True, setjy will show all available models for specific calibrators with specific bands in the Perley-Butler 2017 standard. setjy will look within the directory in which CASA was run for *.im *.mod files for user-defined models or images. If none are found, setjy will confirm this, for this particular dataset we will use a model in CalModels instead.
# Interactive CASA default setjy inp vis='TDEM0025_HI.ms' listmodels=True inp go
# Psuedo-interactive CASA
setjy(vis='TDEM0025_HI.ms',listmodels=True)
#terminal output No candidate modimages matching '*.im* *.mod*' found in . # The single period here refers to the current working directory. Candidate modimages (*) in /home/casa/data/distro/nrao/VLA/CalModels: 3C138_A.im 3C138_L.im 3C138_U.im 3C147_C.im 3C147_Q.im 3C147_X.im 3C286_K.im 3C286_S.im 3C48_A.im 3C48_L.im 3C48_U.im 3C138_C.im 3C138_Q.im 3C138_X.im 3C147_K.im 3C147_S.im 3C286_A.im 3C286_L.im 3C286_U.im 3C48_C.im 3C48_Q.im 3C48_X.im 3C138_K.im 3C138_S.im 3C147_A.im 3C147_L.im 3C147_U.im 3C286_C.im 3C286_Q.im 3C286_X.im 3C48_K.im 3C48_S.im README
Using setjy, we will set the model for the flux calibrator 3C286 (field='0') which was observed in L band.
# Interactive CASA default setjy inp vis='TDEM0025_HI.ms' listmodels=False field='0' spw='0' scalebychan=True fluxdensity=-1 standard='Perley-Butler 2017' model='3C286_L.im' usescratch=True inp go
# Psuedo-interactive CASA
setjy(vis='TDEM0025_HI.ms',listmodels=False,field='0',spw='0',scalebychan=True,fluxdensity=-1,standard='Perley-Butler 2017',model='3C286_L.im',usescratch=True)
Where:
vis='TDEM0025_HI.ms' # the split HI ms.
listmodels='False' # do not list the available models for VLA calibrators.
field='0' # set the model for 3C286, the flux density calibrator for this observation.
spw='0' # calculate the model for one spw.
scalebychan=True # set as default, the model is calculated per channel.
fluxdensity=-1 # uses the flux density standard for recognized sources, and [1,0,0,0] for unrecognized.
standard='Perley-Butler 2017' # Flux density standard used as the default.
model='3C286_L.im' # Use the L band model for 3C286.
usescratch=True # create a MODEL_DATA column containing model data
Within the casalogger, setjy outputs the flux in Janskys for Stokes I for a specific frequency.
2019-03-26 17:32:38 INFO imager Using channel dependent flux densities 2019-03-26 17:32:38 INFO imager Selected 68796 out of 831168 rows. 2019-03-26 17:32:38 INFO imager 1331+305=3C286 (fld ind 0) spw 0 [I=15.028, Q=0, U=0, V=0] Jy @ 1.4103e+09Hz, (Perley-Butler 2017) 2019-03-26 17:32:58 INFO imager Using model image /home/casa/data/distro/nrao/VLA/CalModels/3C286_L.im 2019-03-26 17:32:58 INFO imager Scaling spw(s) [0]'s model image by channel to I = 15.0281, 14.9855, 14.9431 Jy @(1.41037e+09, 1.41838e+09, 1.42638e+09)Hz (LSRK) for visibility prediction (a few representative values are shown). 2019-03-26 17:32:58 INFO imager The model image's reference pixel is 0.000254726 arcsec from 1331+305=3C286's phase center. 2019-03-26 17:32:58 INFO imager Will clear any existing model with matching field=1331+305=3C286 and spw=* 2019-03-26 17:32:58 INFO Clearing model records in MS header for selected fields. 2019-03-26 17:32:58 INFO 1331+305=3C286 (id = 0) not found. 2019-03-26 17:32:58 INFO imager Selected 68796 out of 831168 rows. 2019-03-26 17:32:58 INFO setjy ##### End Task: setjy ##### 2019-03-26 17:32:58 INFO setjy ##########################################
Step 7: Prior Calibration
In this step, we will look at the antenna layout, weather conditions during the observation, generate antenna gain curve corrections, and antenna position corrections. We will summarize opacity corrections, ionospheric TEC corrections, and requantizer gain corrections and why they may not benefit this dataset. Then we'll do an initial assessment of the amplitude versus time to see any RFI, or bad antennas before the calibration stage.
Antenna configuration
Using plotants, we'll look at the overall antenna layout and chose the best reference antenna based on certain criteria. We will want to choose a reference antenna that is closer to the center of the array, so any atmospheric changes are similar for all antennas. For D and C configuration, low declination sources are subject to antenna shadowing and antennas will collect less radiation, causing lower sensitivity which may not be the best choice for the reference antenna. For this guide, we will use ea10 as the reference antenna.
# Interactive CASA default plotants inp vis='TDEM0025_HI.ms' figfile='antlayout.png' logpos=True inp go
# Psuedo-interactive CASA
plotants(vis='TDEM0025_HI.ms',figfile='antlayout.png',logpos=True)
Usually, the array will take a few seconds to stabilize at the start of a scan. Using flagdata with mode quack, we will flag the first 5 seconds at the beginning of each scan.
# Interactive CASA default flagdata inp vis='TDEM0025_HI.ms' mode='quack' quackinterval=5.0 quackmode='beg' scan='' inp go
# Psuedo-interactive CASA
flagdata(vis='TDEM0025_HI.ms',mode='quack',quackinterval=5.0,quackmode='beg',scan='')
Where:
vis='TDEM0025_HI.ms' # the split HI ms.
mode='quack' # specific flagging mode, flag data at the beginning or end of a scan.
quackinterval=5.0 # time in seconds.
quackmode='beg' # flag the first n seconds for each scan designated in the parameter scan.
scan='' # flag data for all scans in vis.
Using plotms, we'll create an elevation versus time plot for all sources. Criteria for the complex gain (amplitude and phase) calibrator should be closer to the source to obtain a similar environment regarding changes in amplitude and phase which will be applied to the source.
# Interactive CASA default plotms inp vis='TDEM0025_HI.ms' xaxis='time' yaxis='elevation' correlation='RR,LL' avgchannel='64' coloraxis='field' plotfile='elevstime.png' inp go
# Psuedo-interactive CASA
plotms(vis='TDEM0025_HI.ms',xaxis='time',yaxis='elevation',correlation='RR,LL',avgchannel='64',coloraxis='field',plotfile='elevstime.png')
Where:
vis='TDEM0025_HI.ms' # the split HI ms.
xaxis='time' # plot time along the x axis.
yaxis='elevation' # plot elevation along the y axis.
correlation='RR,LL' # plot the two parallel hand polarizations, Right and Left polarizations.
avgchannel='64' # average data every 64 channels.
coloraxis='field' # colorize data points by field.
plotfile='elevstime.png' # automatically save this plot to a file titled elevstime.png.
Antenna gain-elevation curve calibration
With elevation, the effective collecting area and surface accuracy of antennas varies as gravity tugs at the surface of the non-rigid antenna. Using gencal, we will write the gain curve solutions to a calibration table.
# Interactive CASA default gencal inp vis='TDEM0025_HI.ms' caltable='antgain.cal' caltype='gc' inp go
# Psuedo-interactive CASA
gencal(vis='TDEM0025_HI.ms',caltable='antgain.cal',caltype='gc')
Where:
vis='TDEM0025_HI.ms' # the split HI ms.
caltable='antgain.cal' # name of output calibration table where values are stored.
caltype='gc' # gain curve calculated per spw.
Opacities and weather conditions
Using plotweather, we'll look at weather conditions throughout the observation that may affect the data. plotweather provides an estimate of the opacity on a per spw basis with units in nepers. The seasonal weight parameter calculates the opacities based on the weather data, seasonal model, or a combination of two. When the seasonal_weight is set to 1 the opacity corrections are derived using only the seasonal data and when set to 0 the opacity corrections are derived from only the weather data.
Atmospheric opacity attenuates incoming radiation and observations at different elevations affect the flux density scale of sources. For lower frequency observation (Ku and lower), this attenuation is small and opacity corrections are usually skipped. Luckily for L band, weather conditions do not significantly affect (unless it’s snow, rain, or strong winds) the data, therefore we can save the weather plots for reference and skip opacity corrections.
# Interactive CASA default plotweather inp vis='TDEM0025_HI.ms' seasonal_weight=0.5 plotName='TDEM0025weathercond.png' doPlot=True inp go
# Psuedo-interactive CASA
plotweather(vis='TDEM0025_HI.ms',seasonal_weight=0.5,plotName='TDEM0025weathercond.png',doPlot=True)
Where:
vis='TDEM0025_HI.ms' # the split HI ms.
seasonal_weight=0.5 # calculate opacities using both the seasonal model and weather data.
plotName='TDEM0025weathercond.png' # name of the generated plot
doPlot=True # create plot. If set to false, do not create plot and output only the calculated opacities.
Requantizer gains calibration
Requantizer gain corrections account for gain changes that occur when resetting the quantizer gains as the correlator changes to a new 3-bit configuration. Requantizer gains need to be redetermined after a change of tuning. To create a calibration table containing corrections for the requantizer gains one would use gencal with caltype=’rq’. However, since this observation uses 8-bit setup this step can be skipped. For information on 8 bit, 3 bit setup observations, and requantizer setup scans here is a link to 8/3-Bit Attenuator and Requantizer Gain Setup Scans
Ionopsheric TEC(total electron content) calibration
The Total Electron Content (TEC) is the total number of electrons present along a path between a radio source and receiver. Ionospheric corrections are important for frequencies below 1GHz or for polarimetry. For this guide, we will not derive or apply TEC corrections.
Applying antenna position corrections
We will apply antenna position corrections using gencal which queries the VLA Archive: Baseline Corrections.
# Interactive CASA default gencal inp vis='TDEM0025_HI.ms' caltype='antpos' caltable='antpos.cal' antenna='' inp go
# Psuedo-interactive CASA
gencal(vis='TDEM0025_HI.ms',caltype='antpos',caltable='antpos.cal',antenna='')
Where:
vis='TDEM0025_HI.ms' # the split HI ms
caltype='antpos' # the type of calibration to solve for, where antpos gathers the ITRF antenna position offsets
caltable='antpos.cal' # create a calibration table titled antpos.cal where the antenna position offsets can be stored
antenna='' # search for all antennas
gencal will report into the casalogger, if there are any antenna position corrections found. In this case, there are position corrections for antenna ea25 that are saved into the calibration table antpos.cal and will be applied in later stages using applycal and gaincal.
- Please note gencal will also report to the logger if there are no antenna position corrections for the observation. This simply means all antenna position corrections were applied by the VLA Operator before the science observation was made.
2019-03-25 21:52:22 INFO gencal::::+ ########################################## 2019-03-25 21:52:22 INFO gencal::::+ ##### Begin Task: gencal ##### 2019-03-25 21:52:22 INFO gencal:::: gencal(vis="TDEM0025_HI.ms",caltable="antpos.cal",caltype="antpos",infile="",spw="", 2019-03-25 21:52:22 INFO gencal::::+ antenna="",pol="",parameter=[],uniform=True) 2019-03-25 21:52:22 INFO calibrater::open ****Using NEW VI2-driven calibrater tool**** 2019-03-25 21:52:22 INFO calibrater::open Opening MS: TDEM0025_HI.ms for calibration. 2019-03-25 21:52:22 INFO Calibrater:: Initializing nominal selection to the whole MS. 2019-03-25 21:52:22 INFO gencal:::: Determine antenna position offsets from the baseline correction database 2019-03-25 21:52:22 INFO gencal:::: offsets for antenna ea25 : -0.00200 0.00000 -0.00120 2019-03-25 21:52:22 INFO calibrater::specifycal Beginning specifycal----------------------- 2019-03-25 21:52:22 INFO Creating KAntPos Jones table from specified parameters. 2019-03-25 21:52:22 INFO Writing solutions to table: antpos.cal 2019-03-25 21:52:22 INFO gencal:::: ##### End Task: gencal #####
Initial inspection of amplitude versus time.
Let's look at amplitude variations with respect to time for each field. We will want to flag significant variations in amplitude and keep track of recurring antennas/timeframes/frequencies/sudden spikes(RFI) in amplitude so these may be flagged as a collection of an issue.
By setting colors per baseline, we notice that one antenna is pretty noisy. Within the plotms GUI, click on the Mark Regions icon, select a few points of the widely varying amplitude, use the Locate icon (the magnifying glass) to output information on the data, and assess common details of the data. We notice that ea24 causes significant noise seen in Figure 4, where the purple data is ea24.
- Please keep in mind when using the flag icon in plotms, which is close to the locate icon, an automatic flagbackup is not set when interactively flagging.
Within the Data tab exclude ea24 by typing !ea24 in the antenna selection. Re-plotting, we can see the overall amplitude is less variable as seen in Figure 5.
# Interactive CASA default plotms inp vis='TDEM0025_HI.ms' xaxis='time' yaxis='amp' correlation='RR,LL' selectdata=True avgchannel='64' coloraxis='baseline' inp go
# Psuedo-interactive CASA
plotms(vis='TDEM0025_HI.ms',xaxis='time',yaxis='amp',correlation='RR,LL',selectdata=True,avgchannel='64',coloraxis='baseline')
Where:
vis='TDEM0025_HI.ms' # the split HI ms
xaxis='time' # plot time along the x axis
yaxis='amp' # plot amplitude along the y axis
correlation='RR,LL' # plot the two parallel hand polarizations, Right and Left polarizations
selectdata=True # expand the parameters to choose from.
avgchannel='64' # average data every 64 channels
coloraxis='baseline' # colorize data points by baseline
We can also iterate per baseline to the reference antenna ea10 and the suspected antenna ea24. by using the green arrow icon within plotms Here we notice that all baselines associated to ea24 vary in amplitude significantly. Figure 6 shows one baseline with ea16&ea24, and iterating through baselines with ea24 all show varying amplitudes.
# Interactive CASA default plotms inp vis='TDEM0025_HI.ms' xaxis='time' yaxis='amp' antenna='ea10,ea24' correlation='RR,LL' selectdata=True avgchannel='64' coloraxis='antenna1' iteraxis='baseline' inp go
# Psuedo-interactive CASA
plotms(vis='TDEM0025_HI.ms',xaxis='time',yaxis='amp',antenna='ea10,ea24',correlation='RR,LL',selectdata=True,avgchannel='64',coloraxis='antenna1',iteraxis='baseline')
Where:
vis='TDEM0025_HI.ms' # the split HI ms.
xaxis='time' # plot time along the x axis.
yaxis='amp' # plot amplitude along the y axis.
antenna='ea10,ea24' # includ antennas ea10 and ea24 only.
correlation='RR,LL' # plot the two parallel hand polarizations, Right and Left polarizations.
selectdata=True # expand the parameters to choose from.
avgchannel='64' # average data every 64 channels.
coloraxis='antenna1' # colorize by the first antenna.
iteraxis='baseline' # create a plot for each baseline to ea10 and ea24.
Now that we've narrowed down the noisy antenna, we will flag ea24 using flagdata which will also output to the casalogger the flagbackup version name.
# Interactive CASA default flagdata inp vis='TDEM0025_HI.ms' mode='manual' antenna='ea24' action='apply' flagbackup=True inp go
# Psuedo-interactive CASA
flagdata(vis='TDEM0025_HI.ms',mode='manual',antenna='ea24',action='apply',flagbackup=True)
Where:
vis='TDEM0025_HI.ms' # the split HI ms
mode='manual' # flag data based on parameters specified.
antenna='ea24' # flag antennas and 24.
action='apply' # apply the flags to the split HI ms.
flagbackup=True # automatically save flags into a predesignated flagversion name.
Now that we have completed initial assessment of the data and flagged bad antennas we will proceed to bandpass calibration.
Step 8: Bandpass Calibration, Complex Gain Calibration, and Fluxscale
Bandpass Calibration
Now, we can run our bandpass calibration. This calibration gives antenna-based solutions against a reference antenna. This will determine the variations of phase and amplitude across frequency.
# Interactive CASA default bandpass inp vis='TDEM0025_HI.ms' caltable='bandpass.cal' field='0' refant='ea10' solint='inf' gaintable=['antgain.cal','antpos.cal'] inp go
# Psuedo-interactive CASA
bandpass(vis='TDEM0025_HI.ms',caltable='bandpass.cal',field='0',refant='ea10',solint='inf',gaintable=['antgain.cal','antpos.cal'])
Where:
vis='TDEM0025_HI.ms' # the split HI ms
caltable='bandpass.cal' # the name you wish to give the calibration table
field='0' # the field id of the bandpass calibrator (or you could give it the field name)
refant='ea10' # the reference antenna you have chosen
solint='inf' # the solution interval time
gaintable=['antgain.cal','antpos.cal'] # gain calibration table(s) to apply on the fly
Now we will plot the bandpass solutions as phase vs channel and amplitude vs channel per antenna.
# Interactive CASA default plotms inp vis='bandpass.cal' xaxis='chan' yaxis='amp' iteraxis='antenna' gridrows=3 gridcols=3 xselfscale=True yselfscale=True inp go
# Psuedo-interactive CASA
plotms(vis='bandpass.cal',xaxis='chan',yaxis='amp',iteraxis='antenna',gridrows=3,gridcols=3,xselfscale=True,yselfscale=True)
Where:
vis='bandpass.cal' # the bandpass calibration table
xaxis='chan' # plot channels across the x axis
yaxis='amp' # plot amplitudes across the y axis
iteraxis='antenna' # pages the plots by antenna
gridrows=3 # places 3 antennas per row
gridcols=3 # places 3 antennas per column
xselfscale=True # create a global xaxis scale for all antennas
yselfscale=True # create a global yaxis scale for all antennas
You may see the phases vs time by changing the yaxis to phase interactively.
Complex Gain Calibration
Next we will calibrate the phases of the antennas using the bandpass and phase calibrators.
# Interactive CASA default gaincal inp vis='TDEM0025_HI.ms' caltable='phasecal.gcal' field='0,1' refant='ea10' calmode='ap' solint='inf' minsnr=3.0 gaintable=['antgain.cal','antpos.cal','bandpass.cal'] inp go
# Psuedo-interactive CASA
gaincal(vis='TDEM0025_HI.ms',caltable='phasecal.gcal',field='0,1',refant='ea10',calmode='ap',solint='inf',minsnr=3.0,gaintable=['antgain.cal','antpos.cal','bandpass.cal'])
Where:
vis='TDEM0025_HI.ms' # the split HI ms
caltable='phasecal.gcal' # the name you wish to give the calibration table
field='0,1' # the field id of the bandpass calibrator (or you could give it the field name)
refant='ea10' # the reference antenna you have chosen
calmode='ap' # the calibration mode that finds both amplitude and phase solutions
solint='inf' # the solution interval time
minsnr=3.0 # the minimum signal to noise ratio to reject solutions
gaintable=['antgain.cal','antpos.cal','bandpass.cal'] # gain calibration table(s) to apply on the fly
Now look at solutions. First we will look at the phase solutions.
# Interactive CASA default plotms inp vis='phasecal.gcal' xaxis='time' yaxis='phase' iteraxis='antenna' gridrows=3 gridcols=3 plotrange=[0,0,-180,180] inp go
# Psuedo-interactive CASA
plotms(vis='phasecal.gcal',xaxis='time',yaxis='phase',iteraxis='antenna',gridrows=3,gridcols=3,plotrange=[0,0,-180,180])
Where:
vis='phasecal.gcal' # the complex gain calibration table
xaxis='time' # plot channels across the x axis
yaxis='phase' # plot amplitudes across the y axis
iteraxis='antenna' # pages the plots by antenna
gridrows=3 # places 3 antennas per row
gridcols=3 # places 3 antennas per column
plotrange=[0,0,-180,180] # sets the plot range
Pressing next through these phase vs time plot we can see that antenna ea16 and ea20 have a scan that shows a phase jump relative to the rest of the scans. If we locate this point we see that it is scan 21 for both antennas. This scan we should flag.
Now look at the amplitudes to see if there are any other irregularities.
# Interactive CASA default plotms inp vis='phasecal.gcal' xaxis='time' yaxis='amp' iteraxis='antenna' gridrows=3 gridcols=3 xselfscale=True yselfscale=True inp go
# Psuedo-interactive CASA
plotms(vis='phasecal.gcal',xaxis='time',yaxis='amp',iteraxis='antenna',gridrows=3,gridcols=3,xselfscale=True,yselfscale=True)
Where:
vis='phasecal.gcal' # the complex gain calibration table
xaxis='time' # plot channels across the x axis
yaxis='amp' # plot amplitudes across the y axis
iteraxis='antenna' # pages the plots by antenna
gridrows=3 # places 3 antennas per row
gridcols=3 # places 3 antennas per column
xselfscale=True # create a global xaxis scale for all antennas
yselfscale=True # create a global yaxis scale for all antennas
Seeing no other things to be concerned about, we will flag these antennas and then rederive the amplitude and phase gaincal solutions.
# Interactive CASA default flagdata inp vis='TDEM0025_HI.ms' mode='manual' antenna='ea16,ea20' scan='21' action='apply' flagbackup=True inp go
# Psuedo-interactive CASA
flagdata(vis='TDEM0025_HI.ms',mode='manual',antenna='ea16,ea20',scan='21',action='apply',flagbackup=True)
Where:
vis='TDEM0025_HI.ms' # the split HI ms
mode='manual' # flag data based on parameters specified.
antenna='ea16,ea20' # flag only antennas ea16 and ea20
scan='21' # flag only scan 21 for both antennas
action='apply' # apply the flags to the split HI ms.
flagbackup=True # automatically save flags into a predesignated flagversion name.
Now you should rerun the gaincal step and the amplitude plotms runs from above. This will rederive the solutions and then you can see that scan 21 for both ea16 and ea20 have been flagged.
We are able to flag this calibration scan and not have to flag any of the surrounding target scans because of the cycle time recommended for L band in the C configuration. The cycle time is ~40 minutes and the time between the 2 phase calibrator scans surrounding scan 21 is ~43 minutes. It is lucky that the observation was set up to have a shorter cycle time than recommended or else we would have to flag all of the scans between calibrator scans 17 and 25 which would have gotten rid of 2/5 of our data set.
Fluxscale
Now that calibration of the observation seems to be done we will use fluxscale to calculate the amplitude of the phase calibrator since we know the amplitude of the flux density scale/bandpass calibrator. We do this by referencing the flux density scale/bandpass calibrator and applying the calibration table solutions we just derived.
# Interactive CASA default fluxscale inp vis='TDEM0025_HI.ms' caltable='phasecal.gcal' reference='0' fluxtable='flux_scale' inp go
# Psuedo-interactive CASA
fluxscale(vis='TDEM0025_HI.ms',caltable='phasecal.gcal',reference='0',fluxtable='flux_scale')
Where:
vis='TDEM0025_HI.ms' # the split HI ms
caltable='phasecal.gcal' # the calibration table used to find amplitude and phase corrections
reference='0' # the field ID of the reference field (where you transfer your flux from)
fluxtable='flux_scale' # the name of the output, flux-scaled calibration table
This is the output to the terminal you will see when you run fluxscale.
{'1': {'0': {'fluxd': array([ 6.91913842, 0. , 0. , 0. ]),
'fluxdErr': array([ 0.00403804, 0. , 0. , 0. ]),
'numSol': array([ 52., 0., 0., 0.])},
'fieldName': 'J1330+2509',
'fitFluxd': 0.0,
'fitFluxdErr': 0.0,
'fitRefFreq': 0.0,
'spidx': array([ 0., 0., 0.]),
'spidxerr': array([ 0., 0., 0.])},
'freq': array([ 1.41832755e+09]),
'spwID': array([0], dtype=int32),
'spwName': array(['EVLA_L#A0C0#0'],
dtype='|S14')}
Reading through this output you can see that for spw 0 the flux of the phase calibrtor is ~6.919Jy. This is very close to the expected value of 6.80Jy in L-band. This expected value can be found by looking in the VLA Calibrator Manual, which you can google, and searching for the source.
