Guide To Redo Calibration For ALMA Cycle 0

How To Re-do ALMA Calibration for Cycle 0 Data Packages

This page describes how to redo ALMA calibration for Cycle 0 data. If you do not need to redo calibration, proceed to the section on Data Concatenation in the Cycle 0 Data Processing Guide.

The data package includes a calibration script in the scripts directory for each measurement set. This script performs all the initial calibrations required prior to combination and imaging of all measurement sets corresponding to a scheduling block. This script is the exact version used to generate the corresponding calibrated measurement set found in the calibrated directory. You can use the packaged calibration script to re-calibrate step-by-step the corresponding raw measurement set (found in the raw directory) and tweak the script as desired for your own purposes. However, for Cycle 0 measurement sets, there are a couple of considerations which need to be taken into account before using the packaged calibration script.

1. While the scripts and the raw data are found in different directories, the calibration script must be run in the same directory as the measurement set on which it is applied. We recommend copying both the measurement set and the calibration script in a new directory where the calibration will be performed, so as to leave original versions intact.
2. The packaged calibration script was generated so as to be used in CASA 3.4, but users are now recommended to use CASA 4.2 for all data processing. There are several differences between CASA 3.4 scripts and CASA 4.2 scripts. To change the packaged calibration script into a CASA 4.2 script, it must be manually modified using a text editor (see the detailed instructions here).
3. The "raw" data supplied with the data package in the raw directory have had several a priori calibrations applied. These include the calibration steps labeled 0-6 in the packaged calibration scripts. In CASA 3.4, the user should hence begin re-calibration of the "raw" data with the step labeled "7" of the calibration script (or "9", if using a CASA 4.2 script). To run one (or several) specific steps in a script, the user needs to define the variable mysteps with a numerical list of desired steps to execute. Below, we show examples corresponding to running the script from the first applicable step all the way through the end of the script:

# In CASA 3.4
 mysteps=[7,8,9,10,11,12,13,14,15] 
 execfile('scriptForCalibration_3.4.py')

# In CASA  4.2
 mysteps=[9,10,11,12,13,14,15,16,17,18]
 execfile('scriptForCalibration_4.2.py')

General considerations on running calibration scripts

1. In addition to CASA tasks, calibration scripts call specific plotting and analysis utilities which are gathered in the analysisUtils package. This package is very easy to download and install. See this page for details.
2. Within the calibration script, you can identify the different steps by the identifier mystep = N. It is possible to run a single step or a sequence of subsequent steps, by defining the mysteps list accordingly. For example:

 mysteps = [9]
 execfile('uid___A002_X554543_X207.ms.scriptForCalibration_4.2.py') # will run only step 9
 mysteps = [10,11,12]
 execfile('uid___A002_X554543_X207.ms.scriptForCalibration_4.2.py') # will run step 10,11,12

1. Steps need to be executed in the order given by the script. For example, step 16 can be executed only after steps 9,10,11,12,13,14 and 15 have been executed.
2. It is also possible to go back and repeat a step that has already been performed, but one must be careful about flags. Flags are applied to the data when the command flagdata is called, but also when calibration tables are applied. This means that if you are going back to perform step 9 after having run steps 9, 10, and 11, the data on which you are applying step 9 will have applied flags that were created in steps 9, 10 and 11. To avoid this, you need to make sure that if you are running step N, you reload the flags as they were at the end of step N-1. This is possible by using the flagmanager task, which allows one to save the state of flags at a given point in the script, and to reload that state of flags later. Several calls to flagmanager are placed throughout the calibration script, and you can add your own. For example, in step 14 of the reduction script (CASA 4.2), you will find a call to flagmanager with mode=save:

 flagmanager(vis = 'uid___A002_X554543_X207.ms.split',
   mode = 'save',
   versionname = 'BeforeGainCalibration')

This command takes a snapshot of the flags just before gain calibration . After running steps 15 and 16 (gain calibration), some additional flags have probably been applied to the data. If you want to re-run step 15 and 16, you will need to reload the state of flags at the end of step 14 using flagmanager with mode=restore:

mysteps=[15,16]  
execfile('uid___A002_X554543_X207.ms.scriptForCalibration_4.2.py')  # first executions of steps 15 and 16
# You decide to modify the script in steps 15 and 16 and run these steps again
flagmanager(vis = 'uid___A002_X554543_X207.ms.split',  # reload flags as recorded at the end of step 14
    mode = 'restore',
    versionname = 'BeforeGainCalibration')
mysteps=[15,16]  
execfile('uid___A002_X554543_X207.ms.scriptForCalibration_4.2.py')  # execute (modified) steps 15 and 16

Example: re-calibration of the TW Hydra data

In our TW HYdra example, there are 3 measurement sets in the raw directory that are ready for calibration, and 3 corresponding calibration scripts in the scripts directory, all written for CASA 3.4. You can find those scripts already updated to CASA 4.2 here:

• File:Uid A002 X554543 X207.ms.scriptForCalibration 4.2.py
• File:Uid A002 X554543 X3d0.ms.scriptForCalibration 4.2.py
• File:Uid A002 X554543 X667.ms.scriptForCalibration 4.2.py

These scripts follow the steps described here. These can be used to calibrate and combine these data sets under CASA 4.2. First you will need to place the datasets and the scripts in the same directory (either the raw directory or another created directory if you don't want to overwrite the original raw datasets). The scripts be run in CASA 4.2 as follows.

# In CASA 4.2
 thesteps = [9,10,11,12,13,14,15,16,17,18]
 execfile('uid___A002_X554543_X207.ms.scriptForCalibration_4.2.py')
 execfile('uid___A002_X554543_X3d0.ms.scriptForCalibration_4.2.py')
 execfile('uid___A002_X554543_X667.ms.scriptForCalibration_4.2.py')