ALMA apply uvcont table

Applying Pipeline Continuum Subtraction

Starting in Cycle 5, ALMA pipeline-calibrated data will be delivered with a calibration table which describes the continuum subtraction the pipeline did. This CASA Guide describes how to apply this pipeline-determined continuum subtraction. The main benefit of using this pre-determined subtractionis that it will save time compared to repeating the hif_uvcontfit() and hif_uvcontsub() steps as described in [1] and [2] (NOTE: THESE ARE LINKS TO THE TWO NON-REVISING OPTIONS OF THE C4 GUIDE; NEED TO UPDATE LINKS TO NEW C4 PAGE AND/OR LINK THE C5 PAGE INVOLVING REVISING THE RANGES). If you wish to revise the continuum subtraction you should instead use the steps described in IWIDJ.

This is a draft page describing how PI's can make use of the UVCONT table that is delivered with their data to uv subtract the data, saving time over doing the uvcont fitting themselves.

STEP 1:

Restore the pipeline calibration as described [[3][here]]. Do not modify the cont.dat file! if you do, the procedure described here will not work as intended.

STEP 2:

Unpack the auxilliary calibration tables, which contain a description of the continuum subtraction.

cd ../../calibration/
gunzip -c (one auxcaltable) | tar xvf -

There will be one such table for each execution, with file names like (MOUS Name).session_2.auxcaltables.tgz, (MOUS Name).session 3.auxcaltables.tgz, etc.; repeat the above steps for each.

STEP 3:

Copy the auxilliary calibration tables into the working directory, and go back to the working directory:

cp *uvcontfit*uvcont.tbl ../calibrated/working

While here, take a look at the files called *auxcalapply*txt. There will be one file for each execution, with names like (Execution UID)_target.ms.auxcalapply.txt. These contain the applycal() statements with which, with minor modifications, you will apply the uv continuum subtraction as a calibration. You will use these in the following step. The contents will look something like this:

applycal(vis='/lustre/naasc/sciops/comm/amcnicho/pipeline/root/2013.1.00722.S_2017_09_13T14_30_33.955/SOUS_uid___A001_X145_X134/GOUS_uid___A001_X145_X135/MOUS_uid___A001_X145_X136/working/uid___A002_X9fddd8_Xc52_target.ms', field='', intent='', spw='17,19,21,23', antenna='0~35', gaintable='/lustre/naasc/sciops/comm/amcnicho/pipeline/root/2013.1.00722.S_2017_09_13T14_30_33.955/SOUS_uid___A001_X145_X134/GOUS_uid___A001_X145_X135/MOUS_uid___A001_X145_X136/working/uid___A002_X9fddd8_Xc52_target.ms.hif_uvcontfit.s27_3.SPT0346-52.uvcont.tbl', gainfield='', spwmap=[], interp='', calwt=False)

Looking ahead, the changes you will make to these statements will be to eliminate the long "full-path" prefix since you will be working with all required files in the "working" directory, which is where you will run the pipeline.

Finally, go back to the working directory

cd ../calibrated/working

STEP 4:

Edit the applycal() statements into the following script in the indicated place:

## Edit the USER SET INPUTS section below and then execute
## this script (note it must be in the 'calibrated/working' directory.

import glob as glob
import os

__rethrow_casa_exceptions = True
pipelinemode='automatic'
context = h_init()

###########################################################
## USER SET INPUTS

## Select a title for the weblog-
context.project_summary.proposal_code = 'PIPELINE CONTSUB'

# if you wish for some reason to restrict the number of SPWs that are imaged-
MySpw=''

############################################################
## Make a list of all uv-datasets appended with *.ms
MyVis=glob.glob('*.ms')

try:
    ## Load the *.ms files into the pipeline
    hifa_importdata(vis=MyVis, pipelinemode=pipelinemode)

    ## Split off the science target data into its own ms (called
    ## *target.ms) and apply science target specific flags
    hif_mstransform(pipelinemode=pipelinemode)
    hifa_flagtargets(pipelinemode=pipelinemode)

    ## Fit and subtract the continuum using revised cont.dat for all spws
    #    we are skipping these in favor of the applycal() which is faster.
    #hif_uvcontfit(pipelinemode=pipelinemode)
    #hif_uvcontsub(pipelinemode=pipelinemode)

    # For CASA 5.0 and 5.1, applycal() does not yet fully support
    #  the new underlying "VI2" infrastructure, requiring an environment
    #  variable "VI1CAL" be set. this limitation will be removed in the
    #  future, obviating the need for this workaround.
    # 
    try:
        vi1cal =  os.environ['VI1CAL']
    except KeyError:
        vi1cal = None
    finally:
        os.environ['VI1CAL']= '1'

    #### PUT THE AUXCALAPPLY.TXT STATEMENTS HERE####
    # the vis and gaintables need to be edited to a valid absolute or relative path.
    #
    applycal(vis='uid___A002_X9f54f7_X183_target.ms', field='', intent='', spw='17,19,21,23', antenna='0~36', gaintable='uid___A002_X9f54f7_X183_target.ms.hif_uvcontfit.s27_1.SPT0346-52.uvcont.tbl', gainfield='', spwmap=[], interp='', calwt=False)
    applycal(vis='uid___A002_X9fddd8_Xc52_target.ms', field='', intent='', spw='17,19,21,23', antenna='0~35', gaintable='uid___A002_X9fddd8_Xc52_target.ms.hif_uvcontfit.s27_3.SPT0346-52.uvcont.tbl', gainfield='', spwmap=[], interp='', calwt=False)
    #
    #### END AUXCALAPPLY STATEMENTS ####

    # now unset VI1CAL if it was originally unset--
    if vi1cal is None:
        del os.environ['VI1CAL']
    else:
        os.environ['VI1CAL']=vi1cal

    ## Make new per spw continuum for revised spw(s) and new aggregate cont
    hif_makeimlist(specmode='mfs',spw=MySpw)
    hif_makeimages(pipelinemode=pipelinemode)
    hif_makeimlist(specmode='cont',pipelinemode=pipelinemode)
    hif_makeimages(pipelinemode=pipelinemode)

    ## Make new continuum subtracted cube for revised spw(s)
    hif_makeimlist(specmode='cube',spw=MySpw,pipelinemode=pipelinemode)
    hif_makeimages(pipelinemode=pipelinemode)

    ## Export new images to fits format if desired.
    hifa_exportdata(pipelinemode=pipelinemode)

finally:
    h_save()

STEP 5:

• cd into the working directory
• verify that each of the gaintables referenced (*ufcontfit*uvcont.tbl) in the applycal() commands you inserted exist in the working directory.
• verify that each of the ms's referenced exists, except you do not need the _target suffix (the _target.ms files are produced by the hif_mstransform() step). In this example we are looking to verify that the files uid___A002_X9f54f7_X183.ms and uid___A002_X9fddd8_Xc52.ms exist in 'working', which should be the case if you successfully ran the restore (STEP 1).
• start up CASA with the pipeline and execute the script

casa --pipeline
execfile(the script)

Result

The result of following the above steps will be a measurement set called (MOUS UID name)_target.ms. The DATA column of this MS will have the fully calibrated but not continuum subtracted visibilities. The CORRECTED column have the fully calibrated and also continuum subtracted visibilities. This is the standard format _target.ms file that the Cycle 4 and 5 pipelines produce. Only science spectral windows and science targets (not calibrators).