ALMA Imaging Pipeline Reprocessing for Manually Calibrated Data (Cycle 10)

About This Guide

This guide describes some examples for creating the interferometric imaging products using the ALMA Cycle 10 Pipeline, for manually calibrated data. And it does NOT work for concatenated measurement set, often named as calibrated_final.ms

Note that the scripts described in this guide have only been tested in Linux.

How to Restore Manually Calibrated Measurement Set

If the delivered data has been manually calibrated, PI should be able to restore the manually calibrated data by running the scriptForPI.py (named member.<uid_name>.scriptForPI.py in /script directory).

Follow instructions in your README for restoring pipeline calibrated data using the scriptForPI.py. NOTE: the SPACESAVING parameter cannot be larger than 1

Once completed, the following files and directories will be present, with specific things about pipeline re-imaging noted:

• calibrated/
  • This directory contains a file(s) called <uid_name>.ms.split.cal (one for each execution in the MOUS) -- these type of files have been split to contain the calibrated pipeline uv-data in the DATA column, and only the science spectral window ids (spws), though importantly the spws have been re-indexed to start at zero, i.e. they will not match spws listed in the pipeline weblog or other pipeline produce products like the science target flag template files (*.flagtargetstemplate.txt) or continuum ranges (cont.dat). Though this type of file has been the starting point for manual ALMA imaging, ms.split.cal files CANNOT BE DIRECTLY USED IN THE EXAMPLES GIVEN IN THIS GUIDE.
  • Provided that the restore is done with a SPACESAVING=1, within the calibrated directory there is a "working" directory which does contain the <uid_name>.ms (i.e. no split has been run on them) that is of the form expected as the starting point of the ALMA imaging pipeline. This directory also contains the *.flagtargetstemplate.txt for each execution which can be used to do science target specific flagging. This is the best location to do ALMA pipeline image reprocessing. (Older data sets may have a <uid_name>.calibration directory rather than "working".)
• calibration/
  • This directory contains the <mous_name>.cont.dat file with the frequency ranges identified by the pipeline as being likely to only contain continuum emission. If a file called cont.dat (i.e. with mous_name stripped off) is present in the directory where pipeline imaging tasks are run, it will be used.
• log/
  • This directory contains the <mous_name>.casa_commands.log which contains all the equivalent casa commands run during the course of the pipeline processing, in particular the tclean commands to make the image products.
• product/
  • The original pipeline image products
• qa/
  • The original pipeline weblog
• raw/
  • The raw asdm(s)
• README
  • File containing information about the QA2 For the MOUS and may contain specific notes about the image quality.
• script/
  • Contains the scriptForPI.py

Getting and Starting CASA

If you do not already have CASA installed on your machine, you will have to download and install it.

Download and installation instructions are available here:

http://casa.nrao.edu/casa_obtaining.shtml

CASA 6.5.4 or later is required to reprocess ALMA data using the scripts in this guide.

NOTE: To use pipeline tasks, you must start CASA with

casa --pipeline

If you are using an NRAO machine, however, you'll need

casa-alma --pipeline

as casa alone will call the latest stable release of CASA, 6.6, rather than the 6.5.4 version needed for the Cycle 10 pipeline.

Re-imaging Examples with Automatically Chosen Parameters by Pipeline

import os
import sys

mymss = ['uid___A002_Xbe2ed7_Xb524.ms']

for myms in mymss:
    if not os.path.exists(myms+'.flagversions'):
        print('Not found: '+myms+'.flagversions')
        sys.exit('ERROR: you must provide the flagversions files for all input MSs.')

h_init()
hifa_importdata(mymss)
hif_makeimlist(intent='CHECK')
hif_makeimages()
hif_mstransform()
hifa_flagtargets()
hifa_imageprecheck()
hif_checkproductsize(maxcubesize=40.0, maxcubelimit=100.0, maxproductsize=500.0)
hif_makeimlist(specmode='mfs')
hif_findcont()
hif_uvcontsub()
hif_makeimages()
hif_makeimlist(specmode='cont')
hif_makeimages()
hif_makeimlist(specmode='cube')
hif_makeimages()
hifa_exportdata()
h_save()

The above example script will produce three types of images: multifrequency synthesis (specmode='mfs') image for each spw without continuum subtraction, continuum (specmode='cont') image for aggregate spws excluding line channels identified by hif_findcont task and linecube (specmode='cube') image for each spw. Depending on the data volume, the channel binning, image size, cell size, number of sources and spectral windows for imaging, can be mitigated by hif_checkproductsize.

For customized imaging with different imaging parameters, please consult the link below that introduces several use cases. Note that the continuum range file, cont.dat either determined by the hif_findcont task in above or by user manually, should exist before making customized imaging.

ALMA Cycle 10 Imaging Pipeline Reprocessing

Also if the user is not satisfied with the clean mask from automasking algorithm, it is possible to change the automasking parameters in hif_makeimages by setting pipelinemode='interactive'. For detailed instructions and guides for using automasking, please consult with the link below.

Automasking Guide

Relevant PL parameters corresponding to the ones in the automasking guide, are hm_sidelobethreshold, hm_noisethreshold, hm_lownoisethreshold, hm_negativethreshold, hm_minbeamfrac and hm_growiterations.