Tclean and ALMA CASA 6.5.4: Difference between revisions

From CASA Guides
Jump to navigationJump to search
Masanche (talk | contribs)
Masanche (talk | contribs)
No edit summary
Line 3: Line 3:
As introduced in the [[First Look at Imaging]] here, starting with CASA 4.7, the imaging capabilities of CLEAN have been refactored and improved in a task called TCLEAN, which is called by the user in largely the same way as clean.  The pipeline uses TCLEAN for all imaging steps.
As introduced in the [[First Look at Imaging]] here, starting with CASA 4.7, the imaging capabilities of CLEAN have been refactored and improved in a task called TCLEAN, which is called by the user in largely the same way as clean.  The pipeline uses TCLEAN for all imaging steps.


In this guide, the differences in syntax between CLEAN and TCLEAN (called hereafter "clean" and "tclean") are made more explicit. At any time, please see the CASA Task Reference Manual [https://casa.nrao.edu/docs/taskref/tclean-task.html here] for more information about tclean.
In this guide, the differences in syntax between CLEAN and TCLEAN (called hereafter "clean" and "tclean") are made more explicit. At any time, please see the CASA Task Reference Manual [https://casa.nrao.edu/docs/taskref/tclean-task.html here] for more information about {{tclean_6.5.4}}.


== Major Syntax Changes ==
== Major Syntax Changes ==
Line 30: Line 30:


'''General data selection:'''
'''General data selection:'''
* The ability to select "field" and "spw" has been moved under selectdata = True in tclean; the default for selectdata is True.
* The ability to select "field" and "spw" has been moved under selectdata = True in {{tclean_6.5.4}}; the default for selectdata is True.
* It is now possible to select the datacolumn to image (data or observed, corrected); the default is "corrected".
* It is now possible to select the datacolumn to image (data or observed, corrected); the default is "corrected".


'''Specifying spectral gridding mode:'''
'''Specifying spectral gridding mode:'''
* Specifying the the spectral gridding type in clean with "mode" (mfs, channel, velocity, frequency) is now available in tclean as "specmode" (mfs, cube, cubedata).
* Specifying the the spectral gridding type in clean with "mode" (mfs, channel, velocity, frequency) is now available in {{tclean_6.5.4}} as "specmode" (mfs, cube, cubedata).
* To restore a cube to a common beam size in every channel, clean uses mode = 'channel' and resmooth = True; in tclean, this has become "restoringbeam = 'common'".
* To restore a cube to a common beam size in every channel, clean uses mode = 'channel' and resmooth = True; in {{tclean_6.5.4}}, this has become "restoringbeam = 'common'".
* To specify the minimum value of the primary beam gain to use for primary beam-based correction and masking, clean uses "minpb"; in tclean, this has become "pblimit".  
* To specify the minimum value of the primary beam gain to use for primary beam-based correction and masking, clean uses "minpb"; in {{tclean_6.5.4}}, this has become "pblimit".  
* In tclean, specmode='cube'
* In {{tclean_6.5.4}}, specmode='cube'
** restfreq, veltype are available as before for cubes
** restfreq, veltype are available as before for cubes
** an additional parameter has been added called chanchunks, which can be used to mitigate the sometimes substantial memory required to  image large spectral cubes. '''In 4.7, tclean assumes that 80% of the computer's memory is available for cube creation'''. chanchunks=1 -- the default-- corresponds to no mitigation; in this case full cubes for all of the imaging products (.residual, .psf, .model, etc.) must fit into this memory allocation. With chanchunks=(a positive integer), "chunks" of channels will be processed together sequentially, reducing the RAM requirements by roughly the value of this integer.  In general it is not recommended to manually choose this integer, but instead to set  chanchunks=-1. In this case a cube will be processed in chunks of channels determined by an internal heuristic based on the size of the available memory (again: 80% of the computer's RAM).
** an additional parameter has been added called chanchunks, which can be used to mitigate the sometimes substantial memory required to  image large spectral cubes. '''In 4.7, {{tclean_6.5.4}} assumes that 80% of the computer's memory is available for cube creation'''. chanchunks=1 -- the default-- corresponds to no mitigation; in this case full cubes for all of the imaging products (.residual, .psf, .model, etc.) must fit into this memory allocation. With chanchunks=(a positive integer), "chunks" of channels will be processed together sequentially, reducing the RAM requirements by roughly the value of this integer.  In general it is not recommended to manually choose this integer, but instead to set  chanchunks=-1. In this case a cube will be processed in chunks of channels determined by an internal heuristic based on the size of the available memory (again: 80% of the computer's RAM).


'''Specifying gridding convolution function and deconvolver:'''
'''Specifying gridding convolution function and deconvolver:'''
Line 47: Line 47:


'''Cell units:'''
'''Cell units:'''
* In clean, the parameter "cell" does not require the user to specify units. Tclean requires the user to specify the units i.e. "1.0arcsec".
* In clean, the parameter "cell" does not require the user to specify units. {{tclean_6.5.4}} requires the user to specify the units i.e. "1.0arcsec".


'''Model images:'''
'''Model images:'''
* In clean, starting with a model image on disk required modelimage = 'model.image'; in tclean, use "startmodel" instead. Use care with setting "startmodel" and overwrite = True.  
* In clean, starting with a model image on disk required modelimage = 'model.image'; in {{tclean_6.5.4}}, use "startmodel" instead. Use care with setting "startmodel" and overwrite = True.  
* Note that in tclean, restarting an imaging run with restart = True and an existing image name in "imagename" will continue with that model, in which case "startmodel" must be set to ' '.
* Note that in {{tclean_6.5.4}}, restarting an imaging run with restart = True and an existing image name in "imagename" will continue with that model, in which case "startmodel" must be set to ' '.


'''Output images:'''
'''Output images:'''
* In clean, the image of the sky sensitivity is called output_name.flux; in tclean, this image is called output_name.pb. If tclean is run with pbcor = True, then the output primary beam-corrected image is called output_name.image.pbcor (equal to output_name.image / output_name.pb).
* In clean, the image of the sky sensitivity is called output_name.flux; in {{tclean_6.5.4}}, this image is called output_name.pb. If {{tclean_6.5.4}} is run with pbcor = True, then the output primary beam-corrected image is called output_name.image.pbcor (equal to output_name.image / output_name.pb).


== Examples ==
== Examples ==


The following are sample tclean calls for a variety of use cases. In each one, 'myvis.ms' is the visibility data, and 'myimage' is the desired output imagename prefix.  
The following are sample {{tclean_6.5.4}} calls for a variety of use cases. In each one, 'myvis.ms' is the visibility data, and 'myimage' is the desired output imagename prefix.  


In every case, we specify chanchunks=-1 to allow the internal heuristics of tclean to determine how many chunks are needed to process the cube. The tclean task will assume that it can access 80% of your computer's memory; memory allocation issues and segmentation faults will result if this is not the case. Leaving chanchunks = 1 (the default value) forces tclean to run with only one chunk and is not recommended for large cubes.  
In every case, we specify chanchunks=-1 to allow the internal heuristics of {{tclean_6.5.4}} to determine how many chunks are needed to process the cube. The {{tclean_6.5.4}} task will assume that it can access 80% of your computer's memory; memory allocation issues and segmentation faults will result if this is not the case. Leaving chanchunks = 1 (the default value) forces {{tclean_6.5.4}} to run with only one chunk and is not recommended for large cubes.  


'''Single field, continuum case'''
'''Single field, continuum case'''
Line 122: Line 122:
In this example, we create a mosaic (gridder='mosaic') continuum (deconvolver='hogbom', specmode='mfs') image. Note that we use deconvolver='hogbom' when the aggregate continuum fractional bandwidth is < 10%; otherwise use deconvolver='mtmfs'. We primary beam correct the image with a primary beam cutoff at 0.2*PB. In this case, the data is called from the 'data' column, which would be the case if calibrations were applied and the 'corrected' column had been split off into a new MS. The cleaning is run with 10000 iterations (niter=10000) or until the threshold is reached (threshold='0.001mJy') in interactive mode (interactive=True), and the weights are set in the same way as clean (robust=0.5, weighting='briggs'). In CASA 5.4 and later, {{tclean_6.5.4}} calls with '''gridder''' = ''''mosaic'''', using Briggs or uniform weighting, have an additional parameter '''mosweight''' with a default of '''True'''. When mosweight = True, the gridder weights each field in the mosaic independently. The mosweight parameter is particularly important for mosaics with non-uniform sensitivity, with rectangular shapes, or when using more uniform values of robust Briggs weighting. For more information on '''mosweight''', please see the {{tclean_6.5.4}} documentation.  
In this example, we create a mosaic (gridder='mosaic') continuum (deconvolver='hogbom', specmode='mfs') image. Note that we use deconvolver='hogbom' when the aggregate continuum fractional bandwidth is < 10%; otherwise use deconvolver='mtmfs'. We primary beam correct the image with a primary beam cutoff at 0.2*PB. In this case, the data is called from the 'data' column, which would be the case if calibrations were applied and the 'corrected' column had been split off into a new MS. The cleaning is run with 10000 iterations (niter=10000) or until the threshold is reached (threshold='0.001mJy') in interactive mode (interactive=True), and the weights are set in the same way as clean (robust=0.5, weighting='briggs'). In CASA 5.4 and later, {{tclean_6.5.4}} calls with '''gridder''' = ''''mosaic'''', using Briggs or uniform weighting, have an additional parameter '''mosweight''' with a default of '''True'''. When mosweight = True, the gridder weights each field in the mosaic independently. The mosweight parameter is particularly important for mosaics with non-uniform sensitivity, with rectangular shapes, or when using more uniform values of robust Briggs weighting. For more information on '''mosweight''', please see the {{tclean_6.5.4}} documentation.  


'''Note that we highly recommend using CASA 5.4 or later for your tclean call when imaging mosaics. In CASA 5.0 and 5.1, the primary beam pattern for mosaics was truncated in error, and could result in flux density errors in the final image or cube. This issue affects mosaic images for all telescopes, but the effect is expected to be smaller for larger antennas. Please see the [https://casa.nrao.edu/casadocs/casa-5.4.0 CASA 5.4 release notes] for more information.'''  
'''Note that we highly recommend using CASA 5.4 or later for your {{tclean_6.5.4}} call when imaging mosaics. In CASA 5.0 and 5.1, the primary beam pattern for mosaics was truncated in error, and could result in flux density errors in the final image or cube. This issue affects mosaic images for all telescopes, but the effect is expected to be smaller for larger antennas. Please see the [https://casa.nrao.edu/casadocs/casa-5.4.0 CASA 5.4 release notes] for more information.'''  


<source lang="python">
<source lang="python">
Line 176: Line 176:


'''Further Notes'''
'''Further Notes'''
* The 4.7 implementation of tclean includes some heuristics to help prevent divergence. If tclean detects:
* The 4.7 implementation of {{tclean_6.5.4}} includes some heuristics to help prevent divergence. If {{tclean_6.5.4}} detects:
   -- divergence within minor cycle iterations  
   -- divergence within minor cycle iterations  
     -> as an increase of 10% from the minimum peak residual so far  
     -> as an increase of 10% from the minimum peak residual so far  
Line 183: Line 183:
     -> as an increase of 50% before/after a major cycle indicating that the model in the minor cycle really doesn't fit the data any more
     -> as an increase of 50% before/after a major cycle indicating that the model in the minor cycle really doesn't fit the data any more
         -> it will stop the entire run after that major cycle.
         -> it will stop the entire run after that major cycle.
* If the optimal size of your mosaic is not square, you must use CASA 5.3 or later for your tclean call (or CASA 4.7, but we recommend using the most recent version when possible). An issue was found in CASA versions 5.0 and later (prior to 5.3) when creating non-square images in mosaic mode. This is fixed in the 5.3 release.
* If the optimal size of your mosaic is not square, you must use CASA 5.3 or later for your {{tclean_6.5.4}} call (or CASA 4.7, but we recommend using the most recent version when possible). An issue was found in CASA versions 5.0 and later (prior to 5.3) when creating non-square images in mosaic mode. This is fixed in the 5.3 release.
== Old Guide for Previous CASA Versions ==
== Old Guide for Previous CASA Versions ==
Old version for this CASA guide can be found here: [[TCLEAN_and_ALMA_CASA_5]].
Old version for this CASA guide can be found here: [[TCLEAN_and_ALMA_CASA_5]].

Revision as of 19:54, 14 March 2024

Last checked on CASA Version 6.5.4

Introduction

As introduced in the First Look at Imaging here, starting with CASA 4.7, the imaging capabilities of CLEAN have been refactored and improved in a task called TCLEAN, which is called by the user in largely the same way as clean. The pipeline uses TCLEAN for all imaging steps.

In this guide, the differences in syntax between CLEAN and TCLEAN (called hereafter "clean" and "tclean") are made more explicit. At any time, please see the CASA Task Reference Manual here for more information about tclean.

Major Syntax Changes

Updates to the task input syntax are called out here with notes below.

clean tclean
mode specmode
resmooth=True restoringbeam='common'
minpb pblimit
gridmode, imagermode gridder
psfmode deconvolver
modelimage startmodel
.flux (output) .pb (output)

General data selection:

  • The ability to select "field" and "spw" has been moved under selectdata = True in tclean; the default for selectdata is True.
  • It is now possible to select the datacolumn to image (data or observed, corrected); the default is "corrected".

Specifying spectral gridding mode:

  • Specifying the the spectral gridding type in clean with "mode" (mfs, channel, velocity, frequency) is now available in tclean as "specmode" (mfs, cube, cubedata).
  • To restore a cube to a common beam size in every channel, clean uses mode = 'channel' and resmooth = True; in tclean, this has become "restoringbeam = 'common'".
  • To specify the minimum value of the primary beam gain to use for primary beam-based correction and masking, clean uses "minpb"; in tclean, this has become "pblimit".
  • In tclean, specmode='cube'
    • restfreq, veltype are available as before for cubes
    • an additional parameter has been added called chanchunks, which can be used to mitigate the sometimes substantial memory required to image large spectral cubes. In 4.7, tclean assumes that 80% of the computer's memory is available for cube creation. chanchunks=1 -- the default-- corresponds to no mitigation; in this case full cubes for all of the imaging products (.residual, .psf, .model, etc.) must fit into this memory allocation. With chanchunks=(a positive integer), "chunks" of channels will be processed together sequentially, reducing the RAM requirements by roughly the value of this integer. In general it is not recommended to manually choose this integer, but instead to set chanchunks=-1. In this case a cube will be processed in chunks of channels determined by an internal heuristic based on the size of the available memory (again: 80% of the computer's RAM).

Specifying gridding convolution function and deconvolver:

  • The gridding parameters "gridmode" (widefield, aprojection; and their respective expandable parameter lists) and "imagermode" (csclean, mosaic; and their respective expandable parameter lists) are now available as "gridder" (standard, wproject, widefield, mosaic, imagemosaic, awproject).
  • The deconvolver parameter "psfmode" (clark, clarkstokes, hogbom) is now available as "deconvolver" (hogbom, clark, multiscale, mtmfs, mem, clarkstokes).
  • Various sub-parameters under each of the gridder and deconvolver options have changed names as these tasks have been expanded.

Cell units:

  • In clean, the parameter "cell" does not require the user to specify units. tclean requires the user to specify the units i.e. "1.0arcsec".

Model images:

  • In clean, starting with a model image on disk required modelimage = 'model.image'; in tclean, use "startmodel" instead. Use care with setting "startmodel" and overwrite = True.
  • Note that in tclean, restarting an imaging run with restart = True and an existing image name in "imagename" will continue with that model, in which case "startmodel" must be set to ' '.

Output images:

  • In clean, the image of the sky sensitivity is called output_name.flux; in tclean, this image is called output_name.pb. If tclean is run with pbcor = True, then the output primary beam-corrected image is called output_name.image.pbcor (equal to output_name.image / output_name.pb).

Examples

The following are sample tclean calls for a variety of use cases. In each one, 'myvis.ms' is the visibility data, and 'myimage' is the desired output imagename prefix.

In every case, we specify chanchunks=-1 to allow the internal heuristics of tclean to determine how many chunks are needed to process the cube. The tclean task will assume that it can access 80% of your computer's memory; memory allocation issues and segmentation faults will result if this is not the case. Leaving chanchunks = 1 (the default value) forces tclean to run with only one chunk and is not recommended for large cubes.

Single field, continuum case

In this example, we create a single field (gridder='standard') continuum (deconvolver='mtmfs', specmode='mfs', nterms=2) image. Note that we use deconvolver='mtmfs' when the aggregate continuum fractional bandwidth is > 10%; otherwise use deconvolver='hogbom'. Note also that nterms > 1 is only available when deconvolver = 'mtmfs'. We primary beam correct the image with a primary beam cutoff at 0.2*PB. In this case, the data is called from the 'data' column, which would be the case if calibrations were applied and the 'corrected' column had been split off into a new MS. The cleaning is run with 1000 iterations (niter=1000) or until the threshold is reached (threshold='0.001mJy') in non-interactive mode (interactive=0), and the weights are set in the same way as clean (robust=0.5, weighting='briggs').

# In CASA 4.7 or later
tclean(vis='myvis.ms',
    imagename='myimage',
    datacolumn='data',
    spw=['17,19,21,23'],
    specmode='mfs', 
    threshold='0.001mJy', 
    imsize=[100, 100], 
    cell=['0.1arcsec'], 
    niter=1000, 
    deconvolver='mtmfs', 
    nterms=2, 
    gridder='standard', 
    weighting='briggs',
    robust=0.5,
    pbcor=True, 
    pblimit=0.2, 
    chanchunks=-1, 
    interactive=0)

Single field, line cube case

In this example, we create a single field (gridder='standard') spectral line (deconvolver='multiscale' for multiscale cleaning, specmode='cube' with width/start/nchan set appropriately and outframe='LSRK') image cube. We primary beam correct the image with a primary beam cutoff at 0.2*PB. In this case, the data is called from the 'corrected' column, which would be the case if calibrations were applied and the 'corrected' column is not split off. The cleaning is run with 1000 iterations (niter=1000) or until the threshold is reached (threshold='0.01mJy') in interactive mode (interactive=True), and the weights are set in the same way as clean (robust=0.5, weighting='briggs'). There will be a consistent beam size for the whole cube (restoringbeam='common').

# In CASA 4.7 or later
tclean(vis='myvis.ms',
    imagename='myimage',
    datacolumn='corrected',
    spw=['25'],
    specmode='cube', 
    width='0.5km/s',
    start='550km/s',
    outframe='LSRK',
    nchan=100,
    threshold='0.01mJy', 
    imsize=[100, 100], 
    cell=['0.1arcsec'], 
    niter=1000, 
    deconvolver='multiscale', 
    gridder='standard', 
    weighting='briggs',
    robust=0.5,
    pbcor=True, 
    pblimit=0.2, 
    restoringbeam='common',
    chanchunks=-1, 
    interactive=True)

Mosaic, continuum case

In this example, we create a mosaic (gridder='mosaic') continuum (deconvolver='hogbom', specmode='mfs') image. Note that we use deconvolver='hogbom' when the aggregate continuum fractional bandwidth is < 10%; otherwise use deconvolver='mtmfs'. We primary beam correct the image with a primary beam cutoff at 0.2*PB. In this case, the data is called from the 'data' column, which would be the case if calibrations were applied and the 'corrected' column had been split off into a new MS. The cleaning is run with 10000 iterations (niter=10000) or until the threshold is reached (threshold='0.001mJy') in interactive mode (interactive=True), and the weights are set in the same way as clean (robust=0.5, weighting='briggs'). In CASA 5.4 and later, tclean calls with gridder = 'mosaic', using Briggs or uniform weighting, have an additional parameter mosweight with a default of True. When mosweight = True, the gridder weights each field in the mosaic independently. The mosweight parameter is particularly important for mosaics with non-uniform sensitivity, with rectangular shapes, or when using more uniform values of robust Briggs weighting. For more information on mosweight, please see the tclean documentation.

Note that we highly recommend using CASA 5.4 or later for your tclean call when imaging mosaics. In CASA 5.0 and 5.1, the primary beam pattern for mosaics was truncated in error, and could result in flux density errors in the final image or cube. This issue affects mosaic images for all telescopes, but the effect is expected to be smaller for larger antennas. Please see the CASA 5.4 release notes for more information.

# In CASA 4.7 or later
tclean(vis='myvis.ms',
    imagename='myimage',
    datacolumn='data',
    spw=['17,19,21,23'],
    specmode='mfs', 
    threshold='0.001mJy', 
    imsize=[1000, 1000],
    cell=['0.1arcsec'], 
    niter=10000, 
    deconvolver='hogbom',     
    gridder='mosaic', 
    weighting='briggs',
    robust=0.5,
    pbcor=True, 
    pblimit=0.2, 
    chanchunks=-1, 
    interactive=True)

Mosaic, line cube case

In this example, we create a mosaic (gridder='mosaic') spectral line (deconvolver='hogbom', specmode='cube' with width/start/nchan set appropriately and outframe='LSRK') image cube. Here we also image two separate MS files (called 'myvis1.ms' and 'myvis2.ms') simultaneously. We primary beam correct the image with a primary beam cutoff at 0.2*PB. In this case, the data is called from the 'corrected' column, which would be the case if calibrations were applied and the 'corrected' column is not split off. The cleaning is run with 100000 iterations (niter=100000) or until the threshold is reached (threshold='0.05mJy') in interactive mode (interactive=True), and the weights are set in the same way as clean (robust=0.5, weighting='briggs'). There will be a consistent beam size for the whole cube (restoringbeam='common').

# In CASA 4.7 or later
tclean(vis=['myvis1.ms','myvis2.ms'],
    imagename='myimage',
    datacolumn='corrected',
    spw=['25'],
    specmode='cube', 
    width='0.5km/s',
    start='550km/s',
    outframe='LSRK',
    nchan=100,
    threshold='0.05mJy', 
    imsize=[1000, 1000],
    cell=['0.1arcsec'], 
    niter=100000, 
    deconvolver='hogbom', 
    weighting='briggs',
    robust=0.5,
    pbcor=True, 
    pblimit=0.2, 
    restoringbeam='common',
    chanchunks=-1, 
    gridder='mosaic', 
    interactive=True)

Further Notes

  • The 4.7 implementation of tclean includes some heuristics to help prevent divergence. If tclean detects:
 -- divergence within minor cycle iterations 
   -> as an increase of 10% from the minimum peak residual so far 
       -> it will stop minor cycle iterations for this plane.
 -- divergence across major cycle boundaries  
   -> as an increase of 50% before/after a major cycle indicating that the model in the minor cycle really doesn't fit the data any more
       -> it will stop the entire run after that major cycle.
  • If the optimal size of your mosaic is not square, you must use CASA 5.3 or later for your tclean call (or CASA 4.7, but we recommend using the most recent version when possible). An issue was found in CASA versions 5.0 and later (prior to 5.3) when creating non-square images in mosaic mode. This is fixed in the 5.3 release.

Old Guide for Previous CASA Versions

Old version for this CASA guide can be found here: TCLEAN_and_ALMA_CASA_5.