Simulating Observations in CASA 3.1: Difference between revisions

From CASA Guides
Jump to navigationJump to search
No edit summary
 
(31 intermediate revisions by 6 users not shown)
Line 3: Line 3:
== Introduction ==  
== Introduction ==  


Simulation capability in CASA follows the usual two-layered structure: there is a beginner-level python <tt>task</tt> interface called [[simdata]], which calls methods in the <tt>sm</tt> C++ <tt>tool</tt>.  The task interface turns a model of the sky (2 to 4 dimensions including frequency and Stokes) into the visibilities that would be measured with ALMA, (E)VLA, CARMA, SMA, ATCA, PdB, etc.  The task also can produce a cleaned image of the model visibilities, compare that image with your input convolved with the synthesized beam, and calculate a fidelity image.  <tt>simdata</tt> can add thermal noise (from receiver, atmosphere, and ground) to the visibilities.
Simulation capability in CASA follows the usual two-layered structure: there is a beginner-level python <tt>task</tt> interface called [[simdata]], which calls methods in the <tt>sm</tt> C++ <tt>tool</tt>.  The task interface turns a model of the sky (2 to 4 dimensions including frequency and Stokes) into the visibilities that would be measured with ALMA, (E)VLA, CARMA, SMA, ATCA, PdB, etc.  The task also can produce a cleaned image of the model visibilities, compare that image with your input convolved with the synthesized beam, and calculate a fidelity image.  <tt>simdata</tt> can add thermal noise (from receiver, atmosphere, and ground) to the visibilities.  


The <tt>sm</tt> tool has methods that can be used to add phase delay variations, gain fluctuations and drift, cross-polarization, and (coming soon) bandpass and pointing errors to your simulated data.  <tt>sm</tt> also has more flexibility in adding thermal noise than <tt>simdata</tt>, for example for new observatories that are unknown to <tt>simdata</tt>.
The <tt>sm</tt> tool has methods that can be used to add phase delay variations, gain fluctuations and drift, cross-polarization, and (coming soon) bandpass and pointing errors to your simulated data.  <tt>sm</tt> also has more flexibility in adding thermal noise than <tt>simdata</tt>, for example for new observatories that are unknown to <tt>simdata</tt>.


<font color="red">New for 3.1.0</font>:  The <tt>simdata</tt> task is the task formerly known in 3.0.2 as <tt>simdata2</tt>. The old version of simdata has been removed.
<font color="red">New for CASA version 3.1.0</font>:  The <tt>simdata</tt> task is the task formerly known in 3.0.2 as <tt>simdata2</tt>. The old version of simdata has been removed.


CASA simulation uses the [http://www.mrao.cam.ac.uk/~bn204/alma/atmomodel.html aatm] atmospheric model, a thin wrapper of Juan Pardo's [http://damir.iem.csic.es/PARDO/class_atm.html ATM] library, to accurately calculate all atmospheric corruption terms (noise, phase delay) accurately as a function of frequency and site characteristics.
CASA simulation uses the [http://www.mrao.cam.ac.uk/~bn204/alma/atmomodel.html aatm] atmospheric model, a thin wrapper of Juan Pardo's [http://damir.iem.csic.es/PARDO/class_atm.html ATM] library, to accurately calculate all atmospheric corruption terms (noise, phase delay) accurately as a function of frequency and site characteristics.
Line 13: Line 13:
Part of CASA's simulation routines are generic ephemeris and geodesy calculations available in python - see [[simutil.py]].
Part of CASA's simulation routines are generic ephemeris and geodesy calculations available in python - see [[simutil.py]].


<font color="green"> Because <tt>simdata</tt> is still actively being developed, documentation may lag reality, please email rindebet at nrao.edu with any questions - It's my job to help you use this software. In particular, you may find that some of the presentations and graphics below show parameter inputs that are slightly different from the latest version of CASA.</font>
'''Note on cleaning:''' just as is the case for real images, cleaning images produced by simdata can lead to a spurious decrease in object fluxes and noise on the image ("clean bias"). This is particularly true for observations with poor coverage of the uv-plane, i.e. using telescopes with small numbers of antennas, such as the ALMA Early Science configurations, and/or in short "snapshot" observations. Users should always clean images with care, using a small number of iterations and/or a conservative (3-5-sigma) threshold, and boxing bright sources.
 
<font color="green"> Because <tt>simdata</tt> is still actively being developed, documentation may lag reality.
Users are encouraged to use the ALMA helpdesk (for ALMA simulations) or the NRAO helpdesk (for simulations using other telescopes) to submit queries or comments. In particular, you may find that some of the presentations and graphics below show parameter inputs that are slightly different from the latest version of CASA.</font>


== Steps to simulation ==
== Steps to simulation ==


<font color="red">Users of the most recent version of CASA, 3.1.0 and later should use simdata. If you are using CASA 3.0.2 and simdata2, you should use simdata2</font>.
<font color="red">Users of the most recent version of CASA, 3.1.0 and later should use simdata. If you are using CASA 3.0.2, you should use simdata2</font>.


{| style="width: 100%; valign: top; " cellpadding=10  
{| style="width: 100%; valign: top; " cellpadding=10  
Line 24: Line 27:


| style="width: 98%; valign: top; background-color:#E0FFFF; border:1px solid #3366FF;" |
| style="width: 98%; valign: top; background-color:#E0FFFF; border:1px solid #3366FF;" |
<big>'''simdata2'''</big> <font color="red">Only works in CASA 3.0.2</font> [https://safe.nrao.edu/wiki/bin/view/Software/ObtainingCASA Obtaining CASA]<br />  
<big>'''simdata'''</big> <font color="red">(named simata2 in CASA 3.0.2)</font> [http://casa.nrao.edu/casa_obtaining.shtml Obtaining CASA]<br />  


1. [[Getting Started in CASA#Installing CASA | Install CASA]]
1. [[Getting Started in CASA#Installing CASA | Install CASA]]


[[simdata2]] inputs look like this (v3.0.2; click to enlarge): [[File:Simdata2.png|100px]]
<tt>simdata (v3.1.0) and simdata2 (v3.0.2)</tt> inputs look like this (click to enlarge): [[File:Simdata_new.png|100px]]      [[File:Simdata2.png|100px]]


The subtasks are modular i.e. as long as you follow a few conventions about filenames, you can run each  
The subtasks are modular i.e. as long as you follow a few conventions about filenames, you can run each  
bit independently and optionally.  For example, you can modify the sky model, then predict ACA visibilities, then run again and predict
bit independently and optionally.  For example, you can modify the sky model, then predict ACA visibilities, then run again and predict
ATCA 12m visibilities and image and analyze both measurement sets together.  You can run once to predict, run interactive clean yourself, and as long as you called your image $project.image, run <tt>simdata2</tt> just to calculate a difference image and analyze the results.
ATCA 12m visibilities and image and analyze both measurement sets together.  You can run once to predict, run interactive clean yourself, and as long as you called your image $project.image, run <tt>simdata</tt> just to calculate a difference image and analyze the results.


2. [[Modify Model]] - relabel (scale) the spectral and spatial coordinates and brightness of the sky model image.
2. [[Modify Model]] - relabel (scale) the spectral and spatial coordinates and brightness of the sky model image.
Line 46: Line 49:
7. [[Analyze]] Calculate and display the difference between output and input, and fidelity image.
7. [[Analyze]] Calculate and display the difference between output and input, and fidelity image.
|}
|}
== Simulating ALMA Observations ==
We will update simdata as ALMA commissioning proceeds. During this period, we expect the noise properties of the telescope to be increasingly better characterized, and its configurations to be refined. Updates will be placed below, under "ALMA updates", along with an estimate of which version of CASA they will be applied to. Configuration files will also be placed here, until they can be incorporated in the next CASA release.
Users should also be aware of the Observation Support Tool (OST) [http://almaost.jb.man.ac.uk/ ]. This is a web-based interface to an ALMA simulator hosted by the University of Manchester, UK. Like simdata, it is based on the CASA sm toolkit, but uses different wrapper scripts, and, in particular, has a different treatment of atmospheric effects. Comparisons to the ALMA sensitivity calculator made in March 2011 suggest that both simdata and the OST give similar noises for observations in bands 3-8, but the OST diverges in bands 9 and 10. <font color="red"> In general, however, because the ALMA sensitivity calculator will be used for the technical assessment of ALMA proposals, only values from it, not simdata or the OST, should be used to estimate exposure times for ALMA Science Goals.</font>
'''ALMA updates'''
<font color="red"> March 2011: the receiver temperatures in ALMA bands 6,7 and 9, and the sideband gain in band 9, have recently been revised in the ALMA sensitivity calculator. These revisions are not in the current version of simdata. Thus, the sensitivities in these bands measured from simdata outputs will be incorrect. (We expect the 3.2 version of CASA will contain the corrected values.)</font>
Simdata in CASA 3.1 does not provide the final versions of the ALMA Early Science (Cycle 0) configurations, though they will be present in CASA 3.2. For those who wish to perform Early Science simulations the two configuration files (compact and extended) are available for download below:
[[File:CompactCycle0.txt]]
[[File:ExtendedCycle0.txt]]


== Tutorials, Recipes, and Example images ==
== Tutorials, Recipes, and Example images ==


{| style="width: 100%; valign: top; background-color:#E0FFFF; border:1px solid #3366FF; " cellpadding=0
{| style="width: 100%; valign: top; background-color:#E0FFFF; border:1px solid #3366FF; " cellpadding=0
| New User's Guide to Simulated ALMA Observations:  fully annotated tutorial<br>
This uses a Spitzer SAGE 8 micron continuum image of 30 Doradus and scales it to greater distance.
| rowspan=2; style="border-bottom:1px solid black;" | [[File:30Dor_ES.png|100px]]
|-
!style="border-bottom:1px solid black;"| [[Simdata New Users Guide 3.1| simdata recipe page]]
|-
| Simulated ALMA Observation of M51 at z = 0.1 and z = 0.3:  fully annotated tutorial<br>
| Simulated ALMA Observation of M51 at z = 0.1 and z = 0.3:  fully annotated tutorial<br>
This uses a BIMA-SONG cube of a nearby galaxy and scales it to greater distance.
This uses a BIMA-SONG cube of a nearby galaxy and scales it to greater distance.
| rowspan=3; style="border-bottom:1px solid black;" | [[File:M51thumb.png|100px]]
| rowspan=3; style="border-bottom:1px solid black;" | [[File:M51thumb.png|100px]]
|-
|-
!style="solid black;"| [[M51 at z = 0.1|simdata page]] &nbsp;&nbsp; [[M51 at z = 0.1 and z = 0.3|simdata2 page]]
!style="solid black;"| &nbsp;&nbsp; [[M51 at z = 0.1 and z = 0.3 (CASA 3.1)|simdata recipe page]]
|-
|-
!style="border-bottom:1px solid black;"| NOTE: increasing the [[etime study|exposure time]] to run faster
!style="border-bottom:1px solid black;"| NOTE: how to run the simulation faster by increasing the [[etime study|exposure time]]
|-
|-


Line 64: Line 92:
| rowspan=2; stype="border-bottom:1px solid black;" | [[File:Psimthumb.png|100px]]
| rowspan=2; stype="border-bottom:1px solid black;" | [[File:Psimthumb.png|100px]]
|-
|-
!style="border-bottom:1px solid black;"| [[PPdisk simdata1| simdata page]] &nbsp;&nbsp; [[PPdisk simdata2| simdata2 page]]
!style="border-bottom:1px solid black;" | [[PPdisk simdata (CASA 3.1)| simdata recipe page]]
|-
|-


Line 72: Line 100:
| rowspan=2; style="border-bottom:1px solid black;" | [[File:N891thumb.png|100px]]
| rowspan=2; style="border-bottom:1px solid black;" | [[File:N891thumb.png|100px]]
|-
|-
!style="border-bottom:1px solid black;"| [[http://casaguides.nrao.edu/images/2/29/N891.simdata.py simdata script]] &nbsp;&nbsp; [[N891 simdata2| simdata2 page]]
!style="border-bottom:1px solid black;"| [[N891 simdata2| simdata recipe page]]
|-
|-
| The face of Einstein: sky model and lightly annotated script<br>
An example of using a non-science image to demonstrate the effects of spatial filtering by ALMA.
| rowspan=2; stype="border-bottom:1px solid black;" | [[File:einstein_fs_cfg8_1hr.gif|100px]]
|-
!style="border-bottom:1px solid black;" | [[Einstein-Face (CASA 3.1) | simdata recipe page]]
|-


| colspan=2; style="border-bottom:1px solid black;" | [[Sim Inputs | Other example input images]]
| colspan=2; style="border-bottom:1px solid black;" | [[Sim Inputs | Other example input images]]
Line 88: Line 125:
* [http://almasimulations.pbworks.com/ Simulation Library] This will become a library of use cases and examples illustrating different science and observation setups.  It is in early stages as of Jan 2010, and we're actively seeking volunteers to turn their simulation projects into use cases.  
* [http://almasimulations.pbworks.com/ Simulation Library] This will become a library of use cases and examples illustrating different science and observation setups.  It is in early stages as of Jan 2010, and we're actively seeking volunteers to turn their simulation projects into use cases.  
* [https://safe.nrao.edu/wiki/bin/view/ALMA/Jan2010Wkshop Jan 2010 workshop] Including slides and discussion of how simdata and Simulator work "under the hood" and plans for development
* [https://safe.nrao.edu/wiki/bin/view/ALMA/Jan2010Wkshop Jan 2010 workshop] Including slides and discussion of how simdata and Simulator work "under the hood" and plans for development
''Italic text''

Latest revision as of 13:28, 30 April 2011


Introduction

Simulation capability in CASA follows the usual two-layered structure: there is a beginner-level python task interface called simdata, which calls methods in the sm C++ tool. The task interface turns a model of the sky (2 to 4 dimensions including frequency and Stokes) into the visibilities that would be measured with ALMA, (E)VLA, CARMA, SMA, ATCA, PdB, etc. The task also can produce a cleaned image of the model visibilities, compare that image with your input convolved with the synthesized beam, and calculate a fidelity image. simdata can add thermal noise (from receiver, atmosphere, and ground) to the visibilities.

The sm tool has methods that can be used to add phase delay variations, gain fluctuations and drift, cross-polarization, and (coming soon) bandpass and pointing errors to your simulated data. sm also has more flexibility in adding thermal noise than simdata, for example for new observatories that are unknown to simdata.

New for CASA version 3.1.0: The simdata task is the task formerly known in 3.0.2 as simdata2. The old version of simdata has been removed.

CASA simulation uses the aatm atmospheric model, a thin wrapper of Juan Pardo's ATM library, to accurately calculate all atmospheric corruption terms (noise, phase delay) accurately as a function of frequency and site characteristics.

Part of CASA's simulation routines are generic ephemeris and geodesy calculations available in python - see simutil.py.

Note on cleaning: just as is the case for real images, cleaning images produced by simdata can lead to a spurious decrease in object fluxes and noise on the image ("clean bias"). This is particularly true for observations with poor coverage of the uv-plane, i.e. using telescopes with small numbers of antennas, such as the ALMA Early Science configurations, and/or in short "snapshot" observations. Users should always clean images with care, using a small number of iterations and/or a conservative (3-5-sigma) threshold, and boxing bright sources.

Because simdata is still actively being developed, documentation may lag reality. Users are encouraged to use the ALMA helpdesk (for ALMA simulations) or the NRAO helpdesk (for simulations using other telescopes) to submit queries or comments. In particular, you may find that some of the presentations and graphics below show parameter inputs that are slightly different from the latest version of CASA.

Steps to simulation

Users of the most recent version of CASA, 3.1.0 and later should use simdata. If you are using CASA 3.0.2, you should use simdata2.

simdata (named simata2 in CASA 3.0.2) Obtaining CASA

1. Install CASA

simdata (v3.1.0) and simdata2 (v3.0.2) inputs look like this (click to enlarge): Simdata new.png Simdata2.png

The subtasks are modular i.e. as long as you follow a few conventions about filenames, you can run each bit independently and optionally. For example, you can modify the sky model, then predict ACA visibilities, then run again and predict ATCA 12m visibilities and image and analyze both measurement sets together. You can run once to predict, run interactive clean yourself, and as long as you called your image $project.image, run simdata just to calculate a difference image and analyze the results.

2. Modify Model - relabel (scale) the spectral and spatial coordinates and brightness of the sky model image.

3. Set Pointings - calculate a mosaic of pointings and save in a text file. You could also make the text file yourself.

4. Predict - Calculate visibilities for a specified array on a specified day

5. Corrupt - Corrupt the measurement set with thermal noise, phase noise, cross-polarization, etc.

6. Image A subset of clean to re-image the visibilities

7. Analyze Calculate and display the difference between output and input, and fidelity image.

Simulating ALMA Observations

We will update simdata as ALMA commissioning proceeds. During this period, we expect the noise properties of the telescope to be increasingly better characterized, and its configurations to be refined. Updates will be placed below, under "ALMA updates", along with an estimate of which version of CASA they will be applied to. Configuration files will also be placed here, until they can be incorporated in the next CASA release.


Users should also be aware of the Observation Support Tool (OST) [1]. This is a web-based interface to an ALMA simulator hosted by the University of Manchester, UK. Like simdata, it is based on the CASA sm toolkit, but uses different wrapper scripts, and, in particular, has a different treatment of atmospheric effects. Comparisons to the ALMA sensitivity calculator made in March 2011 suggest that both simdata and the OST give similar noises for observations in bands 3-8, but the OST diverges in bands 9 and 10. In general, however, because the ALMA sensitivity calculator will be used for the technical assessment of ALMA proposals, only values from it, not simdata or the OST, should be used to estimate exposure times for ALMA Science Goals.


ALMA updates

March 2011: the receiver temperatures in ALMA bands 6,7 and 9, and the sideband gain in band 9, have recently been revised in the ALMA sensitivity calculator. These revisions are not in the current version of simdata. Thus, the sensitivities in these bands measured from simdata outputs will be incorrect. (We expect the 3.2 version of CASA will contain the corrected values.)

Simdata in CASA 3.1 does not provide the final versions of the ALMA Early Science (Cycle 0) configurations, though they will be present in CASA 3.2. For those who wish to perform Early Science simulations the two configuration files (compact and extended) are available for download below:

File:CompactCycle0.txt

File:ExtendedCycle0.txt

Tutorials, Recipes, and Example images

New User's Guide to Simulated ALMA Observations: fully annotated tutorial

This uses a Spitzer SAGE 8 micron continuum image of 30 Doradus and scales it to greater distance.

30Dor ES.png
simdata recipe page
Simulated ALMA Observation of M51 at z = 0.1 and z = 0.3: fully annotated tutorial

This uses a BIMA-SONG cube of a nearby galaxy and scales it to greater distance.

M51thumb.png
   simdata recipe page
NOTE: how to run the simulation faster by increasing the exposure time
Protoplanetary Disk: sky model and lightly annotated script

This uses a theoretical model of dust continuum from Sebastian Wolff, scaled to the distance of a nearby star.

Psimthumb.png
simdata recipe page
Nearby edge-on spiral galaxy: sky model, script, and discussion

This uses a Galactic CO cube from the Galactic Ring Survey and places it at 10Mpc, similar to what NGC891 would look like if it were observable from the southern hemisphere.

N891thumb.png
simdata recipe page
The face of Einstein: sky model and lightly annotated script

An example of using a non-science image to demonstrate the effects of spatial filtering by ALMA.

Einstein fs cfg8 1hr.gif
simdata recipe page
Other example input images
Other example output simulations (scripts to reproduce these are coming)



Technical and Planning

I always welcome input on developing the CASA simulator, and these links are meetings, technical documents, and planning discussions. Much of it won't make sense to a new user of CASA::simdata, but may be of interest to those wanting to delve deeper:

  • Simulation Library This will become a library of use cases and examples illustrating different science and observation setups. It is in early stages as of Jan 2010, and we're actively seeking volunteers to turn their simulation projects into use cases.
  • Jan 2010 workshop Including slides and discussion of how simdata and Simulator work "under the hood" and plans for development

Italic text