Corrupt: Difference between revisions

From CASA Guides
Jump to navigationJump to search
Rindebet (talk | contribs)
Akimball (talk | contribs)
No edit summary
 
(6 intermediate revisions by 3 users not shown)
Line 2: Line 2:
[[Category: Simulations]]
[[Category: Simulations]]


The descriptions on this page are rather anemic at the moment, so please read this
For even more detail than is contained in this wiki page, please read this [https://safe.nrao.edu/wiki/pub/ALMA/SimulatorCookbook/corruptguide.pdf pdf explanation of how this is all done]
[https://safe.nrao.edu/wiki/pub/ALMA/SimulatorCookbook/corruptguide.pdf pdf explanation of how this is all done]


<font color="red"><b>WARNING</b> As of this writing, if you run a simulation a second time with the same name, you will apply the noise twice, even if you delete foo.noisy.ms (simdata in fact does that for you).  We're working on understanding and correcting this problem 2002/02/23.</font>
== Thermal noise ==


The <tt>simobserve</tt> task allows you to add thermal noise from the atmosphere and from the ALMA receivers according to current ALMA specifications according to observing band.  <tt>simobserve</tt> knows the receiver temperature specifications for ALMA and the VLA, and a little about the altitude and pressure at the ALMA site.


== Thermal noise ==
For interferometric simulation, one can select '''thermalnoise''' = ''tsys-manual'' or ''tsys-atm''.  For single dish simulation, only ''tsys-manual'' is currently available.  The manual method requires specifying '''t_ground''', i.e. spillover temperature, '''t_sky''' the physical temperature in the atmosphere, and '''tau0''' the zenith opacity in the center of the simulated band.  ''tsys-atm'' constructs an ATM atmospheric model from the site characteristics and the user specified Precipitable Water Vapor ''user_pwv'', and the user must still specify '''t_ground'''.


'''noise_thermal''', '''t_sky''', and '''tau0''' allow you to add thermal noise from the atmosphere and from the  
<font color="green">NOTE: adding noise can be slow</font> (you have to generate a random number for every visibility).  For purposes of learning the software, and doing zeroeth order parameter studies, consider increasing the integration timeIn particular, choose a very large integration time so that you can explore the simulated parameter space, even though the integration time may be unreasonable large for a real observation.  See [[Etime study]] for details.
ALMA receivers according to current ALMA specifications according to observing band.   
At time of writing, <tt>simdata</tt> knows the receiver temperature specifications for ALMA and EVLA, and a little about the altitude and pressure at the ALMa site.  


For more flexibility and control, use the <tt>sm</tt> tool.  run simdata ''without'' noise, and niter=0 to not produce a clean image.  Optionally make a copy of the noise-free measurement set, and then do the following:
For more flexibility and control, one can use the <tt>sm</tt> tool.  Start by running <tt>simobserve</tt> ''without'' noise to create a noise-free measurement set, and then do the following:


<source lang="python">
<source lang="python">
Line 45: Line 43:
* '''mode'''="screen": a 2-dimensional phase screen is generated with the same fBM mathematical model, and blows across the array at the specified wind speed. This creates much more realistic correlations in time and space between antennas. In practice, what is created is a fluctuation of precipitable water vapor (PWV) screen, and then the aatm library and tropospheric model is used to calculate the corresponding phase delay as a function of frequency.
* '''mode'''="screen": a 2-dimensional phase screen is generated with the same fBM mathematical model, and blows across the array at the specified wind speed. This creates much more realistic correlations in time and space between antennas. In practice, what is created is a fluctuation of precipitable water vapor (PWV) screen, and then the aatm library and tropospheric model is used to calculate the corresponding phase delay as a function of frequency.


 
<!--
old method, unsupported, YMMV: <font color="lightgrey">Rob Reid wrote a python script to add phase noise,  
old method, unsupported, YMMV: <font color="lightgrey">Rob Reid wrote a python script to add phase noise,  
[http://www.cv.nrao.edu/~rreid/ppdisksims/noisify_alma.py noisify_alma.py].  noisify_alma gets its parameters from the ALMA specifications where it can, and an approximate Komolgorov model of the atmosphere such that the phase difference
[http://www.cv.nrao.edu/~rreid/ppdisksims/noisify_alma.py noisify_alma.py].  noisify_alma gets its parameters from the ALMA specifications where it can, and an approximate Komolgorov model of the atmosphere such that the phase difference
between two antennas increases with baseline length.<br>
between two antennas increases with baseline length.<br>
* note that noisify_alma.py also does thermal noise.  don't add twice as much as you intended.</font>
* note that noisify_alma.py also does thermal noise.  don't add twice as much as you intended.</font>
-->


== Gain ==
== Gain ==
Line 69: Line 68:
== Bandpass and Pointing ==
== Bandpass and Pointing ==


coming soon. see alma memo [http://www.alma.nrao.edu/memos/html-memos/abstracts/abs388.html 388], and  
TBD. see alma memo [http://www.alma.nrao.edu/memos/html-memos/abstracts/abs388.html 388], and  
[http://www.alma.nrao.edu/memos/html-memos/abstracts/abs178.html 178], [http://www.alma.nrao.edu/memos/html-memos/abstracts/abs160.html 160]
[http://www.alma.nrao.edu/memos/html-memos/abstracts/abs178.html 178], [http://www.alma.nrao.edu/memos/html-memos/abstracts/abs160.html 160]

Latest revision as of 21:17, 7 November 2012

Simulating Observations in CASA

For even more detail than is contained in this wiki page, please read this pdf explanation of how this is all done

Thermal noise

The simobserve task allows you to add thermal noise from the atmosphere and from the ALMA receivers according to current ALMA specifications according to observing band. simobserve knows the receiver temperature specifications for ALMA and the VLA, and a little about the altitude and pressure at the ALMA site.

For interferometric simulation, one can select thermalnoise = tsys-manual or tsys-atm. For single dish simulation, only tsys-manual is currently available. The manual method requires specifying t_ground, i.e. spillover temperature, t_sky the physical temperature in the atmosphere, and tau0 the zenith opacity in the center of the simulated band. tsys-atm constructs an ATM atmospheric model from the site characteristics and the user specified Precipitable Water Vapor user_pwv, and the user must still specify t_ground.

NOTE: adding noise can be slow (you have to generate a random number for every visibility). For purposes of learning the software, and doing zeroeth order parameter studies, consider increasing the integration time. In particular, choose a very large integration time so that you can explore the simulated parameter space, even though the integration time may be unreasonable large for a real observation. See Etime study for details.

For more flexibility and control, one can use the sm tool. Start by running simobserve without noise to create a noise-free measurement set, and then do the following:

sm.openfromms("my.ms")
sm.setnoise( ... options ... )
sm.corrupt()
sm.done()

sm.setnoise takes as arguments the atmospheric and ground/ambient temperatures, antenna efficiencies, site characteristics, etc. There are three modes:

  • simple: specify simplenoise="1Jy" to get random Gaussian noise with 1Jy RMS
  • tsys-atm: use environment temperatures, antenna parameters, and the aatm library to create a model of the troposphere and add random noise of the appropriate magnitude to the visibilities
  • tsys-manual: specify atmospheric brightness temperature and optical depth yourself (rather than let aatm calculate it for you) and apply noise of the corresponding magnitude.
  • see the pdf mentioned above for the precise equations used

For any of the sm corruption methods, an actual calibration table can be written out, and plotted with plotcal to make sure you're introducing the magnitude and type of corruption that you desire.

Atmospheric Phase Noise

The troposphere is refractive, so introduces a phase delay, and is not uniform, so that phase delay differs between antennas. Excessive phase noise decorrelates the signal. In CASA, you can use sm.settrop to corrupt your measurement set with phase noise in two different ways:

Full corruption will make your data look worse than real ALMA data should look. ALMA will use Water Vapor Radiometers (WVRs) to correct for the atmospheric phase delay in real time during the observation. Once the full hardware and software system is operational, the residual phase errors should be small (a few microns of equivalent PWV variation), roughly Gaussian, and independent on each antenna, so a more realistic portrayal of that may be to use sm.setgain with the real part of the gain equal to zero, or to use mode="individual" below. The full phase screen treatment is interesting to explore the full uncorrected corruption, to test other telescopes, and to test heuristics for WVR calculations. For more information, please see ALMA memos 587, 582, 573, 568, 535, 523, 515, 496, 495, 491, 490, 451, 415, 404, 361, 332, 252, 210, 209, 176 ...

For more information about the atmosphere at Chajnantor, see memos 542, 529, 521, 517, 512, 500, 497, 459, 384, 365, 363, 345, 334, 238, 237, 187, 186 ...

  • mode="individual": the delay varies with time according to fractional Brownian Motion (fBM), or generalized 1/f noise. This mathematical description is very good for many natural stochastic processes, and used very widely for e.g. CGI simulations of fractal landscapes, turbulent motions and structures in clouds and water, electronic gain drift, etc.
  • mode="screen": a 2-dimensional phase screen is generated with the same fBM mathematical model, and blows across the array at the specified wind speed. This creates much more realistic correlations in time and space between antennas. In practice, what is created is a fluctuation of precipitable water vapor (PWV) screen, and then the aatm library and tropospheric model is used to calculate the corresponding phase delay as a function of frequency.


Gain

  • sm.setgain
  • mode="random" generates random Gaussian complex gains with the given real and imaginary RMS
  • mode="fbm" generates a fractional Brownian Motion complex gain drift or time-dependent variation, independently for each antenna

note also alma memos 466, 301

Leakage a.k.a. Cross-polarization

  • sm.setleakage
  • random Gaussian (time-independent) complex cross-polarization leakage terms, with specified real and imaginary amplitudes, and an optional systematic offset from zero
  • at time of writing, sm and simdata only calculate cross-polarization in the linear approximation (i.e. your cross-hands will change but Stokes I should not). Full treatment will be added in the future.

see also alma memo 288

Bandpass and Pointing

TBD. see alma memo 388, and 178, 160