Analysis Utilities: Difference between revisions

From CASA Guides
Jump to navigationJump to search
(53 intermediate revisions by 4 users not shown)
Line 1: Line 1:
[[Category:CASA]] [[Category:ALMA]] [[Category:EVLA]]
[[Category:CASA]] [[Category:ALMA]] [[Category:EVLA]]
Analysis Utilities (or analysisUtils for short) is a small set of Python source code files that provide a number of analysis and plotting utilities.  The utilities were developed for ALMA data reduction and are, in many cases, also useful for EVLA data reduction.  This CASA Guide documents some of the most useful functions contained in the analysisUtils Python module from a user's perspective.
Analysis Utilities (or analysisUtils for short) is a small set of Python source code files that provide a number of analysis and plotting utilities.  The primary file is analysisUtils.py (108000 lines as of March 2021).  The utilities were developed (and continue to be developed) for ALMA commissioning and data reduction and are, in many cases, also useful for EVLA data reduction.  This CASA Guide documents some of the most useful functions contained in the analysisUtils Python module from a user's perspective, but most of the functions have fairly complete inline help.  


=Getting Started=
=Getting Started=


'''If you are working on a machine at NRAO Charlottesville, Socorro, or Santiago, then you can skip Step 1, because the latest modules are already available at: /users/thunter/AIV/science/analysis_scripts. '''  
'''If you are working on a machine at NRAO Charlottesville, then you can skip Step 1, because the latest modules are already available at: /users/thunter/AIV/science/analysis_scripts. '''  


'''Step 1:''' Download Analysis Utilities from [ftp://ftp.cv.nrao.edu/pub/casaguides/analysis_scripts.tar here] and extract the tar ball.  From a Unix command line this can be done with
'''Step 1:''' Download Analysis Utilities from [ftp://ftp.cv.nrao.edu/pub/casaguides/analysis_scripts.tar here] and extract the tar ball.  From a Unix command line this can be done with
Line 13: Line 13:


Seven files will be extracted -- a README file, and six python scripts.  The README file contains the time
Seven files will be extracted -- a README file, and six python scripts.  The README file contains the time
and date that the tar ball was generated, which can be useful for reporting bugs.  The history file available at
and date that the tar ball was generated, which can be useful for reporting bugs.  The [ftp://ftp.cv.nrao.edu/pub/casaguides/history history] file available at the ftp site contains a list of major changes in each version.   
the ftp site contains a list of major changes in each version.   


'''Step 2:''' Edit your casapy initialization file in <tt>$HOME/.casa/init.py</tt>, and add the following
'''Step 2:''' Edit your existing casa initialization file (in CASA5: <tt>$HOME/.casa/init.py</tt>, or in CASA6: <tt>$HOME$/.casa/startup.py</tt>) or create a new file if it does not already exist, and add the following


<source lang="Python">
<source lang="Python">
import sys
import sys
sys.path.append("/PATH_TO_ANALYSIS_SCRIPTS")
sys.path.append("/PATH_TO_ANALYSIS_SCRIPTS/")
import analysisUtils as au
import analysisUtils as au
</source>
</source>


where <tt>/PATH_TO_ANALYSIS_SCRIPTS</tt> is the path to the directory you just extracted from the tar ball.  When this is done, start casapy and you will have access to all the functions contained in the analysisUtils module.
where <tt>/PATH_TO_ANALYSIS_SCRIPTS/</tt> is the path to the directory you just extracted from the tar ball.  An example can be downloaded and edited here: [[File:Init.py]].  When this is done, start casapy and you will have access to all the functions contained in the analysisUtils module.


For help with analysisUtils submit a helpdesk ticket to the CASA department of https://help.almascience.org/.
=Support=
If your question is with regard to '''plotbandpass''', then please provide the version number of this program which is printed
 
at execution time.  Otherwise, provide the release date of your analysisUtils, which can be viewed from inside casapy
Once installed, functions can be searched for by name via minimum match string with the au.help function (where comma denotes a logical AND): e.g. au.help('antenna') or au.help('spw,caltable').  As in python, inline help for an individual function is accessed by typing 'help au.planet'.  For further help with analysisUtils installation or usage, you may submit a helpdesk ticket to the Data Reduction category at https://help.almascience.org/. Be advised that analysisUtils is a commissioning tool, so the level of support available is not as extensive as for CASA, but we do welcome feedback.  If your question is with regard to [[plotbandpass]], then please provide the version number of this program (it is printed to the screen at execution time and on the plots)In all cases, please provide the release date of your analysisUtils, which can be viewed from inside casapy as follows:
as follows:
<source lang="Python">
<source lang="Python">
os.system('more %s/README'%os.path.dirname(au.__file__))
au.version()
</source>
</source>


=Key Functions=
=Script generators=
 
The documentation for the two script generators almaqa2csg.py and almaqa2isg.py, as well as how to provide feedback or get support for their usage, can be found at this link: https://confluence.alma.cl/display/EAPR/ALMA+Data+Analysis+Utilities
; [[plotbandpass]]
:  a faster version of plotcal for bandpass tables, with useful overlay capabilities
 
=Other Functions=


=Some Examples of Functions=
<b>Up-to-date, online help is available for all of these functions (and more) at the CASA prompt by simply using the command: help(au.functionName).</b>  To search for functions, you can type au.help('phrase'), where phrase is case-insensitive and minimum-match. Multiple phrases can be combined with a comma, for example au.help('baseline,stats').
; [[angularSeparationOfFields]]
:  reads an ms and computes the angular separation of all fields, or a subset
; [[buildConfigurationFile]]
:  reads the list of antenna stations in an .ms and creates a .cfg file suitable for [http://casa.nrao.edu/docs/taskref/simobserve-task.html simobserve]
; [[editIntents]]
; [[editIntents]]
:  change the intents for a specified field in an ms (based on John Lightfoot's pipeline script)
:  change the intents for a specified field in an ms (based on John Lightfoot's pipeline script)
; [[getBaselineLengths]]
; [[getBaselineLengths]]
:  returns a dictionary of the baseline lengths in your ms, by default sorted by length
:  returns a dictionary of the baseline lengths in your ms, by default sorted by length
; [https://safe.nrao.edu/wiki/bin/view/ALMA/GetBaselineStats getBaselineStats]
:  returns a bunch of statistics on the baseline lengths in your ms
; [[gjincBeam]]
:  compute the effective restoring beam obtained from the casa command {{sdimaging}} when using the GJINC gridding kernel
; [[lstrange]]
; [[lstrange]]
:  lists the range of LST, UT, MJD for the whole ms, and for scans with ON_SOURCE intent (including the elevation range)
:  lists the range of LST, UT, MJD for the whole ms, and for scans with OBSERVE_TARGET intent (including the elevation range)
; [[obslist]]
; [[obslist]]
:  prints the antenna station coordinates in local offsets from the Center of Array, and computes longest/shortest baselines
:  prints the antenna station coordinates in local offsets from the Center of Array, and computes longest/shortest baselines
; [[planet]]
:  contacts the JPL Horizons telnet service and returns the J2000 position, velocity, angular diameter, range and range rate of a solar system object
; [https://safe.nrao.edu/wiki/bin/view/ALMA/PlanetFlux planetFlux]
:  calls Bryan Butler's solar_system_setjy module in CASA to compute flux density vs. time or flux density vs. frequency
; [[planetPlots]]
:  generates a plots of uv amplitude vs. uv distance for a grid of ALMA configurations and observing frequencies
; [[plotbandpass]]
:  a faster version of plotcal for bandpass tables, with useful overlay capabilities.  This function has been incorporated into CASA as a task.
; [[plotSunDuringTrack]]
:  generates a plot of the Sun's az vs. el during an ms or ASDM
; [[plotconfig]]  
; [[plotconfig]]  
:  plot any standard telescope configuration of observatories known to casa
:  plot any standard telescope configuration of observatories known to casa, and return an array of the sorted baseline lengths
; [[plotmosaic]]
; [[plotmosaic]]
:  shows relative location of pointings in an .ms
:  shows relative location of pointings in an .ms
; [[plotPWV]]
:  reads the PWV from each antenna's WVR from the ASDM_CALWVR table in an ms, and creates a plot vs. time
; [[plotweather]]
; [[plotweather]]
:  plot weather conditions vs. time for your ms
:  plot weather conditions vs. time for your ms
; [[smoothbandpass]]
; [[plotWVRSolutions]]
smooths an existing bandpass table with options for window length and type (Ed Fomalont)
plot the WVR solutions in terms of baseline-based phase corrections
; [[spectralindex]]
:  performs a least-squares fit to the multi-spw output from {{fluxscale}}
; [[timeOnSource]]
; [[timeOnSource]]
:  compute the total time spent integrating on-source for each specified field
:  compute the total time spent integrating on-source for each specified field
<b>A list of additional 1200+ functions, and their brief descriptions, can be found at [https://safe.nrao.edu/wiki/bin/view/Main/CasaExtensions Todd's wikipage]</b>

Revision as of 09:53, 24 May 2021

Analysis Utilities (or analysisUtils for short) is a small set of Python source code files that provide a number of analysis and plotting utilities. The primary file is analysisUtils.py (108000 lines as of March 2021). The utilities were developed (and continue to be developed) for ALMA commissioning and data reduction and are, in many cases, also useful for EVLA data reduction. This CASA Guide documents some of the most useful functions contained in the analysisUtils Python module from a user's perspective, but most of the functions have fairly complete inline help.

Getting Started

If you are working on a machine at NRAO Charlottesville, then you can skip Step 1, because the latest modules are already available at: /users/thunter/AIV/science/analysis_scripts.

Step 1: Download Analysis Utilities from here and extract the tar ball. From a Unix command line this can be done with

$ tar xvf analysis_scripts.tar

Seven files will be extracted -- a README file, and six python scripts. The README file contains the time and date that the tar ball was generated, which can be useful for reporting bugs. The history file available at the ftp site contains a list of major changes in each version.

Step 2: Edit your existing casa initialization file (in CASA5: $HOME/.casa/init.py, or in CASA6: $HOME$/.casa/startup.py) or create a new file if it does not already exist, and add the following

import sys
sys.path.append("/PATH_TO_ANALYSIS_SCRIPTS/")
import analysisUtils as au

where /PATH_TO_ANALYSIS_SCRIPTS/ is the path to the directory you just extracted from the tar ball. An example can be downloaded and edited here: File:Init.py. When this is done, start casapy and you will have access to all the functions contained in the analysisUtils module.

Support

Once installed, functions can be searched for by name via minimum match string with the au.help function (where comma denotes a logical AND): e.g. au.help('antenna') or au.help('spw,caltable'). As in python, inline help for an individual function is accessed by typing 'help au.planet'. For further help with analysisUtils installation or usage, you may submit a helpdesk ticket to the Data Reduction category at https://help.almascience.org/. Be advised that analysisUtils is a commissioning tool, so the level of support available is not as extensive as for CASA, but we do welcome feedback. If your question is with regard to plotbandpass, then please provide the version number of this program (it is printed to the screen at execution time and on the plots). In all cases, please provide the release date of your analysisUtils, which can be viewed from inside casapy as follows:

au.version()

Script generators

The documentation for the two script generators almaqa2csg.py and almaqa2isg.py, as well as how to provide feedback or get support for their usage, can be found at this link: https://confluence.alma.cl/display/EAPR/ALMA+Data+Analysis+Utilities

Some Examples of Functions

Up-to-date, online help is available for all of these functions (and more) at the CASA prompt by simply using the command: help(au.functionName). To search for functions, you can type au.help('phrase'), where phrase is case-insensitive and minimum-match. Multiple phrases can be combined with a comma, for example au.help('baseline,stats').

angularSeparationOfFields
reads an ms and computes the angular separation of all fields, or a subset
buildConfigurationFile
reads the list of antenna stations in an .ms and creates a .cfg file suitable for simobserve
editIntents
change the intents for a specified field in an ms (based on John Lightfoot's pipeline script)
getBaselineLengths
returns a dictionary of the baseline lengths in your ms, by default sorted by length
getBaselineStats
returns a bunch of statistics on the baseline lengths in your ms
gjincBeam
compute the effective restoring beam obtained from the casa command sdimaging when using the GJINC gridding kernel
lstrange
lists the range of LST, UT, MJD for the whole ms, and for scans with OBSERVE_TARGET intent (including the elevation range)
obslist
prints the antenna station coordinates in local offsets from the Center of Array, and computes longest/shortest baselines
planet
contacts the JPL Horizons telnet service and returns the J2000 position, velocity, angular diameter, range and range rate of a solar system object
planetFlux
calls Bryan Butler's solar_system_setjy module in CASA to compute flux density vs. time or flux density vs. frequency
planetPlots
generates a plots of uv amplitude vs. uv distance for a grid of ALMA configurations and observing frequencies
plotbandpass
a faster version of plotcal for bandpass tables, with useful overlay capabilities. This function has been incorporated into CASA as a task.
plotSunDuringTrack
generates a plot of the Sun's az vs. el during an ms or ASDM
plotconfig
plot any standard telescope configuration of observatories known to casa, and return an array of the sorted baseline lengths
plotmosaic
shows relative location of pointings in an .ms
plotPWV
reads the PWV from each antenna's WVR from the ASDM_CALWVR table in an ms, and creates a plot vs. time
plotweather
plot weather conditions vs. time for your ms
plotWVRSolutions
plot the WVR solutions in terms of baseline-based phase corrections
spectralindex
performs a least-squares fit to the multi-spw output from fluxscale
timeOnSource
compute the total time spent integrating on-source for each specified field

A list of additional 1200+ functions, and their brief descriptions, can be found at Todd's wikipage