Updating a script to work with CASA 4.2: Difference between revisions

From CASA Guides
Jump to navigationJump to search
Amoullet (talk | contribs)
No edit summary
Ekeller (talk | contribs)
No edit summary
 
(13 intermediate revisions by 2 users not shown)
Line 1: Line 1:


In the ALMA archive for Cycle 0 data, the packaged calibration script found in the scripts directory is written for execution in CASA 3.4. While it possible to use the packaged calibration script as it is in CASA 3.4, it is now recommended to use CASA 4.2 for data processing. To change the packaged calibration script into a CASA 4.2 script, it must be manually modified using a text editor, following the instructions below. In the script examples below 'measurementset' stands for your uid name (for example, uid___A002_X554543_X207).
In the ALMA archive for Cycle 0 data, the packaged calibration script found in the ''scripts'' directory is written for execution in CASA 3.4. While it possible to use the packaged calibration script as it is in CASA 3.4, it is now recommended to use CASA 4.2 for [[Guide_To_Processing_ALMA_DataGlossary#Calibration|calibration]] and [[Guide_To_Processing_ALMA_DataGlossary#Imaging|imaging]]. To change the packaged calibration script into a CASA 4.2 script, it must be manually modified using a text editor, following the instructions below. In the script examples below 'measurementset' stands for your uid name (for example, uid___A002_X554543_X207). If, after applying those edits and launching the execution, the script either crashes or does not calibrate the data as expected, please contact the CASA HelpDesk.


* Change the list of steps at the beginning of the script to
* Change the list of steps at the beginning of the script to
Line 57: Line 57:


* In step 7:  
* In step 7:  
** the label of the step must be changed to 9 ('mystep = 9')
** the label of the step must be changed to 9 ('mystep = 9') (the steps 0-8 will be skipped since they do not need to be applied on the "raw" data supplied with the data package. Hence one does not need to worry about introducing 'missing' steps)
** add 'if not os.path.exists('measurementset.ms.split.flagversions/Original.flags'):' above the flagmanager call, and ident the flagmanager call. It should now read :
** add 'if not os.path.exists('replaceuidlabelhere.ms.split.flagversions/Original.flags'):' above the flagmanager call, and ident the flagmanager call. It should now read (for the example of uid uid___A002_X554543_X207):
<source lang="python">
<source lang="python">
   if not os.path.exists('measurementset.ms.split.flagversions/Original.flags'):
   if not os.path.exists('uid___A002_X554543_X207.ms.split.flagversions/Original.flags'):
     flagmanager(vis = 'measurementset.ms.split',
     flagmanager(vis = 'uid___A002_X554543_X207.ms.split',
       mode = 'save',
       mode = 'save',
       versionname = 'Original')
       versionname = 'Original')
</source>
</source>


* In step 8:  
* In step 8:  
** the label of the step must be changed to 10 ('mystep = 10')
** the label of the step must be changed to 10 ('mystep = 10')
** any call to tflagdata must be changed to flagdata
** any call to tflagdata must be changed to flagdata


* In step 9:
* In step 9:
** the label of the step must be changed to 11 ('mystep = 11')
** the label of the step must be changed to 11 ('mystep = 11')
** if applicable (when a Solar System Object is used as a flux calibrator), in setjy change the parameter standard = 'Butler-JPL-Horizons 2010' to standard = 'Butler-JPL-Horizons 2012'
** if applicable (when a Solar System Object is used as a flux calibrator), in setjy change the parameter standard = 'Butler-JPL-Horizons 2010' to standard = 'Butler-JPL-Horizons 2012'
** if applicable (when a non solar system object is used as flux calibrator), change the clearcal command to delmod (inputs are unchanged)


* In step 10:
* In step 10:
** the label of the step must be changed to 12 ('mystep = 12')
** the label of the step must be changed to 12 ('mystep = 12')


* In step 11:
* In step 11:
** the label of the step must be changed to 13 ('mystep = 13')
** the label of the step must be changed to 13 ('mystep = 13')
** in the gaincal calls, change any calmode='ap' parameter to calmode='a'
** in the gaincal calls, change any calmode='ap' parameter to calmode='p'
 


* In step 12:
* In step 12:
** the label of the step must be changed to 14 ('mystep = 14')
** the label of the step must be changed to 14 ('mystep = 14')


* In step 13:
* In step 13:
** the label of the step must be changed to 15 ('mystep = 15')
** the label of the step must be changed to 15 ('mystep = 15')
** change the clearcal command to delmod (inputs are unchanged)
** if present, change clearcal command to delmod (inputs are unchanged)
** in the gaincal calls, change any calmode='ap' parameter to calmode='a'
** in the gaincal calls, change any calmode='ap' parameter to calmode='a'
** if a  phaseCalName loop is present, change:
<source lang="python">
    if fc[i].find('Flux density for '+phaseCalName) != -1 and re.search('in SpW=[0-9]+(?: \(ref SpW=[0-9]+\))? is: [0-9]+\.[0-9]+', fc[i]) != None:
        line = (re.search('in SpW=[0-9]+(?: \(ref SpW=[0-9]+\))? is: [0-9]+\.[0-9]+', fc[i])).group(0)
</source>
to
<source lang="python">
      if fc[i].find('Flux density for '+phaseCalName) != -1 and re.search('in SpW=[0-9]+(?: \(.*?\))? is: [0-9]+\.[0-9]+', fc[i], re.DOTALL|re.IGNORECASE) != None:
        line = (re.search('in SpW=[0-9]+(?: \(.*?\))? is: [0-9]+\.[0-9]+', fc[i], re.DOTALL|re.IGNORECASE)).group(0)
</source>


* In step 14:
* In step 14:
** the label of the step must be changed to 16 ('mystep = 16')
** the label of the step must be changed to 16 ('mystep = 16')


* In step 15:
* In step 15:
** the label of the step must be changed to 16 ('mystep = 17')
** the label of the step must be changed to 16 ('mystep = 17')


* Add a step 18 (optional):
* Add a step 18 (optional):
Add this piece of code:
Add this piece of code, where measurementset stands for the uid label (as in, 'uid___A002_X554543_X207'):
<source lang="python">
<source lang="python">
mystep = 18
mystep = 18
Line 104: Line 123:


   os.system('rm -rf measurementset.ms.split.cal')  
   os.system('rm -rf measurementset.ms.split.cal')  
   split(vis = 'uid___measurementset.ms.split',
   split(vis = 'measurementset.ms.split',
     outputvis = 'uid___measurementset.ms.split.cal',
     outputvis = 'measurementset.ms.split.cal',
     datacolumn = 'corrected',
     datacolumn = 'corrected',
     keepflags = T)
     keepflags=True)
</source>
</source>

Latest revision as of 20:46, 29 March 2017

In the ALMA archive for Cycle 0 data, the packaged calibration script found in the scripts directory is written for execution in CASA 3.4. While it possible to use the packaged calibration script as it is in CASA 3.4, it is now recommended to use CASA 4.2 for calibration and imaging. To change the packaged calibration script into a CASA 4.2 script, it must be manually modified using a text editor, following the instructions below. In the script examples below 'measurementset' stands for your uid name (for example, uid___A002_X554543_X207). If, after applying those edits and launching the execution, the script either crashes or does not calibrate the data as expected, please contact the CASA HelpDesk.

  • Change the list of steps at the beginning of the script to
step_title = {0: 'Import of the ASDM',
              1: 'Fix of SYSCAL table times',
              2: 'listobs',
              3: 'A priori flagging',
              4: 'Generation and time averaging of the WVR cal table',
              5: 'Generation of the Tsys cal table',
              6: 'Generation of the antenna position cal table',
              7: 'Application of the WVR, Tsys and antpos cal tables',
              8: 'Split out science SPWs and time average',
              9: 'Listobs, clear pointing table, and save original flags',
              10: 'Initial flagging',
              11: 'Putting a model for the flux calibrator(s)',
              12: 'Save flags before bandpass cal',
              13: 'Bandpass calibration',
              14: 'Save flags before gain cal',
              15: 'Gain calibration',
              16: 'Save flags before applycal',
              17: 'Application of the bandpass and gain cal tables',
              18: 'Split out corrected column'}


  • add "import analysisUtils as au", "import analysisUtils as aU", "import os" and "if 'applyonly' not in globals(): applyonly = False" on line 27. The code under the steps list should now read:
import analysisUtils as au
import analysisUtils as aU
import os
if 'applyonly' not in globals(): applyonly = False  #(AM) added
try:
  print 'List of steps to be executed ...', mysteps
  thesteps = mysteps
except:
  print 'global variable mysteps not set.'


  • Add "if applyonly != True:" in front of any aU and es routines calls. For example:
if applyonly != True: es = aU.stuffForScienceDataReduction() 
if applyonly != True:  es.checkCalTable('measurementset.ms.split.ap_pre_bandpass', msName='measurementse.ms.split', interactive=False)


  • Replace the CASA 3.4 version search to CASA 4.2
if re.search('^4.2', casadef.casa_version) == None: #(AM)
 sys.exit('ERROR: PLEASE USE THE SAME VERSION OF CASA THAT YOU USED FOR GENERATING THE SCRIPT: 4.2')


  • In step 7:
    • the label of the step must be changed to 9 ('mystep = 9') (the steps 0-8 will be skipped since they do not need to be applied on the "raw" data supplied with the data package. Hence one does not need to worry about introducing 'missing' steps)
    • add 'if not os.path.exists('replaceuidlabelhere.ms.split.flagversions/Original.flags'):' above the flagmanager call, and ident the flagmanager call. It should now read (for the example of uid uid___A002_X554543_X207):
  if not os.path.exists('uid___A002_X554543_X207.ms.split.flagversions/Original.flags'):
    flagmanager(vis = 'uid___A002_X554543_X207.ms.split',
      mode = 'save',
      versionname = 'Original')


  • In step 8:
    • the label of the step must be changed to 10 ('mystep = 10')
    • any call to tflagdata must be changed to flagdata


  • In step 9:
    • the label of the step must be changed to 11 ('mystep = 11')
    • if applicable (when a Solar System Object is used as a flux calibrator), in setjy change the parameter standard = 'Butler-JPL-Horizons 2010' to standard = 'Butler-JPL-Horizons 2012'
    • if applicable (when a non solar system object is used as flux calibrator), change the clearcal command to delmod (inputs are unchanged)


  • In step 10:
    • the label of the step must be changed to 12 ('mystep = 12')


  • In step 11:
    • the label of the step must be changed to 13 ('mystep = 13')
    • in the gaincal calls, change any calmode='ap' parameter to calmode='p'


  • In step 12:
    • the label of the step must be changed to 14 ('mystep = 14')


  • In step 13:
    • the label of the step must be changed to 15 ('mystep = 15')
    • if present, change clearcal command to delmod (inputs are unchanged)
    • in the gaincal calls, change any calmode='ap' parameter to calmode='a'
    • if a phaseCalName loop is present, change:
     if fc[i].find('Flux density for '+phaseCalName) != -1 and re.search('in SpW=[0-9]+(?: \(ref SpW=[0-9]+\))? is: [0-9]+\.[0-9]+', fc[i]) != None:
        line = (re.search('in SpW=[0-9]+(?: \(ref SpW=[0-9]+\))? is: [0-9]+\.[0-9]+', fc[i])).group(0)

to

      if fc[i].find('Flux density for '+phaseCalName) != -1 and re.search('in SpW=[0-9]+(?: \(.*?\))? is: [0-9]+\.[0-9]+', fc[i], re.DOTALL|re.IGNORECASE) != None:
        line = (re.search('in SpW=[0-9]+(?: \(.*?\))? is: [0-9]+\.[0-9]+', fc[i], re.DOTALL|re.IGNORECASE)).group(0)
  • In step 14:
    • the label of the step must be changed to 16 ('mystep = 16')


  • In step 15:
    • the label of the step must be changed to 16 ('mystep = 17')


  • Add a step 18 (optional):

Add this piece of code, where measurementset stands for the uid label (as in, 'uid___A002_X554543_X207'):

mystep = 18
if(mystep in thesteps):
  casalog.post('Step '+str(mystep)+' '+step_title[mystep],'INFO')
  print 'Step ', mystep, step_title[mystep]

  os.system('rm -rf measurementset.ms.split.cal') 
  split(vis = 'measurementset.ms.split',
    outputvis = 'measurementset.ms.split.cal',
    datacolumn = 'corrected',
    keepflags=True)