ALMA Pipeline Known Issues

PL2023

CASA

1. ephemeris imaging in MPIcasa remains unreliable. PLWG recommends that due to this, all multi-EB ephemeris data be run in serial mode.

2. The tclean parameter psfphasecenter is still not functioning (CAS-13899). If you want to image a mosaic with no data at phase center, then you should make the image larger and move the phase center to a pointing that has data.

Both Pipelines

1. PL2023 uses python3.8. Some user machines may find dependency issues if they only have python3.6 available.

IF

1. hif_uvcontsub will not skip spws that have "NONE" in cont.dat. This situation doesn't arise in operations, but could arise if someone manually edits cont.dat (PIPE-1898)

2. hif_selfcal will fail for multi-EB single field projects with virtual spws - this is quite rare outside of polarization, and self-calibration of full-polarization projects is not fully validated in any case.

3. If the check source is so faint that no gain solutions can be found, then hifa_gfluxscale will crash (PIPE-1468).

4. An irregular mosaic with no pointing at the geometric center will likely fail, and the pipeline will warn you of this likelihood. Until the CASA bug with psfphasecenter is resolved (CAS-13899), create a larger image and put the phasecenter a pointing with data. (PIPE-98)

5. In hifa_antpos, in the second table "Antenna Position Offsets Sorted By Total Offset", when there are two antennas with the same offset and the Total Offset column has a common value, the second antenna is not bolded in the same way as the first. (PIPE-1631).

6. In the polcal recipe, if the selected session reference antenna is fully flagged on any spw by hifa_gfluxscaleflag, then hifa_gfluxscale will crash. The workaround is to set a different reference antenna (or list of reference antennas) in the hif_refant task in the PPR (PIPE-1664).

7. In the polcal recipe recipe, hifa_gfluxscale can fail silently (pausing pipeline permanently) if the refant has 100% flagging on only 1 of the transfer targets (typically either PHASE or CHECK source) and it still exhibits the best score in hif_refant (PIPE-1805). The workaround is to set geometry=False in hif_refant.

8. On the Spectral Setup weblog page, multi-target datasets with multiple tunings not observed in all targets will not show the Transition names for spws associated with tunings beyond the first tuning (PIPE-1909).

9. For heterogeneous array mosaics with 3 or more PM antennas included, if any field has only 1 baseline of data left unflagged after online flags and any prior stage flagging, then hifa_targetflag will crash (PIPE-1879).

10. In hifa_spwphaseup, (i) if the refant drops out for the majority of solutions, then the Median Phase RMS assessment cannot be made; (ii) if there is low SNR in the bandpass solutions and the refant jumps in phase, then some antennas can incorrectly be identified as outliers by the Median Phase RMS assessment.

11. ACA projects with multiple single-spw FDM tunings might experience odd flagging of non-edge channels (PIPE-1991). The work-around is to set edgespw=False in hifa_flagdata.

--- Items below this point were discovered after the PL2023 google slides were released and frozen: ---

12. In hif_findcont, spws with a single range of continuum channels identified might be declared AllCont even if they contain a significantly lesser amount of continuum than the nominal threshold of 91% (PIPE-2034).

13. In hifa_bandpass, the plots of phase vs. frequency can take a long time to display in the weblog due to a change in behavior of a mathematical function due to an upgrade in the scipy version packaged with the pipeline (PIPE-2035).

14. In full-polarization datasets, the hif_makeimages page for the polcal has images of polarized intensity; the ">_" link which usually produces a popup with the tclean command, in this case just dims the screen without revealing a tclean command, because the image was not made with tclean. Clicking the image background will refresh the screen.

---

---

PL2022.2

2022.2.0.64 used Oct 2022 - April 2023

2022.2.0.68 used April 2023 - Oct 2024 fixes some of the issues below, as noted.

CASA

1. A bug in imaging ephemeris cubes (CAS-13908) will sometimes cause part of the cube being erroneously empty, clearly visible in the pipeline spectrum under "other QA images": PLWG recommends that due to this, all multi-EB ephemeris data be run in serial mode.

2. The tclean parameter psfphasecenter is currently not functioning (CAS-13899). If you want to image a mosaic with no data at phase center, then you should make the image larger and move the phase center to a pointing that has data.

Both Pipelines

PL2022 doesn't import on MacOS - PL was never fully supported on MacOS, and unfortunately this one doesn't work at all. Fixed in 2022.2.0.68

Interferometric Pipeline

1. If the check source is so faint that no gain solutions can be found, then hifa_gfluxscale will crash (PIPE-1468).

2. An irregular mosaic with no pointing at the geometric center will likely fail, and the pipeline will warn you of this likelihood. Until the CASA bug with psfphasecenter is resolved (CAS-13899), create a larger image and put the phasecenter a pointing with data. (PIPE-98)

3. The atmospheric transmission curves overlaid in h_tsyscal, hifa_tsysflag, and hifa_bandpass will be incorrect if any weather stations have non-sensical pressure data. (CAS-13910)

4. This version of the pipeline won’t run on Cycle 2 and earlier data that were labeled J2000 frame because astropy does not recognize J2000. The workaround is to set hifa_flagdata(lowtrans=False) in the PPR (PIPE_1575 -> fixed PL2023).

5. For Single-pol data: (there are none in C8, 2 approved C-rank in C9, so very rare)

• The new calculation of achieved SNR in hifa_spwphaseup will issue a spurious red QA warning that the achieved SNR is zero - the actual calibration and data are fine, this is only a bug in the QA score calculation (PIPE-1623).
• Inaccurate warning and QA score in hifa_spwphaseup (PIPE-1625).
• The new phase offsets QA score in hifa_timegaincal fails (PIPE-1628).
• hifa_tsyscal or hifa_tsysflag *might* fail due to the recent change to single-pol Tsys (ICT-15219), not communicated to the PL team and for which no E2E9 data were acquired.

6. Band 5 data with science spws overlapping with the WVR spws might fail in hif_uvcontfit because of a CASA bug (CAS-13843) in calculating TOPO ranges for such spws (PIPE-1530). The workaround is to set a suitable continuum selection range for (only) the failing spw(s) in cont.dat prior to re-running the pipeline.

7. In hifa_antpos, in the second table "Antenna Position Offsets Sorted By Total Offset", when there are two antennas with the same offset and the Total Offset column has a common value, the second antenna is not bolded in the same way as the first. (PIPE-1631).

8. In hifa_gfluxscaleflag, the flagging summary table does not reflect flagging applied to the TARGET intent when flags for bad baselines are propagated from PHASE cal (PIPE-1630).

9. In hifa_spwphaseup, (i) if several antennas of a CM+PM heterogeneous dataset are flagged on the bandpass calibrator, then the value of the Median Phase RMS in the table will be NaN and the QA score will be 0.0 (PIPE-1633). (ii) if the refant drops out for the majority of solutions the Median Phase RMS assessment cannot be made; (iii) if there is low SNR in the bandpass solutions and the refant jumps in phase then some antennas can incorrectly be identified as outliers by the Median Phase RMS assessment. (iv) The phase structure function plots are calculated from antenna-based gains, so baseline-based flags will not be reflected in the plot.

10. For hifa_makeimages(cubes), if mom8fc score is yellow, it rarely means that continuum subtraction needs to be redone, but merely that in some circumstances the continuum image can be improved to reduce line contamination (see DRM training material).

11. Due to CASA bug CAS-13908, PLWG recommends that all multi-EB ephemeris data be run in serial.

12. Phase structure function plots in hifa_wvrgcalflag are calculated from antenna-based gains, so baseline-based flags will not be reflected (PIPE-1661).

13. In the polcal recipe, if the selected session reference antenna is fully flagged on any spw by hifa_gfluxscaleflag, then hifa_gfluxscale will crash. The workaround is to set a different reference antenna (or list of reference antennas) in the hif_refant task in the PPR. In a related issue, if the session reference antenna is fully flagged due to shadowing on only 1 of the transfer targets (and still wins the best score in hif_refant), then hifa_gfluxscale will crash (PIPE-1805).

14. hifa_imageprecheck will crash on mosaic observations of science targets that have integer names (PIPE-1708).

15. In the polcal recipe recipe, hifa_gfluxscale can fail silently (pausing pipeline permanently) if the refant has 100% flagging on either PHASE or CHECK source (PIPE-1805). The workaround is to set geometry=False in hif_refant.

16. On the Spectral Setup weblog page, multi-target datasets with multiple tunings not observed in all targets will not show the Transition names for spws associated with tunings beyond the first tuning (PIPE-1909).

17. For heterogeneous array mosaics with 3 or more PM antennas included, if any field has only 1 baseline of data left unflagged after online flags and any prior stage flagging, then hifa_targetflag will crash (PIPE-1879).

18. ACA projects with multiple single-spw FDM tunings might experience odd flagging of non-edge channels (PIPE-1991). The work-around is to set edgespw=False in hifa_flagdata.

19. In hif_findcont, spws with a single range of continuum channels identified might be declared AllCont even if they contain a significantly lesser amount of continuum than the nominal threshold of 91% (PIPE-2034).

Single Dish Pipeline

1. The "npiece" parameter determining number of cubic splines to fit can sometimes be miscalculated when the baseline is complex, leading to non-optimal baseline fits. (PIPE-1773, Fixed in 2022.2.0.68)

2. Weights calculated for spectra completely flagged in only one polarization can be inappropriately high, leading to bright and dark artifacts in the images. (PIPE-1771, Fixed in 2022.2.0.68 affected data will be reprocessed)

---

Browser security and viewing file://weblog

More stringent security in most web browsers in 2020 broke the weblog when displayed using a file:// URL, and users may encounter "failure to load" errors (see security update details). As of PL2020, the weblog contains instructions for what users can do if they encounter this error, including a new local html server embedded in the pipeline that users can access through CASA. These are applicable for the older weblogs, even if the weblog itself won't give them to you. (Note that newer versions of firefox (>=v96, ~Feb 2022) no longer have the parameter "privacy.file_unique_origin", but in that case the same workaround described below can be achieved by setting "security.fileuri.strict_origin_policy" to False). For google chrome, it can be started with the following command line arguments: chromium-browser --disable-web-security --user-data-dir="[some directory here]"

In addition to the suggestions below, you can start a local http server with python3, either the one included with casa, or the one installed on your system:

python3 -m http.server 8080 --bind 127.0.0.1 & and then access in your browser the url http://127.0.0.1:8080/

---

---

PL2021.2.0

Oct 2021 - Oct 2022

CASA

1. The bsend mechanism has a limit of 100 MB (introduced in 2014), which can cause the pipeline to fail (PIPE-1337) on MOUS with large numbers of EBs/spws/targets/integrations. So far, it has been seen only on a 42-EB 7m dataset. fixed 2022

Interferometric Pipeline

1. CASA tclean sometimes exits with an unsupported code “0” - this generally is okay, but PL will capture that as a yellow warning in the weblog, and the data reducer should check the images for scientific acceptability. fixed 2022

2. In the hifa_almarenorm PDF summary plot, if the first spw is narrow, subsequent spws even if wide will be missing the vertical segment boundary lines. This only affects the summary plot, so other plots in the PDF will show the lines. fixed 2022

3. When spw mapping is in use, bad data can sometimes escape unassessed by hifa_bandpassflag and appear as bad in hifa_timegaincal and hif_applycal (PIPE-838, to be fixed by PIPE-1154 where spw mapping and combination will be prohibited for all calibrators except the phase calibrator). fixed 2022

4. If an spw is fully flagged for Tsys in hifa_tsysflag, then hifa_gfluxscale can crash (PIPE-1208). This is often due to poor or strongly-varying atmospheric transmission vs. frequency. The possible workarounds are to either: set the offending EB to SemiPass, raise ff_max_limit from 13 to 30 for hifa_tsysflag in the PPR, or flag the poorest transmission parts of the spw (both its Tsys solutions and visibility data). fixed 2022

5. For the hif_applycal stage, the QA subscore for %newly flagged is not included in the overall score evaluation, meaning that a “!” or “?” can appear in the left navigation pane, different from the “By Topic” QA color bar for hif_applycal. fixed 2022

6. As in previous releases, in the hif_applycal weblog, if there are multiple phase calibrators or check sources, their calibrated spectra are overlaid in the same plot (PIPE-432). fixed 2022

7. A bad score for the mom8fc can be erroneously triggered in cube hifa_makeimages when the channelized noise is not uniform in the findcont channels, and has some channels with significantly higher noise (e.g. in regions of deep atmospheric absorption), resulting in more pixels above the Peak/(medianChanMAD) than expected. No workaround - looking at weblog plots should make it clear that there is really no emission in the findcont channels. improved 2022, may still require manual examination

8. full-polarization datasets contain a hifa_lock_refant() task - 'pipelinemode="automatic"' must be removed from that task in casa_pipescript.py in order for casa_pipescript.py to run (PIPE-1226). casa_restorescript.py works without modification. fixed 2022

9. MOUS with large numbers of EBs/spws/fields/integrations can exceed the 100MB memory footprint of bsend in CASA and crash (PIPE-1337). This feature was introduced in 2014 and a small increase has been requested (CAS-13656). improved 2022

10. As in previous releases, in the hif_applycal weblog, if there are multiple phase calibrators or check sources, their calibrated spectra are overlaid in the same plot (PIPE-432). fixed 2022

11. If pointing is not performed in every EB of a full-polarization dataset, hif_makeimages (check sources) can crash (PIPE-1364). fixed 2022

12. If the check source is so faint that no gain solutions can be found, then hifa_gfluxscale will crash (PIPE-1468).

13. An irregular mosaic with no pointing at the geometric center will likely fail, and the pipeline will warn you of this likelihood. You can use tclean with the psfphasecenter parameter to image manually. (PIPE-98)

14. On the Spectral Setup weblog page, multi-target datasets with multiple tunings not observed in all targets will not show the Transition names for spws associated with tunings beyond the first tuning (PIPE-1909).

15. ACA projects with multiple single-spw FDM tunings might experience odd flagging of non-edge channels (PIPE-1991). The work-around is to set edgespw=False in hifa_flagdata.

16. In hif_findcont, spws with a single range of continuum channels identified might be declared AllCont even if they contain a significantly lesser amount of continuum than the nominal threshold of 91% (PIPE-2034).

Single Dish Pipeline

0. Some task parameters in PPR are not selected properly. For example, hsd_blflag fails when spw is specified in PPR (PIPE-1298).

1. hsd_imaging does not properly combine datasets associated with virtual spw. (PIPE-1478)

---

Known Issues for PL2020.1.0

Release date Feb 2021

CASA

CASA 6.1.X known issues

Interferometric Pipeline

0. For polarization datasets, hifa_restoredata does not include the polarization calibrator in its applycal commands. A workaround is being arranged with DRM.

1. If hifa_lowgainflag flags a large number of antennas on a given spw, but not all antennas, then the imaging pipeline can crash downstream in hif_findcont. One workaround is to set niter=2 in hifa_lowgainflag in the PPR. This works as desired (the whole spw is flagged), but it fails to render the weblog page for hifa_lowgainflag. The other alternative is to apply a manual flag for the spw in the regular flag template file.

2. A bad score for the mom8fc can be erroneously triggered in cube hifa_makeimages when the channelized noise is not uniform in the findcont channels, and has some channels with significantly higher noise (e.g. in regions of deep atmospheric absorption), resulting in more pixels above the Peak/(medianChanMAD) than expected. No workaround - looking at weblog plots should make it clear that there is really no emission in the findcont channels.

3. For full-polarization datasets, the amp vs. time detail plots (per antenna) in hif_applycal do not include the calibrator scans with POLARIZATION intent, but the plots of calibration slutions in hifa_timegaincal do include them. The workaround is to run plotms manually if there is bad data in the parent plot (with all antennas overlaid) and you want to determine the subset of antennas involved.

4. The amplitude vs uv distance version of the “before/after flagging” plots are not shown in hifa_polcalflag. Only the ampl. vs time plots are shown.

5. Datasets suffering from the Cycle 7 correlator instrumental problem related to concurrent subarrays can have some outliers escape unflagged. But this issue was apparently fixed in SCCB-1042 (2021-05-20), so it should not affect the remainder of Cycle 7 data.

6. Antenna-based single-polarization, single-integration outliers not apparent in hifa_polcalflag can appear as amplitude outliers on the polarization calibrator in hif_applycal.

7. When spw mapping is used in hifa_spwphaseup, bad data on the bandpass calibrator can escape hifa_bandpassflag unflagged and appear as outliers in hif_applycal, because the timegaincal solutions come from a different spw when applycal is run.

8. The QA message in hifa_gfluxscale on the ratio of S_calibrated / S_catalog has an incorrectly calculated percentage.

9. For Cycle 3 datasets affected by extra field IDs associated with only SQLD data, the new hifa_targetflag stage will show a warning message for these fields (each spw): "Warning! Unable to compute flagging for intent TARGET, field 2, spw 16". The workaround is simply to ignore this warning – it should not happen in new data.

10. Due to CASA changing how tasks are built, several tasks fail to completely render the "inp" input parameters (blank lines appear in the "inp" output). help(taskname) still lists all of the available parameters, and we strongly recommend users refer to the Reference Manual if task parameters are not clear. Affected tasks: hifa_importdata, hifa_wvrgcalflag, hif_lowgainflag, hifa_bandpassflag, hifa_bandpass, hif_applycal, hif_makeimlist, hif_makeimages, hifa_imageprecheck, hif_checkproductsize, hifa_exportdata, hif_mstransform, hif_findcont, hsd_importdata, hsd_flagdata, hsd_applycal, hsd_imaging, hifa_polcalflag, hif_uvcontfit, hif_uvcontsub (dryrun parameter only missing)

11. In the Cycle 6, Cycle 7, and PL2020 releases: datasets with heterogenous antenna diameters will also include the cross-diameter baselines in the science target imaging (rather than merely the 7m-7m or merely the 12m-12m baselines). The calibrator imaging stage correctly includes all baselines, as desired. For 7m array projects with 1 or more 12m antennas included, the effects on the resulting image are: a different effective PB response (slightly smaller), lower noise, a smaller synthesized beam, and (likely) a smaller LAS. Different science use cases may see the net difference as a positive or negative effect.

12. In all pipeline releases through PL2020: FDM spws with 256 channels are being labeled as TDM on the Spectral Setup details page. This is due to faulty logic implemented in CASA::msmd.almaspws.

13. Full-polarization datasets contain a hifa_lock_refant() task - 'pipelinemode="automatic"' must be remove from that task in casa_pipescript.py in order for casa_pipescript.py to run. casa_restorescript.py works without modification.

14. As in previous releases, in the hif_applycal weblog, if there are multiple phase calibrators or check sources, their calibrated spectra are overlaid in the same plot (PIPE-432).

15. If pointing is not performed in every EB of a full-polarization dataset, hif_makeimages (check sources) can crash (PIPE-1364).

16. If the check source is so faint that no gain solutions can be found, then hifa_gfluxscale will crash (PIPE-1468).

17. ACA projects with multiple single-spw FDM tunings might experience odd flagging of non-edge channels (PIPE-1991). The work-around is to set edgespw=False in hifa_flagdata.

18. In hif_findcont, spws with a single range of continuum channels identified might be declared AllCont even if they contain a significantly lesser amount of continuum than the nominal threshold of 91% (PIPE-2034).

Single Dish Pipeline

1. Observed RMS tends to be larger than theoretical RMS (SD pipeline). There are two possible causes. One is that ATM and ripple features remain in the final image. The second is that the pixels located at the map edge are included in the measurement of the observed RMS. The latter will be improved in pipeline2021.

2. Strong emission is flagged in hsd_blflag. The reason of this issue is that some emission components at some channels (i.e. wing component) are not identified as a line. This increases the RMS at corresponding channels. Essentially, improving the line identification algorithm is required to fix this issue. This issue can be avoided by changing the threshold of blflag manually.

3. Wrong position of the open circle indicating peak SN position in Peak SN map. When the scan direction is not along RA/DEC, the open circle indicating peak SN position in Peak SN map and spectrum at peak position are not plotted correctly. However, the information of the peak position is used just as a reference. This does not affect the detection of the contamination and giving “warning”.

4. hsdn_restoredata crashes (Nobeyama Pipeline)

5. executeppr does not support hsd_restoredata task (ALMA-SD Pipeline)