View source for Pipeline: Frequent VLA problems

= General Description =
The [http://go.nrao.edu/vla-pipe VLA pipeline] delivers calibrated data and some initial images of VLA observation runs. The quality of the calibration and imaging products is usually assessed through the weblog that is created in each pipeline run (see also the [https://casaguides.nrao.edu/index.php?title=VLA-S-CASA_Pipeline VLA Pipeline guide]). During the observations, the VLA may have encountered technical problems that are reflected in various ways in the weblog, where graphs show the behavior of the calibration tables as a function of time, frequency, polarization, etc., and analytical numbers describe the amount of flagging, derived fluxes, image statistics, etc. 

Here we would like to briefly describe common VLA observing problems, how they are identified in the pipeline calibration weblog, and how they can be addressed.  

 

== RFI ==

by far the biggest problem is radio frequency interference (RFI). RFI is produced by internal and external sources, can be terrestrial or from satellites that operate at the observed frequency. For the VLA, please find more information on the [https://science.nrao.edu/facilities/vla/docs/manuals/obsguide/rfi Radio Frequency Interference] webpage. Although weak RFI may sonly slightly raise the noise of an image, stronger RFI will produce artifacts that may render the data unusable. An example for strong RFI is shown in Fig. 1. Flagging procedures are also outlines in the [https://casaguides.nrao.edu/index.php/VLA_CASA_Flagging VLA topical CASA guide on flagging].

{|
|[[Image:CASA5.4.0-FLAG-Plotms-amp-time.png|400px|thumb|left|Fig. 1: Example for RFI in an observation.]]
|}

<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: Weak, intermitted RFI will increase the noise and down-weighted in the imaging in the hifv_statwt task. Strong RFI needs to be flagged and only clean data should be calibrated and imaged. Flagging can be manual or automatic. 
</pre>

== Pointing == 
 
At higher frequencies the VLA requires regular pointing calibrations. Each pointing run will reposition the antennas to be centered on a strong source. If the pointing solution fails, the amplitude of the source will drop. A typical graph looks like the one shown in Fig. 2. The pointing solution for the first half of the run failed, which results in the source drifting away from the center of the primary beam. After a pointing update in the middle of the run, the antenna is positioned properly again.   

{|
|[[Image:VLApipe-gains-ea25-CASA6.2.1.png|400px|thumb|left|Fig. 2: Example of a gain table with a failed pointing solution in the first half of the observation]]
|}

<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: If the pointing is only off by a small amount, the gain calibration will take care of it. If it is off by a large amount, the data for this period and antenna needs to be flagged. 
</pre>

== Deformatter Problems ==


The digital transmission system (''DTS'') of each VLA antenna includes a formatting stage to convert the electronic to an optical signal before it is injected on the optical fiber link. On the correlator end the signal will be deformatted back to an electronic signal. Occasionally, the timing on the deformatter can be misaligned which results in very strong amplitude or phase slopes as a function of frequency. Frequently the signal is similar to an ''abs(sin)'', or a 'bouncing' signal across a baseband for one polarization, or, in other terms, various numbers of 'V' shapes in the data. 

	
{| 
|[[Image:VLApipe-step6-BPgain-ea18-CASA6.2.1.png|400px|thumb|left|Fig. 3a: DTS issue in one baseband]]
|[[Image:VLApipe-baddefromatterexampleBP.png|400px|thumb|left|Fig. 3b: An example of a bad deformatter from a different dataset.]]
|}

<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: The data in for this baseband, antenna and polarization needs to be flagged.
</pre>

== Correlator Zeros ==
 
<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>

== Baseband and Subband Edges ==
 
<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>

== Compression ==
 
<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>

== Resolved Calibrators ==
 
<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>

== Wrong Intents ==

If the intents of the data are set incorrectly for the observations, the pipeline will use the wrong calibrators for the calibration. Usually this can be fixed by overwriting the intents. The [http://go.nrao.edu/vla-pipe VLA pipeline webpage] provides instructions and a script to do this. For more complicated setups, like multiple calibrators or bands with separate calibrator scans, data may be split into smaller MSs that contain only the relevant calibrators for each target, or data reduction by hand may be needed.

== Extreme Solution Intervals ==
 

<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>

== Weather ==
 
<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>

== Decorrelation ==

<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>
 

== Shadowing ==
 
<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>


== Switches ==

<pre style="white-space: pre-wrap; background-color: #E0FFFF;">
FIX: 
</pre>