OtherPackages

CASA ships with a few third party packages (NumPy, SciPy, Matplotlib). It can be very useful to add additional packages. The method to do this can vary from package to package and OS to OS, but many of these packages use the same "distutils" approach and we can summarize a general path.

Disclaimer: This is a writeup documenting the experience at NRAO Charlottesville as of mid-2013 and several aspects may now be outdated and incompatible with the current CASA version. Your mileage may vary, but we hope this orients you and at least gets you started. Also in general AVOID OVERWRITING THE PACKAGES INCLUDED WITH CASA. CASA does use these and may not be entirely forward compatible.

Additionally, it has been reported that these instructions are no longer compatible with astropy. To add astropy to CASA we recommend following the instructions here: http://docs.astropy.org/en/stable/install.html#installing-astropy-into-casa

Thanks To: Josh Marvil, Remy Indebetouw, Tom Robitaille (x8 for the script)

The Basic Idea

The process of installing a third party package usually has two steps:

• Building the package using the appropriate version of python for CASA.

• Making sure the package is in a path visible inside CASA so that it can be imported.

The biggest complication is that CASA ships with its own version of python. In order to use a package inside CASA you need to be sure to build that package using this version of python, which may differ from other versions on your system. Another complication, which is usually more minor, is that (depending on permissions) you may need to add the locale of the built package to your path inside CASA. A further, minor, complication is that some of these packages build on each other. For example you may need to go get Cython in order to put the package scikit-image onto your machine.

Calling setup.py

Generally, once you identify the correct python version to use, you will

• Download the buildable version of package in question (usually a ".tar.gz" file).

• Untar and unzip it into a working directory and change directory to that directory.

In that directory you will see a script called "setup.py". You will build the package by calling a variation on

# In the shell
python setup.py install

from the shell.

Generally, an option to install will let you direct where the built package goes. By default it will go to python's "site-package" directory, which is the default home for third party packages. CASA has its own version of this (where it lives exactly depends on your installation), but you may or may not actually have access to this. Try

# In the shell
python setup.py install --help

To see your options. We'll discuss the specific cases of Mac and Linux next.

Your init.py File

Once you have a built version of a package, you need to be sure that CASA can see it. The easiest way to this is to modify your "init.py" file. This is a file that CASA executes every time it starts. It lives in your home directory in a subdirectory called .casa/ so that on a typical system you can see it by

# In the shell
cat ~/.casa/init.py

Now you may not necessarily have an init.py. If you don't, that's no problem you can just create one. If you haven't done so before you just make a text file and include

# python
print "This is my initialization file in ~/.casa/init.py"

Every time CASA starts you should see this line.

Key to installing third party packages, inside this init.py you can add lines like this:

# python
sys.path.append("/users/somebody/somedirectory/")

Then that directory will be in the path that casapy searches when trying to import something. If you don't build to a directory that is automatically in the path, you will need to add a line like this that points to where the built package lives.

How to Tell It Worked

Once you have installed the package, you can tell that it worked if you can import it successfully. For example, if you are installing the AstroPy package try

# python
import astropy

If it works without an error, odds are that you are in good shape. Even better, try running astropy.test().

On the Macintosh

You are in huge luck. Most astronomers can administrate their own Mac and the location of CASA is standardized from Mac to Mac. This allowed Tom Robitaille to write a simple script that makes installing third party packages extremely simple. The script lives here:

https://github.com/astrofrog/casa-python

along with clear, simple instructions! In short:

• Grab this script
• Make it executable
• Use it (casa-python) in place of python when calling setup.py.

Again, you are searching for the buildable versions of packages (basically the .tar.gz file).

# In the shell
casa-python setup.py install

As noted in the docs, this script assumes that CASA is still using Python 2.6. That will need to be upgraded if and when casapy steps forward in version (expected for release 4.2, when we will upgrade to 4.2).

Complications - Compilers on the Mac

You knew it wasn't always going to be that easy. Some of these packages, specifically the ultra-important SciPy*, require a fortran compiler and Mac OS doesn't (at least at the moment) ship with one. In this case you will need to go grab a fortran compiler and install it on your Mac. One good option here is gfortran and you can find instructions to install it by googling gfortran and Mac.

You may also find complaints about your C compiler building things on a Mac. You may need to use XCode to update your command line compilers. It may also be necessary to create a link from a specific gcc version name (e.g., gcc-4.2) to your gcc compiler. Mileage varies from system to system, but this is sometimes needed (e.g., it may be needed for astropy or Cython).

'*' Though you probably shouldn't touch SciPy, this may come up for other examples.

(We'll try to expand this section with more specific information as time goes along.)

On Linux

Linux - especially Linux on a system administered by someone other than you - is more complicated. You will run into two obstacles:

• paths - paths for CASA are less standard, so you will need to do a little detective work to identify the CASA python.

• permissions - you may not have permission to write to the site-packages directory on a system with a shared installation of CASA. In this case you will need to install to somewhere else.

How to Identify Your CASA Python

Your first step on Linux is to figure out CASA's python executable. There's a very easy trick for this:

• Start CASA by running casapy.

• From the CASA prompt execute the shell command "!which python" to see the executable version of python being used by CASA:

# In casapy
!which python

Recall that the exclamation point executes a shell command from inside the casapy shell.

In fact, we can take this one step further. From inside the casapy shell in the directory where you untarred the package to install you can run

# In casapy
!python setup.py install

To build the package (you will usually have the option to append "--dry-run" to test this out). On systems not administered by you you will often encounter some variety of permission error. For example at NRAO we get "[Errno 30] Read-only file system", indicating that the site-packages directory for our CASA install is read-only.

Installing Packages to the Location of Your Choice

We need to build the package in question somewhere else. Easy enough. "setup.py" has an option to specify a destination other than the default site-packages directory. You can specify the target for your build via

# In casapy
!python setup.py install --root some_other_directory

Where "some_other_directory" refers to an alternate root directory (pick one where you have write permissions!). You'll get a whole crazy subdirectory structure that you own coming out of that root directory. Alternatively, you can mess with the prefix or just build and move the package around.

Once this runs successfully you have one more step to go. Go edit your ~/.casa/init.py file and add the relevant directory to your path. For example:

# In your init.py
sys.path.append("/users/aleroy/akl_python/third-party/usr/lib64/casapy/lib/python2.6/site-packages/")

Poking around, you may notice that this path is just a list. If you want one directory to trump another, you can also insert something into the list. For example, to put an item at the top of the list you would just use:

# In your init.py
sys.path.insert(0,"/users/aleroy/akl_python/third-party/usr/lib64/casapy/lib/python2.6/site-packages/")

Packages Already Distributed With CASA

The approach above works well for installing packages that aren't already included with CASA. Things are much complicated when you want to install a new version of one of the included packages:

• NumPy
• SciPy
• matplotlib (pylab)

As of release 4.2 CASA will ship versions 1.7.1 of Numpy, 0.12 of SciPy, and 1.1 of Matplotlib. Your safest path to get future versions is to request that they be included in the CASA distribution via the helpdesk.

Upgrading these packages can be complicated and almost intractable if you don't have permission to edit the site-packages directory. Even when permission are not an issue, upgrading these packages can create errors (you don't always get a clean overwrite of the old package). Matplotlib, in particular, is used heavily by CASA and checked when CASA starts. If you disregard this advice and noodle with these anyways then always, always, always be ready to roll back to clean install of CASA in case your upgrade doesn't work.

Track Record

Successful Implementations

• Installed AstroPy 0.24 using the CASA 4.1 mid-summer stable on the NRAO linux system.
• Installed AstroPy 0.24 using the CASA 4.2 August stable on the NRAO Linux system.
• Installed AstroPy 0.24 on a Mac (required some noodling with compilers including a "sudo ln -s /usr/bin/gcc /usr/bin/gcc-4.2" ... careful with that).
• Installed Cython 1.9.1 on a Mac (also required fooling with the compiler).
• Installed skimage 0.8.2 on a Mac (required Cython first).
• (Not recommended and unnecessary as of 4.2 release) Upgraded NumPy 1.4.1->1.7.1, SciPy to 0.12 (required the Numpy upgrade), and Matplotlib to 1.3.0 on a Mac for CASA 4.1. (Matplotlib required "sudo ln -s /usr/bin/g++ /usr/bin/g++-4.2"; seriously, don't do this.)
• APLpy - required the matplotlib upgrade (will not in 4.2) - installs successfully but some issues with tests.

Unsuccessful Implementations

Right now the binary distribution of CASA for Linux presents a problem when Numpy is a dependency. It looks like the include files are situated in a different place than in the compiled version (or the Mac version), leading to problems with distutils based installations finding needed headers when compiling. Looking into this.

To Test

Once the 4.2 release candidate hits with the upgraded NumPy, SciPy, Matplotlib we should test ability to layer:

• yt - complex because of the install method and heavy dependencies (NumPy+Matplotlib+HDF5+Mercurial) ... this may be better interfaced with outside of CASA.

on top of CASA. None obviously rise to the level of inclusion but all are super useful for post-imaging analysis.

APLpy may also be nonfunctional even if installed. More testing needed.