PythonDataAccess

Back to the PythonOverview.

Preface

We'll take a quick look at saving and loading files focusing on two approaches: reading and writing text files and saving variables via "pickling" (think IDL save/restore). Loosely related, we'll see how to accept input from the user.

Input

Input can be accepted from the command line (or a script paused) using the raw input command.

verb = raw_input("Give me a verb: ") noun = raw_input("Give me a noun: ")

mad_lib = "More fun than "+verb+"ing a "+noun print mad_lib

File Access

Python provides easy basic file access. Grab our example_file.txt for the following example.

Open a file like so:

a_file = open("example_file.txt", "r")

r - means read, w means write. You can do both at once if you want. Read up for more.

Now that it's open we can read the lines in the file into a list like so:

lines = a_file.readlines() print lines

We could have read a single line with readline() or only a fixed set of bytes with read()

Close the file

a_file.close()

We can also write using similar syntax (use an extra "a" to append):

a_new_file = open("new_file.txt", "w") a_new_file.writelines(lines) a_new_file.close()

import os os.system('cat new_file.txt')

note that you need to convert to strings before writing.

Pickling

It's possible to directly save and load variables from a file (without making them into strings and worrying about parsing).

import pickle

Make a dictionary

a_dict = {"field1":100,

         50:[1,2,3,5],
         3.14:"hello"}

Save the dictionary

f = open("pickle.jar","w") p = pickle.Pickler(f) p.dump(a_dict) f.close()

Go ahead and have a look at what it's doing.

import os os.system("cat pickle.jar")

ascii but not english.

Get the stuff back

f = open("pickle.jar","r") u = pickle.Unpickler(f) read_back = u.load() f.close()

print a_dict print read_back

There's also a more compact syntax to just load and dump directly from a file. Options allow binary instead of ascii writing. And there's a faster version called cPickle.

Pickle is stack-based by the way, so:

a = 1 b = 2 c = 3

Save the dictionary

f = open("another_pickle.jar","w") p = pickle.Pickler(f) p.dump(a) p.dump(b) p.dump(c) f.close()

Get the stuff back

f = open("another_pickle.jar","r") u = pickle.Unpickler(f) var1 = u.load() var2 = u.load() var3 = u.load()

... a variable too far:

var4 = u.load()

uhoh!

f.close()

print var1, var2, var3

Of course the disadvantage of pickling is that you need to unpickle it. This is not a generic format to save data and share with other people.

Other Approaches

You don't need to waste a lot of effort duplicating previous work on reading and writing text files. Adam Ginsburg's "readcol.py" (loosely patterned after the IDL version, linked from the page) will save you a lot of effort. The package astroasciidata also looks promising but I have not yet gotten a chance to experiment with it.