PythonDataAccess

Back to the PythonOverview.

Preface

We'll take a quick look at saving and loading files focusing on two approaches: reading and writing text files and saving variables via "pickling" (think IDL save/restore). Loosely related, we'll see how to accept input from the user.

Input

Input can be accepted from the command line (or a script paused) using the raw input command.

verb = raw_input("Give me a verb: ")
noun = raw_input("Give me a noun: ")

mad_lib = "More fun than "+verb+"ing a "+noun
print mad_lib

File Access

Python provides easy basic file access. Grab our example_file.txt for the following example.

Open a file like so:

a_file = open("example_file.txt", "r")

r - means read, w means write. You can do both at once if you want. Read up for more.

Now that it's open we can read the lines in the file into a list like so:

lines = a_file.readlines()
print lines

We could have read a single line with readline() or only a fixed set of bytes with read()

Close the file

a_file.close()

We can also write using similar syntax (use an extra "a" to append):

a_new_file = open("new_file.txt", "w")
a_new_file.writelines(lines)
a_new_file.close()

import os
os.system('cat new_file.txt')

note that you need to convert to strings before writing.

Pickling

It's possible to directly save and load variables from a file (without making them into strings and worrying about parsing).

import pickle

Make a dictionary

a_dict = {"field1":100,
          50:[1,2,3,5],
          3.14:"hello"}

Save the dictionary

f = open("pickle.jar","w")
p = pickle.Pickler(f)
p.dump(a_dict)
f.close()

Go ahead and have a look at what it's doing.

import os
os.system("cat pickle.jar")

ascii but not english.

Get the stuff back

f = open("pickle.jar","r")
u = pickle.Unpickler(f)
read_back = u.load()
f.close()

print a_dict
print read_back

There's also a more compact syntax to just load and dump directly from a file. Options allow binary instead of ascii writing. And there's a faster version called cPickle.

Pickle is stack-based by the way, so:

a = 1
b = 2
c = 3

Save the dictionary

f = open("another_pickle.jar","w")
p = pickle.Pickler(f)
p.dump(a)
p.dump(b)
p.dump(c)
f.close()

Get the stuff back

f = open("another_pickle.jar","r")
u = pickle.Unpickler(f)
var1 = u.load()
var2 = u.load()
var3 = u.load()

... a variable too far:

var4 = u.load()

uhoh!

f.close()

print var1, var2, var3

Of course the disadvantage of pickling is that you need to unpickle it. This is not a generic format to save data and share with other people.

Other Approaches

You don't need to waste a lot of effort duplicating previous work on reading and writing text files. Adam Ginsburg's "readcol.py" (loosely patterned after the IDL version, linked from the page) will save you a lot of effort. The package astroasciidata also looks promising but I have not yet gotten a chance to experiment with it.