File Handling in Python

Everything a program holds in variables disappears when it ends. To keep data — marks, records, a log — a program writes it to a file on secondary storage and reads it back later. This chapter covers the two kinds of file Python works with, how open() and its access modes decide what you may do to a file and where you start, the methods that write and read, how the file pointer moves and is moved with tell() and seek(), and how whole Python objects are stored in binary files with pickle and tabular data in CSV files. The paper tests modes and pointer positions closely, so learn exactly what each mode does to existing content.

Text Files and Binary Files

A file is a named location on secondary storage. Every file is stored as bytes (0s and 1s), but how those bytes are meant to be read divides files into two kinds. A text file holds human-readable characters, each byte or group of bytes encoding a character in ASCII or Unicode; its content is a sequence of lines, each ending with an end-of-line (EOL) character — in Python the newline \n. Source code, .txt files and .csv files are text files and open sensibly in a text editor. A binary file holds bytes that represent something other than text characters — an image, audio, a compiled program, or a Python object stored with pickle. Opened in a text editor it shows meaningless symbols, and a single wrong byte can make the whole file unreadable.

Opening and Closing a File

A file is opened with file_object = open(file_name, access_mode), which returns a file object (also called a file handle) through which all reading and writing happens. The access mode is optional and defaults to "r", read-only text. The file object exposes three attributes: file.closed (True once closed), file.mode and file.name. A file is closed with file_object.close(), which flushes anything still waiting to be written. The with statement closes the file automatically when its block ends, even if an exception occurs inside it: with open("myfile.txt", "r") as f:.

Access modes — add b for a binary file (rb, wb, ab, rb+ …)
ModeAllowsIf the file existsIf it does notPointer starts at
rread onlycontent kepterror (FileNotFoundError)beginning
r+read and writecontent kepterrorbeginning
wwrite onlycontent erasedcreatedbeginning
w+write and readcontent erasedcreatedbeginning
aappend (write) onlycontent keptcreatedend
a+append and readcontent keptcreatedend
⚠️ "w" is destructive
Opening an existing file in "w" or "w+" mode empties it at once, before you write a single character. To add to a file without losing what is there, use "a". "r+" also keeps the content, but writes from the start and overwrites what lies under the pointer.

Writing to and Reading from a Text File

  • write(string) writes one string and returns the number of characters written. It adds no newline of its own, and a number must first be converted with str().
  • writelines(sequence) writes every string of a list or tuple, again without adding newlines, and returns None. f.writelines(["a", "b", "c"]) writes abc.
  • read(n) reads n characters; read() with no argument reads the rest of the file as one string.
  • readline(n) reads one line, including its \n, or at most n characters of it; at the end of the file it returns an empty string.
  • readlines() reads all remaining lines and returns them as a list of strings, each still ending in \n. A file object can also be looped over directly: for line in f:.

For a file notes.txt holding the two lines Hello and World, each followed by a newline, open("notes.txt").readlines() returns ['Hello\n', 'World\n'] — two items, each keeping its newline. Forgetting the \n is the usual slip when such a list is printed or compared.

The File Pointer: tell() and seek()

Python keeps a file pointer (offset) marking where the next read or write will happen; every read or write moves it forward by what was read or written. tell() returns the current position as an integer — the number of bytes from the beginning. seek(offset, reference_point) moves it: reference_point 0 means from the beginning (the default), 1 from the current position and 2 from the end. In a text file, offsets other than zero are used from the beginning only; relative moves belong to binary files.

Worked example. word.txt contains the single word PROGRAMMING. Positions count from 0, so after seek(3) the pointer sits on the G, read(4) returns GRAM, and tell() then gives 7:
f = open("word.txt")
f.seek(3)
print(f.read(4))
print(f.tell())

Binary Files with pickle, and CSV Files

A list, a dictionary or any other Python object cannot be written to a file as it is; it must first be turned into a stream of bytes. That conversion is serialization (pickling), and turning the bytes back into the object is de-serialization (unpickling). The pickle module does both. pickle.dump(data_object, file_object) writes one object to a file opened in binary write mode ("wb" or "ab"); pickle.load(file_object) reads one object back from a file opened in "rb". Each call to load returns the next object, and a load attempted after the last object raises EOFError — so files holding several records are read in a loop that stops on that exception.

The standard reading loop counts every record and stops cleanly at the end:
import pickle
count = 0
with open("rec.dat", "rb") as f:
while True:
try:
obj = pickle.load(f)
count += 1
except EOFError:
break

A CSV (comma-separated values) file is a text file in which each line is a record and the fields are separated by commas — the form spreadsheets export. The csv module reads and writes it. csv.writer(f) gives a writer object whose writerow(list) writes one record and writerows(list_of_lists) several; csv.reader(f) gives an object that yields each row as a list of strings, numbers included. The file is opened with newline="" so that the module controls line endings and no blank rows appear.

⚠️ csv.reader gives strings
The line 101,Asha,88 comes back as ['101', 'Asha', '88'], not [101, 'Asha', 88]. To add marks you must convert with int() first; adding the strings would join them instead.

Key takeaways

  • Text files hold characters in lines ending with \n; binary files hold bytes that are not text, such as images or pickled objects.
  • open() defaults to "r"; "r" and "r+" need an existing file, "w" and "w+" erase it, "a" and "a+" keep it and start at the end.
  • write() returns the number of characters written; neither write() nor writelines() adds a newline.
  • read(n) reads n characters, readline() one line with its \n, readlines() a list of lines.
  • tell() gives the pointer position; seek(offset, 0/1/2) moves it from the beginning, current position or end.
  • pickle.dump() needs "wb"/"ab", pickle.load() needs "rb", and EOFError marks the end of the records; csv.reader yields each row as a list of strings.

Practice questions (8)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. The file word.txt contains only the text PROGRAMMING. What does this code print?
    f = open("word.txt")
    f.seek(3)
    print(f.read(4))

    1. OGRA
    2. GRAMM
    3. GRAM
    4. RAMM
    Show answer

    Answer: C — GRAM

    Positions start at 0, so P, R, O occupy 0 to 2 and seek(3) leaves the pointer on G; read(4) then returns the four characters G, R, A, M. OGRA comes from counting positions from 1, which puts the pointer one place too early. GRAMM reads five characters instead of four, and RAMM starts one place too late.

  2. A program must add new lines to the end of an existing log file without disturbing its earlier contents. In which mode should the file be opened?

    1. "a"
    2. "w"
    3. "r"
    4. "w+"
    Show answer

    Answer: A — "a"

    Append mode keeps whatever the file holds and places the pointer at the end, so every write lands after the old content. "w" and "w+" are the trap: both empty an existing file the moment it is opened, so the log would be lost. "r" opens the file for reading only and any attempt to write would fail.

  3. notes.txt contains two lines: Hello and World, each followed by a newline. What does print(open("notes.txt").readlines()) display?

    1. ['Hello', 'World']
    2. ['Hello\n', 'World\n']
    3. Hello\nWorld\n
    4. ['H', 'e', 'l', 'l', 'o']
    Show answer

    Answer: B — `['Hello\n', 'World\n']`

    readlines() returns a list with one string per line, and each string keeps the newline that ended its line, so the list shows 'Hello\n' and 'World\n'. Dropping the \n is the commonest slip — that would need strip() on each item. A single string with both lines is what read() returns, and a list of characters is what list() of a string would give.

  4. What is printed by the following code?
    f = open("out.txt", "w")
    n = f.write("Hi\n")
    print(n)
    f.close()

    1. 2
    2. None
    3. True
    4. 3
    Show answer

    Answer: D — 3

    write() returns the number of characters it wrote, and "Hi\n" is three characters — H, i and the newline. Answering 2 overlooks the newline, which is a single character however it is typed. None is what writelines() returns, and write() never returns a Boolean.

  5. Which of the following correctly writes a Python list L to a binary file using pickle?

    1. pickle.dump(L, open("d.dat", "w"))
    2. pickle.load(L, open("d.dat", "wb"))
    3. pickle.dump(L, open("d.dat", "wb"))
    4. pickle.dump(open("d.dat", "wb"), L)
    Show answer

    Answer: C — `pickle.dump(L, open("d.dat", "wb"))`

    pickle.dump takes the object first and the file object second, and pickled data is bytes, so the file must be opened in binary write mode, "wb". Opening with plain "w" gives a text file, which refuses the bytes that dump produces. load is the reading function and takes only a file, and swapping the two arguments of dump passes the file where the object should be.

  6. A loop reads records from a binary file with pickle.load() until there are none left. Which exception signals that the end of the file has been reached?

    1. IndexError
    2. EOFError
    3. FileNotFoundError
    4. StopIteration
    Show answer

    Answer: B — EOFError

    Each pickle.load() call returns the next stored object, and a call made after the last one finds no data and raises EOFError, which is why the reading loop sits inside try and breaks on that exception. IndexError belongs to sequences, not files. FileNotFoundError would occur when opening a missing file, not at its end, and StopIteration is what a for loop uses internally, not what load raises.

  7. Assertion (A): Opening an existing file in "w" mode and closing it immediately leaves the file empty. Reason (R): In "w" mode the file pointer is placed at the end of the existing content.

    1. Both A and R are true, and R is the correct explanation of A
    2. Both A and R are true, but R is not the correct explanation of A
    3. A is true, but R is false
    4. A is false, but R is true
    Show answer

    Answer: C — A is true, but R is false

    Opening a file in "w" mode truncates it at once, so even with nothing written the old content is gone and A is true. R describes append mode: in "w" the pointer starts at the beginning of a now-empty file, so R is false. Picking "both true" mixes up w and a, the single most tested distinction in this chapter.

  8. marks.csv holds the line 101,Asha,88. The file is opened as f and read with for row in csv.reader(f): print(row). What is printed for that line?

    1. [101, 'Asha', 88]
    2. 101,Asha,88
    3. ('101', 'Asha', '88')
    4. ['101', 'Asha', '88']
    Show answer

    Answer: D — `['101', 'Asha', '88']`

    csv.reader splits each line at the commas and yields the fields as a list, and since a CSV file is text every field is a string, numbers included. The list with bare 101 and 88 assumes the module converts numbers, which it never does — int() is needed before arithmetic. The raw line is what reading the file directly would give, and the reader yields lists, not tuples.