Reading files¶

Goal: open a text file, read its contents line by line, and turn those lines into data your program can use.

Why this matters¶

Every program you have written so far forgets everything the moment it ends. The shopping list, the phone book, the high score: all gone. Real programs read their data from files and save it back, so it is still there tomorrow. This lesson covers reading. The next one covers writing.

Set up a file to read¶

Programs read files from the folder they are run in. Make a file called shopping.txt in the same folder as your Python files, using Visual Studio Code or any text editor, with this content:

shopping.txt

eggs
bread
apples
oat milk
coffee

The .txt ending means plain text. That is the kind of file this module works with: no formatting, no images, just characters and line breaks. Save it, and make sure your terminal is in the same folder, exactly as you did to run hello.py in Module 1.

Reading the whole file¶

read_whole.py

with open("shopping.txt") as file:
    contents = file.read()

print(contents)
print(len(contents))

eggs
bread
apples
oat milk
coffee

34

Take the first line in two halves. open("shopping.txt") opens the file and gives you a file object. with ... as file: stores that object in the variable file for the duration of the indented block, and closes the file automatically when the block ends. Always use with to open files. It means you never forget to close them, which matters when you start writing to them.

Inside the block, file.read() returns the entire contents as one string. After the block, the file is closed, but the string is still yours to use.

Why is the length 34 when you can count 28 letters? Because line breaks are characters too. Each line ends in an invisible newline character, written \n in Python. Five lines, five newlines, plus 29 visible characters including the space in oat milk. The blank line in the output is the final newline followed by the one print() adds.

Reading line by line¶

Reading everything at once is fine for small files. More often you want to deal with one line at a time, and a file object lets you loop over it directly:

read_lines.py

with open("shopping.txt") as file:
    for line in file:
        print(line)

eggs

bread

apples

oat milk

coffee

Each iteration, line is one line of the file. But the output is double spaced. That is the newline again: each line still ends in \n, and then print() adds its own. The fix is .strip(), which you met in Module 2. It removes spaces and newlines from both ends:

read_lines_strip.py

number = 1

with open("shopping.txt") as file:
    for line in file:
        item = line.strip()
        print(f"{number}. {item}")
        number += 1

1. eggs
2. bread
3. apples
4. oat milk
5. coffee

Almost every line you read from a file should be stripped straight away. Make it a habit.

From lines to a list¶

Combine the loop with .append() and the file becomes a list, ready for everything Module 4 taught you:

read_into_list.py

items = []

with open("shopping.txt") as file:
    for line in file:
        items.append(line.strip())

print(items)
print(f"{len(items)} items to buy.")
items.sort()
print(items)

['eggs', 'bread', 'apples', 'oat milk', 'coffee']
5 items to buy.
['apples', 'bread', 'coffee', 'eggs', 'oat milk']

Note that the printing and sorting happen after the with block, once the file is closed. Read the file into memory first, then work with the data. That keeps file handling in one small place.

Reading structured data¶

A line can hold more than one value. A common simple format puts values on one line separated by commas. Make a second file called scores.txt:

scores.txt

Ada,82
Grace,95
Linus,77
Guido,60
Margaret,91

Each line has a name and a score. .split(",") from Module 4 breaks the line at the comma, and since every line has exactly two pieces, you can unpack them into two variables:

read_scores.py

total = 0
count = 0

with open("scores.txt") as file:
    for line in file:
        name, score = line.strip().split(",")
        score = int(score)
        print(f"{name} scored {score}")
        total += score
        count += 1

print(f"Average: {total / count}")

Ada scored 82
Grace scored 95
Linus scored 77
Guido scored 60
Margaret scored 91
Average: 81.0

Two things to notice. First, .strip() comes before .split(","), so the newline does not end up glued to the score. Second, everything read from a file is a string, just like everything from input(). "95" must become 95 with int() before you can add it to a total.

Files like this, with one record per line and commas between values, are called CSV files, short for comma-separated values. Spreadsheets can save and open them, which makes this a handy bridge between your programs and other people's data.

A reusable reading function¶

Reading a file into a list of stripped lines is so common that it deserves a function, and Module 5 taught you how:

read_function.py

def read_lines(filename):
    """Return the lines of a file as a list, without the newline characters."""
    lines = []
    with open(filename) as file:
        for line in file:
            lines.append(line.strip())
    return lines

shopping = read_lines("shopping.txt")
story = read_lines("story.txt")

print(f"Shopping list: {len(shopping)} lines")
print(f"Story: {len(story)} lines")
print(f"First line of the story: {story[0]}")

Shopping list: 5 lines
Story: 4 lines
First line of the story: The little robot woke up in a quiet room.

This uses a third file, story.txt, which you will need for the exercises:

story.txt

The little robot woke up in a quiet room.
It looked at the door and then at the window.
The window was open, so the robot rolled to the window and looked out.
Outside, the little garden was full of birds, and the robot was happy.

read_lines() takes the filename as a parameter, so one function reads any file. Put it at the top of your programs from now on whenever you need to read a file.

When the file is not there¶

Misspell the filename, or run the program from the wrong folder, and you get this:

Traceback (most recent call last):
  File "read_whole.py", line 1, in <module>
    with open("shoping.txt") as file:
FileNotFoundError: [Errno 2] No such file or directory: 'shoping.txt'

Python looked for shoping.txt in the folder the terminal is in, and it was not there. Check the spelling, check the folder, and check that the file really ends in .txt and not .txt.txt, a common surprise on Windows when file extensions are hidden. Module 7 shows how a program can catch this error and ask for a different name instead of crashing.

Try it¶

Create a file names.txt with five names, one per line. Write a program that reads it into a list, prints how many names there are, and prints them in alphabetical order with a number in front of each.

Common mistakes¶

FileNotFoundError: [Errno 2] No such file or directory

The file is not in the folder your terminal is in, or the name is misspelled. Use cd to move into the right folder.

Every line prints with a blank line after it

Lines from a file end in a newline character. Call .strip() on each line.

TypeError: can only concatenate str (not \"int\") to str, or the total is wrong

Values read from a file are strings. Convert with int() or float() before doing math.

ValueError: not enough values to unpack (expected 2, got 1)

A line did not have a comma, often a blank line at the end of the file. Delete the empty line, or skip lines where line.strip() == "".

ValueError: I/O operation on closed file

You used file after the with block ended. Read the data into a variable inside the block and use the variable after.

Exercises¶

Count words. Print how many lines and how many words story.txt contains. .split() gives the words in a line.
Top scorer. Read scores.txt and print the name of the person with the highest score.
Longest line. Print the longest line in story.txt and how many characters it has.
Word frequency. Count how often each word appears in story.txt, ignoring case, and print the words that appear three times or more. Punctuation is tricky: robot. and robot should count as the same word.

Solution 1

lines = 0
words = 0

with open("story.txt") as file:
    for line in file:
        lines += 1
        words += len(line.split())

print(f"{lines} lines, {words} words.")

Solution 2

best_name = ""
best_score = 0

with open("scores.txt") as file:
    for line in file:
        name, score = line.strip().split(",")
        score = int(score)
        if score > best_score:
            best_name = name
            best_score = score

print(f"Top scorer: {best_name} with {best_score}.")

The find-the-best pattern from Module 4, applied to lines from a file instead of items in a list.

Solution 3

longest = ""

with open("story.txt") as file:
    for line in file:
        line = line.strip()
        if len(line) > len(longest):
            longest = line

print(f"Longest line ({len(longest)} characters):")
print(longest)

Solution 4

counts = {}

with open("story.txt") as file:
    for line in file:
        for word in line.lower().split():
            word = word.strip(".,")
            if word not in counts:
                counts[word] = 0
            counts[word] += 1

for word, count in counts.items():
    if count >= 3:
        print(f"{word}: {count}")

.strip() can take an argument: word.strip(".,") removes any dots and commas from the ends of the word, which is what turns robot. into robot. The two nested loops read as for each line, for each word in that line.

Summary¶

with open("name.txt") as file: opens a file and closes it automatically at the end of the block.
file.read() gives the whole file as one string. for line in file: gives one line at a time.
Every line ends in a newline character. Call .strip() on it.
Everything read from a file is a string. Convert numbers with int() or float().
.split(",") breaks a line of comma-separated values into pieces.
Read the file into a list inside the with block, then work with the list after it.

Next: Writing files