This series of posts is for those who have used a programming language before but are not familiar with Python. In previous posts we’ve introduced the language, installed it, and begun learning how to create our own data types (aka classes).

In this post we’ll dig into a very important topic: reading and writing files.

Let’s jump right in:

Line #3 assigns a full path and filename to filename. Note the letter r prepending the quoted string literal. We’ve been using f-strings (“format strings”), which have the letter f prepended. A prepended r makes a string “raw” — it disables the usual escape sequences [see Filename Notes below].

The built-in open function (line #6) — as its name says — opens a file. In this case, we’re reading an ordinary text file, so we can accept the default parameters for the function, and the only required parameter is the filename.

Assuming the open function succeeds, it returns a file object. As with any Python object, it has methods for interacting with it. On line #9 we call its read method to read the (entire!) file text into a variable named file_content (as one long string).

On line #12, we call the close method to close the file. (This isn’t strictly necessary when reading a file — Python will close the file when the script ends — but it’s good practice.)

Finally, on line #15, we print the text string.

When run, this prints:

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Which is the official ethic of Python.

This is a Python “Easter egg”. In a Python interactive prompt, type import this:

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
...
...

For another Easter Egg, type import antigravity. 😎

Filename Notes:

Escape sequences are two-letter codes that begin with a backslash (e.g. \n for a new-line character). When Python sees a backslash in a normal string, it expects the next character to tell it what special code to insert. If the next character isn’t a special code, Python inserts it as is. Either way, the two-letter pair is replaced by a single character, either a special code or the character after the backslash.

Windows filenames have backslashes, which Python normally sees as escape sequences. What used to happen is that when the character following a pathname backslash was a special code, the two characters were replaced by that special code. Otherwise, the two characters were replaced by just the character following the backslash. Either way, the backslashes were stripped from the filename. A raw string disabled escape sequence processing and preserved the filename intact.

Recently, that changed. Python recognizes Windows filenames, allows them as-is, but prints a warning:

When run, this prints:

Warning (from warnings module):
  File "C:\CJS\prj\Python\blog\hcc\source\fragment.py", line 2
    filename = "C:\Demo\HCC\Python\Python Zen.txt"
SyntaxWarning: invalid escape sequence '\D'
>>> 
======= RESTART: fragment.py =======
C:\Demo\HCC\Python\Python Zen.txt

C:\Demo\HCC\Python\Python Zen.txt

C:\Demo\HCC\Python\Python Zen.txt

C:\Demo\HCC\Python\Python Zen.txt

This is due to line #2, which has a Windows filename in a regular string. Line #7 shows how to avoid the warning when using a regular string — “escape” the backslashes by doubling them.

Line #12 uses a raw string, which treats the backslashes like any other character. Raw strings are useful with filenames and regular expressions (which often also have backslash characters).

Line #19 shows that raw and format strings can be combined. The prepended r and f characters can be in any order.

The first example read the entire file into a single string variable. This is fine for some kinds of processing, but sometimes it’s nice to break a text file into a list of individual lines. Python makes that easy, too:

The readlines method (line #9) returns the file text as a list object where each list item — a text string — is a line of the file.

When we open a text file, the returned file object is list-like:

When we put a (text) file object into a list context, it returns a list of the lines of the file. When run, both these code fragments print:

file_content is a <class 'list'>.
file_content has 21 lines.

Having a list-like file object means we can also do this:

When run, this prints the Python Zen text but with a minor snag — it’s double-spaced; there’s an unexpected blank line between each line of text.

This crops up in many languages. It’s because the print statement normally adds its own new-line character at the end. But each line of the file also has a new-line character, so each line is printed with two new-line characters.

One remedy is to tell print to print nothing at the end:

The end keyword parameter in the print statement lets us control what it does after printing whatever string(s) we send it. Here (line #10) we give it the empty string so it appends nothing (the default is the newline character, '\n').

Another is to use the string rstrip method to remove all whitespace from the right side (i.e. the end) of the string:

Both print the Python Zen text correctly.

At this point, one might reasonably point out that the two examples above seem to violate one of the tenets of Python Zen:

There should be one — and preferably only one — obvious way to do it.

Fair enough, but the two techniques are slightly different. The first prints what’s truly in the file. It just steps out of the way as far as adding anything to that output. The second alters the file text; it removes all whitespace from the right-hand side of the line. This includes spaces and tabs, so it can potentially remove significant characters (if we care about spaces or tabs).

Note also that we used the rstrip method rather than the strip method because we wanted to remove whitespace only from the end of the string. (There is also an lstrip method that removes whitespace only from the front of the string.)

Bottom line, which is the one obvious way depends on context. When in doubt, focus on the first seven lines of the Python Zen, especially the seventh (it’s rule #1 in my book).

We’ve been casual about opening and reading the file. Python raises an Exception if it can’t open the file or if it can’t read data from it.

That’s fine for quick-n-dirty scripts that only read a file. If Python does raise an Exception during opening or reading the file, the script terminates. That means the script never closes the file, but as mentioned above, Python closes any open files it finds when it terminates.

But when we’re writing to a file, we might want to be more certain. If our code raises an Exception for anything besides a file write error, we almost certainly want to be sure to close the file properly.

Wrapping file access in a try block is one way to ensure this:

We’re trying to write the big_list_of_data (line #12) to an output file, and each line of data needs to be processed by two functions (lines #15 and #16) before being written (line #19).

Note on line #6 that when we open a file for writing we set the mode keyword parameter to "w". Its default value is "r", which is why it’s not required for reading text files.

This code has a problem because we haven’t defined big_list_of_data or the two processing functions. Those are inside the try block, so when we run this, it prints:

opened: C:\Demo\HCC\Python\output.txt

ErrType: NameError
ErrText: name 'big_list_of_data' is not defined

closed: C:\Demo\HCC\Python\output.txt

In this case, Python can’t find the function, so line #12 make Python jump down to the try-except block (lines #21 to #24).

The else block (lines #26 and #27) executes only if the try block completes successfully. (This is similar to using else in a for or while loop — see part 4).

The finally block (lines #29 to #33) always executes after either the try block completes successfully (and then the else block if present), or after an Exception triggers the except block. This is a good place to put the close method (line #31).

With file objects, Python offers an even cleaner way to ensure proper file closure:

The with keyword invokes a context manager that automatically cleans up if Python raises an Exception within the context of the manager (i.e. within its block). The context manager for files ensures file closure, we no longer need to do it explicitly (or rely on Python to do it later).

Using the file object context manager, the file write example above becomes:

Which, of course, fails for the same reason as the one above but without the nice error-handling. To get that, we’d need to wrap the with block in a try block (which, in my mind, rather defeats the point of the with block, so I tend not to use with for files).

That said, here’s a more interesting example:

Lines #3 to #5 create the two filenames. Both files are in the same directory, so we use a common pathname for both. All three strings are raw to allow for the backslashes.

Lines #9 to #12 open the input file and read its contents into the file_text string. Reaching the print statement on line #14 means the read worked.

Lines #18 to #49 write the output file, a ROT13 encoded version of the input text. Reaching the print statement on line #51 means the write worked.

We do the ROT13 conversion by converting each character to its index (line #24) and testing that index to see if the character is uppercase (lines #27 to #35) or lowercase (lines #38 to #46). Within each of those we test to see if the character is A-M versus N-Z (or a-m versus n-z). If the character is in the lower half of the alphabet, we add 13 to its index; if it’s in the upper half, we subtract 13.

If the character isn’t alphabetic, we just write it (line #49).

Note that we use the built-in ord function to get a character’s index (aka ordinal number) and the built-in chr function to convert an index back to its character.

In the previous example we had a common pathname and used f-strings to insert it into both filenames. Python has a much better way to do this:

We import the os.path module, which is loaded with useful filename functions, a key one being the join function for building filenames.

When run, this prints:

input: C:\Demo\HCC\Python\Python Zen.txt
output: C:\Demo\HCC\Python\Rotated Zen.txt

Here’s a brief demo of some of the other path functions:

When run, this prints:

BasePath: C:\Demo
DemoPath: C:\Demo\HCC\Python

FileName: C:\Demo\HCC\Python\Python Zen.txt
basename: Python Zen.txt
dirname:  C:\Demo\HCC\Python

Exists: True
IsFile: True
IsDir:  False

FileSize: 878 bytes
DateCreated: 1774017627.2220998
DateUpdated: 1774017693.7498984
LastAccess:  1774032851.6244624

Split:
C:\Demo\HCC\Python
Python Zen.txt

SplitDrive:
C:
\Demo\HCC\Python\Python Zen.txt

SplitRoot:
C:
\
Demo\HCC\Python\Python Zen.txt

SplitExt:
C:\Demo\HCC\Python\Python Zen
.txt

Most should be obvious based on their outputs. The path.getctime, path.getmtime, and path.getatime functions (lines #22 to #24) return a floating-point value that is the number of seconds since the epoch (Midnight, January 1, 1970).

The four split functions (lines #27 to #45) return a tuple made of filename parts. The path.split function (line #28) breaks a filename into its path and name (same as the dirname and basename functions above do individually). The path.splitdrive function breaks a name into drive and path+name parts. The path.splitroot function returns three parts, the drive, the root (if present), and the path+name. Lastly, the path.splitext function divides the file extension (including the period) from the rest of the name.

Suppose we want a function that receives a filename and returns it with an output name (same path and name but with an “out” extension), and a logfile name (different path and extension but same base filename):

given:
C:\Demo\HCC\Python\data.json

function returns:
C:\Demo\HCC\Python\data.json
C:\Demo\HCC\Python\data.out
C:\Demo\logs\hcc\data.log

Easily done with the os.path module:

When run, this prints:

fpath = 'C:\\Demo\\HCC\\Python'
fname = 'data.json'
name = 'data'
ext = '.json'

Filename: C:\Demo\HCC\Python\data.json
Out-Name: C:\Demo\HCC\Python\data.out
Log-Name: C:\Demo\logs\hcc\data.log

Exactly as requested.

Note that Python also has the pathlib module which has even more powerful tools for dealing with filenames.

Our examples so far have involved text files — files with lines of printable text. We can also open binary files — files containing arbitrary byte values. Suppose we have a PNG image file we’d like to read:

To read a binary file, we set the mode argument to 'rb' (line #18). As above, using the file object’s read method (line #19) returns the entire file contents (as bytes). The order of the two characters in the mode argument doesn’t matter ('br' works fine), but we must supply the 'r' here (because just 'b' overrides the default 'r', so the open function doesn’t know whether to read or write the file).

Note that we cannot use the readlines method here because binary files do not have lines of text. We also cannot use a for loop with binary files because that tries to extract lines of text.

When run, this prints:

bitmap-pi.png has 10697 bytes.

read 10697 bytes.
data type is <class 'bytes'>.
datum type is <class 'int'>.

First 10 bytes:
0: 89 (-)
1: 50 (P)
2: 4e (N)
3: 47 (G)
4: 0d (-)
5: 0a (-)
6: 1a (-)
7: 0a (-)
8: 00 (-)
9: 00 (-)

Note that the bytes string is a list of integer values — the ordinal codes of the characters. This is why we use the built-in chr function (line #29) to get the string version of the character and test that for printability (line #30). We replace it with a '-' if it isn’t.

Sometimes, to open a file, we need to scan a folder (or folders) for their contents and proceed depending on what we find (and what we’re trying to do).

Python makes this easy using the listdir function from the os module (the “operating system” module with many useful low-level O/S functions):

When run (on my machine), this prints:

 1: bitmap-pi.png
 2: Christmas Carol.txt
 3: data.json
 4: demo.py
 5: dualnums-1.png
 6: dualnums-2.png
 7: dualnums-test.png
 8: first.py
 9: IDLE demo.lnk
10: IDLE first.lnk
11: multipart-1.txt
12: output.txt
13: Python Zen.txt
14: Rotated Zen.txt
15: second.py
16: Tk window 1.png
17: Tk window 2.png
18: Tk window 3.png
19: Tk window 4.png
20: Tk window 5a.png
21: Tk window 5b.png

The actual output, of course, depends on the actual folder contents.

The listdir function returns a list of files in (case-insensitive) alphabetical order, and note that it returns only the file names, not their paths. If we want full names, we need to use the path.join function:

And given a full path and name, we can list the size and dates:

We’d like commas in the file sizes, so we use an f-string in line #15 to convert the file size (an integer) to a string with commas.

As mentioned above, the getxtime functions return the number of seconds since the epoch, so on line #17 we use the ctime function from the time module to convert that to a standard (locale-based) string.

When run, this prints:

Files in {BasePath}:
 1: bitmap-pi.png       | 10,697 | Wed May 13 22:07:42 2020
 2: Christmas Carol.txt |  4,073 | Fri Mar 20 09:40:10 2026
 3: data.json           |    560 | Thu Feb 12 13:07:12 2026
 4: demo.py             |    817 | Thu Feb 12 12:51:23 2026
 5: dualnums-1.png      | 41,531 | Mon Nov 17 21:38:02 2025
 6: dualnums-2.png      | 48,358 | Mon Nov 17 21:26:44 2025
 7: dualnums-test.png   | 48,887 | Mon Nov 17 21:38:01 2025
 8: first.py            |     89 | Thu Feb 12 13:39:03 2026
 9: IDLE demo.lnk       |  3,154 | Thu Feb 12 13:12:49 2026
10: IDLE first.lnk      |  1,170 | Thu Feb 12 16:32:04 2026
11: multipart-1.txt     |  3,509 | Fri Sep  5 17:00:42 2025
12: output.txt          |      0 | Fri Mar 20 13:32:37 2026
13: Python Zen.txt      |    878 | Fri Mar 20 09:41:33 2026
14: Rotated Zen.txt     |    878 | Fri Mar 20 13:54:11 2026
15: second.py           |      9 | Thu Feb 12 15:28:13 2026
16: Tk window 1.png     |  2,027 | Thu Nov 20 21:02:36 2025
17: Tk window 2.png     |  3,194 | Thu Nov 20 21:18:04 2025
18: Tk window 3.png     |  3,924 | Fri Nov 21 11:11:10 2025
19: Tk window 4.png     |  3,197 | Thu Nov 20 21:47:15 2025
20: Tk window 5a.png    |  5,159 | Thu Nov 20 22:00:06 2025
21: Tk window 5b.png    |  5,218 | Thu Nov 20 22:07:35 2025

The listdir function returns both file names and folder names, which we usually want to differentiate (folder names don’t have sizes, for instance). The path.isfile function and the path.isdir function allow us to separate the two:

Line #11 uses a list comprehension to create a list of full path/file names. This is the equivalent of:

Which does the same thing but takes several lines of code.

Lines #14 and #17 use the built-in filter function to extract the directory names (line #14) and file names (line #17). Because the filter function is one of those functions that returns an iterator rather than a list, we use the list constructor to get an actual list object.

Because we use both results in for loops, we could have just done this:

But in most cases, we’d want the lists for further processing.

As you might imagine, there’s a great deal more we can do with listdir and related functions. As an exercise, try your hand at creating a recursive file scanner that descends into the sub-folders and lists their files as well.

Link: Zip file containing all code fragments used in this post.

∅