Module 3 Python

CHAPTER 2: READING AND WRITING FILES

1. Files and File Paths
2. The [Link] Module
3. The File Reading/ Writing Process
4. Saving Variables with the shelve Module
5. Saving Variables with the [Link]() Function
6. Project: Generating Random Quiz Files
7. Project: Multiclipboard

2.1 Files and File Paths

 A file has two key properties: a filename (usually written as one word) and a path.
 The path specifies the location of a file on the computer.
 Ex: C:\Users\asweigart\Documents\[Link]
 The part of the filename after the last period is called the file’s extension and tells the file’s type.
 Ex: [Link] is a Word document.
 Folders can contain files and other folders.
 Ex: [Link] is in the Documents folder, which is inside the asweigart folder, which is inside
the Users folder.

Backslash on Windows and Forward Slash on OS X and Linux

Dept. of CSE, CBIT, Kolar
19
 Module 3 Python

 On Windows, paths are written using backslashes (\) as the separator between folder names.
 OS X and Linux, however, use the forward slash (/) as their path separator.
 We can handle both using [Link]() function.

 [Link]('usr', 'bin', 'spam') returned 'usr\\bin\\spam'. (Notice that the backslashes are doubled because
each backslash needs to be escaped by another backslash character.)
 If we had called this function on OS X or Linux, the string would have been 'usr/bin/spam'.
 The [Link]() function is helpful if we need to create strings for filenames.

The Current Working Directory

 We can get the current working directory as a string value with the [Link]() function and change it
with [Link]().

 Here, the current working directory is set to C:\Python34, so the filename [Link] refers
to C:\Python34\[Link]. When we change the current working directory
to C:\Windows, [Link] is interpreted as C:\ Windows\[Link].
 Python will display an error if we try to change to a directory that does not exist.

Absolute vs. Relative Paths

Dept. of CSE, CBIT, Kolar

20
 Module 3 Python

 There are two ways to specify a file path.

1. An absolute path, which always begins with the root folder
2. A relative path, which is relative to the program’s current working directory
 There are also the dot (.) and dot-dot (..) folders. These are not real folders but special names that can be
used in a path. A single period (“dot”) for a folder name is shorthand for “this directory.” Two periods
(“dot-dot”) means “the parent folder.”
 The below figure is an example of some folders and files.
 When the current working directory is set to C:\bacon, the relative paths for the other folders and files are
set as they are in the figure.
 The .\ at the start of a relative path is optional. For example, .\[Link] and [Link] refer to the same file.

Creating New Folders with [Link]()

 Our programs can create new folders (directories) with the [Link]() function.

 This will create not just the C:\delicious folder but also a walnut folder inside C:\delicious and
a waffles folder inside C:\delicious\walnut.
 That is, [Link]() will create any necessary intermediate folders in order to ensure that the full path
exists. The below figure shows this hierarchy of folders.

2.2 The [Link] Module

Dept. of CSE, CBIT, Kolar

21
 Module 3 Python

 The [Link] module contains many helpful functions related to filenames and file paths.
 Since [Link] is a module inside the os module, we can import it by simply running import os.

Handling Absolute and Relative Paths

 The [Link] module provides functions for returning the absolute path of a relative path and for checking
whether a given path is an absolute path.
1. Calling [Link](path) will return a string of the absolute path of the argument. This is an easy
way to convert a relative path into an absolute one.
2. Calling [Link](path) will return True if the argument is an absolute path and False if it is a
relative path.
3. Calling [Link](path, start) will return a string of a relative path from the start path to path.
If start is not provided, the current working directory is used as the start path.

 Since C:\Python34 was the working directory when [Link]() was called, the “single-dot” folder
represents the absolute path 'C:\\Python34'.

 Calling [Link](path) will return a string of everything that comes before the last slash in
the path argument.
 Calling [Link](path) will return a string of everything that comes after the last slash in
the path argument.

 Example:

Dept. of CSE, CBIT, Kolar

22
 Module 3 Python

 If we need a path’s dir name and base name together, we can just call [Link]() to get a tuple value
with these two strings, like so:

 Notice that we could create the same tuple by calling [Link]() and [Link]() and
placing their return values in a tuple.

 But [Link]() is a nice shortcut if we need both values.

 Also, note that [Link]() does not take a file path and return a list of strings of each folder. For that,
use the split() string method and split on the string in [Link]. Recall from earlier that the [Link] variable is
set to the correct folder-separating slash for the computer running the program. Example:

 On OS X and Linux systems, there will be a blank string at the start of the returned list:

 The split() string method will work to return a list of each part of the path. It will work on any operating
system if you pass it [Link].

Finding File Sizes and Folder Contents

 Calling [Link](path) will return the size in bytes of the file in the path argument.
 Calling [Link](path) will return a list of filename strings for each file in the path argument. (Note that
this function is in the os module, not [Link].)

 As we can see, the [Link] program on my computer is 776,192 bytes in size, and we have a lot of files
in C:\Windows\system32. If we want to find the total size of all the files in this directory, we can
use [Link]() and [Link]() together.

Dept. of CSE, CBIT, Kolar

23
 Module 3 Python

 As we loop over each filename in the C:\Windows\System32 folder, the totalSize variable is incremented
by the size of each file.

Checking Path Validity

 The [Link] module provides functions to check whether a given path exists and whether it is a file or
folder.
1. Calling [Link](path) will return True if the file or folder referred to in the argument exists and
will return False if it does not exist.
2. Calling [Link](path) will return True if the path argument exists and is a file and will
return False otherwise.
3. Calling [Link](path) will return True if the path argument exists and is a folder and will
return False otherwise.

 We can determine whether there is a DVD or flash drive currently attached to the computer by checking
for it with the [Link]() function.

2.3 The File Reading/ Writing process

 Text files with the .txt extension or Python script files with the .py extension are examples of plaintext
files. These can be opened with Windows’s Notepad or OS X’s TextEdit application. Our programs can
easily read the contents of plaintext files and treat them as an ordinary string value.
 Binary files are all other file types, such as word processing documents, PDFs, images, spreadsheets, and
executable programs. If we open a binary file in Notepad or TextEdit, it will look like scrambled nonsense,
like in figure.

Dept. of CSE, CBIT, Kolar

24
 Module 3 Python

 Since every different type of binary file must be handled in its own way, this book will not go into reading
and writing raw binary files directly.
 There are three steps to reading or writing files in Python.
1. Call the open() function to return a File object.
2. Call the read() or write() method on the File object.
3. Close the file by calling the close() method on the File object.

Opening Files with the open() Function

 To open a file with the open() function, we pass it a string path indicating the file you want to open; it can
be either an absolute or relative path. The open() function returns a File object.
 Try it by creating a text file named [Link] using Notepad or TextEdit. Type Hello world! as the content
of this text file and save it in your user home folder.

 Make sure to replace your_home_folder with your computer username. For example, if username
is asweigart, so I’d enter 'C:\\Users\\asweigart\\ [Link]' on Windows.
 Both these commands will open the file in “reading plaintext” mode, or read mode for short.
 When a file is opened in read mode, Python lets only read data from the file; we can’t write or modify it
in any way.
 e can explicitly specify the mode by passing the string value 'r' as a second argument to open().
So open('/Users/asweigart/ [Link]', 'r') and open('/Users/asweigart/[Link]') do the same thing.
 The call to open() returns a File object. A File object represents a file on your computer; it is simply
another type of value in Python, much like the lists and dictionaries you’re already familiar with.

Reading the Contents of Files

 If we want to read the entire contents of a file as a string value, use the File object’s read() method. Let’s
continue with the [Link] File object you stored in helloFile.

Dept. of CSE, CBIT, Kolar

25
 Module 3 Python

 If we think of the contents of a file as a single large string value, the read() method returns the string that
is stored in the file.
 Alternatively, we can use the readlines() method to get a list of string values from the file, one string for
each line of text. For example, create a file named [Link] in the same directory as [Link] and write
the following text in it:

 Make sure to separate the four lines with line breaks. Then enter the following into the interactive shell:

 Note that each of the string values ends with a newline character, \n, except for the last line of the file. A
list of strings is often easier to work with than a single large string value.

Writing to Files

 Python allows us to write content to a file in a way similar to how the print() function “writes” strings to
the screen.
 We can’t write to a file if we have opened in read mode, though. Instead, we need to open it in “write
plaintext” mode or “append plaintext” mode, or write mode and append mode for short.
 Write mode will overwrite the existing file and start from scratch, just like when we overwrite a variable’s
value with a new value. Pass 'w' as the second argument to open() to open the file in write mode.
 Append mode, on the other hand, will append text to the end of the existing file.
 If the filename passed to open() does not exist, both write and append mode will create a new, blank file.
 After reading or writing a file, call the close() method before opening the file again.

Dept. of CSE, CBIT, Kolar

26
 Module 3 Python

 First, we open [Link] in write mode. Since there isn’t a [Link] yet, Python creates one. Calling
write() on the opened file and passing write() the string argument 'Hello world! /n' writes the string to the
file and returns the number of characters written, including the newline. Then we close the file.
 To add text to the existing contents of the file instead of replacing the string we just wrote, we open the
file in append mode. We write 'Bacon is not a vegetable.' to the file and close it. Finally, to print the file
contents to the screen, we open the file in its default read mode, call read(), store the resulting File object
in content, close the file, and print content.
 Note that the write() method does not automatically add a newline character to the end of the string like
the print() function does. We will have to add this character yourself.

2.4 Saving variables with the Shelve Module

 We can save variables in our Python programs to binary shelf files using the shelve module.
 This way, our program can restore data to variables from the hard drive.
 The shelve module will let us add Save and Open features to your program.
 For example, if we ran a program and entered some configuration settings, we could save those settings to
a shelf file and then have the program load them the next time it is run.

 To read and write data using the shelve module, first import shelve.
 Call [Link]() and pass it a filename, and then store the returned shelf value in a variable.
 We can make changes to the shelf value as if it were a dictionary. When we’re done, call close() on the
shelf value. Here, our shelf value is stored in shelfFile.
 We create a list cats and write shelfFile['cats'] = cats to store the list in shelfFile as a value associated with
the key 'cats' (like in a dictionary). Then we call close() on shelfFile.
 After running the previous code on Windows, you will see three new files in the current working
directory: [Link], [Link], and [Link].
 On OS X, only a single [Link] file will be created.
 These binary files contain the data you stored in our shelf. The format of these binary files is not important;
we only need to know what the shelve module does, not how it does it. The module frees you from
worrying about how to store your program’s data to a file.
 Your programs can use the shelve module to later reopen and retrieve the data from these shelf files. Shelf
values don’t have to be opened in read or write mode—they can do both once opened.

 Here, we open the shelf files to check that our data was stored correctly. Entering shelfFile['cats'] returns
the same list that we stored earlier, so we know that the list is correctly stored, and we call close().

Dept. of CSE, CBIT, Kolar

27
 Module 3 Python

 Just like dictionaries, shelf values have keys() and values() methods that will return list-like values of the
keys and values in the shelf. Since these methods return list-like values instead of true lists, we should
pass them to the list() function to get them in list form. Enter the following into the interactive shell:

 Plaintext is useful for creating files that you’ll read in a text editor such as Notepad or TextEdit, but if you
want to save data from your Python programs, use the shelve module.

2.5 Saving variables with the [Link]() Function

 Using [Link]() will give us a string that we can write to .py file. This file will be our very own
module that we can import whenever you want to use the variable stored in it.
 For example, enter the following into the interactive shell:

 Here, we import pprint to let us use [Link](). We have a list of dictionaries, stored in a variable cats.
To keep the list in cats available even after we close the shell, we use [Link]() to return it as a
string. Once we have the data in cats as a string, it’s easy to write the string to a file, which we’ll
call [Link].
 The modules that an import statement imports are themselves just Python scripts. When the string
from [Link]() is saved to a .py file, the file is a module that can be imported just like any other.
 And since Python scripts are themselves just text files with the .py file extension, our Python programs
can even generate other Python programs. We can then import these files into scripts.

 The benefit of creating a .py file (as opposed to saving variables with the shelve module) is that because it
is a text file, the contents of the file can be read and modified by anyone with a simple text editor.

Dept. of CSE, CBIT, Kolar

28
 Module 3 Python

 For most applications, however, saving data using the shelve module is the preferred way to save variables
to a file.

2.6 Project: Generating Random Quiz Files

 Say you’re a geography teacher with 35 students in your class and you want to give a pop quiz on US state
capitals. Alas, your class has a few bad eggs in it, and you can’t trust the students not to cheat. You’d like
to randomize the order of questions so that each quiz is unique, making it impossible for anyone to crib
answers from anyone else. Of course, doing this by hand would be a lengthy and boring affair. Fortunately,
you know some Python.
 Here is what the program does:
1. Creates 35 different quizzes.
2. Creates 50 multiple-choice questions for each quiz, in random order.
3. Provides the correct answer and three random wrong answers for each question, in random order.
4. Writes the quizzes to 35 text files.
5. Writes the answer keys to 35 text files.
6. This means the code will need to do the following:
7. Store the states and their capitals in a dictionary.
8. Call open(), write(), and close() for the quiz and answer key text files.
9. Use [Link]() to randomize the order of the questions and multiple-choice options.

Step 1: Store the Quiz Data in a Dictionary

 The first step is to create a skeleton script and fill it with your quiz data. Create a file
named [Link], and make it look like the following:

Dept. of CSE, CBIT, Kolar

29
 Module 3 Python

 Since this program will be randomly ordering the questions and answers, you’ll need to import
the random module ❶ to make use of its functions. The capitals variable ❷ contains a dictionary with
US states as keys and their capitals as values. And since you want to create 35 quizzes, the code that
actually generates the quiz and answer key files (marked with TODO comments for now) will go inside a
for loop that loops 35 times ❸. (This number can be changed to generate any number of quiz files.)

Step 2: Create the Quiz File and Shuffle the Question Order
 Now it’s time to start filling in those TODOs.
 The code in the loop will be repeated 35 times—once for each quiz—so you have to worry about only one
quiz at a time within the loop. First you’ll create the actual quiz file. It needs to have a unique filename
and should also have some kind of standard header in it, with places for the student to fill in a name, date,
and class period. Then you’ll need to get a list of states in randomized order, which can be used later to
create the questions and answers for the quiz.

 The filenames for the quizzes will be capitalsquiz<N>.txt, where <N> is a unique number for the quiz that
comes from quizNum, the for loop’s counter. The answer key for capitalsquiz<N>.txt will be stored in a
text file named capitalsquiz_answers<N>.txt. Each time through the loop, the %s placeholder in
'capitalsquiz%[Link]' and 'capitalsquiz_answers%[Link]' will be replaced by (quizNum + 1), so the first quiz and
answer key created will be [Link] and capitalsquiz_answers1.txt. These files will be created with
calls to the open() function at ❶ and ❷, with 'w' as the second argument to open them in write mode.
 The write() statements at ❸ create a quiz header for the student to fill out. Finally, a randomized list of
US states is created with the help of the [Link]() function ❹, which randomly reorders the values
in any list that is passed to it.

Step 3: Create the Answer Options

 Now you need to generate the answer options for each question, which will be multiple choice from A to
D. You’ll need to create another for loop—this one to generate the content for each of the 50 questions on

Dept. of CSE, CBIT, Kolar

30
 Module 3 Python

the quiz. Then there will be a third for loop nested inside to generate the multiple-choice options for each
question. Make your code look like the following:

 The correct answer is easy to get—it’s stored as a value in the capitals dictionary ❶. This loop will loop
through the states in the shuffled states list, from states[0] to states[49], find each state in capitals, and
store that state’s corresponding capital in correctAnswer.
 The list of possible wrong answers is trickier. You can get it by duplicating all the values in
the capitals dictionary ❷, deleting the correct answer ❸, and selecting three random values from this list
❹. The [Link]() function makes it easy to do this selection. Its first argument is the list you want
to select from; the second argument is the number of values you want to select. The full list of answer
options is the combination of these three wrong answers with the correct answers ❺. Finally, the answers
need to be randomized ❻ so that the correct response isn’t always choice D.

Step 4: Write Content to the Quiz and Answer Key Files

 All that is left is to write the question to the quiz file and the answer to the answer key file. Make your
code look like the following:

Dept. of CSE, CBIT, Kolar

31
 Module 3 Python

 A for loop that goes through integers 0 to 3 will write the answer options in the answerOptions list ❶.
The expression 'ABCD'[i] at ❷ treats the string 'ABCD' as an array and will evaluate to 'A','B', 'C', and
then 'D' on each respective iteration through the loop.
 In the final line ❸, the expression [Link](correctAnswer) will find the integer index of the
correct answer in the randomly ordered answer options,
and 'ABCD'[[Link](correctAnswer)] will evaluate to the correct answer’s letter to be
written to the answer key file.
 After you run the program, this is how your [Link] file will look, though of course your questions
and answer options may be different from those shown here, depending on the outcome of
your [Link]() calls:

 The corresponding capitalsquiz_answers1.txt text file will look like this:

2.7 Project: Multiclipboard

 Say you have the boring task of filling out many forms in a web page or software with several text fields.
The clipboard saves you from typing the same text over and over again. But only one thing can be on the
clipboard at a time. If you have several different pieces of text that you need to copy and paste, you have
to keep highlighting and copying the same few things over and over again.
 You can write a Python program to keep track of multiple pieces of text. This “multiclipboard” will be
named [Link] (since “mcb” is shorter to type than “multiclipboard”). The .pyw extension means that
Python won’t show a Terminal window when it runs this program. (See Appendix B for more details.)
 The program will save each piece of clipboard text under a keyword. For example, when you run py
[Link] save spam, the current contents of the clipboard will be saved with the keyword spam. This text

Dept. of CSE, CBIT, Kolar

32
 Module 3 Python

can later be loaded to the clipboard again by running py [Link] spam. And if the user forgets what
keywords they have, they can run py [Link] list to copy a list of all keywords to the clipboard.
 Here’s what the program does:
1. The command line argument for the keyword is checked.
2. If the argument is save, then the clipboard contents are saved to the keyword.
3. If the argument is list, then all the keywords are copied to the clipboard.
4. Otherwise, the text for the keyword is copied to the clipboard.
5. This means the code will need to do the following:
6. Read the command line arguments from [Link].
7. Read and write to the clipboard.
8. Save and load to a shelf file.
 If you use Windows, you can easily run this script from the Run... window by creating a batch file
named [Link] with the following content:
@[Link] C:\Python34\[Link] %*

Step 1: Comments and Shelf Setup

 Let’s start by making a skeleton script with some comments and basic setup. Make your code look like
the following:

 It’s common practice to put general usage information in comments at the top of the file ❶. If you ever
forget how to run your script, you can always look at these comments for a reminder. Then you import
your modules ❷. Copying and pasting will require the pyperclip module, and reading the command line
arguments will require the sys module. The shelve module will also come in handy: Whenever the user
wants to save a new piece of clipboard text, you’ll save it to a shelf file. Then, when the user wants to
paste the text back to their clipboard, you’ll open the shelf file and load it back into your program. The
shelf file will be named with the prefix mcb ❸.

Step 2: Save Clipboard Content with a Keyword

 The program does different things depending on whether the user wants to save text to a keyword, load
text into the clipboard, or list all the existing keywords. Let’s deal with that first case. Make your code
look like the following:

Dept. of CSE, CBIT, Kolar

33
 Module 3 Python

 If the first command line argument (which will always be at index 1 of the [Link] list) is 'save' ❶, the
second command line argument is the keyword for the current content of the clipboard. The keyword will
be used as the key for mcbShelf, and the value will be the text currently on the clipboard ❷.
 If there is only one command line argument, you will assume it is either 'list' or a keyword to load content
onto the clipboard. You will implement that code later. For now, just put a TODO comment there ❸.

Step 3: List Keywords and Load a Keyword’s Content

 Finally, let’s implement the two remaining cases: The user wants to load clipboard text in from a keyword,
or they want a list of all available keywords. Make your code look like the following:
 If there is only one command line argument, first let’s check whether it’s 'list' ❶. If so, a string
representation of the list of shelf keys will be copied to the clipboard ❷. The user can paste this list into
an open text editor to read it.
 Otherwise, you can assume the command line argument is a keyword. If this keyword exists in the
mcbShelf shelf as a key, you can load the value onto the clipboard ❸.
 And that’s it! Launching this program has different steps depending on what operating system your
computer uses. See Appendix B for details for your operating system.
 Recall the password locker program you created in Chapter 6 that stored the passwords in a dictionary.
Updating the passwords required changing the source code of the program. This isn’t ideal because average
users don’t feel comfortable changing source code to update their software. Also, every time you modify
the source code to a program, you run the risk of accidentally introducing new bugs. By storing the data
for a program in a different place than the code, you can make your programs easier for others to use and
more resistant to bugs.

CHAPTER 3: ORGANIZING FILES

1. The shutil Module
2. Walking a Directory Tree
3. Compressing Files with the zipfile Module
4. Project: Renaming Files with American-Style Dates to European-Style Dates

5. Project: Backing Up a Folder into a ZIP File

Dept. of CSE, CBIT, Kolar

34
 Module 3 Python

5.1 The shutil Module

 The shutil (or shell utilities) module has functions to let us copy, move, rename, and delete files in your
Python programs. To use the shutil functions, we will first need to use import shutil.

Copying Files and Folders

 The shutil module provides functions for copying files, as well as entire folders.
 Calling [Link](source, destination) will copy the file at the path source to the folder at the
path destination. (Both source and destination are strings.)
 If destination is a filename, it will be used as the new name of the copied file. This function returns a string
of the path of the copied file.

 The first [Link]() call copies the file at C:\[Link] to the folder C:\delicious.
 The return value is the path of the newly copied file.
 Note that since a folder was specified as the destination ❶, the original [Link] filename is used for the
new, copied file’s filename.
 The second [Link]() call ❷ also copies the file at C:\[Link] to the folder C:\delicious but gives the
copied file the name [Link].
 While [Link]() will copy a single file, [Link]() will copy an entire folder and every folder and
file contained in it.
 Calling [Link](source, destination) will copy the folder at the path source, along with all of its
files and subfolders, to the folder at the path destination.
 The source and destination parameters are both strings. The function returns a string of the path of the
copied folder.

 The [Link]() call creates a new folder named bacon_backup with the same content as the
original bacon folder.

Moving and Renaming Files and Folders

 Calling [Link](source, destination) will move the file or folder at the path source to the
path destination and will return a string of the absolute path of the new location.
 If destination points to a folder, the source file gets moved into destination and keeps its current filename

Dept. of CSE, CBIT, Kolar

35