10 Python File System Methods You Should Know

10 Python File System Methods You Should Know
This article is a guide for current and aspiring developers and data scientists. We’ll highlight 10 essential _os_ and _shutil_ commands so you can write scripts to automate interactions with the file system.

You can write Python programs to interact with the file system to do cool stuff. How to do so isn’t always super clear.

The file system is a bit like a house. Say you’re spring cleaning and you need to move boxes of notebooks from one room to another.

10 Python File System Methods You Should Know

The boxes are like directories. They hold things. In this case, notebooks.

10 Python File System Methods You Should Know

The notebooks are like files. You can read and write to them. You can put them in your directory boxes.

Capiche?

In this guide we’ll look at methods from the os and shutil modules. The os module is the primary Python module for interacting with the operating system. The shutil module also contains high-level file operations. For some reason you make directories with os but move and copy them with shutil. Go figure. 😏

Read Also: A Complete Machine Learning Project Walk-Through in Python

10 File System Methods

The list below follows this pattern:

Method — Description — Equivalent macOS Shell Command

Get Info

  • **os.getcwd()** — get the current working directory path as a string — pwd
  • **os.listdir()** — get the contents of the current working directory as a list of strings — ls
  • **os.walk("starting_directory_path")**— returns a generator with name and path info for directories and files in the the current directory and all subdirectories— no exact short CLI equivalent, but ls -R provides subdirectory names and the names of files within subdirectories

Change Things

  • **os.chdir("/absolute/or/relative/path")** — change the current working directory — cd
  • **os.path.join()**—create a path for later use — no short CLI equivalent
  • **os.makedirs("dir1/dir2")**— make directory —mkdir -p
  • **shutil.copy2("source_file_path", "destination_directory_path")** — copy a file or directory — cp
  • **shutil.move("source_file_path_"_, "destination_directory_path")**— move a file or directory — mv
  • **os.remove("my_file_path")** — remove a file — rm
  • **shutil.rmtree("my_directory_path")**— remove a directory and all files and directories in it —rm -rf

Let’s discuss.

Read Also: Python Tutorial - Python GUI Programming - Python GUI Examples (Tkinter Tutorial)

Get Info

**os.getcwd()**

os.getcwd() returns the current working directory as a string. That one is straightforward. 😄

**os.listdir()**

os.listdir() returns the contents of the current working directory as a list of strings. That one is also straightforward. 😄

**os.walk("my_start_directory")**

os.walk() creates a generator that can return information about the current directory and subdirectories. It works through the directories in the specified starting directory.

os.walk() returns the following items for each directory it traverses:

  1. current directory path as a string
  2. subdirectory names in the current directory as lists of strings
  3. filenames in current directory as a list of strings

It does this for each directory!

It’s often useful to use os.walk() with a for loop to iterate over the contents of a directory and its subdirectories. For example, the following code will print all files in the directories and subdirectories of the current working directory.

import oscwd = os.getcwd()for dir_path, dir_names, file_names in os.walk(cwd):
    for f in file_names:
        print(f)

That’s how we get info, now let’s look at commands that change the working directory or move, copy, or delete parts of the file system.

Read Also: Python Tutorial: Image processing with Python (Using OpenCV)

Change Things

**os.chdir("/absolute/or/relative/path")**

This method changes the current working directory to either the absolute or relative path provided.

If your code then makes other changes to the file system, it’s a good idea to handle any exceptions raised when using this method with try-except. Otherwise you might be deleting directories or files you don’t want deleted. 😢

**os.path.join()**

The [os.path](https://docs.python.org/3/library/os.path.html#module-os.path) module has a number of useful methods for common pathname manipulations. You can use it to find information about directory names and parts of directory names. The module also has methods to check whether a file or directory exists.

os.path.join() is designed to create a path that will work on most any operating system by joining multiple strings into one beautiful file path.😄

Here’s the description from the docs:

Join one or more path components intelligently. The return value is the concatenation of path and any members of *paths with exactly one directory separator (_os.sep_) following each non-empty part except the last…

Basically, if you are on a Unix or macOS system, os.path.join() sticks a forward slash (“/”) between each string you provide to create a path. If the operating system needs a “\” instead, then join knows to use a back slash.

os.path.join() also provides clear information to other developers that you are creating a path. Definitely use it instead of manual string concatenation to avoid looking like a rookie. 😉

**os.makedirs("dir1/dir2")**

os.makedirs() makes directories. The mkdir() method also makes directories, but it does not make intermediate directories. So I suggest you use os.makedirs().

**shutil.copy2("source_file", "destination")**

There are many ways to copy files and directories in Python. shutil.copy2() is a good choice because it tries to preserve as much of the source file’s metadata as possible.

10 Python File System Methods You Should Know

**shutil.move("source_file_"_, "destination")**

Use shutil.move() to change a file’s location. It uses copy2 as the default under the hood.

**os.remove("my_file_path")**

Sometimes you need to remove a file. os.remove() is your tool.

**shutil.rmtree("my_directory_path")**

shutil.rmtree() removes a directory and all files and directories in it.

10 Python File System Methods You Should Know

Careful with functions that delete things! You may want to print what will be deleted as a dry run with print(). Then run substitute in your remove function for print() when you’re sure it won’t delete the wrong files. Hat tip to Al Sweigart for that idea in Automate the Boring Stuff with Python.

Here’s the full list one more time.

Read Also: Python Programming Tutorial - Full Course for Beginners

10 File System Methods Recap

The list below follows this pattern: Method— Description — Equivalent macOS Shell Command

Get Info

  • **os.getcwd()** — get the current working directory path as a string — pwd
  • **os.listdir()** — get the contents of the current working directory as a list of strings — ls
  • **os.walk("starting_directory_path")**— returns a generator with name and path info for directories and files in the the current directory and all subdirectories — no exact short CLI equivalent, but ls -R provides subdirectory names and the names of files within subdirectories

Change Things

  • **os.chdir("/absolute/or/relative/path")** — change the current working directory — cd
  • **os.path.join()**—create a path for later use — no short CLI equivalent
  • **os.makedirs("dir1/dir2")**— make directory —mkdir-p
  • **shutil.copy2("source_file_path", "destination_directory_path")** — copy a file or directory — cp
  • **shutil.move("source_file_path_"_, "destination_directory_path")**— move a file or directory — mv
  • **os.remove("my_file_path")** — remove a file — rm
  • **shutil.rmtree("my_directory_path")**— remove a directory and all files and directories in it —rm -rf

Read Also: An A-Z of useful Python tricks

Wrap

Now you’ve seen the basics of interacting with the file system in Python. Try these commands in your IPython interpreter for quick feedback. Then explain them to someone else to solidify your knowledge. You’ll be less sore than if you had moved boxes of notebooks around your house. 🏠 But the exercise would have been good, so now you can hit the gym instead. 🏋️‍♀️

If you want to learn about reading and writing from files with Python check out the open function. Remember to use a context manager like so: with open(‘myfile’) as file: .😄

I hope you found this intro to Python file system manipulation useful.

Thanks for reading! 👏

Suggest:

Python 3 Tutorial for Beginners

Python Tutorial for Beginners

Python Scripting Tutorial for Beginners

Python Tutorial for Absolute Beginners - Learn Python in 2019

Python Tutorial for Beginners (2019) - Learn Python for Machine Learning and Web Development

Python Tutorials for Beginners - Learn Python Online