Python's many command-line utilities

Did you know that some Python modules can double-up as handy command-line tools?

For example, you can run Python's webbrowser module from the command-line to open up a given URL in your default web browser:

$ python -m webbrowser https://pym.dev/p
Opening in existing browser session.

The Python standard library includes many such module-script hybrids.

Below is a complete list of every module in Python that can be run as a command-line script.

Feel free to jump right to the full list of all scripts in Python at the end.

How -m works

Running Python with the -m command-line argument tells Python to run a given Python module as if it were a Python script.

Some modules do something at import time. For example the antigravity module will open up a web browser for an XKCD comic. Running this module from the command-line would do the same thing as importing it:

$ python -m antigravity

This is called an "import side effect" and most modules avoid import side effects. Fun Easter egg modules like antigravity and this are the exception.

Modules that avoid import side effects need a different mechanism to change their behavior when run as a command-line script or when imported as a module. Python uses a __name__ variable to distinguish between importing a module and running a module as a script.

When Python runs a module as a script, it sets the module's name to the string "__main__" (normally __name__ would contain the module's actual name). See more in defining a main function in Python.

For packages, Python also looks for a __main__.py file to run (there's one in the zipfile package for example).

This distinction between module versus script allows for some really nifty command-line tools.

General-purpose CLI tools

The first tools we'll look at are tools that I use even when I'm not working with Python code.

These are Python's most helpful general-purpose command-line tools.

http.server

Running the http.server module as a script will start a web server on port 8000 that hosts files from the current directory. I use this all the time to preview Sphinx documentation sites (especially when using Sphinx's dirhtml option which is all about subdirectories of index.html files).

$ python -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...

webbrowser

Running the webbrowser module as a script will open a given URL in your default web browser. For example, this would open the page https://pseudorandom.name:

$ python -m webbrowser pseudorandom.name

json.tool

Python's json.tool module can be run as a script to parse a JSON document and print out a version that's formatted nicely for human readability.

$ python -m json.tool /home/trey/Downloads/download.json
[
    {
        "title": "Python's walrus operator",
        "is_premium": false,
        "url": "/using-walrus-operator/"
    },
    {
        "title": "Refactoring long boolean expressions",
        "is_premium": true,
        "url": "/refactoring-boolean-expressions/"
    }
]

calendar

Running the calendar module as a script will print a calendar of the current year by default. It also accepts various arguments to customize its output. Here's a calendar of just one month:

$ python -m calendar 2024 04
     April 2024
Mo Tu We Th Fr Sa Su
 1  2  3  4  5  6  7
 8  9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30

Those 4 scripts are general-purpose tools that I find helpful on any machine. Python also includes a number of tools that are commonly available (or easily installable) on Linux and Mac machines.

Especially handy on Windows machines

Running Python on Windows? Or running Python on a Linux/Mac machine without the ability to easily install common command-line utilities like uuid, sqlite3 and gzip?

These tools are all equivalent to command-line tools that are common on many Linux machines, though the equivalent Linux commands are usually more powerful and more user-friendly.

Note that the command-line interfaces for uuid and sqlite3 were both added in Python 3.12.

I've found the sqlite3 module handy when in a Docker container that didn't have a sqlite3 program installed, but did have Python 3.12.

Working with Python code

These tools are all handy when working with Python code.

pip

The pip module can installs third-party Python packages.

venv

The venv module creates virtual environments.

pdb

The pdb module powers the Python debugger. That's what the built-in breakpoint function starts. Running pdb as a command-line script will set a PDB breakpoint on the first line of your program.

unittest

The unittest module can be used for writing automated tests in Python. When running unittest as a command-line script will, all tests within the current directory will be identified and run automatically.

pydoc

Running the pydoc module as a command-line script will show the documentation for a given module or object. This is the same documentation you would see if you passed the same object name to the built-in help function.

doctest

Running doctest as a command-line script will evaluate all doctests (example code in docstrings) within a given Python file.

ensurepip

The ensurepip script is for folks who found that they've uninstalled pip and need a way to reinstall it (I did this once and it's not fun).

idlelib

Ever wondered how to launch Python's graphical IDLE tool from the command-line? Run python -m idlelib.

zipapp

Want to bundle up a Python module into a ZIP file that can be run directly by Python? Run python -m zipapp my_module.

compileall

Want to warm up the compiled bytecode cache that Python uses to run your modules? Run python -m compileall . to compile all Python files in the current directory to cached bytecode.

Analyzing Python code

Python also includes a handful of other Python-related tools that are specifically for analyzing Python code.

If you wanted to analyze some Python code to see how it ticks, these tools can be useful.

You can think of the tokenize, ast, and dis modules as progressively deeper steps in the process of parsing the code in a Python module.

tokenize

The tokenize module/script will break a Python file into a tree of "tokens":

$ python -m tokenize hello.py
0,0-0,0:      ENCODING     'utf-8'
1,0-1,5:      NAME         'print'
1,5-1,6:      OP           '('
1,6-1,19:     STRING       '"Hello world"'
1,19-1,20:    OP           ')'
1,20-1,21:    NEWLINE      '\n'
2,0-2,0:      ENDMARKER    ''

ast

The ast module/script goes one step further, turning the tokens into an "abstract syntax tree":

$ python -m ast hello.py
Module(
   body=[
      Expr(
         value=Call(
            func=Name(id='print', ctx=Load()),
            args=[
               Constant(value='Hello world')],
            keywords=[]))],
   type_ignores=[])

dis

The dis module/script disassembles the abstract syntax tree into Python's "bytecode":

$ python -m dis hello.py
  0     0 RESUME             0

  1     2 PUSH_NULL
        4 LOAD_NAME          0 (print)
        6 LOAD_CONST         0 ('Hello world')
        8 CALL               1
       16 POP_TOP
       18 RETURN_CONST       1 (None)

I've used tokenize to see how Python initially parses a module. I used the ast module along to create the undataclass tool, along with the ast script which helped me figure out how Python was parsing my file. I've used the dis module to try confirming a claim like "comprehensions generate fewer operations than loops".

inspect

The inspect module can be used as a script to inspect the source code of a given Python object.

$ python -m inspect contextlib:redirect_stdout
class redirect_stdout(_RedirectStream):
    """Context manager for temporarily redirecting stdout to another file.

        # How to send help() to stderr
        with redirect_stdout(sys.stderr):
            help(dir)

        # How to write help() to a file
        with open('help.txt', 'w') as f:
            with redirect_stdout(f):
                help(pow)
    """

    _stream = "stdout"

Unfortunately, it only works on objects that are implemented in Python directly.

$ python -m inspect itertools:zip_longest
Can't get info for builtin modules.

Using the inspect module as a script seems helpful in theory, but I always find myself reaching for the actual code files instead. This may be because it's often helpful to see a bit more context than just the code for an one object: seeing inherited classes, global module state, other functions/classes, and imports are often helpful.

pyclbr

The pyclbr module can be run as a script to get a quick overview of each class, method, and function in a specific Python module:

$ python -m pyclbr timeit
def reindent 81
class Timer [] 86
  def __init__ 104
  def print_exc 139
  def timeit 166
  def repeat 186
  def autorange 212
def timeit 234
def repeat 240
def main 246
  def callback 324
  def format_time 344

That somewhat obfuscated pyclbr name stands for "Python class browser" (it was originally meant just for browsing classes and methods).

Just for fun

These are Python Easter Eggs that work as Python scripts.

__hello__

Want to implement "hello world" in Python? It's already implemented in the __hello__ module!

$ python -m __hello__
Hello world!

this

Want to see the Zen of Python printed out in your terminal? Either import this or run this as a script:

$ python -m this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

antigravity

Importing the antigravity module in Python will open an XKCD comic on Python in your web browser (powered by the webbrowser module mentioned above). Running antigravity as a script works too:

$ python -m antigravity

turtledemo

If you want to see a demo of different drawings you can make with Python's turtle module, run turtledemo as a script:

$ python -m turtledemo

Other Python-related tools

Here are a number of other slightly advanced Python-related tools.

asyncio

If you find yourself working with async/await often in Python, you may find the asynchronous REPL handy.

$ python -m asyncio
asyncio REPL 3.12.0 (main, Nov 30 2023, 17:49:51) [GCC 11.4.0] on linux
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> await asyncio.sleep(1, result='hello')
'hello'

cProfile & pstats

The cProfile script will profile your code by noting how long your code spent on various operations and within various functions.

$ python -m cProfile -s tottime -m http.server 8000
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
^C
Keyboard interrupt received, exiting.
         41242 function calls (40547 primitive calls) in 1.111 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        3    1.075    0.358    1.075    0.358 {method 'poll' of 'select.poll' objects}
       46    0.004    0.000    0.004    0.000 {built-in method marshal.loads}
  213/211    0.003    0.000    0.005    0.000 {built-in method builtins.__build_class__}
       16    0.002    0.000    0.002    0.000 {built-in method _imp.create_dynamic}

... [Over 750 more lines of output]

The pstat command can process and compute statistics on the output of a profile file that's generated by cProfile.

The profile script is equivalent to the cProfile script, but it's written entirely in Python and is slower, so cProfile is preferable over profile.

pickle & pickletools

Have a pickle file and want to see what's in it?

Running the pickle module as a script will show the unpickled data:

$ python -m pickle data.pickle
{'color': 'purple', 'name': 'duck'}

Running the pickletools module as a script will show a detailed explanation of each piece of the pickled data:

$ python -m pickletools data.pickle
    0: \x80 PROTO      4
    2: \x95 FRAME      36
   11: }    EMPTY_DICT
   12: \x94 MEMOIZE    (as 0)
   13: (    MARK
   14: \x8c   SHORT_BINUNICODE 'name'
   20: \x94   MEMOIZE    (as 1)
   21: \x8c   SHORT_BINUNICODE 'duck'
   27: \x94   MEMOIZE    (as 2)
   28: \x8c   SHORT_BINUNICODE 'color'
   35: \x94   MEMOIZE    (as 3)
   36: \x8c   SHORT_BINUNICODE 'purple'
   44: \x94   MEMOIZE    (as 4)
   45: u      SETITEMS   (MARK at 13)
   46: .    STOP
highest protocol among opcodes = 4

Oddly meta tools

Here are even more Python-related tools which are oddly meta.

code

The code module is used for making interactive Python interpreters, so running it will basically run a version of the interactive Python REPL:

$ python -m code
Python 3.12.0 (main, Nov 30 2023, 17:49:51) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>

runpy

The runpy module is used for dynamically running a given Python module by its name. The fact that it has a command-line interface is a bit odd, since it essentially does what Python already does for us!

Here's runpy running runpy running runpy running the unittest module:

$ python -m runpy runpy runpy unittest

----------------------------------------------------------------------
Ran 0 tests in 0.000s

NO TESTS RAN

Less useful tools

The remaining tools are ones that are unlikely to be useful often.

timeit

If you need time how long a single Python expression takes to run, you could use timeit as a script:

$ python -m timeit 'sum([list(range(1000))] * 50, [])'
100 loops, best of 5: 2.2 msec per loop
$ python -m timeit 'import itertools; itertools.chain.from_iterable([list(range(1000))] * 50)'
20000 loops, best of 5: 10.5 usec per loop

You may find it surprising that I include timeit in the list of rarely useful tools. I actually find the timeit module very useful, but I find that I pretty much always need to use it as a module rather than a script. More on using timeit in the documentation.

site

Running the site module as a script will show a bit of information about your current Python environment, including sys.path (which shows the directories in your PYTHONPATH).

$ python -m site
sys.path = [
    '/home/trey/repos/business/screencasts',
    '/home/trey/.pyenv/versions/3.12.0/lib/python312.zip',
    '/home/trey/.pyenv/versions/3.12.0/lib/python3.12',
    '/home/trey/.pyenv/versions/3.12.0/lib/python3.12/lib-dynload',
    '/home/trey/.local/lib/python3.12/site-packages',
    '/home/trey/.pyenv/versions/3.12.0/lib/python3.12/site-packages',
]
USER_BASE: '/home/trey/.local' (exists)
USER_SITE: '/home/trey/.local/lib/python3.12/site-packages' (exists)
ENABLE_USER_SITE: True

The --user-base or --user-site arguments can be passed to the site script to see just the location of those two directories:

$ python -m site --user-base
/home/trey/.local
$ python -m site --user-site
/home/trey/.local/lib/python3.12/site-packages

sysconfig

Running the sysconfig module as a script will show a huge amount of information about your Python installation.

$ python3.12 -m sysconfig | less
Platform: "linux-x86_64"
Python version: "3.12"
Current installation scheme: "posix_prefix"

Paths:
        data = "/home/trey/.pyenv/versions/3.12.0"
        include = "/home/trey/.pyenv/versions/3.12.0/include/python3.12"
        platinclude = "/home/trey/.pyenv/versions/3.12.0/include/python3.12"
        platlib = "/home/trey/.pyenv/versions/3.12.0/lib/python3.12/site-packages"
        platstdlib = "/home/trey/.pyenv/versions/3.12.0/lib/python3.12"
        purelib = "/home/trey/.pyenv/versions/3.12.0/lib/python3.12/site-packages"
        scripts = "/home/trey/.pyenv/versions/3.12.0/bin"
        stdlib = "/home/trey/.pyenv/versions/3.12.0/lib/python3.12"

Variables:
        ABIFLAGS = ""
        AC_APPLE_UNIVERSAL_BUILD = "0"

... [Over 1000 more lines of output]

platform

The platform script will tell you information about your operating system kernel:

$ python -m platform
Linux-6.5.0-1023-oem-x86_64-with-glibc2.35

mimetypes

You can use mimetypes to find the file extension for a given file type:

$ python -m mimetypes -e 'text/markdown'
.md

Or you can use mimetypes to discover the type of a given file:

$ python -m mimetypes README.md
type: text/markdown encoding: None

quopri

The quopri command encode/decode quoted-printable data for raw email data:

$ echo 'Hi! 👋' | python -m quopri
Hi! =F0=9F=91=8B

filecmp

The filecmp script accepts two directories and notes which files are different or the same between them. It has a very primitive command-line interface.

$ python -m filecmp dir1 dir2
diff dir1 dir2
Only in dir1 : ['c', 'sub2']
Only in dir2 : ['d', 'sub3']
Identical files : ['a']
Differing files : ['b']
Common subdirectories : ['sub1']

It's similar to the command-line diff utility but it only works on directories and its output is less readable.

encodings.rot_13

Need to ROT-13 encode/decode some text? Probably not. But Python has a command-line tool for that.

$ echo 'Hello!' | python -m encodings.rot_13
Uryyb!
$ echo 'Uryyb!' | python -m encodings.rot_13
Hello!

tabnanny

Need to check whether a Python file mixes tabs and spaces? Hopefully not!

$ python -m tabnanny example.py
example.py 3 "\tprint('Hi')"

This used to be legal syntax in Python 2, but in Python 3 it's not valid anymore, so a SyntaxError will usually tell you something is wrong without needing to deliberately check.

Every command-line tool in Python

Here's a quick summary of every command-line tool in Python:

I discovered the command-line interface for many of these modules by using this script, which looks for command-line interfaces among all Python standard library modules.

Note that older versions included even more modules that could be run as scripts. The standard library also included scripts for a uu module before Python 3.12 and formatter, binhex, test.pystone, and hotshot.stones existed in Python 2.

These are just the Python scripts included in the Python standard library. Any third-party module that can be run as a script can also be launched via python -m MODULE_NAME as well.