Plumbum: Shell Combinators
Ever wished the compactness of shell scripts be put into a real programming language? Say hello to Plumbum Shell Combinators. Plumbum (Latin for lead, which was used to create pipes back in the day) is a small yet feature-rich library for shell script-like programs in Python. The motto of the library is "Never write shell scripts again", and thus it attempts to mimic the shell syntax ("shell combinators") where it makes sense, while keeping it all Pythonic and cross-platform.
Apart from shell-like syntax and handy shortcuts, the library provides local and remote command execution (over SSH), local and remote file-system paths, easy working-directory and environment manipulation, and a programmatic Command-Line Interface (CLI) application toolkit. Now let's see some code!
This is only a teaser; the full documentation can be found at Read the Docs
Cheat Sheet
Basics
>>> from plumbum import local
>>> local.cmd.ls
LocalCommand(/bin/ls)
>>> local.cmd.ls()
'build.py\nCHANGELOG.rst\nconda.recipe\nCONTRIBUTING.rst\ndocs\nexamples\nexperiments\nLICENSE\nMANIFEST.in\nPipfile\nplumbum\nplumbum.egg-info\npytest.ini\nREADME.rst\nsetup.cfg\nsetup.py\ntests\ntranslations.py\n'
>>> notepad = local["c:\\windows\\notepad.exe"]
>>> notepad() # Notepad window pops up
'' # Notepad window is closed by user, command returns
In the example above, you can use local["ls"] if you have an unusually named executable or a full path to an executable. The local object represents your local machine. As you'll see, Plumbum also provides remote machines that use the same API! You can also use from plumbum.cmd import ls as well for accessing programs in the PATH.
Piping
>>> from plumbum.cmd import ls, grep, wc
>>> chain = ls["-a"] | grep["-v", r"\.py"] | wc["-l"]
>>> print(chain)
/bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l
>>> chain()
'27\n'
Redirection
>>> from plumbum.cmd import cat, head
>>> ((cat < "setup.py") | head["-n", 4])()
'#!/usr/bin/env python\nimport os\n\ntry:\n'
>>> (ls["-a"] > "file.list")()
''
>>> (cat["file.list"] | wc["-l"])()
'31\n'
Working-directory manipulation
>>> local.cwd
<LocalWorkdir /home/tomer/workspace/plumbum>
>>> with local.cwd(local.cwd / "docs"):
... chain()
...
'22\n'
Foreground and background execution
>>> from plumbum import FG, BG >>> (ls["-a"] | grep[r"\.py"]) & FG # The output is printed to stdout directly build.py setup.py translations.py >>> (ls["-a"] | grep[r"\.py"]) & BG # The process runs "in the background" <Future ['/bin/grep', '\\.py'] (running)>
Command nesting
>>> from plumbum.cmd import sudo, ifconfig
>>> print(sudo[ifconfig["-a"]])
/usr/bin/sudo /sbin/ifconfig -a
>>> (sudo[ifconfig["-a"]] | grep["-i", "loop"]) & FG
lo Link encap:Local Loopback
UP LOOPBACK RUNNING MTU:16436 Metric:1
Remote commands (over SSH)
Supports openSSH-compatible clients, PuTTY (on Windows) and Paramiko (a pure-Python implementation of SSH2)
>>> from plumbum import SshMachine
>>> remote = SshMachine("somehost", user = "john", keyfile = "/path/to/idrsa")
>>> r_ls = remote["ls"]
>>> with remote.cwd("/lib"):
... (r_ls | grep["0.so.0"])()
...
'libusb-1.0.so.0\nlibusb-1.0.so.0.0.0\n'
CLI applications
import logging
from plumbum import cli
class MyCompiler(cli.Application):
verbose = cli.Flag(["-v", "--verbose"], help = "Enable verbose mode")
include_dirs = cli.SwitchAttr("-I", list = True, help = "Specify include directories")
@cli.switch("--loglevel", int)
def set_log_level(self, level):
"""Sets the log-level of the logger"""
logging.root.setLevel(level)
def main(self, *srcfiles):
print("Verbose:", self.verbose)
print("Include dirs:", self.include_dirs)
print("Compiling:", srcfiles)
if __name__ == "__main__":
MyCompiler.run()
Sample output
$ python simple_cli.py -v -I foo/bar -Ispam/eggs x.cpp y.cpp z.cpp
Verbose: True
Include dirs: ['foo/bar', 'spam/eggs']
Compiling: ('x.cpp', 'y.cpp', 'z.cpp')
Colors and Styles
from plumbum import colors
with colors.red:
print("This library provides safe, flexible color access.")
print(colors.bold | "(and styles in general)", "are easy!")
print("The simple 16 colors or",
colors.orchid & colors.underline | '256 named colors,',
colors.rgb(18, 146, 64) | "or full rgb colors",
'can be used.')
print("Unsafe " + colors.bg.dark_khaki + "color access" + colors.bg.reset + " is available too.")
10 Sep 14, 2022
5 Dec 22, 2022
4 Apr 06, 2022
15 Oct 15, 2022
56 Nov 19, 2022
5 Jan 04, 2022
192 Jan 07, 2023
7 May 09, 2022
313 Dec 24, 2022
1 Nov 17, 2021
1 Dec 01, 2021
15 Jan 10, 2022
1 Feb 15, 2022
3 Sep 10, 2022
22 Oct 24, 2022
2 Nov 13, 2021
52 Nov 12, 2022
10 Jun 30, 2022
3.8k Jan 07, 2023
1 Sep 11, 2022