4/19/2019 Python Advanced: Pipes in Python

PIPES IN PYTHON

PIPE

Unix or Linux without pipes is unthinkable, or

at least, pipelines are a very important part of
Unix and Linux applications. Small elements
are put together by using pipes. Processes are
chained together by their standard streams, i.e.
the output of one process is used as the input
of another process. To chain processes like
this, so-called anonomymous pipes are used.
The concept of pipes and pipelines was
introduced by Douglas McIlroy, one of the authors of the early command shells, after he
noticed that much of the time they were processing the output of one program as the input to
another. Ken Thompson added the concept of pipes to the UNIX operating system in 1973.
Pipelines have later been ported to other operating systems like DOS, OS/2 and Microsoft
Windows as well.

Generally there are two kinds of pipes:

anonymous pipes
and
named pipes

Anonymous pipes exist solely within processes and are usually used in combination with
forks.

BEER PIPE IN PYTHON

"99 Bottles of Beer" is a traditional song in the United States and Canada. The song is derived
from the English "Ten Green Bottles". The song consists of 100 verses, which are very
similar. Just the number of bottles varies. Only one, i.e. the hundredth verse is slightly
different. This song is often sung on long trips, because it is easy to memorize, especially
when drunken, and it can take a long time to sing.

Here are the lyrics of this song:

Ninety-nine bottles of beer on the wall, Ninety-nine bottles of beer. Take one down, pass it

https://www.python-course.eu/pipes.php 1/7
4/19/2019 Python Advanced: Pipes in Python

around, Ninety-eight bottles of beer on the

wall.

The next verse is the same starting with 98

bottles of beer. So the general rule is, each
verse one bottle less, until there in none left.
The song normally ends here. But we want to
implement the Aleph-Null (i.e. the infinite)
version of this song with an additional verse:

No more bottles of beer on the wall, no more

bottles of beer. Go to the store and buy some
more, Ninety-nine bottles of beer on the wall.

This song has been implemented in all

conceivable computer languages like
"Whitespace" or "Brainfuck". You find the
collection at http://99-bottles-of-beer.net
We program the Aleph-Null variant of the song with a fork and a pipe:
import os

def child(pipeout):
bottles = 99
while True:
bob = "bottles of beer"
otw = "on the wall"
take1 = "Take one down and pass it around"
store = "Go to the store and buy some more"

if bottles > 0:
values = (bottles, bob, otw, bottles, bob, take1, bottles -
1,bob,otw)
verse = "%2d %s %s,\n%2d %s.\n%s,\n%2d %s %s." % values
os.write(pipeout, verse)
bottles -= 1
else:
bottles = 99
values = (bob, otw, bob, store, bottles, bob,otw)
verse = "No more %s %s,\nno more %s.\n%s,\n%2d %s %s." % values
os.write(pipeout, verse)

def parent():
pipein, pipeout = os.pipe()
if os.fork() == 0:
child(pipeout)
else:
counter = 1

https://www.python-course.eu/pipes.php 2/7
4/19/2019 Python Advanced: Pipes in Python

while True:
if counter % 100:
verse = os.read(pipein, 117)
else:
verse = os.read(pipein, 128)
print 'verse %d\n%s\n' % (counter, verse)
counter += 1

parent()

The problem in the code above is that we or better the parent process have to know exactly
how many bytes the child will send each time. For the first 99 verses it will be 117 Bytes
(verse = os.read(pipein, 117)) and for the Aleph-Null verse it will be 128 bytes (verse =
os.read(pipein, 128)

We fixed this in the following implementation, in which we read complete lines:

import os

def child(pipeout):
bottles = 99
while True:
bob = "bottles of beer"
otw = "on the wall"
take1 = "Take one down and pass it around"
store = "Go to the store and buy some more"

if bottles > 0:
values = (bottles, bob, otw, bottles, bob, take1, bottles -
1,bob,otw)
verse = "%2d %s %s,\n%2d %s.\n%s,\n%2d %s %s.\n" % values
os.write(pipeout, verse)
bottles -= 1
else:
bottles = 99
values = (bob, otw, bob, store, bottles, bob,otw)
verse = "No more %s %s,\nno more %s.\n%s,\n%2d %s %s.\n" %
values
os.write(pipeout, verse)
def parent():
pipein, pipeout = os.pipe()
if os.fork() == 0:
os.close(pipein)
child(pipeout)
else:
os.close(pipeout)
counter = 1
pipein = os.fdopen(pipein)
while True:
print 'verse %d' % (counter)
for i in range(4):
verse = pipein.readline()[:-1]

https://www.python-course.eu/pipes.php 3/7
4/19/2019 Python Advanced: Pipes in Python

print '%s' % (verse)

counter += 1
print

parent()

BIDIRECTIONAL PIPES

Now we come to something completely non-alcoholic. It's a simple guessing game, which
small children often play. We want to implement this game with bidirectional Pipes. There is
an explanation of this game in our tutorial in the chapter about loops. The following diagram
explains both the rules of the game and the way we implemented it:

https://www.python-course.eu/pipes.php 4/7
4/19/2019 Python Advanced: Pipes in Python

The deviser, the one who devises the number, has to imagine a number between a range of 1
to n. The Guesser inputs his guess. The deviser informs the player, if this number is larger,
smaller or equal to the secret number, i.e. the number which the deviser has randomly created.
Both the deviser and the guesser write their results into log files, i.e. deviser.log and
guesser.log respectively.

This is the complete implementation:

import os, sys, random

def deviser(max):
fh = open("deviser.log","w")
to_be_guessed = int(max * random.random()) + 1

guess = 0
while guess != to_be_guessed:
guess = int(raw_input())
fh.write(str(guess) + " ")
if guess > 0:
if guess > to_be_guessed:
print 1
elif guess < to_be_guessed:
print -1
else:
print 0
sys.stdout.flush()
else:
break
fh.close()

def guesser(max):
fh = open("guesser.log","w")
bottom = 0
top = max
fuzzy = 10
res = 1
while res != 0:
guess = (bottom + top) / 2
print guess
sys.stdout.flush()
fh.write(str(guess) + " ")
res = int(raw_input())
if res == -1: # number is higher
bottom = guess
elif res == 1:
top = guess
elif res == 0:
message = "Wanted number is %d" % guess
fh.write(message)
else: # this case shouldn't occur
print "input not correct"

https://www.python-course.eu/pipes.php 5/7
4/19/2019 Python Advanced: Pipes in Python

fh.write("Something's wrong")

n = 100
stdin = sys.stdin.fileno() # usually 0
stdout = sys.stdout.fileno() # usually 1

parentStdin, childStdout = os.pipe()

childStdin, parentStdout = os.pipe()
pid = os.fork()
if pid:
# parent process
os.close(childStdout)
os.close(childStdin)
os.dup2(parentStdin, stdin)
os.dup2(parentStdout, stdout)
deviser(n)
else:
# child process
os.close(parentStdin)
os.close(parentStdout)
os.dup2(childStdin, stdin)
os.dup2(childStdout, stdout)
guesser(n)

NAMED PIPES, FIFOS

Under Unix as well as

under Linux it's possible
to create Pipes, which are
implemented as files.

These Pipes are called

"named pipes" or
sometimes Fifos (First In
First Out).

A process reads from and

writes to such a pipe as if
it were a regular file.
Sometimes more than one
process write to such a pipe but only one process reads from it.

The following example illustrates the case, in which one process (child process) writes to the
pipe and another process (the parent process) reads from this pipe.

https://www.python-course.eu/pipes.php 6/7
4/19/2019 Python Advanced: Pipes in Python

import os, time, sys

pipe_name = 'pipe_test'

def child( ):
pipeout = os.open(pipe_name, os.O_WRONLY)
counter = 0
while True:
time.sleep(1)
os.write(pipeout, 'Number %03d\n' % counter)
counter = (counter+1) % 5

def parent( ):
pipein = open(pipe_name, 'r')
while True:
line = pipein.readline()[:-1]
print 'Parent %d got "%s" at %s' % (os.getpid(), line,
time.time( ))

if not os.path.exists(pipe_name):
os.mkfifo(pipe_name)
pid = os.fork()
if pid != 0:
parent()
else:
child()

© 2011 - 2018, Bernd Klein, Bodenseo; Design by Denise Mitchinson adapted for python-course.eu by
Bernd Klein

https://www.python-course.eu/pipes.php 7/7