Saturday, June 06, 2009

Subprocessing in Python

It may sound strange, but Python with all its low-level roots and crossplatform portability still doesn't have pythonic interface for working with child processes. Libraries that were born to do things unix-way usually fail under windows. Let's take a look into history of subprocessing at

Evil os.popen()

POSIX os.popen() call is at holy war with windows - one of the reasons there are almost no crossplatform Python scripts that wrap and control 3rd party processes. For example, study this os.popen() call reported in this thread:

import os,sys

out = os.popen("nslookup", "w")
out.write("google.com\n")
out.close()

On POSIX machine it will call nslookup, send a string on its input and nslookup will print the result on console:

> Server: 10.0.80.11
Address: 10.0.80.11#53

Non-authoritative answer:
Name: google.com
Address: 74.125.45.100
Name: google.com
Address: 74.125.67.100
Name: google.com
Address: 74.125.127.100
>

Under windows nslookup won't output anything on screen, but it will process the input correctly and that can be proved by redirecting output to a file:

import os,sys

out = os.popen("nslookup > output.txt", "w")
out.write("google.com\n")
out.close()

The workaround suggested in Python bug #1366 and used in Mercurial doesn't seems to work with Python 2.5.4 on XP. According to Mercurial patch the following code should give output on console:

import os,sys

out = os.popen('"nslookup 2> NUL:"', "w")
out.write("google.com\n")
out.close()

but it doesn't. It still executes though, but it is not what you want if the main thing your code does is output pagination. To work around this inherently evil feature of os.popen() smart guys invented subprocess module.

import subprocess

out = subprocess.Popen("nslookup", stdin=subprocess.PIPE)
out.stdin.write("google.com\n")
out.stdin.close()

According to Python bugtracker, subprocess is capable to guard you against malicious os.popen() in many different ways finding natural ways to deal with evilness #3144, evilness #602245, #...

..to be continued