Interaction with processes in Python

Running a program

check_output runs the program and returns the output as a binary string, in case execution is successful:

>>> import subprocess
>>> subprocess.check_output('ls')
b'file.txt\npython-regex-popen.txt\n'
>>> subprocess.check_output('ls').decode()
'file.txt\npython-regex-popen.txt\n'
>>> print(subprocess.check_output('ls').decode())
file.txt
python-regex-popen.txt

In case of error an exception is raised:

>>> subprocess.check_output('lsz')
Traceback (most recent call last):
   ...
FileNotFoundError: [Errno 2] No such file or directory: 'lsz'

Passing parameters

Parameters can be passed by specifying a list instead of a string. For example to execute ls -l we can do the following:

 print(subprocess.check_output(['ls','-l']).decode())
total 44
-rw-rw-r-- 1 focardi focardi    40 feb 16 14:23 file.txt
-rw-rw-r-- 1 focardi focardi  5849 feb 16 15:14 python-regex-popen.txt

We cannot pass arguments in a single string unless we specify shell=True, which forces to interpret the string through the shell. However this is dangerous in general.

>>> print(subprocess.check_output('ls -l', shell=True).decode())
total 44
-rw-rw-r-- 1 focardi focardi    40 feb 16 14:23 file.txt
-rw-rw-r-- 1 focardi focardi  7371 feb 16 15:18 python-regex-popen.txt

Popen

Popen executes a program taking input from stdin and sending output to stdout. For example:

>>> subprocess.Popen(['ls','-l'])

>>> total 52
-rw-rw-r-- 1 focardi focardi    40 feb 16 14:23 file.txt
-rw-rw-r-- 1 focardi focardi  8546 feb 16 15:24 python-regex-popen.txt

Input and output can be made available to the Python program by specifying it as subprocess.PIPE. Then, it is possible to communicate with the process by invoking communicate():

>>> p = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE)
>>> p.communicate()
(b'total 136\n-rw-r--r--  1 focardi  staff    269 Feb 16 19:01 README.txt\n-rw-r--r--  1 focardi  staff     26 Feb 16 18:35 file.txt\n-rw-r--r--  1 focardi  staff    174 Feb 16 20:54 popen.py\n-rw-r--r--  1 focardi  staff  11062 Feb 16 19:03 python-regex-popen-wordpress.html\n-rw-r--r--  1 focardi  staff  32547 Feb 16 19:24 python-regex-popen.html\n-rw-r--r--  1 focardi  staff  11168 Feb 16 21:13 python-regex-popen.txt\n', None)
>>>

communicate returns a pair corresponding to stdout and stderr (that is not redirected).

Redirecting both input and output

We can also pass data to process via communicate, in the form of a bytestream.

The following example runs cat and sends b'hello'. The effect is to receive as output b'hello' which is the expected behaviour of cat:

>>> p1 = Popen(["cat"], stdin=PIPE,stdout=PIPE)
>>> p1.communicate(b'hello')
(b'hello', None)
>>>

Simulating a shell pipeline

We can refer to stdint and stdout of processes so to simulate a shell pipeline.

For example let us simulate ls | grep txt:

>>> from subprocess import *
>>> p1 = Popen(["ls"], stdout=PIPE)
>>> p2 = Popen(["grep", "txt"], stdin=p1.stdout, stdout=PIPE)
>>> p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits (PIPE is dup2-ed on standard output)
>>> output = p2.communicate()[0]
>>> print(output.decode())
file.txt
python-regex-popen.txt

Interacting with a process

In order to interact with a program it is possible to use read and write but it is important to observe that:

The following example interacts 10 times with cat:

>>> p = subprocess.Popen('cat',stdin=subprocess.PIPE,stdout=subprocess.PIPE)
>>> for i in range(10):
...     w = p.stdin.write(b'ciao\n')
...     p.stdin.flush()
...     print(p.stdout.readline().decode(), end='')
...
ciao
ciao
...
ciao

Exercise

Write a python program that interacts with the following C program and wins the game:

#include 
#include 
#include 

int main() {
	srand(time(NULL));   

	int i,d;

	for (i=0;i<10;i++) {
		int r = rand() % 10;      
		printf("Write %i: ",r);
		fflush(stdout);
		scanf("%d",&d);
		if (d != r) {
			printf("WRONG!\n");
			exit(1);
		}
	}
	printf("Great! You did it!\n");
	exit(0);
}