child = subprocess.Popen(command,
shell=True,
env=environment,
close_fds=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
bufsize=1,
)
subout = ""
with child.stdout:
for line in iter(child.stdout.readline, b''):
subout += line
logging.info(subout)
rc = child.wait()
some times (intermittently) this hangs forever.
not sure if it hangs on iter(child.stdout.readline) or child.wait()
i ps -ef for the process it Popens and that process no longer exists
my guess is that it has do with bufsize so that child.stdout.readline is going on forever but i have no idea how to test it and as this happens intermittently
I could implement alarm but i m not sure if that's appropriate as i cant really tell whether the popen'd process is just slow or hanging
let's assume that either child.stdout.readline or wait() hangs forever, what actions could i take besides alarm ?
2 Answers
You're likely hitting the deadlock that's explained in the documentation:
Popen.wait():Wait for child process to terminate. Set and return
returncodeattribute.Warning: This will deadlock when using
stdout=PIPEand/orstderr=PIPEand the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. Usecommunicate()to avoid that.
The solution is to use Popen.communicate().
Add close_fds=True to the call to subprocess.Popen() as mentioned here.
