I have a bash script that sets an environment variable an runs a command
LD_LIBRARY_PATH=my_path
sqsub -np $1 /homedir/anotherdir/executable
Now I want to use python instead of bash, because I want to compute some of the arguments that I am passing to the command.
I have tried
putenv("LD_LIBRARY_PATH", "my_path")
and
call("export LD_LIBRARY_PATH=my_path")
followed by
call("sqsub -np " + var1 + "/homedir/anotherdir/executable")
but always the program gives up because LD_LIBRARY_PATH is not set.
How can I fix this?
Thanks for help!
(if I export LD_LIBRARY_PATH before calling the python script everything works, but I would like python to determine the path and set the environment variable to the correct value)
3 Answers
bash:
LD_LIBRARY_PATH=my_path
sqsub -np $1 /path/to/executable
Similar, in Python:
import os
import subprocess
import sys
os.environ['LD_LIBRARY_PATH'] = "my_path" # visible in this process + all children
subprocess.check_call(['sqsub', '-np', sys.argv[1], '/path/to/executable'],
env=dict(os.environ, SQSUB_VAR="visible in this subprocess"))
You can add elements to your environment by using
os.environ['LD_LIBRARY_PATH'] = 'my_path'
and run subprocesses in a shell (that uses your os.environ) by using
subprocess.call('sqsub -np ' + var1 + '/homedir/anotherdir/executable', shell=True)
There are many good answers here but you should avoid at all cost to pass untrusted variables to subprocess using shell=True as this is a security risk. The variables can escape to the shell and run arbitrary commands! If you just can't avoid it at least use python3's shlex.quote() to escape the string (if you have multiple space-separated arguments, quote each split instead of the full string).
shell=False is always the default where you pass an argument array.
Now the safe solutions...
Method #1
Change your own process's environment - the new environment will apply to python itself and all subprocesses.
os.environ['LD_LIBRARY_PATH'] = 'my_path'
command = ['sqsub', '-np', var1, '/homedir/anotherdir/executable']
subprocess.check_call(command)
