Skip to main content
  1. Posts/

Why Cannot I Get the Value of A Env Variable in Python?

··678 words·4 mins·
Table of Contents

I found a strange issue when I wanted to get the value of an env variable from Python script.

To reproduce, we can first set the value of FOO in the Bash shell:

$ FOO=123

Then execute the following python command:

$ python -c "import os; print(os.getenv('FOO', "None"))"

The output is None. So the python process can not see the variable FOO. However, if we run the two command in one line:

$ FOO=123 python -c "import os; print(os.getenv('FOO', "None"))"

The output will be 123.

If we use export FOO=123 instead of FOO=123, we can also get the value of FOO in Python.

But why? What is the difference here? This is in fact a bit complexer than what I think.

Bash built-in and external commands
#

First, we need to know that, when Bash executes commands, it treats the built-in and external commands differently. Built-in commands are those that are, literally, built into the Bash executable. External commands are those commands that are not part of the Bash shell.

When Bash executes built-in commands, it does not create a new process. When it executes external commands, it will create a new shell and then execute the external commands. The new shell will inherit the environment variables of its parent shell (see here on why Bash needs to create a new process to execute an external command).

How do we check if a command is a Bash builtin or an external command?

First, we can check the output of help, it will list all Bash built-in commands. We can also use the type command to check if a command is a built-in command:

type -t some_command

For built-in commands, it will output builtin. For external commands, it will output different info, for example, file (see help type for more info).

Ref:

When will variable value be available in a process?
#

If we define variable without using export, it is not part of the environment variable. When Bash create a new process, this variable will not be available there.

One exception is that if you set the variable and run the command in the same line, it will be available only to that command. From the Bash manual:

The environment for any simple command or function may be augmented temporarily by prefixing it with parameter assignments, as described in Shell Parameters. These assignment statements affect only the environment seen by that command.

If a variable is not exported, it is only a shell variable. When it is exported, it is an environment variable that will be available in all subprocesses.

Ref:

Back to the issue
#

Now with sufficient background knowledge on how Bash works, we are able to understand why certain ways work or fail.

# Fails
$ FOO=123
$ python -c "import os; print(os.getenv('FOO', "None"))"

The above way to get the value of FOO fails, because python is an external command for Bash. Bash will execute python in a new process. However, FOO defined via FOO=123 is only available in the shell that invoking python command, not the sub-process (also a Bash shell) that actually run python command.

# work
$ FOO=123 python -c "import os; print(os.getenv('FOO', "None"))"

The above way works because the variable definition is in the same line with python command. It will be available temporily for this command. If you execute python -c "import os; print(os.getenv('FOO', "None")) in a second line, it will not work any more.

$ export FOO=123
$ python -c "import os; print(os.getenv('FOO', "None"))"

This way also works because we set an environment variable FOO via export. So FOO will be available to all subprocesses created by this shell.

References
#

Related

Bash Script Note 1
··445 words·3 mins
Why don't settings inside bashrc or bash_profile take effect?
··1008 words·5 mins
Creating Beautiful Bash Prompts
·403 words·2 mins