I found a strange issue when I want to get the value of variable from Python script.

To reproduce, we can first set the value of foo in the Bash shell:

$ FOO=123

Then execute the following python command:

$ python -c "import os; print(os.getenv('FOO', "None"))"

The output is None. So the python command can not see the variable FOO.

However, if we run the two command in one line:

$ FOO=123 python -c "import os; print(os.getenv('FOO', "None"))"

The output will be 123.

If we use export FOO=123 instead of FOO=123, we can also access the value of FOO in Python.

But why? What is the difference here? This is in fact a complex issue.

Bash builtin and external commands

First, we need to know that, when bash execute commands, it treats the builtin and external commands differently. Builtin commands are those are literally, built into the bash executable. External commands are those commands that are not part of the bash shell.

When bash executes builtin commands, it does not create a new process. When it executes external commands, it will create a new shell and then execute the external commands. The new shell will inherit the environment variables of its parent shell (see here on why bash needs to create a new process to execute an external commands).

How do we check if a command is bash builtin or external command?

First, we can check the output of help, it will list all bash builtin commands. We can also use the type command to check if a command is a builtin command:

type -t some_command

For builtin commands, it will output builtin. For external commands, it will output other output such as file (see help type for more info).

Ref:

When will variable value be available in a process?

If we define variable without using export, it is not part of the environment variable. When bash create a new process, this variable will not be available.

One exception is that if you set the variable and run the command in the same line, it will be available only to that command. From the Bash manual:

The environment for any simple command or function may be augmented temporarily by prefixing it with parameter assignments, as described in Shell Parameters. These assignment statements affect only the environment seen by that command.

If a variable is not exported, it is only a shell variable. When it is exported, it is an environment variable that will be available in all subprocesses.

Ref:

Back to the issue

Now with sufficient knowledge on how bash works, we are able to know why certain ways work or fail.

$ FOO=123
$ python -c "import os; print(os.getenv('FOO', "None"))"

The above way to get the value of FOO failed, because python is an external command to Bash. Bash will execute python in a new process. However, FOO defined via FOO=123 is only available the shell that invoking python command, not the sub-process (also a Bash shell) that actually run python command.

$ FOO=123 python -c "import os; print(os.getenv('FOO', "None"))"

The above way works because the variable definition is in the same line with python command. It will be available temporily for this command. If you execute python -c "import os; print(os.getenv('FOO', "None")) in a second line, it will not work.

$ export FOO=123
$ python -c "import os; print(os.getenv('FOO', "None"))"

This way works because we set an environment variable FOO via export. So FOO will be available to all subprocesses created by this shell.

References