2 Environments

Last edited

What is a python environment?

An interpreter and modules (standard library and third-party package if you installed any).
These two allow you execute a python program.

Every Python program runs “inside” a Python environment: the interpreter in the environment executes the program’s code, and import statements load modules from the environment.
You select the environment by launching its interpreter.

Running on an interpreter

Python offers two mechanisms for running a program on an interpreter.

# Pass the python script as an argument
py hello.py
# Pass a module with -m (provided the interpreter can import the module)
py -m hello

Also many python applications install an entry-point script in your PATH

hello

This method has a drawback, if you have programs in multiple environments the first environment on PATH windows, py -m hello offers more controls

Parts of a python install

The interpreter

The executable that runs Python programs is named python.exe on Windows.
On Linux and macOS, the interpreter is named python3.x and stored in the bin directory with a python3 symbolic link.

# Dump a bunch of info on the python environment / interpreter
py -m sysconfig

# Or via an interactive session
>>> import sys
>>> sys.executable # location of the interpreter
>>> sys.implementation.name # e.g., cpython, pypy
>>> sys.version_info # Version of python language
>>> sys.path # List of directories searched when importing python modules
['', '/usr/lib/python313.zip', '/usr/lib/python3.13', '/usr/lib/python3.13/lib-dynload', '/usr/local/lib/python3.13/dist-packages', '/usr/lib/python3/dist-packages']

Modules

Modules are containers of Python objects that you load via the import statement.

Simple modules

a single file containing python source code.

# executes string.py once
# binds `string` to the module object
import string

Packages

directories with __init__.py, they let you organize modules in a hierarchy.

# executes email/__init__.py then email/message.py
# binds `email`(reach message via email.message)
import email
email.message.EmailMessage()

Namespace packages

directories with modules (python file) but no __init__.py

Unlike with regular packages, you can distribute each module (python file) in a namespace package separately.

# same importing a single module, just we've added the namespace
import acme.unicycle

Extension modules

compiled code from low-level language like C, that can be imported like regular modules. math is an example.

They’re shared libraries with a special entry point that lets you import them as modules from Python. These exist for performance or to bind to other language libraries like C.

Their extensions are .pyd on windows, .so on Linux

Builtin Modules

Some modules from the standard library, such as the sys are compiled into the interpreter.

Frozen Modules

Some modules from the standard library are written in Python but have their bytecode embedded in the interpreter. Such as os and io

Byte code caching

Note

Bytecode is an intermediate representation of Python code that is platform-independent and optimized for fast execution.
The interpreter compiles pure Python modules to bytecode when it loads them for the first time. Bytecode modules are cached in the environment in .pyc files under pycache directories.

Entry-point scripts

Entry-point script: A python file that has a single purpose: launch an application by importing the module with it’s entry-point function and call that function.

Package installers like pip can generate entry-point scripts for third party packages they install. Package authors just have to say which function the script should run.

On linux they’re regular python files with x permissions, on windows they’re exe (PE). The binary launches the interpreter with the embedded code.

#!usr/local/bin/python3.12
import pydoc
if __name__ == "__main__":
    pydoc.cli()

# The OS uses the shebang to locate and launch the interpreter.

A tour of environments

Per-User Environments

Install third-party package for a single user ~/.local/bin

py -m pip install --user
# If the per-user environment doesn’t exist yet pip creates it for you.
# The per-user script directory may not be on PATH by default

The bad part about them, by design they’re not isolated

import requests  # comes from the system
import rich      # comes from your user install

Note

Don’t use this for projects, basically just use for personal command-line tools. Even still just use pipx it’s better.

Virtual Environments

Unlike system-wide and per-user environments, virtual environments isolate your projects, avoiding dependency conflicts.

Virtual environments are a lightweight Python environment

  • It stores third-party packages and delegates most other things to a full installation
  • Packages in virtual environments are visible only to the interpreter in the environment
# Create it using the venv module 
py -m venv <dir_name> # conventionally .venv

What’s in the virtual env?

Here’s the main parts

Bin
./venv
└── bin
    ├── activate      # setup your shell, purely convenient
    ├── activate.csh      # 1) just updates your PATH with this folder (for python/pip)
    ├── activate.fish     # 2) sets the VIRTUAL_ENV var so tools  know its a virtual env
    ├── Activate.ps1      # 3) updates the shell prompt, customize this with --prompt
    │   
    ├── pip         # entry point script for pip, installed by default by venv
    ├── pip3        # all the same script just convenience
    ├── pip3.13
    │   
    ├── python -> python3 # convenience link so you can use either name
    ├── python3.13 -> python3 # convenience link so you can use either name
    └── python3 -> /usr/bin/python3 # link to system interpreter

You can create a virtual environment without pip and just use an external installer:

python -m venv .venv --without-pip  # no pip
pip --python=.venv install httpx    # using global pip
pyvenv

pyvenv.cfg is the metadata file for a virtual environment. Python uses it to recognize the venv and locate the standard library via the base interpreter. While keep third-party packages isolated to the venv.

./venv
└── pyvenv.cfg
third-party packages
./venv
└── lib
    └── python3.13
        └── site-packages
            └── pip
            ....

This is where third party packages are installed for the venv, right now i only have pip installed but they all get stored here.

Pipx

Applications tend to depend on more packages than libraries, and they can be quite picky about the versions of their dependencies.

Ideally we’d have a virtual environment for each package but managing and activating a separate virtual environment for every application is a lot of work. Wouldn’t it be great if we could confine applications to virtual environments and still have them available globally?

That’s what pipx does:
it copies or symlinks the entry-point script for the application from its virtual environment into a directory on your search path

How it works

This is what it does for you, say we wanted to install black:

mkdir -p ~/.local/bin
export PATH="$HOME/.local/bin:$PATH"
py -m venv black
black/bin/python -m pip install black
cp black/bin/black ~/.local/bin
# the copied entry point has a shebang that points back to the "black" venv we made

Install/Use

Download through your package manager (apt)

pipx install black
pipx upgrade black
pipx uninstall black
pipx list
pipx upgrade-all
pipx uninstall-all

Managing Environments with uv

uv is a drop-in replacement for core Python packaging tools. It’s way faster and compiled to a single binary.

pipx install uv

uv venv # create a virtual env .venv
uv pip install httpx # install packages

Finding Python Modules

Module discovery the interpreter searches the directories on sys.path for the module:

  1. first the directories that contain the modules of the standard library
  2. then the site-packages directory with third-party packages — in a virtual environment, that environment’s site-packages
# Shows you where your packages will get imported from
py -IPm site

Module importing (import statement)

The interpreter translates every use of import X into a call of the __import__ function from importlib, which returns a module object.

  1. CACHE - Check the cache for X

    sys.modules["X"]

    If found return the already built module object (its functions/classes/vars in __dict__). Done

  2. FIND - if we missed, search sys.path (above) for X usually .py, once we find it output a type.ModuleSpec which contains:

    • __name__ - the name email.message
    • __file__ - the location of the .py
    • __cached__ - the path to the bytecode .pyc, etc.
    • __package__ - fully qualified name of the containing package

  3. LOAD - load and execute the module’s code. note that code inside a def or class just get defined.

Edit this page
© 2026 Monjiapawne