Virtual Environments #

When you install Python packages globally, every project on your computer shares the same installation. That means if Project A needs requests==2.28 and Project B needs requests==2.31, they can’t coexist in the same environment — one of them will break. A virtual environment solves this by creating an isolated Python environment per project — the interpreter, pip, and all packages live inside the project directory and don’t affect the global installation or other projects. In this article we cover everything you need to know: from the built-in venv, good dependency management, to uv as a much faster modern tool.

Why Virtual Environments Are Necessary #

Imagine you have two Python projects at the same time:

Without a virtual environment:

Computer
└── Global Python
    └── site-packages/
        ├── django==3.2     ← Project A needs this
        ├── django==4.2     ← Project B needs this (conflict!)
        └── requests==2.28

Only one version can be installed → one of the projects will definitely break.

With virtual environments:

Computer
├── ProjectA/
│   └── .venv/
│       └── site-packages/
│           └── django==3.2   ← isolated for Project A
└── ProjectB/
    └── .venv/
        └── site-packages/
            └── django==4.2   ← isolated for Project B

Both run without conflicts.

To clarify this isolation concept architecturally, look at how a virtual environment maps the local interpreter and separates the library directory (site-packages) by redirecting the sys.prefix variable:

flowchart TD
    subgraph Global ["Global Python Environment (No Isolation)"]
        sys_py["System Python Interpreter"]
        sys_packages["Global site-packages<br/>(django==3.2 at risk of being overwritten by django==4.2)"]
        sys_py --> sys_packages
    end

    subgraph ProjectA ["Project A Isolation (.venv)"]
        a_py["Local Python Copy / Symlink"]
        a_prefix["sys.prefix = /projectA/.venv"]
        a_packages["Project A site-packages<br/>(django==3.2)"]
        a_py --> a_prefix
        a_prefix --> a_packages
    end

    subgraph ProjectB ["Project B Isolation (.venv)"]
        b_py["Local Python Copy / Symlink"]
        b_prefix["sys.prefix = /projectB/.venv"]
        b_packages["Project B site-packages<br/>(django==4.2)"]
        b_py --> b_prefix
        b_prefix --> b_packages
    end

    sys_py -.->|Reference Core Libraries| a_py
    sys_py -.->|Reference Core Libraries| b_py

Through this mechanism, each project has its own “sandbox”. Although Python’s core libraries (standard library) are still referenced from the global system, third-party packages are installed independently inside each project’s folder.

venv — Python’s Built-in Virtual Environment #

venv has been available since Python 3.3 with no installation needed:

# Create a virtual environment in the .venv folder (the modern convention)
python -m venv .venv

# Or with another name
python -m venv env
python -m venv venv

The Directory Structure Created #

project/
├── .venv/
│   ├── bin/                    # macOS/Linux
│   │   ├── python              # symlink to Python
│   │   ├── python3
│   │   ├── pip
│   │   └── activate            # activation script
│   ├── Scripts/                # Windows
│   │   ├── python.exe
│   │   ├── pip.exe
│   │   └── activate.bat
│   ├── lib/
│   │   └── python3.12/
│   │       └── site-packages/  # installed packages live here
│   └── pyvenv.cfg             # venv configuration
├── src/
│   └── main.py
└── requirements.txt

Activating and Deactivating #

# === Activation ===

# macOS / Linux
source .venv/bin/activate

# Windows (Command Prompt)
.venv\Scripts\activate.bat

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

# After activation, the prompt changes:
# (.venv) user@computer:~/project$

# Verify — make sure Python points to .venv
which python        # macOS/Linux → /path/to/project/.venv/bin/python
where python        # Windows     → C:\...\project\.venv\Scripts\python.exe

# === Deactivation ===
deactivate

The name .venv (with a leading dot) is the convention recommended by PEP 8 and is recognized automatically by many editors (VS Code, PyCharm) and tools like uv. The names venv or env are also commonly used.

Managing Packages with pip #

With the virtual environment active, all pip operations happen inside .venv:

# Install packages
pip install requests
pip install "django>=4.2,<5.0"   # with version constraints
pip install flask==3.0.0          # specific version

# Install several packages at once
pip install fastapi uvicorn pydantic

# Upgrade packages
pip install --upgrade requests
pip install -U pip                # upgrade pip itself

# Uninstall packages
pip uninstall requests
pip uninstall requests django -y  # -y to skip confirmation

# List all installed packages
pip list
pip list --outdated               # packages with newer versions available

# Detail info for one package
pip show requests

# Search packages on PyPI
pip search "http client"          # (may not be available in some versions)

requirements.txt — Documenting Dependencies #

A requirements.txt file documents all the packages a project needs so other developers can reproduce the same environment:

# Generate from the active environment
pip freeze > requirements.txt

# Install from requirements.txt
pip install -r requirements.txt

Contents of a pip freeze-Generated requirements.txt #

# requirements.txt (output of pip freeze)
certifi==2024.2.2
charset-normalizer==3.3.2
idna==3.6
requests==2.31.0
urllib3==2.2.1

pip freeze includes every package including transitive dependencies with exactly pinned versions. That’s great for reproducibility, but it makes the file large and hard to maintain by hand.

Splitting requirements.txt per Environment #

A better practice is to split dependencies by purpose:

project/
├── requirements/
│   ├── base.txt      ← core dependencies always needed
│   ├── dev.txt       ← development tools (pytest, black, mypy)
│   └── prod.txt      ← production dependencies (gunicorn, sentry-sdk)
└── requirements.txt  ← optional: shortcut to base.txt

# requirements/base.txt
fastapi==0.110.0
pydantic==2.6.3
sqlalchemy==2.0.28

# requirements/dev.txt
-r base.txt          # include base.txt
pytest==8.1.1
pytest-asyncio==0.23.5
black==24.2.0
mypy==1.9.0
ruff==0.3.2

# requirements/prod.txt
-r base.txt
gunicorn==21.2.0
sentry-sdk==1.43.0

# Install for development
pip install -r requirements/dev.txt

# Install for production
pip install -r requirements/prod.txt

pip-tools — Better Dependency Management #

pip-tools separates the dependencies you declare (*.in) from the fully resolved result (*.txt), making maintenance easier:

pip install pip-tools

# requirements.in — only the direct dependencies you need
fastapi
pydantic>=2.0
sqlalchemy

# Generate requirements.txt from requirements.in
pip-compile requirements.in
# → produces requirements.txt with all transitive dependencies pinned

# Update all dependencies to the latest versions
pip-compile --upgrade requirements.in

# Sync the environment with requirements.txt (install + remove unneeded)
pip-sync requirements.txt

Output from requirements.in to requirements.txt:

# requirements.txt (generated by pip-compile)
#
# This file is autogenerated by pip-compile with Python 3.12
# Do not edit this file directly!
# Run pip-compile to regenerate.
#
anyio==4.3.0
    # via starlette
fastapi==0.110.0
    # via -r requirements.in
...

uv — A Very Fast Modern Tool #

uv is the newest Python package manager, written in Rust — 10–100x faster than pip for package installation. It replaces pip, pip-tools, and virtualenv all at once:

# Install uv
# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

# Or via pip
pip install uv

# Create a virtual environment
uv venv                         # creates .venv with the default Python
uv venv --python 3.12           # a specific Python version

# Install packages (even without activating!)
uv pip install requests
uv pip install -r requirements.txt

# Create a new project with pyproject.toml
uv init project-name
cd project-name

# Add dependencies (updates pyproject.toml automatically)
uv add fastapi
uv add --dev pytest black mypy   # dev dependencies

# Remove dependencies
uv remove requests

# Sync the environment with pyproject.toml
uv sync

# Run a script in the project environment (no manual activation)
uv run python main.py
uv run pytest

Speed comparison (installing 100 packages):

pip              → ~45 seconds
pip + cache      → ~8 seconds
uv               → ~0.5 seconds   ← ~100x faster than pip

uv uses the pyproject.toml format as the source of truth for dependencies — this is the modern Python standard (PEP 517/518/621) that’s more structured than requirements.txt. For new projects, using uv from the start is highly recommended.

pyproject.toml — The Modern Project Configuration Standard #

pyproject.toml is the unified configuration file for modern Python projects, replacing setup.py, setup.cfg, and requirements.txt:

# pyproject.toml
[project]
name = "cashier-system"
version = "1.0.0"
description = "Python-based cashier system"
requires-python = ">=3.11"

dependencies = [
    "fastapi>=0.110.0",
    "pydantic>=2.0",
    "sqlalchemy>=2.0",
    "python-dotenv>=1.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "black>=24.0",
    "mypy>=1.9",
    "ruff>=0.3",
]

[tool.black]
line-length = 88
target-version = ["py311"]

[tool.mypy]
python_version = "3.11"
strict = true

[tool.ruff]
line-length = 88

pipenv — An All-in-One Alternative #

pipenv combines a virtual environment and dependency management in a single tool:

pip install pipenv

# Create the environment and install packages
pipenv install requests
pipenv install pytest --dev      # dev dependency

# Activate a shell inside the environment
pipenv shell

# Run commands without activating
pipenv run python main.py
pipenv run pytest

# Generate requirements.txt from Pipfile.lock
pipenv requirements > requirements.txt

Pipenv uses two files: Pipfile (the dependencies you declare) and Pipfile.lock (pinned versions for reproducibility).

# Pipfile (managed automatically by pipenv)
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
requests = "*"
fastapi = ">=0.110"

[dev-packages]
pytest = "*"
black = "*"

[requires]
python_version = "3.12"

.gitignore for Virtual Environments #

Virtual environments must never be committed to git — they’re large and contain platform-specific binaries:

# .gitignore
# Virtual environments
.venv/
venv/
env/
ENV/

# pip
*.egg-info/
dist/
build/
*.egg

# pyc files
__pycache__/
*.py[cod]
*$py.class
*.pyo

# Environment variables
.env
.env.local
.env.*.local

# IDE
.idea/
.vscode/
*.swp

Never commit the .venv/ or venv/ directory to a repository. It can be hundreds of MB and contains binaries that aren’t portable across operating systems. Commit only requirements.txt or pyproject.toml — that’s all another developer needs to reproduce the environment.

Full New-Project Workflow #

Using venv + pip (Traditional) #

# 1. Create the project directory
mkdir my-project && cd my-project

# 2. Create a virtual environment
python -m venv .venv

# 3. Activate it
source .venv/bin/activate          # macOS/Linux
# or: .venv\Scripts\activate       # Windows

# 4. Upgrade pip
pip install --upgrade pip

# 5. Install dependencies
pip install fastapi uvicorn pydantic

# 6. Record the dependencies
pip freeze > requirements.txt

# 7. Create a .gitignore
echo ".venv/" >> .gitignore

# 8. Start coding...

Using uv (Modern — Recommended) #

# 1. Create a new project
uv init my-project && cd my-project

# 2. Add dependencies
uv add fastapi uvicorn pydantic

# 3. Add dev dependencies
uv add --dev pytest black mypy ruff

# 4. Sync the environment (automatically creates .venv if missing)
uv sync

# 5. Run the code
uv run python src/main.py

# 6. Create a .gitignore
echo ".venv/" >> .gitignore

Cloning an Existing Project #

# Clone the repository
git clone https://github.com/user/project.git
cd project

# === If the project uses requirements.txt ===
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# === If the project uses pyproject.toml + uv ===
uv sync

# === If the project uses a Pipfile ===
pipenv install

Troubleshooting Common Issues #

# Issue 1: "python" not found after activation
# Try: python3 instead of python
python3 -m venv .venv
source .venv/bin/activate
python3 --version

# Issue 2: PowerShell refuses to run scripts (Windows)
# Run as admin:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
# Then try again: .venv\Scripts\Activate.ps1

# Issue 3: "pip: command not found" after activation on Ubuntu
# Install python3-venv first
sudo apt install python3-venv python3-pip -y
python3 -m venv .venv

# Issue 4: VS Code doesn't detect the virtual environment
# Press Ctrl+Shift+P → "Python: Select Interpreter"
# Choose the path to .venv/bin/python

# Issue 5: Package installed but can't be imported
# Make sure the virtual environment is active when running code
which python      # must point to .venv, not the global Python
pip show requests # verify the package is indeed installed in this venv

Virtual Environment Tool Comparison #

Summary #

• Always create a virtual environment for every project — even small ones. Don’t install packages into the global Python.
• Use .venv as the name — the modern convention recognized automatically by editors and tools.
• Don’t commit .venv/ to git — only commit requirements.txt or pyproject.toml containing the dependency list.
• uv is the best choice for new projects — 100x faster than pip, manages venv + dependencies at once, and uses the modern pyproject.toml standard.
• pip freeze > requirements.txt pins every version exactly — good for reproducibility but hard to maintain. Consider pip-tools for larger projects.
• Split requirements.txt per environment (base, dev, prod) so you don’t install dev tools in production.
• pyproject.toml is the modern standard replacing setup.py and requirements.txt — use it for new projects.
• Verify the active environment with which python (macOS/Linux) or where python (Windows) before installing or running code.

← Previous: Regex   Next: Keywords →