14. Converting poetry.lock to requirements.txt for Legacy CI/CD Pipelines

Poetry modernizes Python dependency management with pyproject.toml and poetry.lock for reproducible builds. However, legacy CI/CD pipelines (Jenkins, Travis CI, older GitLab) rely on pip install -r requirements.txt. This guide shows how to export poetry.lock to requirements.txt, preserving exact versions and hashes for reliable deploys.

Why Convert poetry.lock to requirements.txt?

• Reproducibility: poetry.lock pins exact versions; exporting prevents “works on my machine” issues.
• Legacy Compatibility: No Poetry in old pipelines—use familiar pip.
• Security: Include hashes to verify packages.
• CI Speed: Pre-generate file, avoid runtime Poetry install.

Prerequisites

• Poetry installed locally: pipx install poetry or curl -sSL https://install.python-poetry.org | python3 -
• Project with pyproject.toml and poetry.lock.

Method 1: poetry export (Recommended)

Poetry’s built-in exporter handles groups, extras, and hashes.

# Basic export (main group)
poetry export -f requirements.txt --output requirements.txt --without-hashes

# With hashes (secure, recommended for CI)
poetry export -f requirements.txt --output requirements.txt

# Specific groups (dev, test)
poetry export -f requirements.txt --output requirements-dev.txt --dev --without-hashes

# All groups
poetry export -f requirements.txt --output requirements-all.txt --all-extras

Verify:

# Create venv
python -m venv tmp-venv
source tmp-venv/bin/activate  # Linux/macOS
# tmp-venv\\Scripts\\activate  # Windows

pip install -r requirements.txt
pip list  # Compare with `poetry show --latest`
deactivate
rm -rf tmp-venv

Method 2: Scripted Export for Custom Needs

For automation or filtering:

# export_lock.py
import tomllib
import json
from pathlib import Path

def poetry_to_reqs(lockfile="poetry.lock"):
    with open(lockfile, "rb") as f:
        data = tomllib.load(f)
    
    reqs = []
    for pkg in data["package"]:
        name = pkg["name"].replace("-", "_")
        version = pkg["version"]
        hash_ = pkg.get("content-hash")  # Or files hashes
        if hash_:
            reqs.append(f"{name}{version} --hash=sha256:{hash_}")
        else:
            reqs.append(f"{name}{version}")
    
    Path("requirements.txt").write_text("\n".join(reqs))

poetry_to_reqs()

Run: python export_lock.py

Integrating in CI/CD Pipelines

GitHub Actions Example

# .github/workflows/ci.yml
name: CI
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with: {python-version: '3.12'}
    - run: pip install poetry  # Or pre-commit hook to generate
    - run: poetry export -f requirements.txt --output requirements.txt
    - run: pip install -r requirements.txt
    - run: pytest

Jenkins Pipeline

pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        sh 'pipx install poetry'
        sh 'poetry export -f requirements.txt --output requirements.txt'
        sh 'pip install -r requirements.txt'
      }
    }
  }
}

Commit requirements.txt to repo (gitignore poetry.lock if desired, but regenerate on changes).

Troubleshooting

• Hash mismatches: Use --without-hashes for dev; add hashes for prod.
• Dev deps missing: Add --dev.
• Platform-specific: Export on target OS or use poetry lock --no-update.
• Large lockfiles: Split by group.
• Version drift: Always export from latest poetry.lock.

Conclusion

Exporting poetry.lock to requirements.txt bridges modern Poetry with legacy pipelines. Automate via pre-commit hooks: pre-commit.com/hooks or GitHub Actions. For full Poetry CI, upgrade pipelines. Questions? Comment below. Check Poetry docs.