The go-to resource for upgrading Python, Django, Flask, and your dependencies.

Converting poetry.lock to requirements.txt for Legacy CI/CD Pipelines


You’ve adopted Poetry for its superior dependency resolution and poetry.lock pinning, moving beyond pip’s unpredictability. But legacy CI/CD pipelines—like Jenkins, Travis CI, or older GitLab CI—still expect the traditional pip install -r requirements.txt. In this article, we’ll export your poetry.lock to a pip-compatible requirements.txt, maintaining reproducibility and adding security via hashes where needed.

Why Convert poetry.lock to requirements.txt?

Poetry’s lockfile captures a complete, resolved dependency tree—far superior to loose specifiers in requirements.txt. Exporting it lets you develop with Poetry’s power while deploying via pip in constrained environments.

Benefits:

  • Reproducibility: Exact versions and sub-dependencies prevent environment drift—what installs today should install tomorrow.
  • Broad compatibility: Pip works everywhere; skip Poetry installation in air-gapped or minimal CI runners.
  • Security hardening: Optional hashes verify package integrity against supply-chain attacks.
  • CI efficiency: No bootstrap overhead; faster builds by minutes.

Trade-offs:

  • Duplicate lockfiles: requirements.txt must sync with poetry.lock—use hooks to automate.
  • Hash fragility: Platform or index differences can break installs; omit for dev, include for prod.
  • Misses Poetry features: No dev groups or plugins in basic exports.

Of course, upgrading CI to Poetry support avoids exports altogether—but that’s not always feasible.

Prerequisites

  • Poetry 1.2+: $ pipx install poetry (preferred) or $ curl -sSL https://install.python-poetry.org | python3 -
  • Project with pyproject.toml and poetry.lock.

Poetry’s built-in exporter handles groups, extras, and hashes.

$ poetry export -f requirements.txt --output requirements.txt --without-hashes   # main group only
$ poetry export -f requirements.txt --output requirements.txt                    # with hashes (recommended for prod)
$ poetry export -f requirements.txt --output requirements-dev.txt --dev --without-hashes  # dev dependencies
$ poetry export -f requirements.txt --output requirements-all.txt --all-extras   # all groups and extras

Expected requirements.txt (example):

requests==2.32.3
click==8.1.7

(Versions/hashes from your lockfile; cat the file to inspect.)

Verify:

$ python -m venv tmp-venv
$ source tmp-venv/bin/activate  # Linux/macOS; Windows: tmp-venv\\Scripts\\activate

$ pip install -r requirements.txt

$ pip list  # Should match `poetry show --latest` output

$ deactivate
$ rm -rf tmp-venv

Tip: If lists differ, your export missed groups—use --dev or --all-extras.

Method 2: Scripted Export for Custom Needs

For automation or filtering:

# export_lock.py (basic version; for production, use `poetry export`)
import tomllib
from pathlib import Path

def poetry_to_reqs(lockfile="poetry.lock", include_dev=False):
    with open(lockfile, "rb") as f:
        data = tomllib.load(f)
    
    reqs = []
    for pkg in data["package"]:
        if not include_dev and pkg.get("category") == "dev":
            continue
        name = pkg["name"]  # Pip names use '-', no replacement needed
        version = pkg["version"]
        reqs.append(f'{name}=={version}')
    
    Path("requirements.txt").write_text("\n".join(reqs) + "\n")
    print("Exported to requirements.txt")

if __name__ == "__main__":
    poetry_to_reqs()

Note: Skips full hashes/sources—use for dev prototyping. Handles deps by category; toggle include_dev=True.

Quick Alternatives Comparison

Before scripting, weigh options:

  • poetry export (default): Full-featured, official ✓
  • pip freeze > reqs.txt (post-poetry install): Easy but impure (env state)
  • poetry2reqs pip: Third-party; check compatibility

Prefer official when possible.

$ python export_lock.py
Exported to requirements.txt
$ cat requirements.txt
requests==2.32.3
click==8.1.7
...

(Matches your lockfile’s resolved packages.)

Integrating in CI/CD Pipelines

GitHub Actions Example

# .github/workflows/ci.yml
name: CI
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with: {python-version: '3.12'}
    - run: pip install poetry  # Or pre-commit hook to generate
    - run: poetry export -f requirements.txt --output requirements.txt
    - run: pip install -r requirements.txt
    - run: pytest

Jenkins Pipeline

pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        sh 'pipx install poetry'
        sh 'poetry export -f requirements.txt --output requirements.txt'
        sh 'pip install -r requirements.txt'
      }
    }
  }
}

Best practices:

  • Commit requirements.txt to repo for prod deploys.
  • .gitignore: Keep both lockfiles; regenerate reqs on Poetry changes.
  • Automate: Pre-commit hook ensures sync:
# .pre-commit-config.yaml
repos:
  - repo: local
    hooks:
      - id: poetry-export
        name: 'Export Poetry lock to requirements.txt'
        entry: poetry export -f requirements.txt --output requirements.txt --without-hashes
        language: system
        files: ^poetry\\.lock$
        pass_filenames: false

Install: $ pre-commit install

  • Security: Use --hash for prod; --without-hashes dev/CI caches.
  • Legacy limits: Older pipelines lack Python caching—exports add little overhead vs full Poetry.

Troubleshooting Common Issues

Here are pitfalls we’ve encountered and fixes:

  • “No poetry.lock found”: Run $ poetry lock first to generate/update it.
  • Hash verification fails on pip install: Platform/wheel mismatch—use $ poetry export --without-hashes or export on CI target OS.
  • Missing dev/test deps: Specify $ poetry export --dev or --only=main explicitly.
  • requirements.txt too large/slow: Export groups separately: --only=main, --dev; pip handles duplicates fine.
  • Version drift between exports: Lockfile unchanged? Use $ poetry lock --no-update to avoid resolutions.
  • TOML parse errors: Outdated Poetry—upgrade: $ poetry self update.
  • Source repos (git/private): Hashes may miss; export includes URLs, but test pip install.

Pro tip: Always verify with the venv test from Method 1 before committing.

Wrapping Up

You’ve now got reliable ways to span Poetry and pip worlds. Here’s what to remember:

Key Takeaways:

  • poetry export handles 90% of cases—prioritize it over scripts.
  • Always verify: Isolated venv pip list should mirror poetry show.
  • Automate exports: Pre-commit hooks keep files in sync without manual toil.
  • Hashes boost security but demand careful platform matching.
  • Long-term: Migrate CI to Poetry for richer features like parallel installs.

Try it: Export your project’s lockfile, spin up a Docker container mimicking your CI (e.g., docker run python:3.12-slim), and test pip install.

Further Reading:

Questions or edge cases? Share in the comments—we learn together.

Sponsored by Durable Programming

Need help maintaining or upgrading your Python application? Durable Programming specializes in keeping Python apps secure, performant, and up-to-date.

Hire Durable Programming