LETSQL - Migrating to uv

As Python packaging tools evolve, uv has emerged as a promising alternative to Poetry, offering faster dependency resolution and simplified environment management. This guide walks through migrating a Python project from Poetry to uv, including handling maturin build backend, pre-commit hooks, and CI/CD configuration.

Motivation

uv offers significantly faster dependency resolution, simpler environment management, and better alignment with modern Python packaging standards. However, two decisive factors drove our migration from Poetry:

Nix Integration: The team at LETSQL heavily uses nix, and the poetry2nix is currently unmaintained ¹. The project even suggests the use of uv2nix as an alternative.
Duplicate Dependency Management: Our project uses Poetry for dependency management and maturin as a build backend. This configuration results in duplicated dependencies because maturin does not (and will not²) support synonyms in the pyproject.toml file, and Poetry lacks support for PEP-621³.

Prerequisites

Before starting the migration, install uv using the standalone installer in macOS and Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

Verify the installation:

uv --version

More details about installing it can be found here.

Additionally, since we are using Poetry ensure you have your Poetry project’s pyproject.toml and poetry.lock files.

Project Setup

The project is a hybrid Rust/Python package generated with maturin new --mixed and has the following structure:

Python package code in the python/ directory
Pre-commit hooks for quality checks
GitHub Actions CI for testing

The entire project can be found here.

Migration

When executing the migration process, it’s advisable to do so in stages, with each stage represented as a commit in your version control system (such as Git). This approach allows for easy rollback if needed. Additionally, ensure you create a backup of your pyproject.toml file before proceeding. After completing the migration, you must conduct comprehensive testing to verify that everything works as expected.

pyproject.toml

The first stage is to migrate the pyproject.toml file. In our case, it has the following content:

[build-system]
requires = ["maturin>=1.7,<2.0"]
build-backend = "maturin"

[tool.poetry]
name = "migrate-uv-demo"
version = "0.1.0"
description = ""
authors = ["Daniel Mesejo <mesejoleon@gmail.com>"]
readme = "README.md"
packages = [
    { include = "migrate_uv_demo", from = "python" },
]

[tool.poetry.dependencies]
python = "^3.10"
pyarrow = "^18.1.0"
attrs = "^24.3.0"
duckdb = {version = ">=0.10.3,<2", optional = true}

[tool.poetry.group.dev.dependencies]
pytest = "^8.3.4"
black = "^24.10.0"
pre-commit = "^4.0.1"
maturin = "^1.8.1"

[tool.poetry.extras]
duckdb = ["duckdb"]

[tool.maturin]
python-source = "python"
features = ["pyo3/extension-module"]

For migrating the pyproject.toml , we can use the uvx command like the following:

uvx pdm import pyproject.toml

The uvx command invokes a tool without installing it. More info can be found in the guides of uv.

After that, we have to perform the following cleanup:

Change back the build-system section to maturin, pdm import sets the build-backend to pdm.backend.
Check the proper configuration is set for the version. In this case, we want a single source of truth from the Cargo.toml, so the version must be dynamic = ["version"].
Remove duplicated dependencies specifications.
Remove all tool.poetry and tool.pdm sections.

After the pyproject.toml has been updated, execute (inside the project directory):

uv sync

Syncing ensures that all project dependencies are installed and up-to-date with the lock file. It will be created if the project virtual environment (.venv) does not exist.

To verify the migration of the pyproject.toml is successful, do:

uv run pytest

Additionally, if you wish, you can activate the virtual environment by doing:

source .venv/bin/activate  # Unix

A note on pip

uv does not install pip by default when creating a virtual environment, so any dependency using pip, and assuming it is available, will fail. maturin assumes that pip is available when executing develop, but recently they introduced the --uv option for using uv ⁴.

In any other case, to add pip to the dev group, execute:

uv add pip --dev

.pre-commit-config.yaml

The second stage is to migrate⁵ the pre-commit configuration (.pre-commit-config.yaml), in our case it has the following content:

repos:
-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
    -   id: trailing-whitespace
    -   id: end-of-file-fixer
    -   id: check-yaml
    -   id: check-added-large-files
-   repo: https://github.com/python-poetry/poetry
    rev: 1.8.2
    hooks:
    -   id: poetry-check
    -   id: poetry-lock
        args: ["--no-update"]
    -   id: poetry-install
-   repo: https://github.com/python-poetry/poetry-plugin-export
    rev: 1.8.0
    hooks:
    -   id: poetry-export
        args: ["--with", "dev", "--no-ansi", "--without-hashes", "-f", "requirements.txt", "-o", "requirements-dev.txt"]

uv already has integrations with pre-commit hooks to emulate the poetry one described in the config above, replace with:

-   repo: https://github.com/astral-sh/uv-pre-commit
    # uv version.
    rev: 0.5.4
    hooks:
      # Update the uv lockfile
      - id: uv-lock
      - id: uv-export
        args: ["--frozen", "--no-hashes", "--no-emit-project", "--all-groups",  "--output-file=requirements-dev.txt"]

Test the pre-commit hooks by doing the following:

pre-commit run --all-files

At this stage, you’ll be able to commit the work already done.

ci-test.yaml (GitHub actions)

Following the instructions in the GitHub actions of the uv documentation. The ci-test.yaml file defined in the repo should be updated to match the content below:

name: ci-test

on:
  push:
    branches:
      - main
      - master
    tags:
      - '*'
  pull_request:
  workflow_dispatch:

permissions:
  contents: read

jobs:
  linux:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        target: [x86_64]
        python-version: ["3.10"]
    steps:
      - uses: actions/checkout@v4

      - name: Rust latest
        run: rustup update

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: install uv
        uses: astral-sh/setup-uv@v5
        with:
          enable-cache: true

      - name: install dependencies
        run: uv sync --all-extras --all-groups --no-install-project
        working-directory: ${{ github.workspace }}

      - name: maturin develop
        run: uv run maturin develop --uv
        working-directory: ${{ github.workspace }}

      - name: pytest
        run: uv run pytest --import-mode=importlib
        working-directory: ${{ github.workspace }}

One thing to note is the choice of arguments for uv sync:

uv sync --all-extras --all-groups --no-install-project

The --no-install-project option excludes the project, but all of its dependencies are still installed. This avoids a (duplicated) maturin build; the installation of the package is done in a separate step:

uv run maturin develop

At this stage, you can commit and push to a new branch.

renovatebot (optional)

If you have configured renovatebot for your repo, then the PRs may fail with the following error:

Command failed: uv lock --upgrade-package pytest-cov
Using CPython 3.13.1 interpreter at: /opt/containerbase/tools/python/3.13.1/bin/python3
  × Failed to build `koenigsberg @
  │ file:///tmp/renovate/repos/github/mesejo/koenigsberg`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `maturin.prepare_metadata_for_build_editable` failed (exit
      status: 1)

The reason is that Rust is not installed; a partial solution is creating a workflow that modifies the PRs created by the renovatebot. A possible config is:

name: uv lock

on:
  pull_request:
    types: [opened, synchronize, reopened]
  workflow_dispatch:


jobs:
  uv-lock:
    runs-on: ubuntu-latest
    if: startsWith(github.head_ref, 'renovate')

    permissions:
      contents: write
      pull-requests: write

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          ref: ${{ github.head_ref }}
          fetch-depth: 0
          token: ${{ secrets.UV_LOCK_PAT_TOKEN }}

      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Rust latest
        run: rustup update

      - name: Install uv
        uses: astral-sh/setup-uv@v3
        with:
          enable-cache: true

      - name: uv lock
        run: uv lock

      - name: Check for changes
        id: git-check
        run: |
          git diff --quiet || echo "changes=true" >> $GITHUB_OUTPUT

      - name: Commit changes
        if: steps.git-check.outputs.changes == 'true'
        run: |
          git config --local user.email "github-actions[bot]@users.noreply.github.com"
          git config --local user.name "github-actions[bot]"
          git add .
          git commit -m "chore: uv lock"
          git push

Notice that in the checkout step, the token must be a Personal Access Token (PAT) with the proper permissions, otherwise the workflow will not trigger the other GitHub actions.

Conclusion

Migrating from Poetry to uv requires manual configuration but offers benefits like faster dependency resolution and simpler environment management.

Remember that uv is actively developed, so check the official documentation for the latest features and best practices.

Footnotes

see the README ↩︎
see maturin issue 632 ↩︎
see poetry issue 3332 ↩︎
https://github.com/PyO3/maturin/issues/1959↩︎
If you try to commit the changes done to the pyproject.toml without changing the pre-commit config, the hooks will fail with the following error: [tool.poetry] section not found↩︎

--- title: "Migrating to uv - A Short Guide" format: html: code-fold: false code-tools: true authors: - name: Daniel Mesejo date: "2025-01-07" description: | A simple guide migrating a Poetry and Maturin project to uv categories: [tutorial] date-modified: last-modified image: "images/poetry_to_uv.png" mainfont: Inter --- As Python packaging tools evolve, uv has emerged as a promising alternative to Poetry, offering faster dependency resolution and simplified environment management. This guide walks through migrating a Python project from Poetry to uv, including handling maturin build backend, pre-commit hooks, and CI/CD configuration. ## Motivation uv offers significantly faster dependency resolution, simpler environment management, and better alignment with modern Python packaging standards. However, two decisive factors drove our migration from Poetry: 1. **Nix Integration:** The team at LETSQL heavily uses nix, and the poetry2nix is currently unmaintained ^[see the [README](https://github.com/nix-community/poetry2nix?tab=readme-ov-file#maintenance-status-%EF%B8%8F-unmaintained-%EF%B8%8F)]. The project even suggests the use of uv2nix as an alternative. 2. **Duplicate Dependency Management:** Our project uses Poetry for dependency management and maturin as a build backend. This configuration results in duplicated dependencies because maturin does not (and will not^[see [maturin issue 632](https://github.com/PyO3/maturin/issues/632)]) support synonyms in the pyproject.toml file, and Poetry lacks support for PEP-621^[see [poetry issue 3332](https://github.com/python-poetry/poetry/issues/3332)]. ## Prerequisites Before starting the migration, install uv using the standalone installer in macOS and Linux: ```bash curl -LsSf https://astral.sh/uv/install.sh | sh ``` Verify the installation: ```bash uv --version ``` More details about installing it can be found [here](https://docs.astral.sh/uv/getting-started/installation/). Additionally, since we are using Poetry ensure you have your Poetry project's `pyproject.toml` and `poetry.lock` files. ## Project Setup The project is a hybrid Rust/Python package generated with `maturin new --mixed` and has the following structure: - Python package code in the `python/` directory - Pre-commit hooks for quality checks - GitHub Actions CI for testing The entire project can be found [here](https://github.com/mesejo/migrate-uv-demo). ## Migration When executing the migration process, it’s advisable to do so in stages, with each stage represented as a commit in your version control system (such as Git). This approach allows for easy rollback if needed. Additionally, ensure you create a backup of your `pyproject.toml` file before proceeding. After completing the migration, you must conduct comprehensive testing to verify that everything works as expected. ### pyproject.toml The first stage is to migrate the `pyproject.toml` file. In our case, it has the following content: ```toml [build-system] requires = ["maturin>=1.7,<2.0"] build-backend = "maturin" [tool.poetry] name = "migrate-uv-demo" version = "0.1.0" description = "" authors = ["Daniel Mesejo <mesejoleon@gmail.com>"] readme = "README.md" packages = [ { include = "migrate_uv_demo", from = "python" }, ] [tool.poetry.dependencies] python = "^3.10" pyarrow = "^18.1.0" attrs = "^24.3.0" duckdb = {version = ">=0.10.3,<2", optional = true} [tool.poetry.group.dev.dependencies] pytest = "^8.3.4" black = "^24.10.0" pre-commit = "^4.0.1" maturin = "^1.8.1" [tool.poetry.extras] duckdb = ["duckdb"] [tool.maturin] python-source = "python" features = ["pyo3/extension-module"] ``` For migrating the `pyproject.toml` , we can use the `uvx` command like the following: ```bash uvx pdm import pyproject.toml ``` The `uvx` command invokes a tool without installing it. More info can be found in the guides of [uv](https://docs.astral.sh/uv/guides/tools/#running-tools). After that, we have to perform the following cleanup: 1. Change back the build-system section to maturin, `pdm import` sets the `build-backend` to [`pdm.backend`](https://backend.pdm-project.org/). 2. Check the proper configuration is set for the version. In this case, we want a single source of truth from the `Cargo.toml`, so the version must be `dynamic = ["version"]`. 3. Remove duplicated dependencies specifications. 4. Remove all `tool.poetry` and `tool.pdm` sections. After the `pyproject.toml` has been updated, execute (inside the project directory): ```bash uv sync ``` Syncing ensures that all project dependencies are installed and up-to-date with the lock file. It will be created if the project virtual environment (.venv) does not exist. To verify the migration of the `pyproject.toml` is successful, do: ```bash uv run pytest ``` Additionally, if you wish, you can activate the virtual environment by doing: ```bash source .venv/bin/activate # Unix ``` ### A note on pip uv does not install pip by default when creating a virtual environment, so any dependency using `pip`, and assuming it is available, will fail. `maturin` assumes that pip is available when executing `develop`, but recently they introduced the `--uv` option for using uv ^[https://github.com/PyO3/maturin/issues/1959]. In any other case, to add `pip` to the dev group, execute: ```bash uv add pip --dev ``` ## .pre-commit-config.yaml The second stage is to migrate^[If you try to commit the changes done to the `pyproject.toml` without changing the pre-commit config, the hooks will fail with the following error: `[tool.poetry] section not found`] the pre-commit configuration (`.pre-commit-config.yaml`), in our case it has the following content: ```yaml repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v5.0.0 hooks: - id: trailing-whitespace - id: end-of-file-fixer - id: check-yaml - id: check-added-large-files - repo: https://github.com/python-poetry/poetry rev: 1.8.2 hooks: - id: poetry-check - id: poetry-lock args: ["--no-update"] - id: poetry-install - repo: https://github.com/python-poetry/poetry-plugin-export rev: 1.8.0 hooks: - id: poetry-export args: ["--with", "dev", "--no-ansi", "--without-hashes", "-f", "requirements.txt", "-o", "requirements-dev.txt"] ``` `uv` already has [integrations](https://docs.astral.sh/uv/guides/integration/pre-commit/) with pre-commit hooks to emulate the poetry one described in the config above, replace with: ```yaml - repo: https://github.com/astral-sh/uv-pre-commit # uv version. rev: 0.5.4 hooks: # Update the uv lockfile - id: uv-lock - id: uv-export args: ["--frozen", "--no-hashes", "--no-emit-project", "--all-groups", "--output-file=requirements-dev.txt"] ``` Test the pre-commit hooks by doing the following: ```bash pre-commit run --all-files ``` At this stage, you'll be able to commit the work already done. ## ci-test.yaml (GitHub actions) Following the instructions in the GitHub actions of the uv [documentation](https://docs.astral.sh/uv/guides/integration/github/). The `ci-test.yaml` file defined in the repo should be updated to match the content below: ```yaml name: ci-test on: push: branches: - main - master tags: - '*' pull_request: workflow_dispatch: permissions: contents: read jobs: linux: runs-on: ubuntu-latest strategy: matrix: target: [x86_64] python-version: ["3.10"] steps: - uses: actions/checkout@v4 - name: Rust latest run: rustup update - uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: install uv uses: astral-sh/setup-uv@v5 with: enable-cache: true - name: install dependencies run: uv sync --all-extras --all-groups --no-install-project working-directory: ${{ github.workspace }} - name: maturin develop run: uv run maturin develop --uv working-directory: ${{ github.workspace }} - name: pytest run: uv run pytest --import-mode=importlib working-directory: ${{ github.workspace }} ``` One thing to note is the choice of arguments for `uv sync`: ```bash uv sync --all-extras --all-groups --no-install-project ``` The `--no-install-project` option excludes the project, but all of its dependencies are still installed. This avoids a (duplicated) maturin build; the installation of the package is done in a separate step: ```bash uv run maturin develop ``` At this stage, you can commit and push to a new branch. ### renovatebot (optional) If you have configured [renovatebot](https://docs.renovatebot.com/) for your repo, then the PRs may fail with the following error: ```bash Command failed: uv lock --upgrade-package pytest-cov Using CPython 3.13.1 interpreter at: /opt/containerbase/tools/python/3.13.1/bin/python3 × Failed to build `koenigsberg @ │ file:///tmp/renovate/repos/github/mesejo/koenigsberg` ├─▶ The build backend returned an error ╰─▶ Call to `maturin.prepare_metadata_for_build_editable` failed (exit status: 1) ``` The reason is that Rust is not installed; a partial solution is creating a workflow that modifies the PRs created by the renovatebot. A possible config is: ```yaml name: uv lock on: pull_request: types: [opened, synchronize, reopened] workflow_dispatch: jobs: uv-lock: runs-on: ubuntu-latest if: startsWith(github.head_ref, 'renovate') permissions: contents: write pull-requests: write steps: - name: Checkout repository uses: actions/checkout@v4 with: ref: ${{ github.head_ref }} fetch-depth: 0 token: ${{ secrets.UV_LOCK_PAT_TOKEN }} - uses: actions/setup-python@v5 with: python-version: '3.12' - name: Rust latest run: rustup update - name: Install uv uses: astral-sh/setup-uv@v3 with: enable-cache: true - name: uv lock run: uv lock - name: Check for changes id: git-check run: | git diff --quiet || echo "changes=true" >> $GITHUB_OUTPUT - name: Commit changes if: steps.git-check.outputs.changes == 'true' run: | git config --local user.email "github-actions[bot]@users.noreply.github.com" git config --local user.name "github-actions[bot]" git add . git commit -m "chore: uv lock" git push ``` Notice that in the checkout step, the token must be a Personal Access Token [(PAT)](https://github.blog/security/application-security/introducing-fine-grained-personal-access-tokens-for-github/) with the proper permissions, otherwise the workflow will not trigger the other GitHub actions. ## Conclusion Migrating from Poetry to uv requires manual configuration but offers benefits like faster dependency resolution and simpler environment management. Remember that uv is actively developed, so check the [official documentation](https://docs.astral.sh/uv/) for the latest features and best practices.