↓ Skip to main content
jdhao’s digital space
  1. Posts/

Migrate Python Project from Pip to Uv

·801 words·4 mins·
Python Pip Uv
Table of Contents

Previously, our projects are still using the old Python 3.10, which will be deprecated soon this year. We are also using the good old pip and venv for the project/package management. This is already posing some issues for us:

  • venv is a pure virtual env tool without the ability to manage python versions (it just picks your local python), so it is hard to enforce the local env of each developer
  • pip’s requirements.txt is not a lockfile and thus does not ensure a fully reproducible python environment
  • pip as a package management tool is not performant enough in resolving and installing packages

In order to solve these problems, I took the effort to migrate our project management to uv. Here are some of the things that I learned in the process as a first-time uv user.

Install
#

There are a lot of ways to install uv, depending on the OS, the env (Docker or not). The official install guide is quite comprehensive already.

Migrate project from pip to uv
#

For existing project, you need to have pyproject.toml under the root of project. You need to have [project] section in pyproject.toml with name, version, description, readme and requires-python field.

set up Python version
#

Then pin the project to use specific python version: uv python pin 3.13. This will add a .python-version file for your project.

Note that the version in .python-version is definite source which python version is used, not the one specified in requires-python in pyproject.toml, which is usually a version range. See also here and here. For version specifiers, see this python packing doc.

Install packages
#

Most probably, you already have requirements.txt file for the project. Use uv add to add dependencies and development package/tools:

# add normal dependencies
uv add -r requirements.txt

# add development dependencies
uv add -r requirements_dev.txt

Finally, run uv sync to set up the project. Under the hood, uv will create the virtual env, resolve and install Python and all the dependencies.

Ref:

managing development tools (pytest, ruff)
#

To add development tools for your project, use uv add --dev <tool-name>, for example, uv add --dev pytest. This will create a group dependency named dev for your project.

If you only want to install dev dependencies, you can then run:

uv sync --group dev
# or use the following
# uv sync --dev

uv and Dockerfile
#

Warning

By default, unlike what pip does, uv does not compile the package to bytecode. During startup of the application in Kubernetes, the process may be killed by Kubernetes due to memory spike. This is a dangerous and rather difficult issue to diagnose if you do not know deeply about the internal working of uv.

It is better to always add --compile-bytecode option to avoid the issue. For me, I had one OOMkilling issue when the container is deployed to Kubernetes.

So when installing the dependencies in Dockerfile, instead of RUN uv sync --locked, make sure that you use RUN uv sync --locked --compile-bytecode.

Ref:

Do I need to activate the virtual env manually?
#

By default, the virtual env created by uv is named .venv. You can source to activate it, like what you usually do:

source .venv/bin/activate

However, activating the environment is not required by uv. The officially recommended way is to use uv run to run everything. Uv will pick the correct virtual env and run the tool for you. For example, uv run black src/ to format your project. Before uv run, it will install all the dependencies if they are not installed.

There is also uv issue proposing to add a uv shell to activate the virtual env.

uv add vs uv pip install
#

uv add is a project api, and uv pip install is a low level thing. uv pip install will install a package to your virtual env, but it will not change pyproject.toml, while uv add will install a package and update pyproject.toml.

uv add should be used if you are using pyproject.toml for project dependency management. You can use uv pip install if you are still using requirements.txt, but you want the speed benefit of uv.

See also:

lock and sync
#

uv.lock is a lockfile that contains the package versions (including the dependency of packages), so others can reproduce the project. pyproject.toml alone is not enough to ensure the same environment is used across the team.

To check if your uv.lock file is in sync with the pyproject.toml spec, use the following command:

uv lock --check

Ref:

benchmark
#

Benchmark command:

uv pip install --no-cache --compile-bytecode -r requirements.txt pip3 install --no-cache-dir -r requirements.txt

Time taken:

  • pip: 37.5s
  • uv: 13.8s

Related