Poetry

Page content

1. Intro

Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.

Install on OpenBSD as root

… and update path

doas su -
pip3 install --user poetry
pip3 install poetry -U

cat << EOF >> ~/.profile

# Poetry, added $(date)
export PATH=\$PATH:/root/.local/bin
EOF

. ~/.profile

Install as User poetry

doas pip3 install --user poetry
pip3 install poetry -U

cat << EOF >> ~/.profile

# Poetry, added $(date)
export PATH=\$PATH:~/.local/bin
EOF

. ~/.profile

Install Debian

Run as User or Root

sudo apt-get install -y python3 python3-distutils
curl -sSL https://install.python-poetry.org/ | python3 -

cat << EOF >> ~/.bashrc

# Poetry, added $(date)
export PATH=\$PATH:~/.local/bin
EOF

source ~/.bashrc

Basic Usage

$ poetry --version
Poetry version 1.1.11

$ poetry self update
You are using the latest version

$ poetry self update --preview
Updating to 1.2.0a2

  RuntimeError

  Could not find poetry-1.2.0a2-linux.sha256sum file

  at ~/.poetry/lib/poetry/console/commands/self/update.py:260 in _update
      256│         try:
      257│             r = urlopen(base_url + "/{}/{}".format(version, checksum))
      258│         except HTTPError as e:
      259│             if e.code == 404:
    → 260│                 raise RuntimeError("Could not find {} file".format(checksum))
      261│
      262│             raise
      263│
      264│         checksum = r.read().decode().strip()

# Downgrade
$ poetry self update 0.8.0

poetry new

$ poetry new poetry-demo
Created package poetry_demo in poetry-demo

$ tree poetry-demo/
poetry-demo/
├── poetry_demo
│   └── __init__.py
├── pyproject.toml
├── README.rst
└── tests
    ├── __init__.py
    └── test_poetry_demo.py

2 directories, 5 files

or if you have an existing project, you can do the poetry init

poetry init

$ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [test]:
Version [0.1.0]:
Description []:  just a little test
Author [None, n to skip]:  That's Me
License []:
Compatible Python versions [^3.9]:

Would you like to define your main dependencies interactively? (yes/no) [yes]
You can specify a package in the following forms:
  - A single name (requests)
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Search for package to add (or leave blank to continue):

Would you like to define your development dependencies interactively? (yes/no) [yes]
Search for package to add (or leave blank to continue):

Generated file

[tool.poetry]
name = "test"
version = "0.1.0"
description = "just a little test"
authors = ["That's Me"]

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"


Do you confirm generation? (yes/no) [yes]

poetry add

$ poetry add pendulum
Creating virtualenv test-nWsx-QRW-py3.9 in /home/stoege/.cache/pypoetry/virtualenvs
Using version ^2.1.2 for pendulum

Updating dependencies
Resolving dependencies... (0.5s)

Writing lock file

Package operations: 4 installs, 0 updates, 0 removals

  • Installing six (1.16.0)
  • Installing python-dateutil (2.8.2)
  • Installing pytzdata (2020.1)
  • Installing pendulum (2.1.2)

poetry show tree

$ poetry show --tree
pendulum 2.1.2 Python datetimes made easy
├── python-dateutil >=2.6,<3.0
│   └── six >=1.5
└── pytzdata >=2020.1

poetry show –latest

$ poetry show --latest
pendulum        2.1.2  2.1.2  Python datetimes made easy
python-dateutil 2.8.2  2.8.2  Extensions to the standard Python datetime module
pytzdata        2020.1 2020.1 The Olson timezone database for Python.
six             1.16.0 1.16.0 Python 2 and 3 compatibility utilities

poetry build

$ poetry build
Building test (0.1.0)

  ModuleOrPackageNotFound

  No file/folder found for package test

  at ~/.poetry/lib/poetry/_vendor/py3.9/poetry/core/masonry/utils/module.py:63 in __init__
       59│                             "from": str(src.relative_to(self._path)),
       60│                         }
       61│                     ]
       62│                 else:
    →  63│                     raise ModuleOrPackageNotFound(
       64│                         "No file/folder found for package {}".format(name)
       65│                     )
       66│
       67│         for package in packages:

poetry publish

$ poetry publish
No files to publish. Run poetry build first or use the --build option.

set python version

poetry env use $(which python3.9)

-> pyproject.toml -> python = "^3.9"

set python version 3.9

poetry env use $(which python3.9)

set python version 3.10

poetry env use $(which python3.10)

set python version 3.11

poetry env use $(which python3.11)

Use an Existing requirements.txt File

$ cat requirements.txt
beautifulsoup4==4.9.3
certifi==2020.12.5
chardet==4.0.0

$ poetry add `cat requirements.txt`
Creating virtualenv rp-require-0ubvZ-py3.9 in ~/Library/Caches/pypoetry/virtualenvs

Updating dependencies
Resolving dependencies... (6.2s)

Writing lock file

Package operations: 7 installs, 0 updates, 0 removals

  • Installing certifi (2020.12.5)
  • Installing chardet (4.0.0)
  • Installing idna (2.10)
...

or

awk -F '==' '!/sha256/{print $1}' requirements.txt |xargs -n1 poetry add

or

poetry add $(awk -F '==' '!/sha256/{print $1}' requirements.txt |tr '\n' ' ')

Create requirements.txt From poetry.lock

poetry export --output requirements.txt

Command Reference

$ poetry --version  Show the version of your Poetry installation.
$ poetry new          Create a new Poetry project.
$ poetry init         Add Poetry to an existing project.
$ poetry run          Execute the given command with Poetry.
$ poetry add          Add a package to pyproject.toml and install it.
$ poetry update       Update your project’s dependencies.
$ poetry install      Install the dependencies.
$ poetry show         List installed packages.
$ poetry lock         Pin the latest version of your dependencies into poetry.lock.
$ poetry lock --no-update   Refresh the poetry.lock file without updating any dependency version.
$ poetry check        Validate pyproject.toml.
$ poetry config --list      Show the Poetry configuration.
$ poetry env list     List the virtual environments of your project.
$ poetry export       Export poetry.lock to other formats.

Venv

check in-project settings

poetry config virtualenvs.in-project
poetry config virtualenvs.create

set in-project settings

# In Project
poetry config virtualenvs.in-project true
poetry config virtualenvs.in-project

# Create Virtual
poetry config virtualenvs.create true
poetry config virtualenvs

2. Move Project to .venv

you have your Virtual Env somewhere in your Home Folder and like to move it to the Project Folder ?

List Virtual Env

poetry env list
app-rBXXtrXX-py3.10 (Activated)

Remove Virtual Env

poetry env remove app-rBXXtrXX-py3.10
Deleted virtualenv: /Users/XXXXXXXX/Library/Caches/pypoetry/virtualenvs/app-rBXXtrXX-py3.10

Set in-project true

poetry config virtualenvs.in-project true

Check Config

poetry config -vvv
# poetry config -vvv
Loading configuration file /home/USERNAME/.config/pypoetry/config.toml

# cat /home/USERNAME/.config/pypoetry/config.toml
[virtualenvs]
in-project = true

Poetry Lock (why ?)

poetry lock
Creating virtualenv app in /Users/XXXXXXXX/YOUR_REPO_NAME/.venv
Updating dependencies
Resolving dependencies... (1.7s)

Poetry Install

poetry install
poetry install
Installing dependencies from lock file

Package operations: 65 installs, 0 updates, 0 removals

  • Installing idna (3.4)
  • Installing sniffio (1.3.0)
  • Installing anyio (3.6.2)
  • Installing pycparser (2.21)
  • Installing certifi (2022.9.24)
  • Installing cffi (1.15.1)

Update Poetry on OpenBSD 7.2

before

poetry --version
Poetry version 1.1.13

upgrade

pip install poetry -U

oder

poetry self update

Update Poetry on OpenBSD

doas su -
poetry --version
poetry self update
poetry --version

then switch to the appropriate User and check Virtual Env / Update if needed

su - webmaster
cd /home/webmaster/web_domain_stoege_net
poetry shell
poetry install

after upgrade …

poetry --version
Poetry (version 1.2.2)
Poetry (version 1.3.1)
Poetry (version 1.3.2)
Poetry (version 1.4.0)
Poetry (version 1.5.0), Mai 2023
...

Downgrade Poetry

Poetry v.1.4.1 had some issues with debugpy: “Installing debugpy (1.6.6): Failed”.

poetry self update 1.4.0

3. Customization

Command Prompt

show current config

poetry config --list
...
virtualenvs.prompt = "{project_name}-py{python_version}"

show default prompt

user@host /tmp/gugus1234 (main) $ poetry shell
Spawning shell within /tmp/gugus1234/.venv

(gugus1234-py3.11) user@host /tmp/gugus1234 (main) $

update prompt

poetry config virtualenvs.prompt "hello-from-py{python_version}"

kill .venv

rm -rf .venv

build new .venv

user@host /tmp/gugus1234 (main) $ poetry shell
Creating virtualenv gugus1234 in /tmp/gugus12344/.venv
Spawning shell within /tmp/gugus1234/.venv

. /tmp/gugus1234/.venv/bin/activate

user@host /tmp/gugus1234 (main) $ . /tmp/ugus1234/.venv/bin/activate
(hello-from-py3.11) user@host /tmp/gugus1234 (main) $ 

4. Strange Stuff

Collection of strange behaviours & errors. hope i can keep this section small ;)

Install Error Fastapi

(gugus2-py3.10) puffy202$ poetry add [email protected]                                 

Updating dependencies
Resolving dependencies... (0.9s)

Package operations: 10 installs, 0 updates, 0 removals

  • Installing exceptiongroup (1.1.3)
  • Installing idna (3.4)
  • Installing sniffio (1.3.0)
  • Installing typing-extensions (4.7.1)
  • Installing annotated-types (0.5.0)
  • Installing anyio (3.7.1)
    • Installing pydantic-core (2.6.3): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel
  
  Running `maturin pep517 build-wheel -i /tmp/tmp4eqst_ob/.venv/bin/python --compatibility off`
  📦 Including license file "/tmp/tmpb2vbe9ix/pydantic_core-2.6.3/LICENSE"
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings
  🐍 Found CPython 3.10 at /tmp/tmp4eqst_ob/.venv/bin/python
  📡 Using build options features, bindings from pyproject.toml
     Compiling autocfg v1.1.0
     Compiling proc-macro2 v1.0.64
  Traceback (most recent call last):
    File "/home/stoege/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/stoege/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/stoege/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/tmp4eqst_ob/.venv/lib/python3.10/site-packages/maturin/__init__.py", line 111, in build_wheel
      return _build_wheel(wheel_directory, config_settings, metadata_directory)
    File "/tmp/tmp4eqst_ob/.venv/lib/python3.10/site-packages/maturin/__init__.py", line 90, in _build_wheel
      result = subprocess.run(command, stdout=subprocess.PIPE)
    File "/usr/local/lib/python3.10/subprocess.py", line 505, in run
      stdout, stderr = process.communicate(input, timeout=timeout)
    File "/usr/local/lib/python3.10/subprocess.py", line 1141, in communicate
      stdout = self.stdout.read()
  KeyboardInterrupt
  

  at ~/.local/lib/python3.10/site-packages/poetry/installation/chef.py:147 in _prepare
      143│ 
      144│                 error = ChefBuildError("\n\n".join(message_parts))
      145│ 
      146│             if error is not None:
    → 147│                 raise error from None
      148│ 
      149│             return path
      150│ 
      151│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pydantic-core (2.6.3) not supporting PEP 517 builds. You can verify this by running 'pip wheel --use-pep517 "pydantic-core (==2.6.3)"'.

Solution

  • install lower Version First (0.85.0)
  • upgrade afterwars
(gugus2-py3.10) puffy202$ poetry add [email protected]  

Updating dependencies
Resolving dependencies... (0.4s)

Package operations: 3 installs, 0 updates, 0 removals

  • Installing pydantic (1.10.12)
  • Installing starlette (0.20.4)
  • Installing fastapi (0.85.0)

Writing lock file
(gugus2-py3.10) puffy202$ poetry add [email protected] 

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 0 installs, 2 updates, 0 removals

  • Updating starlette (0.20.4 -> 0.27.0)
  • Updating fastapi (0.85.0 -> 0.103.0)

Writing lock file

do i have to know why? no does it work ? yes

KeyRing Backend

on Day, i couldn’t use poetry anymore. whatever i did, i mostly got such an error:

(poetry-py3.10) $ poetry add package
Using virtualenv: /var/www/virtual/.../.venv

  Error

  g-io-error-quark: Cannot autolaunch D-Bus without X11 $DISPLAY (0)

and the persistent fix

cat << EOF >> ~/.profile

# fix strange poetry error, $(date), https://blog.stoege.net/posts/poetry/#keyring-backend
export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring >> ~/.profile
EOF

Full Error Log, Fix & confirmation

user@host$ poetry install
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing config (0.5.1): Failed

  Error

  g-io-error-quark: Cannot autolaunch D-Bus without X11 $DISPLAY (0)

  at ~/.local/lib/python3.9/site-packages/keyring/backends/libsecret.py:134 in get_credential
      130│         Otherwise, it will return the first username and password combo that it finds.
      131│         """
      132│         query = self._query(service, username)
      133│         try:
    → 134│             items = Secret.password_search_sync(
      135│                 self.schema, query, Secret.SearchFlags.UNLOCK, None
      136│             )
      137│         except GLib.Error as error:
      138│             quark = GLib.quark_try_string('g-io-error-quark')

The following error occurred when trying to handle this error:


  KeyringLocked

  Failed to unlock the item!

  at ~/.local/lib/python3.9/site-packages/keyring/backends/libsecret.py:140 in get_credential
      136│             )
      137│         except GLib.Error as error:
      138│             quark = GLib.quark_try_string('g-io-error-quark')
      139│             if error.matches(quark, Gio.IOErrorEnum.FAILED):
    → 140│                 raise KeyringLocked('Failed to unlock the item!') from error
      141│             raise
      142│         for item in items:
      143│             username = item.get_attributes().get("username")
      144│             try:

user@host$ export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring
user@host$ poetry install
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing config (0.5.1)

Installing the current project: tutorial-flaskr (0.1.0)
user@host$

[Errno 2] No such file or directory: ‘python’

$ poetry shell

$ rm -rf $(poetry config virtualenvs.path)/*

Alternative: venv

Create Venv

python3 -m venv env

Activate venv

source ./env/bin/activate

Install Packages

pip3 install fastapi

Deactivate venv

deactivate

Cleanup

rm -rf venv

Any Comments ?

sha256: dd082caac3ae7b25baf8894df57d2e1da976d8c940d3d53da116c78f1cb716a4