My Python Development Environment in 2021
Choosing which tools to use in your Python development environment might be the toughest part of Python programming. The Python tooling ecosystem continues to evolve rapidly. I’ll share what I need my tools to do, what tools are a part of my development environment, and why I use each one.
Table of Contents
I develop web applications as my day job, specifically a Django application. I also maintain some open source packages and want to do more open source in the future. I use Ubuntu for my development operating system and avoid the system’s Python interpreter like the plague. In my younger years, I bricked my computer messing with the system interpreter. I love easy-to-use tools that improve code quality such as Flake8 and Black and want them to be seamlessly integrated into my development flow.
I want my tools to help:
- Install and switch between multiple Python interpreter versions while avoiding the system interpreter.
- Install and manage the version of other languages' package managers and run times such as Node.js.
- Make Python CLI utilities globally available without using the system interpreter or having to switch between virtual environments.
- Simplify publishing of Python packages because I don’t want to mess with a
- Manage numerous Python virtual environments.
- Run and maintain linters and formatters for multiple languages.
Asdf is a CLI tool that can manage multiple language runtime versions on a per-project basis. Initially, I just used Pyenv to manage my Python versions, but on some projects, I need to manage my Node.js version as well. I started using Nvm until a colleague showed me Asdf. The naming isn’t best, but naming things is hard. Other than the name, the tool rocks and having a single tool to manage all of my runtime versions simplifies my development environment. On top of that, Asdf’s Python plugin uses Pyenv to install Python versions and integrates Pyenv’s config formats which eased my transition.
- Install Asdf
- Add the Python plugin
asdf plugin add python
- Install your favorite Python version
asdf install python 3.9.6
- Set it globally
asdf global python 3.9.6
- Validate your favorite version is global
- Add the Node.js plugin
asdf plugin add nodejs
asdf install nodejs lts
- Set the LTS version globally
asdf global nodejs lts
- Validate it
Pipx helps you install and run CLI tools written in Python within isolated environments. I use Python CLI tools such as Awscli and Pre-commit, and I definitely don’t want these tools installed to the system interpreter. Installing these tools to their own virtual environment and manually switching quickly becomes annoying and inconvenient. Pipx fixes this by managing the isolated virtual environments and making these CLI tools globally available.
- Install pipx
- Use pipx to install Pre-commit
pipx install pre-commit
- Validate Pre-commit installed
Poetry helps manage and install dependencies of Python projects and allows you to publish Python packages. Why not just use Pip to install and manage your Python dependencies? Poetry builds on top of Pip with some nice extra features.
- management of per-project virtual environments.
- switching between virtual environments with different Python versions.
- maintaining a top-level record of both production dependencies and development dependencies.
- publishing of Python packages.
Why is a top-level record of dependencies important?
Pip freeze produces a file with all dependencies including
dependencies of dependencies. I want to keep my project’s dependencies up-to-date and without this top-level record,
updating packages becomes a struggle. With Poetry, I run
poetry update, and I’m up to date. I use docker to deploy
a lot of my projects, and image size always becomes an issue. Poetry separates production dependencies from development
dependencies which helps reduce image sizes. I love Pytest, but it doesn’t
need to be installed into my production containers.
- Install Poetry
- I recommend against installing with Pipx. For more info, read over this issue
- Start your new project
- Add a new dependency
poetry add httpx
- Add a new development dependency
poetry add --dev pytest
- Build and publish your Python package to PYPI
poetry publish --build --username laactech --password super_secret
- Install Pre-commit with Pipx
pipx install pre-commit
- Use my Pre-commit config or define your own
- Install the hook in your git repository
- Run the hook across your entire code base
pre-commit run --all-files
For a beginner starting their Python journey, this setup is overkill. For me, all of these tools make my day-to-day development life easier. The learning curves aren’t too steep and investing the time pays off. The Python tooling ecosystem continues to change rapidly. Pipsi got abandoned in favor of Pipx. Poetry and Pipenv duke it out for Python packaging dominance while other tools continue to pop up. I hope that I’m not writing this same article two years from now with all new tools.