Introduction
Python is a powerful language. During my internship, I was assigned to work on a Python project. I am familiar with Python as a script. So, it was not a big deal for me to work out the project which could meet most of the specifications. However, when it comes to deployment, it is different. For now, I uploaded it to test-pypi, used poetry for dependency management, and pyenv for developing and testing.
The whole thing is quite interesting and I am also hoping it could be helpful for someone.
(You don’t need any of the below things if you are using Python as a script)
Deployment
I am going to talk about deployment first because it worked fine on a single environment for one project. But when I finished most of my developing job and tried to put it into a pipeline, a question came out:
How to run this project on another machine?
I was thinking if I could download it via pip install <package_name>
and I found the instructions at official tutorials. Following the instructions, I refactored the project and made it a module to be downloaded via pip
.
However, after one week’s work, I decided to make some changes: use poetry to set up the project instead because it simplifies the whole process in some aspects:
- it can build the project with
poetry build
- it can publish with configuring it once
- it has only one configuration file
pyproject.toml
- it can add and install dependencies just like
npm
oryarn
does
I was amazed by this fancy tool and quickly change the current packaging workflow to poetry.
Development
As mentioned before, I didn’t manage the dependencies at first until I had a lot of problems after I deployed the tool on a different machine.
- Some dependencies are missing
- Using a development version rather than the deployed version
- How can I test in other Python environment?
Therefore, I started to use virtual environment to handle the above issues. Of course, if you are only using Python as a script, it is not worthy to get a new environment; if you are working on a single project, it might be meaningless to get an isolated environment. But, if you are working on a different projects, want to test in different environment, virtual environment could help a lot.
There are basically 3 tools to introduce:
- **pyenv **- downloads versions of Python
- **venv **- creates isolated virtual environment
- **virtualenv **- creates an virtual environment with a specified Python version
Also, if you are using Linux or mac, there is a plugin named pyenv-virtualenv to do all these work with only pyenv. Unfortunately, according to feature: pyenv virtualenv command · Issue #61. pyenv-win doesn’t support the plugin and might would not support it.
Here is the process of getting a virtual environment for developing and testing
- Download a version of Python:
pyenv install 3.9.0
- if we want to know where is it, looking for
C:\Users\%USERNAME%\.pyenv\pyenv-win\versions\3.9.0\python.exe
- Or, you can set the local python with
pyenv local 3.9.0
and get the location with commandpyenv which 3.9.0
. It should give you the location.
- if we want to know where is it, looking for
- Then create a virtual environment
virturalenv -p=<python_location> <your_venv_name>
. A folder should be generated in your project with name<your_venv_name>
and this is your virtual environment. - To activate, just run command
<your_venv_name>\Scripts\activate
and you will be in. Try runningpip list
to see if it is isolated. If you want to leave the environment, just rundeactivate
In case you didn’t notice, I did’t use venv
here because venv
can only create virtual environment with the same python within your environment. So we used virturalenv
as a workaround. You can pick up your tools based on your situations.
Acknowledgement
I am not using conda
which is also a well-known tool for this for no reason. I just randomly chose a tool and recorded everything’s come up to my mind. I might add new stories if I start to use conda. Let’s stay tuned.