CI for a Python project of the National Institute of Standards and Technology
Interesting coincidences.
Just a few days ago as I started to work on the Open Source Developer Course I started to collect Open Source projects by governments and today the first project I found on PyDigger that I wanted to try to contribute to was a project of the NIST - the National Institute of Standards and Technology.
Side note: I find it funny that in this age of globalization when anyone can easily bump into any organization (barring language barrier), organizations still call themselves "National" without including an indication of which nation, in their name. At least on the web site of NIST you can see US Department of Commerce in their logo.
The process
Anyway, what did I do?
I noticed that the project has a folder called tests but that it also has a file called runtests.py. I am not sure why do some projects have their own test-runner instead of just running pytest, but I seem to recall I saw this in other Django-related projects as well.
So I set started a Docker container that I use for better isolation of the foreign code I run from my computer and ran
python runtests.py
Not surprisingly it failed complaining about missing Django.
So I went ahead and installed all the requirements I could find:
pip install -r requirements.txt pip install -r requirements.core.txt
Running again
python runtests.py
still failed. This time it was missing psycopg2 which is the Python package to connect to PostgreSQL.
So I tried to install it:
pip install psycopg2
It failed, but it pointed me at the installation documentation. That's a very nice touch!
From that website I understood I need to install libpq. So I ran
apt-cache search libpq
(I use an Ubuntu 22.10 based Docker container for my experiments.)
In the results of this command I saw libpq-dev. This is the development package. Libraries usually need that so I installed it:
apt-get install -y libpq-dev
and ran the installation again:
pip install psycopg2
After that I ran the tests again:
python runtests.py
The output looked very much like an output from a test-run and it ended with
Ran 86 tests in 0.271s
It was not hard at all.
I created the GitHub Actions configuration file using a Docker container and configured python 3.11. I let the developers of the package decide which other versions of Python they might want to test with and if they might want to test on Windows and macOS as well.
I sent the Pull-Request
GitHub Actions
examples/python/core_curate_app/ci.yml
name: CI on: push: pull_request: workflow_dispatch: schedule: - cron: '42 5 * * *' jobs: test: strategy: fail-fast: false matrix: python-version: ["3.11"] runs-on: ubuntu-latest name: OS Python ${{matrix.python-version}} container: python:${{matrix.python-version}} steps: - name: Checkout uses: actions/checkout@v3 - name: Install dependencies run: | apt-get install -y libpq-dev pip install -r requirements.txt pip install -r requirements.core.txt pip install psycopg2 - name: Check Python version run: python -V - name: Run tests run: python runtests.py
Conclusion
I am already quite experienced setting up CI in general and GitHub Actions in particular, but I think even people who are new to this area could do similar task within a few hours. So I think these will be reasonable assignments in the Open Source Development Courses.
Published on 2022-12-13