Development processes¶

For software development purposes alexandria3k can also be installed and used through its Github repository, rather than as a Python package.

Installation¶

git clone https://github.com/dspinellis/alexandria3k.git
cd alexandra3k/src
pipenv install

Development environment¶

Perform the following steps for setting up a virtual environment with the required development dependencies.

# While in alexandria3k/src directory

# Install development dependencies
pipenv install --dev

# Launch a shell in the virtual environment
pipenv shell

You can then run the command-line version from the source distribution on the top-level directory as follows:

bin/a3k --help

Continuous integration¶

GitHub actions are used for running unit tests, linting, verifying the code’s format, as well as building the package and the documentation when a new commit is pushed to GitHub. Before doing so, it’s advisable to perform these actions locally, as detailed in the following sections. In addition, testing, formatting, and linting can be easilly configured to be run as a fast Git pre-commit hook by running the following commands at the top-level directory.

printf '#!/bin/sh\nbin/pre-commit\n' >.git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

Debugging and logging¶

Much of the alexandria3k code is executed by means of callbacks through the SQLite apsw module. This makes it difficult to trace and debug. The liberal use of print statements is your friend. In addition, alexandria3k offers many debug options, which you should be using for troubleshooting. Run a3k --help to see which options are available.

Testing¶

Python unit and integration tests¶

# While in the top-level directory
python3 -m unittest discover -s .

SQL unit tests¶

To run SQL unit tests install rdbunit and the SQLite command-line tool.

# While in the top-level directory
for t in tests/*.rdbu; do rdbunit --database=sqlite $t | sqlite3 ; done

Code formatting¶

# While in the top-level directory
find src -name '*.py' | xargs black -l 79

Linting¶

# While in the top-level directory
find src -name '*.py' | xargs python -m pylint --rcfile .pylintrc

Plugin documentation¶

When data source or processing plugins are added the Python API documentation must be updated as follows.

# While in the top-level directory
bin/update-python-api

For users on MacOS, this might require installing the GNU sed package.

Application examples documentation¶

When new application examples are added in the examples directory documentation must be updated as follows.

# While in the top-level directory
bin/update-app-eg

This requires an installed version of Pandoc.

Database schema diagrams¶

After the database schema has changed, its relational diagrams in the documentation must be updated as follows.

# While in the top-level directory
bin/update-schema

This requires an installed version of the GraphViz dot command.

Building¶

# While in the top-level directory
hatch build dist/

This will result in built files being placed in the dist directory.

Documentation building¶

The process for converting the documentation into HTML and a Unix man page is documented in the installation instructions.

Adding a new data source¶

Adding a new data source to alexandria3k involves the addition of a new plugin, as described in Plugin development.