Framework
JSON-driven process definitions called from Jupyter notebooks. No code changes needed to reconfigure workflows — edit a JSON file and re-run.
Database setup
Step-by-step guide to installing PostgreSQL, creating an Anaconda environment, and defining schemas and tables using Xspatula from a Jupyter notebook.
Setup processes
Register the translate-and-manage process catalogue in the database — these process definitions are the prerequisite for the user-data import workflow.
User data
Add your own tabular (Excel) data to the database: the translate step converts rows to JSON process files; the manage step inserts them using the registered processes.
AI4SoilHealth database
Step-by-step guide to seeding the EU-funded AI4SoilHealth (AI4SH) PostgreSQL database — 9 schemas covering soil observations, eDNA, landscape, and more — using the Xspatula framework.
AI4SoilHealth processes
Register the AI4SH process catalogue in the database — root processes, translate, community, utility, observation utility, observation, and eDNA process families.
AI4SoilHealth data import
Two-step translate-then-manage workflow for importing soil observation data: utility catalogues, dataset metadata, campaigns, sampling logs, and measurements.
Open source
Code under MIT License, data under CC-BY. Clone the framework from GitHub and adapt it to your own domain.
What is Xspatula?
Xspatula is a Python framework for building database-integrated modelling workflows. The core idea is simple: all execution logic lives in JSON files, not in code. You define schemes, jobs, pilots, and processes in JSON; a Jupyter notebook calls the framework; the framework reads the JSON and runs the pipeline.
This separation makes it straightforward to:
- swap datasets without touching Python
- version-control your experimental configurations separately from the codebase
- hand workflows to non-programmers who edit JSON rather than code
The framework uses PostgreSQL as its backbone. Every result, parameter set, and process definition is stored in the database, which gives you full audit trails and makes it easy to query what ran, when, and with what inputs.
Documentation
The full documentation is at xspatula.github.io/setup_core_db_docs and covers:
- Framework architecture — scheme files, job files, pilot files, process files, notebook interface
- Database setup — PostgreSQL, Anaconda,
.netrccredentials, schemas and tables - Setup processes — register the translate-and-manage process catalogue; prerequisite for the user-data workflow
- User data — translate Excel rows to JSON process files, then manage them into the database
AI4SoilHealth case study
xspatula.github.io/xspatula_ai4sh_docs documents seeding the EU Horizon-funded AI4SoilHealth database using the Xspatula framework:
- Setup DB — defines 9 schemas and all tables (utility, community, process, landscape, observation, eDNA and their utility schemas)
- Setup processes — registers all framework processes in the database
- Import data — translate-then-manage workflow for importing utility catalogues, dataset metadata, and soil observation records
Licenses
- Code: MIT License
- Data: Creative Commons Attribution (CC-BY)