Tooling#

The configuration in pyproject.toml is the source of truth for every rule marked ⚙ in AGENTS.md; this page explains the intent of each block so that the configuration can be changed deliberately. If this page and pyproject.toml disagree, pyproject.toml wins and this page is out of date.

Environment#

  • uv manages Python (3.14) and dependencies; uv sync creates the environment, uv add / uv add --dev change it, uv lock --check runs in CI. Never pip install by hand; never edit uv.lock.

  • The package version comes from git tags (hatch-vcs, falling back to 0.0.0 before the first tag); there is no version string to edit.

  • The target-specific native backend, solprop-native, is the repository’s second distribution, built from native/, and a required dependency. The root is a uv workspace with native/ as its member and one uv.lock; hatch_build.py pins solprop-native to solprop’s own version in the built metadata, and both take that version from the repository’s tags. A plain uv sync builds and installs it editable. It removes a hand-installed Pyroraptor (uv sync --inexact and uv run --no-sync keep it). uv run --no-project tools/setup_light.py is the lightweight setup for Python and documentation contributors, with no native package and no compiler: it supports the reference backend and the lint, type, test and documentation checks, with the environment activated or UV_NO_SYNC=1, because a plain uv run would build native. poe fetch-wheel installs the wheel CI built for the newest commit in this checkout’s history, using gh and no compiler, and refuses it if native/ or a solprop module that solprop_native imports differs from that commit, committed or not (contributing).

  • Dependencies resolve from PyPI. No local-path or private source is committed.

  • Supported platforms: where solprop-native installs, as wheels for osx-arm64, linux-64, linux-aarch64, win-64 and win-arm64; it builds from source on Linux and macOS; the Queue and Slurm scan executors are POSIX-only and refuse to start on Windows.

The gate: uv run poe check#

A documentation-only change needs only the documentation checks.

Runs, in order: ruff check --fix . → ruff format . → ruff check . → basedpyright → numpydoc lint (on src/solprop and the figure scripts) → codespell → reuse lint → poe changelog-check → poe size-check → poe test (fast and medium tiers, doctests included, under xdist) → sphinx-build -b html -W -n --keep-going docs docs/_build/html. CI runs exactly the same sequence plus poe test-slow and the CPU coverage lane; keep the two identical.

Other tasks: poe test-medium (the medium tier alone, timed against its budget), poe test-slow, poe cov (the default lane with coverage), poe bench (the benchmark plan in benchmarks/; track runs it on a quiet machine; manual), poe docs, poe docs-clean, poe figures (redraw the committed structure and geometry SVGs) and poe docs-examples (execute the stale gallery scripts and refresh their committed output).

The native package has a gate of its own, uv run poe check-native (native/scripts/check.sh, bash, so Linux and macOS): its ruff and strict pyright, clang-format, clang-tidy, the native tests, ASan and UBSan, its pytest suite on an editable build, and the distribution wheel’s build and isolated smoke test. uv run poe check-all runs check and then check-native; a change that touches native/ runs it. Its rules are in native/AGENTS.md.

ruff#

line-length = 88, target-version = "py314", src = ["src", "tests"], excluding the generated documentation trees. Rule families: pycodestyle, pyflakes, isort, pep8-naming, pydocstyle (numpy convention), annotations, pyupgrade, bugbear, simplify, comprehensions, pathlib, no-print, tidy-imports (absolute only), return, unused-arguments, PGH (no blanket noqa), NumPy (no legacy random), pylint subset (max-positional-args = 5: past five positional parameters a signature is keyword-only, PLR0917), ruff-specific (stale noqa). Ignored: PLR2004 (magic values in numerics), PLR0913 (the parameter count: keyword-only parameters read well at every call site, so PLR0917 counts the positional ones), ANN401 (a commented, confined Any), F722 (pyflakes reads jaxtyping shape strings such as " n_omega" as broken forward annotations; jaxtyping’s own documentation asks for this ignore) and UP037 (quoted annotations: the jaxtyping/beartype hook evaluates annotations before a class body or a TYPE_CHECKING import binds the name, so the quotes it would remove are needed; see below). Per-file: tests drop docstring, annotation, magic-value, unused-argument and positional-count rules (dummies implement protocols with unused arguments, and pytest passes fixtures and parameters by name); the independent oracles under tests/oracles/ are held to the benchmark rules; docs/conf.py drops docstring and annotation rules; gallery scripts and worked examples drop them too and allow print and cell-style imports; benchmarks allow print.

basedpyright#

include = ["src", "tests"] without the oracles, typeCheckingMode = "standard" (strict once the baseline is clean), pythonVersion = "3.14", failOnWarnings = true, reportMissingTypeStubs = false because pyfluids ships no stubs (narrow ignores are still preferred over broad ones); scipy-stubs supplies SciPy’s. The checker is pinned to an exact version and bumped in its own commit.

pytest#

testpaths = ["tests", "src"] (so --doctest-modules sees the Examples sections); addopts = "-ra --strict-markers --strict-config --doctest-modules -m 'not slow and not gpu and not benchmark'"; random order via pytest-randomly; filterwarnings = ["error"] with targeted ignores; tier markers medium, slow and lane markers gpu, benchmark; xfail_strict = true. poe test and poe test-slow use pytest-xdist (-n auto --dist loadgroup), grouping tests by the module-scoped fixtures they share (testing.md), explicit groups for independent heavy cases, and one thread per worker for every pool the suite opens (SOLPROP_NATIVE_THREADS, OpenMP, OpenBLAS, MKL, Accelerate) locally and in CI; plain pytest, whose addopts carries no -n, runs serially. Tier choice is by purpose, with budgets in testing.md.

JUnit reports use junit_family = "legacy" so the numerical record_property fixtures keep their bounds and measurements when --junitxml is requested. The default xunit2 format refuses those properties under the suite’s warnings-as-errors policy.

coverage#

One configuration: coverage-cpu.ini (source = solprop, branch = True) for poe cov, one pass of the default lane; in CI, poe cov-part per shard and poe cov-report, which combines them. The [tool.coverage] tables in pyproject.toml are the shared defaults. The lane requires 100 % of statements and branches of the whole package (fail_under = 100); # pragma: no cover requires a reason on the same line.

numpydoc validation#

[tool.numpydoc_validation] checks = ["all", "ES01", "SA01", "EX01", "GL01", "GL02"] (“all” minus extended summary, see-also and examples being mandatory, and minus the two quote-placement rules that conflict with ruff’s numpy docstring style), excluding __init__, test functions and private names: any dotted name with a single-underscore component ((^|\.)_[^_], so a private module, class or function and everything inside it), which keep a short docstring instead (AGENTS.md, “Docstrings and comments”). The same exclusions hold for the front end in gui/pyproject.toml. It is run by numpydoc lint in the gate and by the pre-commit hook. Sphinx renders docstrings with the numpydoc extension alone (napoleon is not loaded: the two conflict).

jaxtyping + beartype#

tests/conftest.py installs the jaxtyping import hook for the whole session, with beartype configured for the PEP 484 numeric tower (beartype.beartype(conf=beartype.BeartypeConf(is_pep484_tower=True))), before importing the package, so shape and dtype annotations are checked at runtime during tests only; production imports are unaffected. SOLPROP_TEST_TYPECHECK=0 leaves the hook out, which makes the reference backend’s tests several times faster; CI checks in its coverage lane only (ADR-053). numpydoc’s PR02 fires on any dataclass documented with a Parameters section (the hook cannot see the generated __init__); document fields under Attributes, or add # numpydoc ignore=PR02 on the class line.

What the hook means for annotations:

  • A plain float or an empty tuple is not an ArrayLike under beartype: annotate such parameters ArrayLike | float or Sequence[float] | np.ndarray.

  • An int is accepted where a float is annotated (the PEP 484 numeric tower above).

  • A return annotation that names the enclosing class is quoted (-> "Grid"): the hook evaluates it while the class body is still running (UP037 is ignored for this).

  • numpy.typing.NDArray cannot be used: beartype cannot interpret NumPy’s generic alias. Use jaxtyping, the one array-annotation style.

  • A dataclass whose __post_init__ validates shape agreement between its fields gives them independent axis names, so that the validation stays reachable in production, where no hook runs.

  • The hook is session-wide, so a branch written before a rule it enforces passes its own gate and fails on main: run the gate on the merged tree.

pre-commit#

ruff (check with --fix, then format), numpydoc validation, codespell, uv-lock, and reuse lint-file on the staged files. Check the rev pins against current releases when updating.

Native C++ tooling#

The native backend’s C++ (native/) has its own tools, judged against this project: a double-precision Kokkos code with one process-wide runtime, no untrusted input, a Python oracle for every result, and end-to-end runs as the measure of speed.

tool

status

where, or why not

clang-format (pinned 23.1.0)

adopted

every tracked C++ file, in check-native’s lint stage; poe fmt-native formats in place

clang-tidy (pinned 23.1.4), whole families

adopted

bugprone, performance, modernize, misc, readability, clang-analyzer and cppcoreguidelines-special-member-functions, each exclusion stated in native/.clang-tidy; warnings are errors; the tidy stage; poe tidy-native for changed files

compiler warnings

adopted

-Wall -Wextra -Wpedantic -Wconversion -Wsign-conversion -Wshadow -Wundef -Wnon-virtual-dtor -Woverloaded-virtual, with Clang’s -Wimplicit-fallthrough and GCC’s -Wduplicated-cond -Wduplicated-branches -Wlogical-op; errors in development and CI builds

ASan + UBSan with Kokkos bounds checks

adopted

the cpu-sanitize preset over the GoogleTest suite, UBSan stopping at its first report

compute-sanitizer (memcheck, racecheck, initcheck, synccheck)

adopted

.github/workflows/cuda.yml, on a device

Clang source coverage

scheduled

.github/workflows/native-deep.yml, weekly: GoogleTest and native pytest, a report whose zero-count lines are the dead-code candidates

ASan + UBSan on the extension

scheduled

native-deep.yml: the native pytest suite through the bindings

misc-include-cleaner

scheduled

native-deep.yml, a report; too noisy across Kokkos’s umbrella headers to gate

perf, Hotspot, Instruments, Nsight Systems and Compute, rocprof, optimisation remarks, llvm-mca, Compiler Explorer

ad hoc

profiling and code-generation questions, by hand; the deciding number is python -m benchmarks ab

cppcheck

rejected

overlaps clang-tidy and misreads Kokkos lambdas

include-what-you-use (the binary)

rejected

misc-include-cleaner gives the same report from the pinned clang-tidy

CodeChecker, scan-build

rejected

front ends to the Clang static analyser, which clang-analyzer-* runs

GCC -fanalyzer

rejected

C-oriented and noisy on C++ templates; clang-analyzer-* covers it

CodeQL, SonarQube

rejected

security analysis of untrusted input, which this code has none of

ThreadSanitizer

rejected for the gate

needs an instrumented libomp, FFTW and BLAS; the thread-invariance probes cover the outcomes. An on-demand campaign only

MemorySanitizer, TypeSanitizer

rejected

MSan needs every library instrumented; TySan is experimental and strict aliasing is not a risk here

Valgrind

rejected

ASan covers it, much faster, and it does not run on Apple silicon

Clang thread-safety annotations

rejected

there is one mutex

Google Benchmark

rejected

microbenchmarks mislead here; the deciding number is the end-to-end benchmarks ab

Tracy, HPCToolkit

rejected

Kokkos Tools and the profilers above cover them

fuzzers (libFuzzer, RapidCheck)

rejected

the seam’s inputs are generated by Hypothesis in the Python tests

MPFR

rejected

mpmath oracles in the Python tests give the high-precision references

Sphinx#

docs/conf.py: myst-parser, numpydoc (without napoleon), autosummary (recursive, generate = True), mathjax, intersphinx (numpy, scipy, matplotlib, h5py, python), viewcode, matplotlib plot directive, sphinx-gallery, pydata-sphinx-theme. The three galleries, lessons (docs/introduction/tutorials/ → docs/introduction/lessons/), tutorials (docs/tutorials/ → docs/auto_tutorials/) and worked examples (docs/examples/ → docs/auto_examples/), have their generated pages, figures and checksums committed: a normal build reuses them, executes nothing and refuses a stale one, so the build that publishes the manual runs no propagation. poe docs-examples (SOLPROP_RUN_EXAMPLES) executes the stale scripts, or one by name, and CI’s docs job runs every lesson and tutorial again (SOLPROP_RUN_EXAMPLES=tutorials). Warnings are errors and missing references are warnings (-W -n). docs/dev/ is included in the manual under “Development”. Intersphinx inventories are fetched from each project’s site first and read from the vendored copies under docs/_inventories/ when that fails, so the build works offline and survives a flaky mirror; refresh the copies occasionally.

Size#

poe size (tools/size_report.py, standard library only) reports where the Python lines go, per top-level package of src/solprop (--all adds the native adapter and the front end, --json prints JSON): code, public and private docstring, comment and blank lines; private docstrings with numpydoc sections and docstrings longer than their body; the largest files and functions; noqa comments by rule; private names imported across packages; __all__ lines; cloned code (windows of 50 tokens seen twice, with names renamed and as written) with the file pairs that share most; and ruff’s complexity counts. A name is private by exactly the rule numpydoc’s validation excludes (a single-underscore component of module.qualname), so the report and poe docstrings agree; both count a nested function as private only when its name starts with an underscore.

poe size-check, in poe check and CI’s static job, holds src/solprop to the ratchets in tools/size_budget.toml: the number of private docstrings with numpydoc sections, a 4,000-line cap on a module, and the number of private names imported across packages. The modules already larger are listed, each capped at its size plus a headroom of 2 % (at least 50 lines); the check also fails on an entry more than three headrooms above its module’s size, or for a module that is gone or now under the cap, so a module that shrinks by more than twice its headroom tightens its entry in the same change. A cap is lowered when the code improves and never raised to let a change through. .github/workflows/size.yml adds a soft budget: a pull request whose net growth of src/solprop against its merge base exceeds 1,500 lines fails unless a maintainer gives it the size-ok label.

Dead code is looked for by hand, not in the gate: uvx vulture src/solprop --min-confidence 80. Its findings need reading, since public names, protocol members and names used only from tests or the native package look unused to it.

Environment variables the tools honour#

  • SOLPROP_THREADS, the thread budget the native backend’s pools stay within, and SOLPROP_OVERSUBSCRIBE, which lets an explicit count exceed the CPUs (ADR-052).

  • The native backend’s plans and neighbours: SOLPROP_FFTW_PLAN (estimate in CI and tests, measure by default, read by native/include/solprop_native/fft_plan.hpp), SOLPROP_FFTW_WISDOM (a file name, beside which the native backend keeps the plans its runs measure instead of the calibration store, ADR-076) and SOLPROP_BLAS_THREADS (the width of the other BLAS libraries while a native backend lives).

  • In the test suite: SOLPROP_TEST_BACKEND (the backend the driver tests select; by default what backend="auto" selects, native where it is installed), SOLPROP_TEST_TYPECHECK=0 (leave out the runtime type checks, ADR-053), SOLPROP_TEST_SHARD=i/n (run shard i of n, whole modules, as CI does) and SOLPROP_WRITE_RECORDS=1 (rewrite the committed convergence records, announced at the end of the session).

  • In the docs build: SOLPROP_RUN_EXAMPLES (stale or a script name, set by poe docs-examples; tutorials in CI’s docs job).

CI#

.github/workflows/ci.yml runs on every push to main and every pull request, with SOLPROP_FFTW_PLAN=estimate. Its first job, changes, compares the commit with its base: when only docs/ and Markdown files changed, static, docs and docs-tests run (the test modules that open a documentation file: the SEP structure, the guide’s snippets, the accuracy page, the catalogues, the kernel contract and the figures), and the test lanes below are skipped. A push to main that merges a pull request runs no further jobs when its tree is the pull request head’s tree and CI passed on that head: the pull request was up to date with main, so its run tested exactly what was merged. A pull request that was behind main merges to a new tree, which runs in full. A change confined to the front end (gui/, with docs/ and Markdown files) runs static, docs and gui only. Any other change, or a push whose base is unknown, runs every job except docs-tests, gui included, so a change to solprop is checked against the front end.

The long lanes run as shards, parallel jobs that each take whole modules: SOLPROP_TEST_SHARD=i/n keeps shard i of n (tests/shards.py, ADR-053). The shards are balanced by the seconds per module in tests/data/shard_weights.json. Every shard uploads a JUnit report, and uv run python tests/shards.py junit-*.xml rewrites the table from a run’s reports when the balance drifts. The runtime type checks (the jaxtyping hook) run in the coverage lane only; the other test lanes set SOLPROP_TEST_TYPECHECK=0 (ADR-053).

  • static (Linux): uv lock --check, ruff (lint and format check), poe types, poe docstrings, poe spell, poe reuse, poe changelog-check, poe size-check.

  • docs (Linux): poe docs, with the wheel installed, so a lesson or tutorial on backend="auto" runs on native.

  • native-wheel (Linux x86-64, macOS arm64, Windows x64, Windows on Arm): this tree’s native/ built as its CPU distribution wheel, cached under a hash of the native sources, so a change outside native/ reuses the last build. On Linux the build is the native gate’s wheel stage (the build and its isolated smoke test); on macOS and Windows it is cibuildwheel, as in wheels.yml. The docs and test lanes install the wheel. A documentation-only change builds only the Linux wheel. poe fetch-wheel installs the newest one from this checkout’s history that fits the tree, for a contributor without a compiler.

  • test (macOS arm64, Windows x64 and Windows on Arm, three shards each): poe test, the default lane, with the wheel installed, so backend="auto" and the driver tests select native.

  • coverage (Linux x86-64): poe cov as parallel jobs, with the wheel installed (its paths in solprop count towards the 100 %) and SOLPROP_TEST_BACKEND=native. Six shards run poe cov-part. coverage-report combines their data (poe cov-report), requires 100 % of statements and branches, and uploads the report to Codecov for the badge.

  • slow (Linux and macOS): poe test-slow with the wheel installed; on macOS only its functional tests, the statistics-cost timing being Linux’s. On Linux, SOLPROP_TEST_BACKEND=native selects the ladders’ backend.

  • oldest (Linux, three shards): the package installed with uv pip install --resolution lowest-direct, so every dependency sits at its floor, then the wheel with --no-deps, then the default lane (dependencies.md).

  • no-native (Linux): a few modules of the default lane with no native package and SOLPROP_TEST_BACKEND=numpy_ref: that backend="auto" refuses there, and a real run on numpy_ref by name.

  • native-quality (Linux, three jobs): uv run poe check-native, the native package’s gate (native/scripts/check.sh), split by SOLPROP_NATIVE_CHECK_STAGES. The jobs are its lint and strict types, clang-format and its pytest suite (lint pytest); clang-tidy and the native tests (tidy); and ASan and UBSan (sanitize). The distribution wheel’s stage is native-wheel’s Linux job.

  • device-compile (Linux, two jobs): on a change to native/, compiles the cuda preset under CUDA 12.8 and the hip preset under ROCm 7.2, tests included, with warnings as errors and no device. About 25 and 11 minutes, with no cache. A fault only a card shows, or a test that fails on one, still needs a run on a device.

  • gui (Linux x86-64, macOS arm64, Windows x64): uv run poe check-gui, the front end’s gate (its types against gui/pyproject.toml, its docstrings and its tests, with the layers without Qt held to 100 % coverage), after uv sync --extra gui --group gui-test, with the widgets drawn offscreen. Its lint, format, spelling and licence checks are static’s.

  • ci-passed (Linux): runs after every lane, always, and fails if any of them failed or was cancelled. It is the check the main ruleset requires, beside the Read the Docs build: which lanes run depends on the change, and a skipped matrix job reports its unexpanded name, so no fixed list of job names suits every pull request.

The release-note collector (tools/release_notes.py) uses only the standard library. poe changelog-check validates filenames, categories, entry text and the Unreleased structure; poe changelog-preview prints the assembled changelog without changing files. Fragments are Markdown, so a change confined to fragments takes the documentation lanes. The rolling release PR keeps unrelated implementation PRs from editing the same changelog lines. This prevents those conflicts; it does not reuse a previous commit’s CI result or bypass required checks.

.github/workflows/wheels.yml builds the native release wheels (Linux x86-64 and aarch64, macOS arm64, Windows x64 and Arm) on pull requests that change the wheel recipe (native/ci/, native/pyproject.toml, the CMake files, the workflow), weekly, on release tags and by hand, and tests each installed wheel. It also builds solprop’s sdist and wheel and checks a fresh install of each, builds solprop-gui’s and checks that solprop[gui] installs from the files and answers at the version, and on a release tag, once enabled, publishes solprop-native, then solprop, then (once PUBLISH_GUI is set) solprop-gui (Releasing and the native wheels, whose pre-flight checklist sets publishing up).

native runs in the coverage lane’s driver tests; the parity tests check it against numpy_ref in every job. GPU tests run by hand on a node with a device (pytest -m gpu). The self-hosted .github/workflows/cuda.yml, started by hand, builds the CUDA extension and runs the native package’s pytest suite and Compute Sanitizer on it. Keep the gate’s steps identical to poe check; a change to one is a change to the other, in the same commit.