Building Ligase

For working on Ligase. To use it, you want installation — the published wheels need no Rust toolchain and no C compiler.

Requirements


Python	3.11 or newer
Rust	a stable toolchain (rustup )
A C compiler	`cc`, for the bundled SQLite
Node 22	for the documentation site in `web/`
curl	for fetching test data
minimap2, samtools, bcftools	only to run the HIV-1 reference demo

Rust and a C compiler are needed for a source build. The cache bundles SQLite and compiles it rather than linking whatever version the machine happens to have: that costs a C toolchain at build time and buys an identical SQLite everywhere, which matters for an on-disk format the cache has to keep reading.

A source checkout


git clone git@github.com:Neoforge-Labs/ligase.git
cd ligase
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

The first build takes a minute or two while pyo3 compiles. Later builds are incremental.

Run the checks

The same gate CI runs, in the order it runs them:


cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
ruff check .
ruff format --check .
mypy
pytest
shellcheck scripts/*.sh
cd web && npm run check          # the docs site: palette, build, links, assets, discovery

All of them should pass on a clean checkout. If clippy complains about a missing doc comment on something you just added, that is deliberate — the workspace sets missing_docs to warn and CI runs clippy with -D warnings, so public items without documentation fail the build.

Warningpytest passing is not the pipeline working

Nine tests need the demo’s tools or its data: the five acceptance tests in tests/test_demo.py, and four in test_types.py and test_pipeline.py that want the real HXB2 or a BAM a real aligner wrote. They skip where either is missing, so a green pytest on a bare checkout says nothing about the pipeline. To insist instead:


LIGASE_DEMO_REQUIRED=1 pytest

Under that variable each of those skips becomes a failure naming what was missing. The demo job in CI sets it, which is what stops a green build from meaning “politely declined”.

Rebuilding after a Rust change

An editable install does not recompile on import. After editing anything under crates/, rebuild:


maturin develop

Or pip install -e . again, which is slower but equivalent. Forgetting this is the single most common confusion in a mixed Rust/Python project: your Python changes take effect immediately and your Rust changes do not.

Test data

The reference pipeline runs on real HIV-1 data, and there is nothing to run first. The pipeline says where its data comes from:


hxb2 = lg.remote("ncbi:K03455.1", kind=Reference, digest="1f460b25...")
reads = lg.remote("sra:SRR8525904", kind=Reads, digest="a7800f3d...", samples=["SRR8525904"])

So ligase check and ligase run fetch it, verify every digest before any step runs, and keep it in the content-addressed store under .ligase/. The second run fetches nothing. See remote inputs.

Warming the store for the tests

The test suite is a different matter. A test that needs the demo’s inputs is offline by default, because a suite that downloads a hundred megabytes because you ran it is a suite people stop running — so it skips instead, and one command makes it stop skipping:


python scripts/fetch_demo_data.py


fetching the demo's inputs into /path/to/ligase/.ligase
  ncbi:K03455.1 ready
  sra:SRR8525904 ready
  ...
done; the store is warm and the tests will stop skipping

You get about 130 MB in .ligase/store: HXB2 (NCBI K03455.1, 9,719 bp), four paired runs from PRJNA517147 with published resistance results, and four from PRJNA1218827 with Sanger ground truth. Useful flags:


python scripts/fetch_demo_data.py --list                     # the manifest
python scripts/fetch_demo_data.py --project PRJNA1218827     # the small set
python scripts/fetch_demo_data.py --reference-only           # just HXB2

Set LIGASE_DEMO_STORE to keep it somewhere else — useful if you keep large files on another volume. CI sets LIGASE_DEMO_REQUIRED, which both allows the fetch and turns a skip into a failure.

What the pins guarantee

Every input is verified against a digest in examples/hiv-resistance/pipeline.py, and a mismatch is an error before any step runs. Re-running is cheap: an input already in the store with a matching digest is not fetched again, so this doubles as an integrity check on data you fetched weeks ago.

Each digest is over the input’s canonical form rather than its bytes — the sequence of a FASTA, the sorted records of a FASTQ. That is decision D4 applied to an input, and D39 is why: NCBI can reflow a FASTA or reword a defline without the biology changing, and an alarm that fired on that would teach you to bump pins without reading. Such a rewrite is reported as a note. A real change to the sequence fails, loudly.

FASTQs come from ENA over HTTPS, so no SRA toolkit is required — and since the engine fetches them itself, not even curl.

The tools the demo runs

The reference pipeline shells out to three external tools. The rest of Ligase does not need them; the demo and the nine tests that drive it do.

Tool	Version the demo is signed off on
minimap2	2.31
samtools	1.24
bcftools	1.24


brew install minimap2 samtools bcftools              # macOS
 
micromamba install -c conda-forge -c bioconda \
    minimap2=2.31 samtools=1.24 bcftools=1.24        # anywhere

Which versions you have matters more here than it usually would. A step’s cache key includes its tools’ versions, so upgrading minimap2 invalidates every alignment it produced — correctly, since a new aligner may genuinely align differently. That is also why CI pins all three from bioconda in .github/demo-tools.yml rather than taking whatever apt offers: an aligner that moved on its own would present as a Ligase regression, and the job is there to catch Ligase regressions.

Build the docs

The site is Nextra, in web/, and it is what serves ligase.neoforgelabs.tech :


pip install -e ".[docs]"
cd web
npm ci
npm run dev

npm run check is the gate: it regenerates the API reference, the CLI reference, the captured diagnostics, the quickstart, the ranked-list captures and the rendered notebooks, builds, and then fails on a broken link, a dangling anchor, a page nothing links to, or a Markdown link that never resolved. Before the build it runs the source-only checks (palette contrast, the two themes declaring the same tokens, the mark’s two SVGs, the stylesheet’s furniture and print rules), so a palette mistake fails in a second; after it, the generated PNGs, the sitemap and the machine-readable prose. Every generator imports ligase, so the virtualenv has to be active — that is what the docs extra and the pip install -e above are for.

Troubleshooting

No module named 'ligase' after a successful install. You are probably in a different interpreter than the one you installed into. Check with python -c "import sys; print(sys.executable)".

Rust changes have no effect. Run maturin develop. See above.

error: linker 'cc' not found. No C toolchain. On Debian or Ubuntu, apt install build-essential; on macOS, xcode-select --install.

The Nextra build fails on _meta. A page is named in a _meta.ts and does not exist, or the reverse. Nextra validates both directions, which is deliberate.