Installation¶
Requirements¶
| Python | 3.11 or newer |
| Rust | a stable toolchain (rustup) |
| A C compiler | cc, for the bundled SQLite |
| curl | for fetching test data |
| minimap2, samtools, bcftools | only to run the HIV-1 reference demo |
Rust is needed because installing builds the engine from source. There are no pre-built wheels yet.
The C compiler is needed because the cache bundles SQLite and compiles it, rather than linking whatever version the machine happens to have. That costs you a C toolchain at install time and buys an identical SQLite everywhere, which matters for an on-disk format the cache has to keep reading. If you are missing one, see below.
Install¶
git clone git@github.com:Neoforge-Labs/ligase.git
cd ligase
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
The first build takes a minute or two while pyo3 compiles. Later builds are
incremental.
Check it worked:
The distribution and the import have different names
The PyPI distribution is ligase-bio but you import ligase. So it
is pip install ligase-bio and import ligase. The distribution name
ligase is already taken on PyPI by an unrelated project, and the import
name is the one you actually live with, so that is the one that stayed
short.
Run the checks¶
The same gate CI runs, in the order it runs them:
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
ruff check .
ruff format --check .
mypy
pytest
All seven should pass on a clean checkout. If clippy complains about a
missing doc comment on something you just added, that is deliberate — the
workspace sets missing_docs to warn and CI runs clippy with -D warnings,
so public items without documentation fail the build.
pytest passing is not the pipeline working
Nine tests need the demo's tools or its data: the five acceptance tests
in tests/test_demo.py, and four in test_types.py and
test_pipeline.py that want the real HXB2 or a BAM a real aligner
wrote. They skip where either is missing, so a green pytest on a bare
checkout says nothing about the pipeline. To insist instead:
Under that variable each of those skips becomes a failure naming what
was missing. The demo job in CI sets it, which is what stops a green
build from meaning "politely declined".
Rebuilding after a Rust change¶
An editable install does not recompile on import. After editing anything
under crates/, rebuild:
Or pip install -e . again, which is slower but equivalent. Forgetting this
is the single most common confusion in a mixed Rust/Python project: your
Python changes take effect immediately and your Rust changes do not.
Test data¶
The reference pipeline runs on real HIV-1 data. It is downloaded rather than
committed, into a gitignored data/:
==> Reference: HXB2 (K03455.1)
downloading from NCBI
verified: 9719 bp, canonical SHA-256 matches
==> Reads: PRJNA1218827
SRR32217714_1.fastq.gz verified
...
4 runs ready in data/reads/PRJNA1218827
You get:
| Path | What |
|---|---|
data/refs/HXB2.fasta |
HIV-1 HXB2 reference, NCBI K03455.1, 9719 bp |
data/reads/PRJNA517147/ |
4 paired runs, ~100 MB — published resistance results |
data/reads/PRJNA1218827/ |
4 paired runs, ~30 MB — Sanger ground truth |
Useful flags:
scripts/fetch_test_data.sh --list # show the manifest
scripts/fetch_test_data.sh --project PRJNA1218827 # just the small set
scripts/fetch_test_data.sh --reference-only # just HXB2
scripts/fetch_test_data.sh --force # re-download
Set LIGASE_DATA_DIR to put the data somewhere else — useful if you keep
large files on another volume.
What the script guarantees¶
Every file is verified against a checksum pinned in the script, and the script exits non-zero if one does not match. Re-running is safe and cheap: files already on disk that verify are not downloaded again, so this doubles as an integrity check on data you fetched weeks ago.
Reads are pinned by ENA's own per-file MD5. The reference is pinned by the SHA-256 of its canonical sequence — sequence only, uppercased, line breaks removed — rather than of the raw file. This is decision D4 applied to an input: NCBI can reflow a FASTA or reword a defline without the biology changing, and a cosmetic change upstream must not break your build. A real change to the sequence still fails, loudly.
FASTQs come from ENA over HTTPS, so no SRA toolkit is required — only curl.
The tools the demo runs¶
The reference pipeline shells out to three external tools. The rest of Ligase does not need them; the demo and the nine tests that drive it do.
| Tool | Version the demo is signed off on |
|---|---|
| minimap2 | 2.31 |
| samtools | 1.24 |
| bcftools | 1.24 |
brew install minimap2 samtools bcftools # macOS
micromamba install -c conda-forge -c bioconda \
minimap2=2.31 samtools=1.24 bcftools=1.24 # anywhere
Which versions you have matters more here than it usually would. A step's
cache key includes its tools' versions, so upgrading minimap2 invalidates
every alignment it produced — correctly, since a new aligner may genuinely
align differently. That is also why CI pins all three from bioconda in
.github/demo-tools.yml rather than taking whatever apt offers: an
aligner that moved on its own would present as a Ligase regression, and the
job is there to catch Ligase regressions.
Build the docs¶
Then open http://127.0.0.1:8000. CI builds with mkdocs build --strict, so
a broken internal link fails the build rather than shipping.
Troubleshooting¶
No module named 'ligase' after a successful install.
You are probably in a different interpreter than the one you installed into.
Check with python -c "import sys; print(sys.executable)".
Rust changes have no effect.
Run maturin develop. See above.
error: linker 'cc' not found.
No C toolchain. On Debian or Ubuntu, apt install build-essential; on macOS,
xcode-select --install.
mkdocs build --strict fails on a link you just added.
Internal links are relative to the file they live in, and must include the
.md extension — ../decisions.md, not /decisions.