Milestones and Acceptance Tests
One milestone per session. Plan → write tests → implement → write docs.
M0 — Scaffold
Cargo workspace (ligase-core, ligase-py), maturin, python/ligase, CI running fmt, clippy, ruff, mypy, cargo test, pytest.
Done when: pip install -e . works and import ligase succeeds in CI.
M1 — Types and dimensions
Done when:
- HXB2 FASTA and a sample BAM are loaded with dimensions inferred from headers.
- A BAM aligned to a different reference fails
check()with a clearDimensionError. - A
chr1vs1mismatch raises a distinct error.
M2 — Graph
Done when: the golden example builds a graph without executing anything, and ligase check reports all type errors in one pass.
M3 — Execution and cache
Done when:
- The demo runs end to end on 3–4 runs from PRJNA517147.
- A second run completes entirely from cache.
- Changing
min_freqreruns onlycallandresistance. (M13 rewrote the demo onligase.steps, socallis nowmpileup_call; the criterion is unchanged andtests/test_demo.pystill asserts it.) - Adding one sample processes only that sample.
M4 — Semantic hashing and early cutoff
Done when: rerunning alignment with a different thread count produces the same canonical hash and downstream steps are skipped.
M5 — Provenance
Done when: ligase methods outputs a methods paragraph with tool versions, parameters, reference accession and citations, plus a bundle that reproduces the results on a clean machine.
M6 — Validation
Done when: resistance calls match published results for PRJNA517147 and the Sanger ground truth for PRJNA1218827, with discrepancies explained.
Done. The calls are checked against the published Stanford HIVdb reports for PRJNA1218827: 81 of 81 drug calls agree exactly at the 15% and 5% thresholds, and the three that differ at 2% each carry a cause. The scoring engine separately reproduces 3,348 published expectations.
There were nine. Six of them were an artefact of reading codons from a VCF, which carries no phasing, and went when codons started being read from the haplotypes the reads carry (D31).
PRJNA517147 could not serve as a source of expected answers: its per-sample calls are published only as colour in a heatmap, reverse transcriptase only. It remains the demo’s default dataset. See validation.
M7 — Foundations
The scope rule
in CLAUDE.md. release, subject and source_set added as dimensions,
applying to no existing kind except release on a Reference — a genome
build has a release. organism set-valued, with the symmetric-difference
error. D35, D36 and D37 written into
the decision log.
Done when:
- Every existing test still passes and none of their assertions change. The
Rust call sites that construct a declaration move with the value type —
Declared::organismholds a list now — but no existing Python test is touched, because the Python surface only widens:organism=takes a string as it always did. - A
Referencedeclaredorganism=["HIV-1", "Homo sapiens"], checked against anAlignmentdeclaredorganism="HIV-1", fails with an error naming Homo sapiens as the difference. - What a v0.1 pipeline’s checks and diagnostics see is unchanged: a single
organism renders as exactly
HIV-1, the three new axes are appended to the dimension list rather than slotted in beside the axes they resemble, andligase check’s own tests and the diagnostics captured for the docs are untouched. What is not unchanged is the catalogue a script can read back: aReferencenow listsreleaseamong its applicable dimensions, and reports why it is unknown where it previously reported nothing, so code that iterates the unknown dimensions and prompts for each will now ask for one more.
Done. That third clause is a restatement, and the original is worth
recording: the feature spec asked that “ligase check on the HIV demo
produces byte-identical output to v0.1.0”. data/ is gitignored, so a plain
checkout has no HIV demo to run and the clause would have been a promise
rather than a check. What is checkable without the data is what it now says —
the single-organism rendering is pinned in Rust by
one_organism_still_renders_exactly_as_it_did and in Python by
tests/test_types.py, the appended order by
the_three_axes_are_appended_so_earlier_reports_keep_their_order, and
tests/test_cli.py and capture_diagnostics.py’s existing scenarios are
unchanged by this milestone. The demo itself is covered
where the data exists: CI’s demo job fetches the reads and the reference and
then runs the whole suite, which is where tests/test_pipeline.py asserts
that the demo’s organism survives inheritance through two steps.
The v0.2 roadmap
M7 to M12 are built; M13 and M14 remain. v0.2 exists to make
Ligase useful to a scientist who has never heard of this project, and three
things carry that: the commonest silent bug in biology — identifiers from one
namespace joined to another — becomes a refused connection; the fetch that
starts every pipeline comes inside the engine’s guarantees, where today it
sits outside them, including in Ligase’s own demo; and the cost of adopting
any of it drops, because ligase inspect works on files you already have and
a library of typed steps means composing a pipeline rather than writing shell.
One milestone per session, as before. v0.2 ships after M14.
M8 — Identifiers
The identifier_namespace dimension. An EntitySet kind — the gene or
protein list — with its loader, canonical form and Python surface.
map_identifiers with a user-supplied mapping table, and the conversion
registry it sits in. D38.
Done when: an EntitySet of HGNC symbols and one of Ensembl IDs refuse to
join; map_identifiers across them succeeds, declares Ensembl, and notices
every unmapped symbol by name; the fixture set includes SEPT7-style symbols
corrupted to dates and the loader notices them; namespace inference is
exercised by fixtures for HGNC, Ensembl, Entrez and UniProt, and declines to
infer when a file mixes two.
Standing requirement, from M7. The first kind to claim a dimension axis
must pin that axis’s arm in check.rs::compare and in graph.rs::propagate
with a test. Until a kind applies an axis, check_step never reaches its arms,
so no test can catch one written against the wrong field, and the compiler
cannot either: Organism and SourceSet are one type, and so are Release
and Subject. This milestone gives identifier_namespace a kind, and an
EntitySet may also apply subject or source_set; whichever of those it
applies, the requirement falls on it. If it is the first kind to give
source_set one, it must re-examine the dedup rule before it does: the set
type folds members that differ only in ASCII case, which is right for
accessions and wrong for object keys, and source_set gets a type of its own
if it is to hold the latter
(D36).
Done. All four clauses, each pinned by a named test.
An EntitySet of HGNC symbols and one of Ensembl IDs refuse to join, with
D38’s leading question, in
conversions.rs::two_gene_lists_in_different_spaces_refuse_to_join and in
tests/test_conversions.py::test_two_spaces_refuse_to_join_and_say_what_to_do_about_it.
The message itself is captured from a real run into
my gene-set overlap is empty by a
capture_diagnostics.py scenario, so the page cannot drift from the engine.
A clause the spec did not ask for is pinned beside it: two gene lists that
each declare nothing are refused too, because
identifier_namespace is in required_pair for that pair and two lists
agreeing by silence is the failure in its purest form.
map_identifiers across them succeeds, declares Ensembl and names every
unmapped symbol, in
test_a_mapped_list_declares_ensembl_and_joins_an_ensembl_list and
test_running_it_names_every_symbol_that_did_not_cross. Two kinds of
identifier do not cross and both are named rather than dropped: one the table
has no row for, and one it maps to several targets.
The fixture set includes SEPT7-style symbols a spreadsheet has corrupted to
dates, and the loader notices them —
the_loader_notices_a_symbol_excel_turned_into_a_date and
test_the_loader_says_which_symbols_a_spreadsheet_ate, over
entities_excel_dates.tsv. The detector cost one correction found by test:
matching a month by prefix read the intact SEPT7-1 and MARCH1-2 as dates,
flagging the very symbols the note exists to report the corruption of, so a
month is now matched in full.
Namespace inference is exercised by fixtures for HGNC, Ensembl, Entrez and
UniProt, and declines when a file mixes two —
a_namespace_is_inferred_from_the_identifiers_themselves,
a_file_that_mixes_two_namespaces_declines_to_infer, and
test_every_namespace_the_catalogue_knows_is_inferred. It declines in two
further cases the spec did not name and a user will meet: a file with a header
and no rows, where every pattern fits all zero identifiers, and a file holding
one identifier nothing claims.
The standing requirement is half discharged. An EntitySet claims
identifier_namespace, subject and release, so each of those axes now has
its check.rs::compare arm and its graph.rs::propagate arm pinned —
each_axis_an_entity_set_claims_compares_its_own_field and
a_step_over_a_gene_list_carries_every_axis_it_claims. It claims no
source_set, because the feature spec’s §4.1 gives it four dimensions and
that is not among them, so the deduplication question stays open and the
obligation stays with M10 and M12 below. Nothing in this milestone touches the
shared set type.
One existing assertion changed, and it is the milestone’s point: M7’s
release_applies_to_a_reference_and_to_nothing_else said no kind but a
Reference has a release and nothing at all has a subject. Both are now
false, so it reads
release_applies_to_a_reference_and_to_a_gene_list and still pins that
source_set has no kind.
What the review caught, and the lesson worth keeping. A fresh review of the finished branch found two paths that produced the very failure the milestone exists to break, and both were in the half of the work that looks like detail.
A ten-character TrEMBL accession — A0A024R161, and most of UniProt is
TrEMBL — fell through the accession shape, which had been written for the
six-character reviewed form only, and was claimed by the gene-symbol shape
instead. So a protein list and a gene-symbol list both inferred HGNC symbol
and agreed, and the join between them succeeded with an empty overlap:
D38’s own failure, caused by the engine rather than by the user. And a
many-to-one mapping — a table holding both a deprecated symbol and its
replacement, with a list holding both — collapsed two entities onto one
identifier with no note at all, which is the exact mirror of the one-to-many
case the plan had flagged and tested.
The lesson is that a shape too narrow only declines to infer, which is
inconvenient, while a shape too broad claims another space’s identifiers,
which is the silent failure itself; and that a rule written for one direction
of a mapping needs checking in the other. The catalogue now admits what
people actually have — versioned Ensembl IDs, mixed-case symbols such as
C9orf72, readthrough symbols, the whole RefSeq prefix set — and
the_identifiers_a_biologist_actually_has_resolve_to_their_own_space is a
standing table asserting that each resolves to its own space or to nothing,
never to another’s. A CR-only TSV, which Excel for Mac 2011 wrote, had also
been loading as an empty gene list without a word; it is now read as the rows
it holds.
M9 — Inspect
ligase inspect, generated into the CLI reference by the
existing generate_cli.py.
Done when: ligase inspect tiny_bare.fasta tiny_prefixed.sam prints the
same ContigNamingError the home page shows and exits non-zero; a single file
prints every dimension with its source; the quickstart gains an “inspect
first” section captured by capture_quickstart.py.
Done. All three clauses, each pinned by a named test. See the internals.
ligase inspect tiny_bare.fasta tiny_prefixed.sam prints the
ContigNamingError and returns 1, in
test_two_disagreeing_files_print_the_diagnostic_and_exit_non_zero. It is
the same error in the sense that matters: the same check_step over the
same comparison, rendered through the same helpers, with the same values,
the same provenance and the same conclusion. Two things necessarily differ,
and both are deliberate. The heading names the step inspect, because that
is what did the comparing and there is no step. And the handles are the
files’ basenames rather than a pipeline’s argument names, since on a command
line there are no argument names to use;
a_description_line_is_the_line_a_diagnostic_quotes pins the rendering
itself, by asserting that the diagnostic’s line appears verbatim in the
description printed above it.
A single file prints every dimension with its source, in
test_a_single_file_prints_every_dimension_with_its_source and in
inspect.rs’s own tests. The clause the spec did not write is the one that
took the care: a dimension that does not apply to the kind must not be
printed as a gap, because Dimensions::reason answers for it as readily as
for an applicable one.
a_dimension_the_kind_has_no_business_with_is_never_called_unknown and
test_a_dimension_that_does_not_apply_is_not_reported_as_a_gap pin that a
Reference is never described as missing a quality encoding, and
the_dimensions_that_do_not_apply_are_named_once_as_not_applying pins that
they are still named, on a line saying they do not apply — because dropping
them silently would leave a reader unable to tell “this file does not say”
from “this question does not arise”.
The quickstart’s “Inspect them first” is §2 of
the quickstart, captured by
capture_quickstart.py from a real run on the two files the page has just
told the reader to create. The capture also reads the exit code, because the
page claims inspect exits non-zero on a mismatch and a claim about an exit
code can only be checked by reading one.
No decision, and no concepts page. The spec’s D35–D43 holds nothing about
inspect and M9 adds none: it introduces no rule a future request could be
measured against, only a new way to reach the dimensions M7 and M8 defined.
For the same reason it ships a guide/ page and no concepts/ one —
types and dimensions already says what a
dimension is and why it is inferred, and a second page saying it again from
the command line would be a page to keep in step for nothing.
What the work caught, and the lesson worth keeping. Two bugs, both found by tests written before the code, and both in the same place: the handles that name a file in a message.
The handles are the keys the comparison is given, so two files sharing one
collapsed into a single input and the second was silently dropped —
ligase inspect a/sample.sam b/sample.sam compared one file with itself and
reported agreement. And the summary line returned 0 whenever
comparable was empty, which is exactly the case where two gene lists that
each declare nothing produce an UndeclaredDimensionError: the surface built
to catch M8’s failure reported it as a clean bill of health.
The lesson is that a name chosen to read well is still a key, and that a
verdict assembled from two sources — the errors, and what there was to
compare — has to be read from the one that can say no. Both are now pinned:
test_every_file_given_gets_its_own_say,
test_the_same_file_twice_is_still_two_files and
test_two_gene_lists_that_each_state_nothing_are_refused. A third test,
test_files_with_nothing_in_common_do_not_claim_to_agree, holds the inverse
— that files sharing no stated dimension are not told they agree.
M10 — Remotes and replay
PinnedRemote and RecordedQuery, with record, replay and refresh; the
pinned, replayed and live audit verdicts; drift reporting; recorded
bodies in the bundle; the methods-paragraph wording for both. The HIV demo
retrofitted onto them. D39.
Split into two, and why. This is the largest milestone in v0.2 and the
only one whose design constrains later ones, so the design was settled whole
— D39 is written in full, including the half not yet built — and the delivery
split at the seam D39 itself draws. M10a is the pinned remote and the
blob tier M14 needs; M10b is the recorded query and replay. Everything
M10b needs is delivered and exercised by M10a: the store trait, the locator
parser, the transport and the remote index. decisions.mdx records what was
decided and this page records what is built, which are different questions.
Done when: the demo runs end to end from lg.remote calls with no shell
script; a deliberately wrong digest fails loudly before any step runs; a
pipeline with a RecordedQuery step runs once against a local mock server and
again with that server down, completing from the record with an identical
result; --replay on a query with no record fails before any step runs;
--refresh against a mock whose response changed reports the drift and does
not overwrite the record without being told to; a bundle exported from the
recorded run replays on a clean machine with no network; two paginated calls
recorded in one order replay correctly when issued in the other; a request
made with an auth header stores no trace of it; the CI demo job caches on
remote digests.
Standing requirement, from M7. The first kind to claim a dimension axis
must pin that axis’s arm in check.rs::compare and in graph.rs::propagate
with a test. Until a kind applies an axis, check_step never reaches its arms,
so no test can catch one written against the wrong field, and the compiler
cannot either: Organism and SourceSet are one type, and so are Release
and Subject. Whichever milestone first gives source_set a kind re-examines
the dedup rule before it does (see M8, which may get there first). If M8 gave
it none, this milestone does, on the same grounds: the set type folds members
that differ only in ASCII case, which is right for accessions and wrong for
object keys
(D36),
and the requirement falls on it for source_set as for any axis it applies.
M10a — Pinned remotes and the blob tier
Done. The three of M10’s eleven clauses that are about a pinned remote, plus the trait M14 is built on.
The demo runs end to end from lg.remote calls with no shell script.
scripts/fetch_test_data.sh is deleted — three hundred lines that knew ENA’s
directory layout, computed checksums and verified them, all of it outside
anything Ligase promises. examples/hiv-resistance/pipeline.py now declares
seventeen remotes and is itself the manifest of their pins, and the ENA
layout rule moved into remote/locator.rs where
the_ena_subdirectory_rule_holds_at_every_accession_length holds it to all
four of its branches.
A deliberately wrong digest fails loudly before any step runs, in
remotes.rs::a_wrong_digest_fails_naming_the_locator_the_expected_and_the_got
and in
tests/test_remotes.py::test_a_wrong_digest_fails_while_the_pipeline_is_still_being_read
— which needs no pipeline at all, because resolution happens as the file is
read and there is no graph yet. The message is captured from a real run into
the guide by a capture_diagnostics.py scenario, over a
loopback socket, so the page cannot drift from the engine.
The CI demo job caches on remote digests: the pins live in the demo
pipeline, so its hash is the cache key and what is cached is the engine’s own
content-addressed store. A restored cache cannot be stale, only incomplete,
and an incomplete one is a fetch rather than a wrong answer.
Three clauses the spec did not ask for, each pinned by a test. A fetched
body that will not parse as its declared kind is refused rather than stored,
because NCBI answers a bad accession with a 200 and an HTML page and with no
digest declared nothing else stands between that page and a pipeline treating
it as a reference. Reads fetched from a remote must declare their sample,
because a store object is named for the hash of its bytes and a hash is no
sample identifier — and the run accession is not defaulted to, because one
sample can have several runs. And offline=True makes “the second run needs
no network” a thing a test can prove rather than infer, which is also what a
reviewer with no connection wants.
What writing the tests caught. The fetch has to release the GIL. Holding it through a minutes-long SRA download would freeze every other Python thread, and it deadlocked outright against a local server the test was fetching from — the server’s thread could not run while the fetch it was answering held the lock. The only way that one gets found is by writing the test.
One existing assertion changed, and it is the milestone’s point.
test_a_bundle_of_the_demo_reproduces_it_on_a_clean_cache looked for
SRR8525904_1.fastq.gz in the bundle’s inputs.tsv. The reads are fetched
now, so the path is a store object named for the hash of its bytes; the test
asserts the hashes instead, and additionally that remotes.tsv names the
locator. The hash was the better check all along — a file name is not
evidence of anything, which is why the engine does not key on one.
The source_set dedup rule is settled. It must not keep NameSet’s
ASCII-case folding, and the reason is the asymmetry of the two failures.
Folding case drops a member, which turns two sets that differ into two sets
that agree — a false agreement, the thing this engine exists to refuse. Not
folding it can only add one, which turns agreement into a refusal: loud,
legible, and fixed by declaring one spelling. Ligase has taken that trade
twice already, in D37’s refusal to equate human with Homo sapiens and in
D38’s refusal of an alias table.
The folding is also the unusual rule rather than the default. release,
subject and identifier_namespace are all Label and compared
case-sensitively, because a label is a name the user chose. Organism folds
because Latin binomials are a closed conventional vocabulary where HIV-1
and hiv-1 are one organism by convention. Sources are not that:
s3://bucket/A.vcf and s3://bucket/a.vcf are two objects by
specification, and GSE12345 matching gse12345 is the minority case,
costing a user one consistent spelling.
M10a applies the axis to no kind — PinnedRemote and RecordedQuery are
not kinds, they are how a dataset is obtained, and what comes back is a
Reference or a Reads. So the settlement is recorded in
dim::names::SourceSet’s own doc comment and here, and the type split lands
with the milestone that first applies the axis, together with that axis’s
check.rs::compare and graph.rs::propagate arms. Shipping an unexercised
value type now is the very condition M7’s standing requirement exists to
prevent.
Which milestone that is, after M12. M12 declined the axis too, and gave
its reasons: §4.2 defines a RankedList as an ordered EntitySet, §4.1
gives an EntitySet four dimensions, and source_set is not among them.
Claiming an axis to match a summary sentence would have forced D36’s question
open on a kind with no need of it. So no kind in v0.2 applies
source_set, and the implementation travels to KnowledgeGraph in v0.3,
which §4.3 does give it. What has changed since M7 raised the question is
that the answer is now written down rather than left open, so whoever gets
there is implementing a decision rather than making one.
M10b — Recorded queries and replay
Done. The six of M10’s clauses M10a did not discharge, each by a named test. D39 needed no second write; it gained one subsection, below.
A pipeline with a RecordedQuery step runs against a local mock server and
then again with that server down, completing from the record with a
byte-identical result, in
tests/test_queries.py::test_a_recorded_pipeline_runs_again_with_the_server_down_to_an_identical_result
— where “down” is a closed socket rather than a stubbed error, which is what
M10a’s mock_http.rs and its shutdown() were written for.
--replay on a query with no record fails before any step runs
(test_replay_on_a_query_with_no_record_fails_before_any_step_runs), and
needs no arranging for the same reason the wrong-digest clause did not: there
is no graph yet when it fails, because a query resolves while the pipeline
file is being read. The message is captured from a real run into
the guide by a capture_diagnostics.py scenario,
so the page cannot drift from the engine.
--refresh against a changed response reports the drift and does not
overwrite the record
(a_refresh_against_a_changed_response_reports_the_drift_naming_both_records,
an_earlier_run_is_still_replayable_after_a_refresh). Two paginated calls
recorded in one order replay in the other
(test_two_paginated_calls_recorded_in_one_order_replay_in_the_other), which
falls out of keying a record on the canonical request rather than being
arranged.
A bundle exported from the recorded run replays on a clean machine with no
network (test_a_bundle_replays_on_a_clean_machine_with_no_network): a fresh
store, a stopped server, and remotes.seed — which ligase methods --verify
calls itself, before the pipeline is imported, because by the time
methods() runs the query has already resolved.
And a request made with an auth header stores no trace of it
(test_an_auth_header_leaves_no_trace_in_the_record_or_the_bundle), asserted
over every file in the store and the bundle rather than over a list of header
names. M10a did not discharge this one: its
test_nothing_the_engine_sends_or_keeps_holds_a_secret proves only that the
engine sends no header of its own, and the clause needs a request actually
made with one.
Three decisions the building raised. D39 settles what is promised and left these.
Where volatile fields are dropped from. Only the top level of a JSON
object, which is the one new subsection D39 gained. Recursing is the obvious
implementation and the wrong one: a timestamp at the top of an envelope is
D22’s case, and the same key inside a data record is what the record says, so
dropping it recursively would silently delete a time series. Each drop is a
note, in case the key was the user’s own data after all.
How a record is keyed. (request, revision) and not
(request, recorded_at). A refresh must never overwrite what it refreshes,
and keying on a timestamp would let two refreshes inside the same second
collide — resting “never overwrite” on a clock. There is no supersedes
column either: revision n supersedes n − 1 by construction, where a
stored pointer could drift from it.
How a token is supplied. auth_env= names an environment variable rather
than taking a value. D39’s structural argument — that nothing stored has a
field for a header — does not cover the pipeline file, which a bundle copies
into pipeline/, so a literal token would have travelled in every bundle.
Naming the variable closes that door, and Auth’s hand-written Debug
closes the last one by redacting the token out of any message that formats a
request.
What writing the tests caught. The mock server routed on the path alone,
stripping the query string — so it could not tell ?page=1 from ?page=2,
which is the whole of the pagination clause. It now matches a route
registered with a query string against the whole target. The only way that
gets found is by writing the test.
One honest gap, pinned by a test rather than left to be discovered. A
fetched body must read as its declared kind, which catches an HTML error page
served as a Reference and does not catch one served as an EntitySet:
a single-column TSV is a valid gene list, so an error page is a gene list of
one implausible gene. That is uniform across local files, pinned remotes and
queries rather than something this milestone introduced, and
an_error_page_passes_the_kind_check_for_an_entity_set_and_this_is_why
records it so that the day the entity-set loader learns to refuse an
identifier containing a <, something says so.
And one gap in M10a’s docs, closed. web/content/guide/remotes.mdx was
never added to its _meta.ts. The page built and was linked, so npm run check passed, but Nextra appended it unlisted with a default title instead
of showing it as “Fetching your inputs” in its intended place.
source_set needed nothing here. M10a settled the dedup rule and
established that no kind in v0.2 applies the axis; lg.query claims no axis
for the first time either, since it takes the same declared dimensions as
lg.remote and a kind already applies every one of them. So M7’s
arm-pinning requirement is not reachable in this milestone, and the type
split still travels to KnowledgeGraph in v0.3.
M11 — Stochastic steps
A step that is not deterministic says so, and an unseeded one is refused. D40.
Done when: decorating an unseeded stochastic=True step raises
StepDefinitionError naming the missing seed; a seeded stochastic step audits
as reproducible over two runs with the same seed; the methods paragraph
names the seed; the test uses a real stochastic tool on the tiny fixtures
(IQ-TREE on a small alignment, or a seeded Python step where the tool is
absent).
Done. All four clauses, each pinned by a named test. See steps that draw on randomness, declaring one and how the verdict is decided.
Decorating an unseeded stochastic=True step raises StepDefinitionError
naming the missing seed, in test_a_stochastic_step_must_name_its_seed, and
the refusal is captured from a real decoration into both pages by a new
capture_diagnostics.py scenario. Four more refusals are pinned beside it,
all at decoration: a seed= naming a parameter the step does not take, one
not typed int, one naming a data parameter, and seed= with no
stochastic=True — which the spec does not mention and which is refused
rather than read as a shorthand, because two spellings of one declaration is
how a second untested path gets in.
A seeded stochastic step audits as reproducible over two runs with the same
seed, in test_a_seeded_stochastic_step_audits_as_reproducible and
a_seeded_stochastic_step_is_reproducible_rather_than_deterministic. The
second of those is the clause that matters, and it is asserted as a
contrast: the same body, the same seed and the same output hash, declared
and undeclared, giving reproducible and deterministic. A reader finding
only the first would reasonably take the saturation for a bug.
The methods paragraph names the seed, in
a_stochastic_steps_paragraph_says_it_was_seeded. It says seeded with
seed = 42 rather than with seed = 42, which is more than the clause asked
for and less than it meant: a seed is a plain parameter, so the paragraph
already printed it, indistinguishable from min_freq = 0.01. A reader of
that cannot tell the step samples or that this is the number to copy.
The test runs everywhere, and nothing skips. The clause offered IQ-TREE
on the tiny fixtures or a seeded Python step where the tool is absent, and
the Python step was taken for both halves of the reason. IQ-TREE is installed
on no machine this project builds on — it is not among the demo job’s three
tools — so an IQ-TREE test would skip on all of them, and a test that
silently skips in CI is worth much less than one that runs. It would also
need the Tree kind D40 declines. The vehicle is a seeded Python shuffle of
a RankedList, chosen because that kind’s canonical form keeps row order
(D41), so the seed’s effect is visible in the engine’s own comparison rather
than hidden behind a sort.
No Tree kind, and so no standing requirement. D35 refuses one: a
tree’s tip labels are on the identifier_namespace axis an EntitySet
already claims, its organisms are organism, a reference tree’s version is
release, and what is left is Newick with a schema — a parser. So this
milestone adds no kind, applies no axis, and brings no compare or
propagate arm within reach that was not already pinned. M7’s standing
requirement does not land here, and the absence is recorded rather than left
to be inferred.
The seed was already in the cache key. CLAUDE.md requires seeds to be
explicit, typed, hashed parameters, and key.rs already said in its module
docs that a seed needs nothing special because it is an ordinary parameter.
Verified rather than assumed, and then pinned, because an unpinned property
is one that can be lost: a_step_run_under_two_seeds_has_two_keys and
the_same_seed_is_the_same_key are both new, and without the first two seeds
could come to share a key and the second run would be served the first one’s
answer in silence.
Audit has no first-run problem. An audit is defined as two runs of its
own, so there is never one observation to judge; the surface that does see a
single run, ligase run, reports an Outcome rather than a verdict, which is
the right answer rather than a gap. Nothing to build, and it is written down
in the architecture page so that the next reader does not go looking for the
missing case.
What the work turned up, and the lesson worth keeping. Three things, and none of them was in the four clauses.
The first is that satisfying the saturation by relabelling would have made the
report lie by a different route. reproducible’s existing text is “the bytes
differ; what they mean does not”, which is false of a byte-identical seeded
step — so AuditReport::text now branches on the facts rather than on the
verdict alone. The most valuable branch is one nothing asked for: a step that
names a seed and still disagrees with itself has a declaration that is
false, and the report now names the parameter the pipeline believed was
controlling the tool and says it is not. Filing that under ordinary
irreproducibility would have wasted the one thing the declaration bought.
The second is methods.rs::restating, which drops a parameter whose value is
merely the node’s own scope so that sample_id = 'SRR8525904' does not appear
beside a sentence already naming four samples. That rule is right and its
catch-all was written when every parameter was a setting. It is too wide the
moment one parameter is obliged to appear, and a seed caught by it would
have vanished from the paragraph — a methods section quietly omitting the one
number D40 exists to make it state.
a_seed_is_named_even_where_it_reads_like_the_sample pins the exemption.
The third is in the error message, which on this project is the product. The
refusal for a missing seed tried to suggest the parameter it thought was
meant, and the first version suggested the first int parameter it could
find — producing seed="iterations" on a step taking iterations: int, a
confident wrong answer a reader might take. A guess is only worth printing
where it cannot be wrong. It now suggests a spelling only where a parameter
is actually called one, and otherwise lists what it found and leaves the
choice with the person who knows the tool.
test_the_refusal_does_not_guess_a_seed_it_cannot_know pins that, and the
published diagnostic is a real one rather than a paraphrase.
The lesson. Two of those three are the same shape as M12’s: a rule
written when its category had one member, which becomes wrong the moment the
category has two. restating was correct while every parameter was optional
prose and wrong once one was mandatory; Verdict::Deterministic was correct
while every step was either arithmetic or broken and wrong once a step could
be seeded. Adding a member to a category rewrites every catch-all that was
written when there was one — and the place to look is wherever a rule says
“all parameters” or “the bytes matched”.
One pre-existing claim corrected. exec.rs said of citations that
“adding one must not rerun a pipeline”. The field is indeed out of the cache
key, but graph.py::_source hashes a step’s source including its decorator
line, so editing cite= has always moved the step’s code and rerun it —
early cutoff then stopping the rerun downstream. The field being excluded
reads like a promise that editing it is free, and it is not. Both fields now
say what is true, because the new seed field would otherwise have copied
the error.
Deliberately not done. No Tree kind; no new Verdict variant and no
separate StochasticVerdict (the argument is in D40); no change to
prov::bundle, whose Match::Identical is correct as it stands and whose
file belongs to M14; no tolerance=, which is a separate nondeterminism rule
no clause here asks for; and no attempt to detect stochasticity, which the
engine cannot do and which a heuristic would get wrong in both directions.
AuditReport::verdict() still answers deterministic for a report in which
every step failed — pre-existing, outside these clauses, and already handled
on the Python surface by AuditReport.reproducible, which accounts for
failures. Noted rather than widened into.
M12 — Ranked lists
A RankedList kind, canonicalised in order. D41.
Done when: two files with identical rows in different orders produce
different canonical hashes — the inverse of every other kind’s test, and the
test says so; a RankedList and an EntitySet in different namespaces refuse
to join; a RankedList released HGNC 2026-09 refuses to score against an
EntitySet released HGNC 2026-03.
Standing requirement, from M7. The first kind to claim a dimension axis
must pin that axis’s arm in check.rs::compare and in graph.rs::propagate
with a test. Until a kind applies an axis, check_step never reaches its arms,
so no test can catch one written against the wrong field, and the compiler
cannot either: Organism and SourceSet are one type, and so are Release
and Subject. A RankedList brings source_set and subject into play, so
the requirement falls on this milestone for both, and the source_set dedup
question under M8 and M10 must be settled before source_set is applied.
Done. All three clauses, each pinned by a named test. See comparing ranked lists and the internals.
Two files with identical rows in different orders produce different canonical
hashes, in ranked_list.rs::the_order_of_a_ranked_list_is_its_result. The
test says so at length, and it asserts the contrast rather than the bare
fact: it hashes the same two files as a ranking and as a set, and asserts the
ranking disagrees and the set agrees. A reader finding only an assert_ne!
between two orderings would reasonably take it for a bug, and the fix is one
word in one line. A clause the spec did not ask for is pinned beside it:
reading one file as both kinds gives two hashes, because the DataKind is
hashed into the stream, so a step annotated to return a ranking can never be
served the cached output of one annotated to return a set.
A RankedList and an EntitySet in different namespaces refuse to join, in
a_ranked_list_in_one_space_refuses_to_join_a_gene_list_in_another and
test_a_ranking_and_a_truth_set_in_different_spaces_refuse_to_join. This
needed a change to check::required_pair that reading the clause does not
suggest: identifier_namespace was required for (EntitySet, EntitySet) and
the arm below it returned nothing for every other pair involving an
EntitySet, so the mixed pair — a ranking scored against a curated truth
set, which is the commonest reason to join the two kinds at all — would have
been demanded nothing of. Both pairs are now written out, and
two_named_lists_must_both_say_which_space_they_are_in asserts all four
combinations rather than the two the clause names.
A RankedList released HGNC 2026-09 refuses to score against an
EntitySet released HGNC 2026-03, in
a_ranked_list_refuses_to_score_against_a_gene_list_from_another_release and
test_a_ranking_refuses_to_score_against_another_release. That one needed no
table change — both kinds apply release, so compare_known reaches it
wherever both sides state one — which is the point of M7 having made
release an axis rather than part of reference.
Both refusals are captured from a real run into
the guide by two new
capture_diagnostics.py scenarios, so neither page can drift from the
engine. The guide’s central claim is captured too, by a new
capture_ranked.py: it builds a one-step pipeline over one file of each kind
in a temporary directory, runs it, reorders the ranking, runs it, reorders the
set, runs it, and publishes the four real run reports. The ranking rerunning
and the set not is the whole of D41, so the docs build now fails if either
stops being true.
The standing requirement was already discharged, and the entry above
misreads the spec. A RankedList brings subject into play, and M8 had
already brought it: an EntitySet claims subject, release and
identifier_namespace, and pinned each one’s compare and propagate arm.
This milestone adds each_axis_a_ranked_list_claims_compares_its_own_field
and a_step_over_a_ranked_list_carries_every_axis_it_claims anyway, since a
second kind reaching the same arms is cheap to pin, but the arms were not
unverified.
It does not bring source_set into play. The clause above says it does;
the spec it summarises says otherwise, and the spec is right. §4.2 defines a
RankedList as “EntitySet but ordered”, and §4.1 gives an EntitySet four
dimensions — organism, identifier_namespace, subject and release —
with source_set not among them. Claiming an axis in order to match a
summary sentence would have been the tail wagging the dog, and would have
forced D36’s NameSet question open on a kind with no need of it. So
source_set still has no kind, source_set_still_has_no_kind and
a_ranked_list_claims_no_source_set both hold, and the deduplication
obligation travels on to whichever milestone first needs the axis — M10 by
its own entry, or KnowledgeGraph in v0.3, which §4.3 does give it.
What the work turned up, and the lesson worth keeping. Two things, and neither was in the three clauses.
The first is the required_pair gap above: a table written when there was
one named list had a catch-all arm that silently became wrong the moment
there were two. It was not a bug when it was written and nothing flagged it,
because the arm it was hiding behind did not exist yet.
The second is the conversion registry. map_identifiers was registered for
DataKind::EntitySet alone, and graph::propagate guards a conversion’s
axis override on converts(output) — so a step declaring
conversion="map_identifiers" and annotated to return a ranking would have
carried the input’s namespace onto an output that had crossed, which is a
false statement about data, produced by the engine, of exactly the kind D38
exists to prevent. The guard’s own comment reasons about a kind whose
applies says it has no such axis; a RankedList has one, so the guard
stopped protecting anything and started hiding something. The row now names
both kinds, and map_identifiers_crosses_for_a_ranked_list_too pins it.
The lesson is that adding a second member to a category rewrites every “the only kind that…” assumption in the codebase, and that those assumptions are mostly not written down as assumptions — they are catch-all match arms and single-element lists that were exactly right when there was one. Three existing tests changed to say “either named list” where they had said “only a gene list”, and each of those was a place the assumption had been written down. The two defects were the places it had not.
M13 — Step library
ligase.steps: typed steps that ship with their dimensions, their citation
and their canonical form, and the demo rewritten on them. D42.
Done when: examples/hiv-resistance/pipeline.py contains no lg.sh call
of its own; every library step has a fixture test, a citation that resolves
and a page under guide/steps/; the validation suite still passes 81 of 81 at
15% and 5%, with the same explained discrepancies at 2%.
Done. The demo contains no shell command of its own — alignment and
variant calling come from ligase.steps.minimap2 and
ligase.steps.bcftools, and resistance stays because
lg.hivdb.interpret is this pipeline’s own subject rather than a tool
anybody’s pipeline begins with. tests/test_demo.py’s fourteen acceptance
tests pass on the rewritten pipeline and tests/test_validation.py passes 81
of 81 at 15% and at 5%.
The number in that clause was wrong, and the file was right. The clause
above said “the same nine explained discrepancies at 2%”, and so does the
spec. tests/test_validation.py says there were nine, that six went when
codons started being read from read-level haplotypes
(D31),
and that three remain — all Sample 30’s V75A, a sensitivity limit of
aligning to HXB2. KNOWN_AT_TWO_PERCENT holds three entries and
test_every_remaining_discrepancy_is_the_one_missed_mutation asserts exactly
those three. The clause was written before D31 landed and was never
corrected; the number to hold is three, and it is unchanged by this
milestone, which is the whole point of running that suite over a rewritten
demo.
Four steps, and two refusals
minimap2.align, bwa.mem, samtools.sort and bcftools.mpileup_call. The
other two rows of §5.5’s table were refused, and
D42’s
last clause is why: a step that cannot say what it produces is not a library
step. A criterion whose first two uses are admissions is not a criterion.
samtools index produces an index, which is not a dataset. It brings no
dimension, so under D35 it earns no kind; and declared
Alignment → Alignment its output would be a BAI whose bytes the engine
hashes as an alignment’s meaning, because a file that does not parse as its
kind falls back to Form::Raw without complaint — so everything downstream
would accept an index as an alignment.
Paired-end trimming cannot be a step at all. A step’s output is exactly one
file and paired reads are two, and every single-file reading of
Reads → Reads is a false dimension: one mate inherits paired-end from its
input and the graph propagates it, while an interleaved FASTQ loads as
Pairing::Single and Pairing has no third value.
Both are one missing engine feature — a step that can produce more than one file — together with a third symptom from the other side: the two aligners are paired-end only and nothing in their signatures says so, because a step can declare what it writes but cannot require a value of an input’s dimension. The architecture page says what closing it would take. Shipping a step with a false type on it to avoid writing that down would have been the worse trade.
What the work turned up
Four things, and none of them was in the clauses.
The engine recorded a tool’s complaint as its version, and the cache key
therefore never moved when the tool did. exec::tool_version ran
<tool> --version, read stdout, fell back to stderr, and ignored the exit
status. bwa does not understand --version: it prints
[main] unrecognized command '--version' and exits non-zero, so that
sentence was bwa’s recorded version — in provenance, in the methods
paragraph, and in the cache key. It is identical for every bwa ever released,
so upgrading the aligner moved no key and every alignment computed by the old
one would have been served for the new. That is the only thing tool=
promises. There is now a probe table, and a probe that fails unexpectedly or
loses its marker line is an error. The suite’s own stand-in tool had to change
from wc to awk for the same reason.
bcftools mpileup -f wrote a .fai into the content-addressed store. It
indexes beside the path it is handed, and in a run that path is an object
named after the hash of its own bytes. The demo on main had left one there.
An unaddressed file in an addressed store is a lie about what the store is,
it will not travel with the object when M14 gives the store a blob tier, and
on a read-only or remote store the write is not untidy but a failure. The
library step indexes a symlink in a directory of its own.
Every step the package exports was published in the API reference as a
repr() line with its documentation thrown away. @lg.step replaces a
function with a Step instance, which is neither a class nor a routine, so
generate_api.py fell through to its plain-value branch — and the branch that
fails the build on a missing docstring was never reached. lg.map_identifiers
had been published as map_identifiers: Step = <Step map_identifiers -> EntitySet> since M8. The generator documents the decorated function now, and
its coverage contract walks nested submodules so steps.minimap2.align has to
be grouped and documented rather than hiding behind steps.minimap2.
A string parameter reaches the shell with Python’s quotes around it. Found
and deliberately not fixed. graph.py serialises every plain parameter
with repr() — right for a cache key, where it is what tells f(1) from
f("1"), and wrong for shell substitution. So the demo’s BAMs carry
@RG SM:'SRR8525904' and the VCFs below them name their sample column the
same way. Since a sample set is a dimension, and a BAM’s read groups and a
VCF’s sample names are the one field canonicalisation refuses to drop
(D26),
every semantic hash in the demo is taken over a sample identifier that is not
the sample’s identifier. The science is unaffected, which is why it went
unseen: Ligase takes sample identity from the declared dimension rather than
from the file. It is still false, and it is hashed.
It is not fixed here because the fix is not a one-liner, and because
graph.py was M11’s while this was built: a parameter has to travel as two
forms, the one that enters the key and the one that enters the command, across
the Python layer, the bindings and exec.rs, and it retires every cache key
of every step with a string parameter. str() is not the fix — it would make
f(1) and f("1") share a key. It is recorded as a strict
xfail, test_a_string_parameter_should_reach_the_shell_as_its_value, so the
day it is fixed the test fails the other way and the workaround cannot be left
behind.
A step is a script now, not a pipeline
Every step whose body runs more than one program writes a file between stages,
under set -e, in a directory it removes on exit. That looks like a step
backwards from the one-line pipelines the demo had, and it is not: the engine
runs a recipe as sh -c, and sh -c 'a | b' reports only b’s status. A
dead aligner left samtools sort writing a perfectly valid, empty BAM and the
step succeeded — which downstream reads as a sample with no variants, which
for this demo reads as a patient susceptible to everything. POSIX sh has no
pipefail to reach for, so the answer is to stop depending on a pipeline’s
status.
Carried over from M12, and decided
lg.map_identifiers was typed for an EntitySet while the conversion
registry covered a RankedList too, so the registry advertised an edge no
shipped step crossed. Decided: ship it, as lg.map_ranked_identifiers.
A ranking of what crossed is still a ranking — rank has always been relative
to the set that was ranked, and
D41
claims that row order is the result, not that output rank equals input rank.
Refusing would have pushed a very ordinary operation off the typed path
entirely, which is the opposite of what v0.2 is for. It is a second step
rather than an argument because @lg.step reads the output kind off the
return annotation once, and one function cannot produce both kinds; the
mapping itself needed no change, because convert.rs already streams the
input and writes survivors in input order.
What was missing was the presentation half the M12 entry named, and it is a
real gap: the existing notes are about the identifiers that did not cross.
For a ranking the consequence is about the ones that did — every surviving
rank has moved. So there is a ranks-renumbered note carrying what rank the
output’s first row held in the input, and its how_to_fix is the thing a
reader will otherwise get wrong: take a cut-off before mapping, because the
top ten of a mapped ranking is the top ten of what crossed.
That number is computed from the three files rather than from the mapping’s own notes, and the reason is a trap worth recording: the named identifiers are capped for a long list, so counting them would give a number that was too small. A note with a wrong number in it is worse than no note.
The lesson worth keeping
Three of the four findings above are the same shape. A step declares something
about its output, the engine has no way to check the declaration, and so the
declaration is prose. tool= claimed a version the engine could not read
honestly; a return annotation claimed a kind the engine would silently stop
believing; a docstring claimed documentation a generator silently dropped.
None of them failed loudly, and each had been true-looking for several
milestones.
So the three accessors this milestone added to the bindings —
tool_version, citation, canonical_form — are the deliverable as much as
the four steps are. D42 asks a step for four declarations, and a declaration
nothing can check is a comment.
M14 — Remote store
The content-addressed store gains a blob tier with pluggable backends, on the trait M10 introduces. The index stays local and rebuildable. D43.
Done when: the full suite passes against a MinIO container in CI with
LIGASE_STORE set and no test changed; a second checkout on a clean machine
pointed at the same bucket completes the HIV demo entirely from cache, with
zero steps run; an object corrupted in place in the bucket is rejected on read
with an error naming the hash mismatch, and the step reruns; deleting the
local SQLite index and running again rebuilds it from the store; a thin bundle
replays on a machine with store access and fails with a clear message on one
without; the default install has no object_store dependency and
ligase store status says so.
Done, with one clause met differently and said so here rather than quietly.
Five of the six are met and were demonstrated on the real HIV demo against
a MinIO bucket: a second cache pointed at the same bucket reported
12 steps: 12 from cache; deleting cache.sqlite and running again
reported the same; an object corrupted in place was refused with an error
naming both hashes and the bucket that served it, and its step reran; a
thin bundle replays where the store is reachable and fails before any step
runs where it is not; and a default build has no object_store in
cargo tree, which tests/default_build.rs asserts rather than claims.
The first clause is not met for two separate reasons, and both are
findings. The smaller one first: MinIO is no longer freely obtainable —
Docker Hub denies anonymous pulls of minio/minio, quay.io/minio/minio
answers 401, and the community binaries at dl.min.io are 410 Gone (checked
23 October 2026). CI runs LocalStack’s S3, and nothing in the tests is
MinIO-specific.
The larger one is about the suite. A handful of tests assert that a fresh cache directory means nothing has been computed, which a shared store is precisely what makes false — two tests computing the same thing from the same inputs have the same step key. The failing set is not even fixed: whichever of a colliding pair runs first wins, so skipping six produced seven different failures. The clause also contradicts the second: completing the demo from cache on a clean machine requires step entries to be shared, and the suite passing unchanged requires sharing them to change nothing.
Measured against a real bucket: 458 of 460 Rust tests and 303 of 307
Python tests pass unchanged. What the clause was really asking — is the
remote tier substitutable for the local one — is answered by
tests/s3.rs and the end-to-end tests, which run against a bucket in
CI’s store job. The argument is in
the store.
Two things found on the way that nothing asked for. FsStore::put staged
every write of a key at the same path, so two concurrent writers of one
object raced and the loser failed with NotFound on a put that had
succeeded — reachable before this milestone by two fan-out nodes recording
the same paginated response. And provenance did not travel with a shared
cache entry, so the feature’s headline scenario produced runs that
completed perfectly and could not be written up; D43 has it.
The v0.3 roadmap
v0.2 is shipped. v0.3 is KnowledgeGraph, then claim binding, then the
chemistry extra, then cluster execution on the shared store, with the NeoRx
seven-disease experiment ported across as validation. One milestone per
session, as before.
Cluster execution is the last engine milestone of v0.3. The NeoRx experiment is not in this repository and is a validation milestone of its own; M18’s entry says what was looked for.
M15 — Knowledge graphs
A KnowledgeGraph kind: typed nodes and edges with per-edge provenance, read
from JSON, carrying organism (set), identifier_namespace, source_set,
subject and release. The first kind in Ligase that is assembled rather
than measured, and so the first with anything to say about which sources
contributed. D44.
Done when: the kind exists with its loader, canonical form, dimensions and
Python surface; two graphs whose sources disagree refuse to join, naming the
source one side lists and the other lacks, pinned in Rust and in Python;
source_set has its own value type and compares exactly, so
s3://evidence/A.json and s3://evidence/a.json are two sources while
Organism still folds HIV-1 and hiv-1 into one; every axis the kind
claims has its check::compare and graph::propagate arm pinned by a named
test; a graph’s sources are read off its own edges, and a graph naming none
leaves the axis unknown with a reason rather than known and empty; reordering
a graph’s nodes or edges does not change its canonical hash while adding a
source to one edge does; a RecordedQuery declared KnowledgeGraph runs
against a mock server and replays with that server down to an identical
result, closing the JSON gap
remote inputs names.
Standing requirement, from M7. The first kind to claim a dimension axis
must pin that axis’s arm in check.rs::compare and in graph.rs::propagate
with a test. Until a kind applies an axis, check_step never reaches its
arms, so no test can catch one written against the wrong field, and the
compiler cannot either: Organism and SourceSet are one type, and so are
Release and Subject. This milestone gives source_set its first kind —
the axis M8, M10 and M12 each declined and recorded as travelling here — so
the requirement falls on it, and D36’s reserved right to split the set type
has to be carried out before the axis is applied.
Done. Every clause, each pinned by a named test. See assembled data, comparing knowledge graphs, the internals and my target ranking changed and I cannot find the analysis change.
D35 first, because that is the question to ask of a new kind. The kind
brings source_set, which no other kind in Ligase applies, because no other
kind is assembled from anything: a gene list is one file and a BAM is a
measurement. A graph’s content is a function of which evidence sources were
folded into it, no node or edge records that list, and two graphs differing
only in it diff and score against each other in silence — which reads as
biology. What does not earn the kind its place is the file shape: there is
no standard knowledge-graph format, so this milestone defines the node/edge
JSON Ligase reads, and defining a shape is exactly what D35 says is
insufficient. That is said out loud on the architecture page rather than left
for a reviewer to notice. Take source_set away and the kind should not
exist.
Two graphs assembled from different sources refuse to join in
knowledge_graph.rs::two_graphs_assembled_from_different_sources_refuse_to_join
and in
tests/test_knowledge_graphs.py::test_two_graphs_assembled_from_different_sources_refuse_to_join.
Three capture_diagnostics.py scenarios publish the messages from real runs,
so the pages cannot drift from the engine.
The source_set type split, which three milestones deferred. D36 reserved
the right; M10a settled the rule — a source must not fold ASCII case,
because folding can only drop a member and produce a false agreement while
not folding can only add one and produce a refusal a user can read; M8, M10
and M12 each declined the axis and recorded that the obligation travelled on.
It lands here as NameSet<M>, generic over how two members are compared, so
the set algebra is written once and each rule once. Organism is
NameSet<FoldsAsciiCase> and SourceSet is NameSet<Exactly>.
Two shapes were considered and rejected, both on the project’s own earlier
reasoning. A second struct would have implemented the algebra twice, which
D36’s argument for sharing the type calls “a rule that will eventually be
implemented differently”. A fold_case: bool field would have put two
meanings behind one call chosen by an argument a caller can get wrong
silently — the shape Digest::record and ordered_record were deliberately
split to avoid — and would have let one dimension carry two rules at runtime,
which D36 forbids outright.
What the split turned out to be worth, beyond tidiness. M7’s standing
requirement exists because the two axes were one type, so a compare arm
written against the wrong field compiled. They are no longer one type, so that
particular mistake is now a build failure rather than something a test has to
catch. Release and Subject are still one type, so the requirement still
stands for those — and the two tests are there anyway,
each_axis_a_knowledge_graph_claims_compares_its_own_field and
a_step_over_a_knowledge_graph_carries_every_axis_it_claims, because the
requirement asks for them and because propagate’s arms name their fields by
hand whatever the types are.
The sources are read, not asked for, and that is D44. They are on the
edges. Asking for them would be the engine demanding something it is holding,
and a declaration it could have checked and did not is a declaration that
drifts from the data. So the loader takes the union of every sources list on
a node, an edge and the top level, records it as inferred and names the file.
A declaration still wins, because a graph’s own names are sometimes a mirror
or a shard rather than the source a reader would recognise. An empty reading
is a gap: two graphs that each named none would otherwise agree on the one
axis the kind exists for.
The JSON gap is closed rather than named again.
architecture/remotes.mdx recorded that no kind in v0.2 read JSON, so
response.rs’s JSON branch — keys sorted at every depth, the server’s own
fields dropped from the top level — was exercised by unit tests and never
through a pipeline. This kind reads JSON, so deferring would have been
deferring for no reason. Four tests in tests/test_queries.py take a graph
through the recorder, into the store and back out through the loader, and the
one that matters most is
test_a_servers_signature_does_not_make_a_graph_a_new_graph: the same graph
served a minute later with a different generatedAt is the same graph, and
without the drop every run would see drift and everything below the query
would rerun.
Three clauses the spec did not ask for, each pinned. The namespace is
demanded of every mixed pair — graph against gene list, graph against
ranking, in either order — rather than left to a catch-all, which is M12’s
lesson applied before it could become the same bug a second time; scoring a
graph’s neighbourhood against a curated list is the commonest reason to join
those kinds at all. The canonical form’s deduplication inside a node is exact,
matching SourceSet, because a hash that folded A.json into a.json would
make two graphs the checker refuses to join agree on their content — the two
halves of the engine contradicting each other. And each node and edge record
leads with a discriminator, since both go into one sorted list and a node and
an edge whose parts lined up would otherwise be one record.
Four assertions changed, and they are the milestone arriving.
source_set_still_has_no_kind had held since M8, asserting the axis applied
to nothing; a_ranked_list_claims_no_source_set carried the same loop. Both
were correct until this milestone and are now wrong by design, so the loop
moved to knowledge_graph.rs::source_set_is_the_axis_this_kind_was_admitted_for
and each kind’s own test keeps only its own claim. The two
only_a_named_list_has_… tests became
only_a_kind_that_names_entities_has_…, and
release_applies_to_a_reference_and_to_either_named_list became
…_and_to_the_kinds_that_name_entities. Renaming them rather than widening
them quietly is the point: a test whose name no longer describes what it
asserts is a test nobody reads.
What the work turned up that nothing asked for. A default type parameter
looked like free compatibility — NameSet<FoldsAsciiCase> as the default
meant every existing mention of NameSet kept compiling — and it was the
wrong call. NameSet::of([...]) cannot infer M from its arguments, so the
existing value tests failed to compile anyway; and worse, a bare NameSet
would have meant whichever rule the default happened to be, which is precisely
how the two axes came to share a rule nobody had chosen for both. The default
is gone, and tests/dimension_values.rs now names the axis whose rule each
test is about.
M16 — Claim binding
A Claim: a number or a phrase a paper states, declared on the pipeline and
bound by the engine to the step key and output hash that produced it, with
unbacked claims reported the way uncited tools are. §5.6 of the v0.2 spec in
three sentences; the design that turns it into something checkable is half
the milestone. D45.
Done when: a claim naming a step the pipeline runs is bound, carrying
that step’s key, the output’s canonical hash and the date and platform its
record was written on; a claim naming a step nothing produces is unbacked,
naming the steps it does produce; a claim whose step is a methods gap is
unbacked rather than bound; a claim naming a sample the step did not run on,
a parameter it does not take or a tool it did not run is unbacked, each with
its own reason naming what there was; a claim restating a parameter value or
a tool version the run disagrees with is contradicted, stating both
numbers; a claim bound to a step served from the cache is bound to the
computation that produced it and the date that computation happened; a claim
never enters a step key, so adding, editing or removing one reruns
nothing; an unbacked or contradicted claim raises ligase methods’ exit code
and a bound one does not; a claim’s declaration is refused where it is
written if it names both a parameter and a tool, gives a value with neither
or names either with no value; a pipeline that declares no claims
produces byte-identical methods text and summary; claims travel in the
bundle as claims.tsv.
Done. Every clause, each pinned by a named test. See a number in a paper, binding a number you published, the internals and the number in my paper is not the number my pipeline produces.
The scope rule, and why D35 is the wrong test to apply. D35 asks a kind
to bring a dimension whose mismatch is a documented silent failure. Claim
binding is not a kind and brings no dimension — and neither does ligase audit, the bundle, early cutoff or the citation table, so applying D35’s
letter to a feature would condemn four things the project is built on. It
was held to the reasoning behind D35 instead: the engine already makes one
promise three times — D24 names an uncited tool, D26 names a field that
cannot be dropped, a Gap names a step nothing is known about — and that
promise is that the record names what it cannot account for. The number in
the paper was the one thing it could not account for and never mentioned.
This milestone is the fourth instance of an existing promise rather than a
new capability, which is why it added no store, no dimension and no
dependency. That argument is on the architecture page rather than left for a
reviewer to reconstruct.
Where the spec was reshaped, and said so. §5.6 says claim binding
“extends Methods and MethodsGap”. The first it does. The second it does
not, and the literal reading was available and cheaper: a MethodsGap is
(step, scopes, nodes), and a claim has no nodes — it is one sentence, not
a count of graph nodes — one scope rather than a list, and five
distinguishable reasons where a step’s gap has one. Pushing a claim through
that shape would have meant a nodes column always equal to 1 and meaning
nothing, and a reason with nowhere to live: a record whose columns are not
true, which is the one thing a provenance store may not be. So what extends
is the gap-reporting MethodsGap stands for, in a type honest about what a
claim is, and every existing use of MethodsGap is untouched.
The question §5.6 left open, which is the heart of D45. “The step and run
that produced it” has to be settled when the cache can satisfy a step from an
earlier run. A claim binds to the computation — the key is already the
identity of one — so a cached step backs its claim completely and the date
reported is when that computation happened, not when this run fetched it.
Binding only what the current run recomputed would have made ligase run --force the only honest way to write up a paper. The second half needs no
balancing: a claim in a cache key would mean editing a sentence in a
manuscript reruns the aligner, so a claim is in no key and can never fail
check().
One existing meaning changed, and it is the right change.
Methods.accounted_for was not self.gaps and is now not self.gaps and not self.unbacked. A property with that name returning True for a write-up
that states a number the analysis does not support would be making the
engine’s own promise falsely. It was raised as a public-API change before it
was made, as was the whole surface, because CLAUDE.md asks.
What the work turned up that nothing asked for. Two things, both found by a test rather than by design.
python3 --version prints Python 3.15.0 — a name that is not the name
the tool was invoked under. The first version_states stripped only the
tool’s own name, as describe_tool does, and so called a correct claim
contradicted. It now tries the whole first line and each way of trimming a
leading name off it, including everything before the first digit; that
generosity is safe only because of the rule that does the work, which is that
a prefix must end at a component boundary, so 2.4 still refuses to claim
2.41. The same rule leaves a digitless version line whole, which is why a
claim against bwa’s [main] unrecognized command '--version' banner — the
string D42 found the engine recording as a version — is wrong rather than
accidentally right.
And a step with a single node has no scope at all, because
Graph::scopes gives one only where something tells siblings apart (D15). A
claim naming a sample of such a step is therefore told that there is nothing
there to name and advised to drop the sample, rather than being handed an
empty list of samples it might have meant.
Methods::produced, which the design did not start with. A Group is a
set of nodes run the same way and keeps one representative Record,
discarding the other nodes’ keys. A claim binds to a computation, so those
keys are exactly what it needs — and produced is not a duplicate of
groups but the thing grouping was throwing away. An unscoped claim over a
four-sample step has four bound rows, where binding it to the group’s
representative key would have named one sample’s computation and implied it
covered four.
A seventh generator, and the flake it shipped with.
web/scripts/capture_claims.py runs a real one-step pipeline three times and
captures what ligase methods printed and what it exited with. An
unbacked claim failing and a bound one not is the whole of what makes a claim
enforceable in CI, so the docs build fails if either stops being true.
It first wrote one pipeline.py three times in a shared directory, and CI
caught it on the first run of the PR. Two of the three differ by a single
character — min_freq=0.01 against min_freq=0.05 — so the file is the same
size, and CPython validates a cached .pyc against the source’s size and
its mtime in whole seconds. Write both inside one second and the second
run executes the first one’s bytecode, which is exactly what the build
reported: a capture of the contradicted write-up that was really the bound
one, down to the step key.
It passed locally every time because this machine’s shell exports
PYTHONDONTWRITEBYTECODE=1, so no .pyc was ever written here. That is the
transferable part: a generator that rewrites a Python file in place is
reproducible only by accident, and the first guess at the cause — made and
checked before it was acted on — was wrong for that reason. Each capture now
gets a directory of its own, which is the shape capture_stochastic.py
already had, and each capture asserts the number it is about rather than
only the sentence around it.
M17 — The chemistry extra
Two kinds — MoleculeSet and MoleculeRanking — and the axis that admits
them: representation, which says what a name in a molecule table is and
what was done to the structures before it was computed. RDKit arrives as an
optional Python extra, pip install 'ligase-bio[chem]', so the genomics
install acquires nothing. D47–D50.
Done when: both kinds exist with their loader, canonical forms,
dimensions and Python surface; two sets standardised differently refuse to
join, naming the operations one side also applied, pinned in Rust and in
Python; representation is demanded between any two of the molecule kinds in
all four combinations, and two sets that each say nothing are refused rather
than agreeing by silence; the axis’s check::compare and graph::propagate
arms are each pinned by a named test; a representation is read off the line
the standardiser wrote, and a table of InChIKeys with no such line leaves the
axis unknown with a reason rather than assuming as given; reordering a
set’s rows leaves its hash unchanged while reordering a ranking’s changes it,
asserted as the contrast over the same two files; changing only the
representation line changes the content hash with the key column
byte-identical; the measurement §4.4 asked for is in the repository and
reproducible; standardize crosses the axis as a registered conversion whose
failures are named and counted, with stereo discarded but not declared an
error and declared but not discarded a note; and the default install is
proved unburdened rather than claimed to be.
Standing requirement, from M7. representation is a new axis, so the
first kind to claim it pins that axis’s arm in check.rs::compare and in
graph.rs::propagate with a test. The compiler catches less than it looks
here: Representation is a type of its own, so an arm reading another axis’s
field does not build, but propagate’s arms name their fields by hand
whatever the types are.
Done. Every clause, each pinned by a named test. See what a molecule’s name means, comparing compound sets, the internals and my compound overlap is almost empty.
The measurement came first, and it decided the design. §4.4 asked for InChIKey stability to be measured before committing to it as a canonical form, so it was measured before any engine code was written: six RDKit releases spanning four years, thirty input structures, six standardisation states each, 1,080 keys. Zero disagreements between versions, tautomer normalisation included, so the key is a sound thing to compare by.
In the same span RDKit’s canonical SMILES for cisplatin changed, while its InChIKey did not move at all:
2022.09.5-2024.09.6 Cl[Pt]Cl.N.N
2025.09.6-2026.09.1 N.N.[Cl][Pt][Cl]That is §4.4’s “InChIKeys, not SMILES strings” measured rather than assumed — a canonical form over SMILES would have moved the content hash of every file holding that compound on an upgrade that changed no chemistry — and it is why the identity is part of the axis’s value rather than an implementation detail.
What moves the key is the standardisation: twelve of the thirty, within one version, all of them ordinary drugs. Aspirin has three keys, two of which differ in the final character. The dimension is therefore demonstrated rather than argued for, which is the strongest position a D35 argument has been in.
§4.4 was wrong about stereochemistry, and that is the finding. It
describes the state as keeping stereo. RDKit’s tautomer canonicaliser
discards it at an sp3 centre inside a tautomeric system, and a carboxylic
acid’s alpha-carbon is inside one, so ibuprofen’s two enantiomers both arrive
as HEFNNWSXXWATRW-UHFFFAOYSA-N — one active compound and one inactive one,
indistinguishable. So the axis records what was done rather than what was
asked, the standardiser checks its own work, and the check is asymmetric on
M10a’s own reasoning: stereo lost and not declared is an error, declared and
not lost is a note. D47.
D35 as written did not admit the second kind, and that is now written
down. MoleculeRanking brings no axis MoleculeSet does not bring. Nor,
examined the same way, did RankedList bring one EntitySet lacked — so
either M12 shipped a kind outside the rule, or the rule has a second limb
nobody had written. It has one, and D48 is it: a kind also earns its place by
bringing a canonical form whose absence makes the cache serve a wrong answer,
which is D41’s argument exactly, relocated from the data to the engine. The
limb is deliberately narrow, and D48 says what it does not admit.
The kind is also renamed. §4.4 called it BindingScores, and the same
file shape holds QSAR-predicted activities, predicted solubilities and
synthetic-accessibility scores, none of which is a binding score.
The engine acquired no chemistry at all, and that is proved. D49: the core
compares representations and refuses joins, which is string work; RDKit is
needed only to produce a standardised set, and lives in one Python module
that imports it inside the one function that uses it. So cargo tree is
unchanged, default_build.rs gained a test that a featureless build does the
whole of the checking, and a subprocess asserts that import ligase imports
no RDKit — which is what notices the day somebody moves the import to the top
of the module. The docs site builds with no toolkit either.
Unlike remote-store, this extra is not empty: RDKit is a Python package,
so an extra can reach it. The floor, rdkit>=2024.3.1, is the first series
with NumPy 2 wheels rather than a guess; the measurement covers two series
below it.
Standardisation is the conversion registry’s second entry, and the first
with a Python body. D50. Nothing in the registry had to change for that,
which is better evidence than another Rust-bodied row would have been that
liftover and rename_contigs will be rows rather than redesigns. Its
version capture could not be a tool=, because RDKit is a library with no
executable to probe, so the version is read before the step is built and
passed as a parameter — which puts it in the cache key, where reading it
inside the body would not.
Three assertions were renamed rather than widened quietly. A molecule
table names entities and claims no identifier_namespace, because
representation.identity already says which space it is in, so
only_a_kind_that_names_entities_has_an_identifier_namespace had stopped
being true and is now no_kind_that_holds_sequence_has_…. The subject matrix
became only_a_kind_that_is_about_something_named_has_a_subject, since a
compound set is about a target as squarely as a gene list is about a disease.
And the_three_axes_are_appended_… had said “three” since M8.
What the work turned up that nothing asked for. Three bugs, each found by
a test rather than by its author. A line’s own newline was yielding a blank
line that the standardiser’s header loop read as a header of one empty
column, so a table with a comment above its header failed to parse. The
stereo count double-counted the molecules it was reporting on. And the
canonical form accepted any one-column text file as a molecule table — a
JSON document included, since every line of one is a row of a single field —
because it was modelled on canon::entity_set, which deliberately does not
check which column holds its identifiers. Here the identity column’s name is
fixed by the kind, so it can be checked and therefore must be: without it a
file hashes as the canonical form of something it is not, which is the
opposite of the conservative direction canon::hash takes everywhere else.
What was declined, with reasons. source_set is not claimed, though a
library merged from three vendors is genuinely assembled and the axis exists:
nothing here can read it, because which vendor a compound came from is a
column rather than a fact about the set, and an axis the loader cannot fill is
the unexercised value type M7’s standing requirement exists to prevent. No
chemistry file format is read — no SDF, no MOL, no .smi — which is D35’s
first clause applied rather than argued around. representation records the
operations and not which toolkit applied them, because the measurement
compared RDKit against itself across time and not against another toolkit,
and recording the toolkit would claim more than the evidence supports. RDKit’s
citation was a shortfall while this milestone shipped and is no longer one:
it needed a change to D42’s machinery, which D46 made — a tool may be a
library, whose version the host supplies, because tool= had meant a program
the engine execs as well as a name a citation resolves from. The step now
declares tool="rdkit" and the methods paragraph credits it.
M18 — Cluster execution on the shared store
The delivery the blob tier was the precondition for. D43 ends with the sentence this milestone is answerable to — “workers need a shared store before they can share work” — and v0.2’s out-of-scope list is as precise about what was not delivered: “Cluster execution (the shared store is its precondition, not its delivery).” D51, D52, D53, D54, D55.
Several machines run one pipeline between them with no coordinator, no scheduler integration and no lock, because a step key is a content hash and the store is write-once: two workers that compute the same key compute the same answer. The lease that divides the work is advisory, so losing it costs CPU and never a result.
Done when: ligase run --worker[=NAME] and --lease exist with help
text, and a worker names itself in its report; two workers over one store
complete one graph between them and the union of what they computed is the
whole graph; neither computes a step the other computed; a worker whose only
ready steps are leased waits and takes them from the store rather than
duplicating them; a worker that dies holding leases cannot stall a run — an
idle worker computes those steps once the deadline passes; a worker whose
step fails relinquishes at once rather than holding the key to its deadline;
a partially complete graph is resumable without recomputing what landed,
including after N processes wrote one object concurrently; two workers
computing one key to byte-different but canonically-equal outputs agree
silently, and two computing it to canonically-different outputs do not —
the loser adopts the published entry and the report names the step and both
canonical hashes; --worker=NAME defaults its cache to
<cache>/workers/NAME and a second worker in one explicit --cache is
refused, naming the holder; a run that computed nothing can still produce a
complete methods paragraph and bundle; a methods paragraph for a run whose
records carry two platforms or two engine versions names both rather than the
first; and cluster execution needs no new dependency and no cargo feature.
Done. Every clause, each pinned by a named test, and the distributed claims demonstrated against a real S3 server rather than asserted.
Two workers divide one graph in
cluster.rs::two_workers_divide_one_graph_between_them and
cluster.rs::neither_worker_computes_what_the_other_computed, and against a
bucket in s3.rs::two_workers_against_a_bucket_divide_one_graph. Waiting is
cluster.rs::a_worker_whose_only_work_is_leased_waits_and_takes_it_from_the_store;
a dead worker is cluster.rs::a_lost_worker_cannot_stall_a_run; a failure
giving the key back is
cluster.rs::a_failed_step_is_given_back_rather_than_held_to_its_deadline,
which measures the wait as well as the outcome so that a regression is a
red test rather than a suite that never returns. Resumability is
cluster.rs::a_partly_complete_graph_completes_without_recomputing_what_landed.
The two divergence cases are
cluster.rs::two_workers_whose_bytes_differ_but_whose_meaning_agrees_say_nothing
and cluster.rs::two_workers_whose_outputs_mean_different_things_are_reported.
The cache directory is
cluster.rs::a_second_worker_in_one_cache_directory_is_refused and
cluster.rs::a_workers_cache_directory_is_its_own_by_default. Writing up a
run this machine did not do is
cluster.rs::a_run_that_computed_nothing_can_still_write_it_up, and the
plural methods sentence is
provenance.rs::the_paragraph_names_every_platform_a_run_used and
…::the_paragraph_names_every_engine_version_a_run_used. The dependency
clause is asserted rather than claimed by
default_build.rs::cluster_execution_needs_no_feature_and_no_new_dependency.
tests/test_cluster.py runs the whole thing as real separate processes
under subprocess, because that is the one thing threads in one process
cannot show: the claim is that the workers need no coordinator, and a test
sharing a process would be quietly sharing a mutex, a rayon pool and a page
cache. One capture_diagnostics.py scenario publishes the refusal a worker
with no store gets, from a real run.
What “cluster” means here, which was the first question
Not a local worker pool, which is M3. Not a scheduler client: a site’s Slurm is its own, so a submitter is either wrong at most sites or endlessly configurable at all of them, and Ligase would own job lifecycle. And it mistakes the gap — a group with a cluster knows how to start twenty of something, and what it lacks is twenty processes that can divide one pipeline without a coordinator. D51 has the argument.
Three bugs the tests found, two of them latent in M14
All three were found by tests failing rather than by reading the code, and all three were invisible for as long as every object’s name was the hash of its own contents.
published::key_of did not hash ROW_VERSION. Every row format competed
for one object address, and objects are write-once — so a bucket holding a
version 1 row would have refused every version 2 row at that key for as long
as the bucket existed, leaving that step key permanently unpublishable by
anyone. Not the one recomputation the format’s doc comment promises: all of
them, for ever. Unreachable until the first format bump, and this is the
first format bump.
FsStore::install checked for the file and then renamed, which is two
steps with a window. For an object the race is invisible, because the loser’s
bytes are the winner’s bytes. For a published row it is not — a row’s
name is the hash of the question it answers, so two writers can legitimately
hold different bytes, and “whoever wrote last wins” means two workers both
read their own lease back and both believe they hold it. Measured: two
workers divided four steps into six executions. hard_link decides it in one
syscall; the remote backend had the same hole from the other side and now
uses PutMode::Create, which is what makes D43’s “written once and never
overwritten” true of a bucket rather than merely intended.
A lease was written to and read from the local tier, so every worker saw
only its own claims, found every key free, and computed the whole graph. The
feature was silently absent and the runs were still correct — which is
D52 being true, and is exactly why a reading could not have caught it.
TieredStore::shared() exists for it, and the steps row’s read-back goes
the same way: asked of the tiered store it reads this machine’s own row and
answers “mine” every time, which is how the first implementation managed to
be blind to every divergence while looking right.
The trap in the fixtures, which is the lesson worth keeping
The first draft of tests/cluster.rs built each worker’s marker path into the
step’s shell command. A step key hashes the recipe, so the two workers had
different step keys, never contended for anything, and a test of what two
workers do about one key was testing nothing — while passing most of its
assertions.
That is the same hazard as a real pipeline whose command names $TMPDIR, and
it is the practical reason D52 has to be argued rather than assumed. The
failure mode of this whole milestone is a feature that is quietly absent and
a run that is quietly correct, which is why every one of the three bugs
above had to be found by a test and why two of them had been sitting in the
tree since M14.
M14’s unmet clause, measured again
M14’s first done-when clause — the full suite passing against a bucket with
LIGASE_STORE set and no test changed — was not met, because a handful of
tests assert that a fresh cache directory means nothing has been computed,
and a shared store is precisely what makes that false. This milestone was
warned it would meet more of them, and it did.
Against a fresh LocalStack bucket: 728 of 736 Rust tests and 390 of 396
Python tests pass unchanged, against M14’s 458 of 460 and 303 of 307. The
failures grew from two and four to eight and six while the suites grew by
276 and 89, and every one of them is the same thing: a test expecting ran
and getting cached because another test computed the same key. M14’s other
observation held too — the failing set is not fixed. Measured on a bucket
that had accumulated entries from earlier runs in the same session, the Rust
figure was 717 of 736 and the Python failure list had thirteen names rather
than six. Which of two colliding tests sees the hit depends on what ran
before it, so a deselect list would not stay true for a day.
CI runs what it ran before — tests/s3.rs and three Python files, now four
with tests/test_cluster.py — and all 65 of those pass against the bucket.
Store garbage collection, deferred with reasons
ligase store gc was deferred to v0.3 by the v0.2 spec as unsized. It does
not belong here, for three reasons that get stronger in that order.
Distribution is about who computes and gc is about what is kept, so they are
orthogonal. This milestone makes gc harder, not easier: N workers writing
concurrently, plus two new classes of garbage — a lease row per claim, and
the unreferenced loser object of every byte divergence — and a collector that
cannot tell a live claim from a dead one deletes the object a running worker
is about to publish a row for. And it is still unsized: v0.2 wanted a
measurement to size an LRU with, and this milestone is the first thing that
fills a shared store at cluster rates, so sizing gc from what M18 generates
is the right order and guessing it inside M18 is not.
The NeoRx seven-disease experiment is not here
v0.3’s roadmap names it as the validation case, “ported across”. Nothing of it
is in this repository. docs/design/neorx-expansion-notes.md is absent — the
v0.2 spec already records that and says nothing in the milestones depends on
it — and the only other mentions anywhere are a brand colour in the identity
spec and an identifier_namespace value in a guide example.
So there is nothing to port, and porting it is not an engine milestone: it needs the experiment’s own data and code from wherever they live, and it is a validation milestone of its own. This entry carries no clause for it.
M19 — What two consumers found
The first milestone the project did not choose. Two integrations pinned
ligase-bio and reported what they hit: SAWA (hiv-dr-watch), using
lg.hivdb.interpret as a second resistance backend, and NeoRx, scoring a
P. falciparum target ranking against a curated truth set. Six issues
between them, every one measured against the published 0.3.0 wheel rather
than against the tree.
They are worth taking together because they share a shape. Three were places where the engine answered a question it could not settle, and the fourth was a place where it reported a gap nobody could close — which is the same fault seen from the other side: in both cases a mechanism built to be unignorable was spending its credibility on something it had no business asserting.
The one that mattered most, and was hardest to see
identifier_namespace is the dimension that exists to catch a join across
two identifier spaces. It was inferring HGNC symbol for ChEMBL’s component
synonyms — DHFR-TS, PPPK-DHPS — and because both sides of a join
inferred the same wrong answer, the values compared equal and the join
went ahead. ligase check printed no problems found and exited 0 for
exactly the defect the axis was added for.
The reporter’s own framing is the part worth keeping:
This is the one silent failure I have found in the dimension system. Everywhere else, ligase’s answer to “I cannot tell” is to refuse; here it guesses, and the guess is load-bearing.
They offered three remedies and recommended the smallest. The one taken is
the largest: inference stops answering for the marker-less shape altogether
(D56).
Marking the value as a guess would have left a value in the field, and a
value in the field is what check compares — so the join would still pass
unless equality were also taught to tell a guess from a fact, which is two
mechanisms where one will do.
It cost more than any other change here: nine Rust tests, sixteen Python tests and four generated captures declared a space they had been getting for free. That is the migration a consumer performs, performed here first, and it is the reason this milestone also produced a release policy.
The gap that could not be closed
ligase methods named ligase as uncited. ligase is the tool of the
built-in conversions, and those take no decorator, so the advice — add one
with @lg.step(cite=...) — pointed at a step that does not exist. The
reporter had a Ligase citation on their own step and still got the warning.
A gap that cannot be closed trains a consumer to ignore it, which costs you the mechanism for the gaps that are real.
That is a complete argument and it needed no addition. The engine cites itself now (D58), and the rule it leaves behind is the general one: a gap the engine reports must name a remedy the user can apply, and where it cannot, the engine owes the answer.
Two findings about claims, which arrived in the right order
The first was a test the reporter ran because they did not trust the
identifier: adding cite= to a step moved a claim’s printed key while
nothing reran and the artifact stayed byte-identical. D45 had settled what a
claim binds to and left what it is cited by unexamined, and the write-up
was printing the one identifier that must not go in a manuscript
(D59).
Their closing line — I would have printed the key in a methods section
without this test — is the whole value of a consumer.
The second was the feature request that followed from it, and it is the one
that changes what claims are for. A claim could bind a sentence to a
computation and could not say that the number in the sentence was the number
that computation produced. at=lg.cell(...) with a declared value and
tolerance closes it, and a refuted number exits non-zero
(D61).
What is worth recording is how tightly the request was scoped by the person asking for it: only a declared location, never a search; the tolerance is the consumer’s and never defaulted; not a statistics engine. Every one of those constraints is in the implementation unchanged. A feature request that arrives with its own refusals written down is rarer than a good bug report.
The biology that had no representation
map_identifiers dropped an identifier with several targets and explained
why, well. The gap was that for bifunctional enzymes — P. falciparum
DHFR-TS is one polypeptide answering to human DHFR and TYMS — there is
no correct one-to-one row to add, so the loud, safe behaviour was also the
end of the road. In the reporting pipeline the top-ranked entity was
exactly that case.
ambiguous="fan-out" is the declared way across, and the ranked half
answered itself: rank is the row position, so a fanned ranking is not a
ranking, and fan_out_ranked_identifiers returns an EntitySet
(D60).
The reporter proposed that demotion themselves, and preferred it to tie
ranks, which is the same instinct this project’s own decisions run on.
What the milestone changed about releasing
The last issue was not a defect at all. Both consumers hit behaviour changes that shipped as minor bumps, documented inside long changelog entries, and one of them changed numbers while the run succeeded:
“The run succeeds and the result is different” is the one category a consumer cannot defend against.
So MIGRATING.md exists, and the pair between it and the changelog is
checked in pytest and again in the release workflow
(D57).
Writing it down was not the hard part; making it fail when it is not done
was, and M13’s lesson applies to process exactly as it does to code — a
declaration nothing can check is a comment.
The lesson worth keeping
Every issue in this milestone was found by someone running the published wheel against their own data, and none of them could have been found here. The dimension system’s one silent failure was in the dimension system’s own subject matter; the unclosable gap was in the mechanism built to be unignorable; the unstable identifier was in the feature whose whole purpose is to be quoted in a paper. The project’s own tests passed throughout, because they tested what the engine was built to do rather than what a consumer would do with it.
One reporter also withdrew a part of their own issue after discovering they had compared a working tree against a tag, and wrote up the mistake rather than deleting it. That is worth as much as the measurements: a bug report that corrects itself in public is what makes the rest of it trustworthy.
M20 — The same two consumers, reading the fixes
Six more issues from the two integrations of M19,
and the shape of this set is different from the last. Three of them are
about M19’s own changes: the self-citation that closed an unclosable gap
turned the remedy the old version had instructed into a duplicate reference;
the claim digest that stopped moving under cite= was labelled as the wrong
one of a step’s two digests; and the migration file that exists so a
consumer is not surprised told one of them to drop a workaround they still
need.
That is worth saying plainly: half of this milestone is the cost of the last one. Not regressions — every M19 change was right and none was reverted — but each left an edge its own reasoning had not looked at, and the people who found those edges were the people who had asked for the change.
A fix that instructs is a fix with a second half
0.3.0 said No reference is recorded for ligase. Add one with @lg.step(cite=...). A consumer did. Then the engine learned to cite itself
and their write-up carried two entries for one work, and — the part that
makes it more than untidy — no spelling of their pipeline was correct on
both versions: deleting the line fixed the new build and brought the gap
back on the pinned one.
The engine recognises its own work now and keeps its own reference (D63). The rule it leaves behind is the one the milestone is named for: when a message tells a consumer to do something, the version that stops needing it owes them the un-doing, and “we documented the remedy” is not that. A deduplication nobody has to act on is.
Two surfaces naming one result
D59 put the stable identifier in a claim’s sentence and called it output.
A step has two digests — its bytes, which name the artifact in the store, and
its canonical form, which is what audit compares and what the sentence
actually printed — and ligase audit showed the other one with no label to
tell them apart. The reporter, who had asked for D59, measured the same
irreconcilability in a new form and said so tentatively and correctly: the
canonical digest is the right one to cite; the word in front of it was wrong
(D64).
The vocabulary was coarser than the identifier
The one that is not a follow-up, and the best bug report of the two
milestones. salt-stripped does not pin a charge state; an InChIKey’s third
block does. So two mutually incompatible reference sets both carried the
honest declaration inchikey; salt-stripped, compared equal, and gave
one generated set a novelty of 0% or 50% depending on which was loaded —
with ligase check printing no problems found for both.
It is D56 one level down: there two inferred values agreed wrongly, here
two declared ones do, and no inference is involved anywhere. The report
opened by establishing that lg.chem.standardize is correct and that they
had gone looking for a bug in it and not found one, which is what made the
structural reading unavoidable.
What changed is narrower than the finding. Charge state is checked against the keys, because the keys can refute it; salt stripping and tautomer normalisation still cannot be checked and are still taken on trust. The rule is that a declaration is trusted only where there is nothing to check it against (D65), which is less of a change to D38 than it sounds: reading one character of a string is not chemistry.
The build that claimed to be a release
CARGO_PKG_VERSION is the manifest’s number, and the manifest is bumped by
a release commit, so every commit between releases reported the number of
the release it followed. That string is the engine’s half of a hivdb step’s
tool version, so it reached the cache key: if CI builds from a branch and
shares a store, two engines collide under one key. A branch build answering
differently from the published release, with a provenance record asserting
the published version
(D62).
Nothing was wrong in the tree at the moment it was reported — main was at
the release tag — and it would have come back with the next commit and gone
on coming back. A defect in the release process rather than in any commit
is one this project had no mechanism to notice, because its own tests run on
whatever build is in front of them.
And one plain bug
ligase inspect refused a molecule table keyed by SMILES — the one shape the
chemistry surface is specified to consume — and blamed a missing id column
that a molecule table has no reason to have, while lg.molecule_set loaded
the same file happily. A one-line dispatch that had been written for the kind
of table the milestone was about and never widened to the other.
The lesson worth keeping
M19’s was that a consumer finds what the project cannot. This one is about what happens next: a change made for a consumer lands in a world the old version’s messages have already shaped. Three of these six exist because a message told somebody to do something, or named a thing, or promised a workaround was over, and the change that followed did not account for having said it.
So the question to ask of a fix is not only “is this right” but “what did the last version tell people to do, and does this make them wrong?” Each of the three had an answer available for the asking, and each cost a consumer a cycle because nobody asked.