Binding a number you published

Your manuscript states numbers. Your pipeline produces them. Nothing has ever connected the two, so when the pipeline moves the manuscript does not follow. This page is how you connect them, and how CI catches it when they part company.

Declare the claim beside the pipeline

A claim goes on the Pipeline, next to the outputs it is about:

import ligase as lg
from ligase.types import Alignment, Reference
 
hxb2 = lg.reference("refs/HXB2.fasta", organism="HIV-1", build="HXB2")
 
 
@lg.step
def call(ref: Reference, min_freq: float = 0.01) -> Alignment:
    """Call variants above a minimum allele frequency."""
    return lg.sh("cat {ref} > {out} && echo {min_freq} > /dev/null")
 
 
pipe = lg.Pipeline(
    call(hxb2, min_freq=0.01),
    claims=[
        lg.Claim(
            "variants were called above 1% allele frequency",
            step="call",
            parameter="min_freq",
            value=0.01,
        ),
    ],
)

On the pipeline rather than passed to methods(), for one practical reason: ligase methods loads your pipeline file and finds the Pipeline objects in it, so a claim declared here needs no new flag, no claims file and no second format to keep in step.

Run it:

ligase methods

The claim is bound: call is a step this pipeline produces, min_freq is a parameter it takes, and the run held it at 0.01.

The identifier the sentence gives is the canonical digest — the digest of the canonical form of what the step produced, which is what the sentence calls it. That is the one to put in a manuscript, and the reason is worth a paragraph of its own.

Then the threshold moves

Six months later the threshold is 5%. Nothing fails — the pipeline is correct, the output is correct, and the sentence in the paper is now wrong:

pipe = lg.Pipeline(
    call(hxb2, min_freq=0.05),
    claims=[
        lg.Claim(
            "variants were called above 1% allele frequency",
            step="call",
            parameter="min_freq",
            value=0.01,
        ),
    ],
)

Both numbers are stated, and ligase methods exits non-zero.

Three shapes of claim

A number you can restate. A threshold, a window, a cutoff — anything that is a parameter of a step. The engine compares it:

lg.Claim(
    "variants were called above 1% allele frequency",
    step="call",
    parameter="min_freq",
    value=0.01,
)

A tool version. The sentence in every methods section that goes stale when a cluster is upgraded:

lg.Claim(
    "reads were aligned with minimap2 2.31",
    step="align",
    tool="minimap2",
    value="2.31",
)

The comparison is textual and allows a build suffix, so 2.31 matches the 2.31-r1287 that minimap2 --version prints — but 2.4 does not match 2.41, because a prefix has to end at a component boundary. A version that is nearly right is reported as wrong.

A number the engine cannot read. A count of mutations, a median coverage, a figure from a plot. There is no parameter to compare, so give the step alone:

lg.Claim("three resistance mutations were found in SRR8525904",
         step="resistance", scope="SRR8525904")

This is still a real binding: the claim carries that sample’s canonical digest, key and date, so a reviewer with your bundle can find the computation and rerun it. What it is not is a verification of the number three — the engine does not open your report and count. Nothing on this page pretends it does.

Naming one sample

scope= says which sample a claim is about:

lg.Claim("SRR8525904 carried M184V", step="resistance", scope="SRR8525904")

Without it the claim is about the step across every sample it ran on, which is right for “resistance was interpreted for every sample” and wrong for a count from one of them. There is no inference either way — scope is how you say which you mean.

When a claim has nothing behind it

The analysis was rewritten and the sentence stayed:

The complaint names what the pipeline does produce, so a renamed step is a one-line fix. Five things can put a claim here, and each says which:

Reading the claims from a script

methods = pipe.methods()
 
for check in methods.claims:
    print(check.verdict, check.claim.text)
    for bound in check.bound:
        print("   ", bound.key, bound.recorded_at)
 
if not methods.accounted_for:
    for check in methods.unbacked:
        print(check.reason)

accounted_for covers claims as well as gaps: a write-up that states a number the analysis does not support has a hole in it, whatever else is complete. check.reason is the sentence ligase methods prints, so a script and the command line say the same thing.

Editing a claim reruns nothing

A claim is read when the pipeline is written up and at no other time. It is in no cache key:

ligase methods          # the write-up, and every step from cache

Edit the sentence, add a claim, remove one — the steps are still cached, because nothing about the computation changed (D45). For the same reason a claim can never make check() fail: a pipeline whose write-up names a renamed step is not a pipeline that must refuse to run.

Claims in the bundle

--bundle writes them as claims.tsv, one row per computation, with both identifiers in full rather than the twelve characters a sentence prints:

ligase methods --bundle paper/bundle

$ cat paper/bundle/claims.tsv
claim	verdict	step	scope	key	canonical
variants were called above 1% allele frequency	bound	call		206c6ae6…	a41f…

An unbacked claim is in that table too, with an empty key, rather than left out of it — which is the same promise the methods paragraph makes about a step it cannot account for.

Verifying the number itself

Everything above binds a sentence to a computation. None of it says that 0.333 is the number that computation produced — and that is the gap that makes published numbers wrong in practice: the pipeline is rerun after the table is pasted into the manuscript, and nothing notices.

Tell the claim where its number lives and the engine will read it:

lg.Claim(
    "malaria F1 = 0.333 against seven curated targets",
    step="validate",
    value=0.333,
    at=lg.cell(column="f1", row=0),
    tolerance=5e-4,
)

The verdict is verified, which is bound and more: the number in the sentence is the number in the bytes. The write-up gives the number it read and the tolerance it was allowed, so a reader can check the comparison as well as the claim.

Rerun the analysis, leave the manuscript alone, and:

ligase methods exits non-zero, so CI fails when a manuscript’s number stops matching its artifact.

The three parts are one declaration

value, at and tolerance are refused in any incomplete combination, where they are written:

>>> lg.Claim("F1 = 0.333", step="validate", value=0.333, at=lg.cell(column="f1"))
ValueError: the claim 'F1 = 0.333' names a cell of the output and no
`tolerance=`. State the precision the paper states: a claim to three
decimals is `tolerance=5e-4`, and defaulting it would invent a precision
nobody wrote

The tolerance is yours and is never defaulted — a claim to three decimals and a claim to machine precision are different claims. And at beside parameter= or tool= is two claims in one, so it is refused too.

What a wrong location does

It reports that it could not check, not that the claim is wrong — and names what was there instead:

"malaria F1 = 0.333" has nothing behind it: `validate` is in this pipeline
and its output has no column named `f_one`; its columns are precision, f1.
Point the claim at a cell that holds the number, or drop `at` and let the
claim be bound without being verified.

A column that is not there says nothing about whether the sentence is true, so the verdict is unbacked rather than contradicted (D61).

What this does not do

• It does not go looking for a number it was not pointed at, and it parses one format: a TSV with a header. A claim that cannot say where its number is stays step-only, which is a legitimate state.
• It does not recompute anything, or check that an F1 is a well-formed F1. Just: the number you published is the number in the bytes.
• It does not know your sentence has gone stale. It knows the value you restated has, which is why parameter= and tool= are worth giving whenever there is one to give.
• It does not add a verdict to ligase methods --verify. A claim names a canonical digest, and whether that result still reproduces on another machine is what results.tsv’s verdict already says.

See also

• A number in a paper — why this exists.
• The number in my paper is not the number my pipeline produces — the failure, by symptom.
• Writing up a pipeline — the methods paragraph a claim is printed beside.