Skip to Content
GuidesDeclaring a stochastic step

Declaring a stochastic step

Your tool samples: a tree search, a chain, a docking run, a subsample, a training run. Two keywords make that a fact the engine knows, and the difference shows up in what ligase audit and ligase methods are then allowed to say.

@lg.step(tool="seqtk", stochastic=True, seed="seed") def subsample(reads: Reads, fraction: float = 0.1, seed: int = 20260101) -> Reads: return lg.sh("seqtk sample -s{seed} {reads.r1} {fraction} > {out}")
  • stochastic=True — there is randomness in this tool.
  • seed="seed" — and this parameter is what holds it still.

Both or neither. Everything else on this page follows from those two lines.

Before you start: nothing. The refusals below happen as a pipeline file is imported, so you can try them in a REPL.

The two things to get right

Name a parameter, not a value

seed= takes the name of one of the step’s own parameters. The engine is not being handed a seed to look after; it is being told which parameter already is one.

@lg.step(tool="iqtree", stochastic=True, seed="rng_seed") # the name def tree(aln: Alignment, rng_seed: int = 42) -> Intervals: ...

Type it int

@lg.step(stochastic=True, seed="seed") def sampled(reads: Reads, seed: str = "42") -> Reads: # refused ...
diagnosticraised by @lg.step, as the pipeline is imported
step `sampled` names `seed` as its seed, but `seed` is str rather than `int`. A seed has to be a value the engine can hash into the cache key and a methods section can print, which is what makes two runs of a stochastic step comparable at all

A float seed, a timestamp or a None is a seed somebody will set differently by accident. The annotation is the cheapest place to say so.

What happens when you forget

The refusal is raised by the decorator, so it arrives when the pipeline file is read — not when the step runs, and not as a warning:

diagnosticraised by @lg.step, as the pipeline is imported
step `tree` declares itself stochastic but names no seed, so nothing says what makes it repeat. Add `seed=` naming the parameter the tool draws its randomness from. It takes `seed: int`, so `seed="seed"` is probably what you meant.

It suggests a parameter only when it can see one actually called a seed. Given a step that takes iterations: int, it lists what it found and leaves the choice to you, because seed="iterations" would be a confident wrong answer.

The other direction is refused too. A seed= with no stochastic=True is not read as a shorthand:

@lg.step(seed="seed") # refused def sampled(reads: Reads, seed: int = 42) -> Reads: ...
diagnosticraised by @lg.step, as the pipeline is imported
step `sampled` names `seed` as its seed but does not declare itself stochastic, so the engine has been told how the step repeats without being told that it needs to. Add `stochastic=True` beside it, or drop `seed=` if the tool draws on no randomness

One declaration, one spelling.

Auditing it

ligase audit runs the step twice and compares both the bytes and the meaning. For a correctly seeded step the bytes match — and the verdict is still reproducible:

ligase audit, on a declared stochastic step
auditing `permute` in pipeline.py
 
reproducible `permute`
the same bytes both times 6c636bdb27d9
canonical form c14286600727
because `seed` was held at 20260101, not because the step cannot wobble
 
1 step: 1 reproducible.

The second line is the whole of the declaration’s value. Without it, the same run reports this:

the same body, with nothing declared
auditing `permute` in pipeline.py
 
deterministic `permute`
the same bytes both times 6c636bdb27d9
canonical form c14286600727
 
1 step: 1 deterministic.

Identical hash, different verdict — because deterministic is a claim about the step, and said of a step that samples it is false. See why.

When the seed does not hold

If the tool ignores the flag you named — renamed between releases, or a library reading the global generator — the audit says so, and says which parameter you were wrong about:

a seed the tool does not draw on
auditing `permute` in pipeline.py
 
not reproducible `permute`
the two runs mean different things bfdb8834a6bb / 2486f7101d9d
`seed` was held at 20260101 and the result moved anyway: the seed does not control this tool
 
1 step: 1 not reproducible.

What to do, in order:

  1. Check the flag. -seed, --seed, -s, --random_seed: they are all in use, and a tool that does not recognise yours usually says nothing.
  2. Check the threads. Many tools are deterministic per seed only at a fixed thread count, because a parallel reduction adds in whatever order it finishes. If so, the thread count is a parameter too.
  3. Check for a second source. A step that shells out twice, or calls a library that seeds itself, has randomness your parameter never reaches.

Until it is fixed the step is honestly reported as not reproducible, which is better than a deterministic that would have been luck.

From Python

report = pipe.audit("subsample") for step in report.steps: if step.seed is not None: parameter, value = step.seed print(f"{step.step}: {step.verdict}, with {parameter} = {value}")

step.seed is None for a step that declared none, and step.verdict is never "deterministic" for one that did.

Writing it up

ligase methods names the seed as a seed:

ligase methods
writing up 1 step in pipeline.py
 
Analyses were run with Ligase 0.4.0+14.g6c6a8b4 on linux-x86_64.
 
`permute` ran on one input, by a function in the pipeline itself, seeded
with seed = 20260101.
 
References
 
Njeri K. Ligase: dimension-checked, reproducible computational biology
pipelines (version 0.4.0+14.g6c6a8b4).
https://github.com/Neoforge-Labs/ligase

Seeded with rather than with, because a reader of with seed = 20260101 cannot tell whether that number is a sampling seed or a cutoff — and the one thing a methods section owes a stochastic analysis is the number to copy.

Changing the seed

A seed is an ordinary typed parameter, so it is in the cache key:

lg.Pipeline(subsample(reads, seed=1)).run() # ran lg.Pipeline(subsample(reads, seed=1)).run() # from cache lg.Pipeline(subsample(reads, seed=2)).run() # ran

Two seeds are two results, so they are two keys. Nothing was arranged to make that so; it falls out of the seed being a parameter, which is why it has to be one.

Next

Last updated on