Skip to Content
Start hereQuickstart

Quickstart

Two files that disagree, inspected in one command before a line is written — then the same mistake caught inside a pipeline before anything computes, then that pipeline running, and then not running because nothing changed. No aligner, no reference genome, no downloads. Everything below is files you create here, and every output on this page came from running exactly these commands.

pip install ligase-bio mkdir ligase-quickstart && cd ligase-quickstart

1. Two files that do not agree

A tiny reference, and an alignment against it:

mkdir refs aln cat > refs/genome.fasta <<'EOF' >1 contig one ACGTACGTACGTACGTACGT >2 contig two TTTTCCCCGGGGAAAATTTT EOF
printf '@HD\tVN:1.6\tSO:coordinate @SQ\tSN:chr1\tLN:20\tM5:a965a71aa3690f605935c54d320905ab @SQ\tSN:chr2\tLN:20\tM5:89c2d6e069f386bafb1a0c811ba821f4 @RG\tID:rg0\tSM:sample01\tPL:ILLUMINA read1\t0\tchr1\t1\t60\t10M\t*\t0\t0\tACGTACGTAC\tIIIIIIIIII\tRG:Z:rg0 ' > aln/sample01.sam

Look carefully and you can see the problem: the FASTA calls its contigs 1 and 2, the SAM calls them chr1 and chr2. Same sequence, same lengths, different names. In a real project those two files come from different places months apart, and nobody looks carefully.

2. Inspect them first

Before writing anything at all, ask what Ligase makes of the two files:

ligase inspect refs/genome.fasta aln/sample01.sam
Reference refs/genome.fasta
genome.fasta.reference = 2 contigs, 40 bp (from FASTA contigs: refs/genome.fasta)
genome.fasta.contig_naming = bare (Ensembl/NCBI style) (from FASTA deflines: refs/genome.fasta)
genome.fasta.organism is unknown (a FASTA does not record its organism; declare it)
genome.fasta.release is unknown (a FASTA does not record which release it came from; declare it)
coordinates, sample_set, pairing, quality_encoding, subject, source_set, identifier_namespace and representation do not apply to a Reference.
 
Alignment aln/sample01.sam
sample01.sam.reference = 2 contigs, 40 bp (from SAM @SQ header: aln/sample01.sam)
sample01.sam.contig_naming = chr-prefixed (UCSC style) (from SAM @SQ header: aln/sample01.sam)
sample01.sam.coordinates = 1-based closed (from SAM/BAM record convention: aln/sample01.sam)
sample01.sam.sample_set = [sample01] (from SAM @RG header: aln/sample01.sam)
sample01.sam.organism is unknown (not recorded in SAM/BAM headers; declare it)
sample01.sam.pairing is unknown (recorded in the SAM flags, which reading the header alone does not reach)
quality_encoding, release, subject, source_set, identifier_namespace and representation do not apply to an Alignment.
 
ContigNamingError in step `inspect`:
genome.fasta.contig_naming = bare (Ensembl/NCBI style) (from FASTA deflines: refs/genome.fasta)
sample01.sam.contig_naming = chr-prefixed (UCSC style) (from SAM @SQ header: aln/sample01.sam)
The contigs are otherwise identical. Add a renaming step, or use one naming style throughout.
 
2 files, compared on reference and contig_naming: 1 disagreement.

That is the whole of Ligase in one command, and it needed no pipeline. Read it in three parts.

Each file gets a block headed by what it was read as. Inside it, the dimensions Ligase worked out — 2 contigs, 40 bp, chr-prefixed (UCSC style), [sample01] — each with where it read that from, which is the part that lets you go and look rather than guess.

Then the dimensions that have no value, with the reason. Those are an invitation: organism is unknown (a FASTA does not record its organism; declare it) is how you find out what declaring is for. And the last line of each block names the dimensions that do not apply at all — a FASTA is not missing a quality encoding, it has no such property to have.

Finally the comparison: the same ContigNamingError a pipeline would have given you, and a line saying which dimensions the verdict covers. It exits non-zero, so ligase inspect refs/*.fasta aln/*.bam && snakemake refuses to start a run over files that disagree.

3. A pipeline

pipeline.py
"""A two-step pipeline, with a mistake in it.""" from pathlib import Path import ligase as lg @lg.step def call(aln: lg.Alignment, ref: lg.Reference) -> lg.Variants: """Pretend to call variants. Real pipelines shell out; this one does not.""" def write(out: Path) -> None: out.write_text( "##fileformat=VCFv4.3\n" "##contig=<ID=1,length=20>\n" "#CHROM\tPOS\tID\tREF\tALT\tQUAL\tFILTER\tINFO\tFORMAT\tsample01\n" "1\t5\t.\tA\tG\t60\tPASS\t.\tGT\t1\n" ) return lg.write(write) ref = lg.reference("refs/genome.fasta", organism="test") aln = lg.alignment("aln/sample01.sam", organism="test") pipeline = lg.Pipeline(call(aln, ref))

Three things are doing work here.

The type annotations are not decoration. aln: lg.Alignment and ref: lg.Reference are what let the engine know what each argument is and which dimensions apply to it, and -> lg.Variants is how it knows what comes out. The graph is built from these signatures.

lg.write hands the engine a function to call with the path to write to, so this step needs no external tool. A real step would use lg.sh and name the tool it runs.

lg.Pipeline(...) builds the graph. It does not run anything — calling call(aln, ref) records a call rather than performing one.

4. Check it

ligase check
ContigNamingErrorin stepcallnothing ran
aln.contig_naming=chr-prefixed (UCSC style)from SAM @SQ header: aln/sample01.sam
ref.contig_naming=bare (Ensembl/NCBI style)from FASTA deflines: refs/genome.fasta
The contigs are otherwise identical. Add a renaming step, or use one naming style throughout.

The same mismatch as in step 2, and that is the point: inspect compared two files you named, while check compared the inputs the pipeline itself wires together — every connection, every step, every sample, under the argument names your step declares. On a pipeline over four hundred samples that is a different question with the same answer, and the one run refuses to proceed past.

It took milliseconds, and the step body never ran. Without the check, this pipeline would have succeeded: tools match contigs by name, so the variant caller would have found no overlap between the alignment and the reference, written a valid empty VCF, and exited zero.

5. Fix it

Rename the contigs in the SAM to match the reference:

sed -i.bak 's/SN:chr/SN:/; s/\tchr1\t/\t1\t/' aln/sample01.sam ligase check
checked 1 step in pipeline.py
 
no problems found.

6. Run it

ligase run
running 1 step in pipeline.py
 
rancall0ms
 
1 step: 1 ran.

Now run it again, without changing anything:

running 1 step in pipeline.py
 
cachedcall
 
1 step: 1 from cache.

Nothing recomputed. A step’s key is built from its code, its parameters, its inputs’ contents and its tools’ versions — and none of those moved. Edit the step body, or a byte of the input, and it runs again.

7. Say how it was produced

ligase methods
writing up 1 step in pipeline.py
 
Analyses were run with Ligase 0.4.0+14.g6c6a8b4 on linux-x86_64.
 
`call` ran on one input, by a function in the pipeline itself, against the
reference (fingerprint aa185c059780, 2 sequences, 40 bp).
 
References
 
Njeri K. Ligase: dimension-checked, reproducible computational biology
pipelines (version 0.4.0+14.g6c6a8b4).
https://github.com/Neoforge-Labs/ligase

On a real pipeline that paragraph names every tool at the version it actually ran at, every parameter as it was hashed, and every reference by accession and checksum. --bundle packages it so somebody else can check the results.

Last updated on