Skip to content

SDRF experimental design

Every other reader in pyMzLib answers what did the search find. SDRF-Proteomics answers what was searched: which sample, which organism part, which replicate, which instrument settings. That is the half you need before results from two experiments can be compared at all.

>>> import pymzlib
>>> doc = pymzlib.sdrf.read("PXD000070.sdrf.tsv")
>>> doc.row_count, len(doc.columns)
(6, 31)
>>> doc.value("characteristics[organism part]")[0]
'human erythrocytes'

Pool several experiments into one table, giving each a name you choose:

>>> pooled = pymzlib.sdrf.pool(
...     {"PXD000070.sdrf.tsv": "malaria", "PXD026824.sdrf.tsv": "colon"}, limit=4
... )
>>> pooled.document_count, pooled.row_count, pooled.labels
(2, 24, ['malaria', 'colon'])

Every >>> example on this page runs in CI against output recorded from the real bridge, on SDRF files from mzLib's and pyMzLib's test data. Blocks titled Not run say why.

What you can ask of an SDRF

Each question is answered by one mzLib type, projected once in the bridge so that Python, Rust and R get the same answer:

You want to know Call mzLib answers with
What does the file say, cell for cell? read() SdrfDocument
Several experiments in one table pool() SdrfCollection.Merge
Is this file well-formed? validate() SdrfValidator
Does it describe its samples at all? assess() SdrfSampleInformativeness
Do these files write the same thing the same way? lint() SdrfDriftLint
What does it say about each sample? samples() SdrfSampleBlock
How old was each sample, in years? parse_ages() SdrfAge
Can this file drive a quantification, and with what design? design() SdrfLabelFreeDesign (#1363)

validate, assess and samples also come as *_many, which reads a whole corpus in one call.

The three judging functions are blind in different places, which is why there are three. A file whose sample columns are all "not available" passes validate(), because reserved words are the specification's correct way to say nothing, and passes lint(), because a reserved word cannot drift. Only assess() sees that it answers no biological question. Twenty files can each pass validate() and still be unpoolable; only lint() looks across files. Every result carries caveats naming the blind spot it leaves.

This module is row-major, and every other reader is columnar

read_records() and its siblings hand back columns, a name-to-values dict. This one hands back columns (a list of names) and rows (a list of cell lists). That is not a style choice; the format forces it.

SDRF column names are data, not a schema, and they repeat. In the curated corpus of bigbio/sdrf-annotated-datasets, about half the files carry comment[modification parameters] more than once, up to eight times in a single file, and one file repeats an empty name 23 times. A dict keyed by name would keep one occurrence of each and silently drop the rest. PXD000070 is one of them:

>>> doc.has_repeated_columns
True
>>> len(doc.indexes_of("comment[modification parameters]"))     # positions in the header
8
>>> doc.value("comment[modification parameters]")[0]            # the FIRST one, in row 0
'NT=Carbamidomethyl;AC=UNIMOD:4;TA=C;MT=Fixed'
>>> len(doc.all("comment[modification parameters]")[0])         # ALL of them, in row 0
8

So position is what links a name to a cell. records gives you the convenient dict-per-row shape, but it is lossy when a name repeats: later positions overwrite earlier ones. has_repeated_columns tells you whether that applies before you trust it.

Three things to know before you index anything

Rows are ragged

len(row) may be less than len(columns). Real files are like this, and mzLib preserves it rather than padding, so the file round-trips byte for byte; pyMzLib preserves it in turn. PXD059974, in mzLib's own fixtures, has a 46-column header and 17 of its 22 rows carry only 42 cells:

>>> r = pymzlib.sdrf.read("PXD059974.sdrf.tsv")
>>> r.row_count, len(r.columns), r.ragged_row_count
(22, 46, 17)
>>> next(c for c in r.caveats if "SHORT" in c)[:60]
'17 of 22 rows are SHORT: they carry fewer cells than column_'

value() returns None for a position a row does not reach. ragged_row_count counts the short rows you were given, and the caveats list names the count for the whole document even when you page with limit and offset.

Cells are raw strings, never interpreted

The SDRF key=value grammar arrives exactly as written:

>>> doc.value("comment[modification parameters]")[0]
'NT=Carbamidomethyl;AC=UNIMOD:4;TA=C;MT=Fixed'

It is not decoded, and that is deliberate rather than unfinished. A cell containing = and ; cannot be told apart from a CV term by its shape: comment[file uri] cells in the corpus carry pre-signed download URLs whose query strings contain Signature=, Expires= and Id=. Anything that decoded "cells that look like CV terms" would decode those.

Why this matters more than it sounds

readers.read_records() will also open a .sdrf.tsv, because mzLib dispatches the type. It gives you two columns, header and cells, each semicolon-joined, and since SDRF's own grammar is semicolon-delimited, that string cannot be split back into cells. Use pymzlib.sdrf.read() for SDRF; the joined form is not recoverable.

A reserved word is a real value

"not available" and "not applicable" mean the experiment stated an absence. That is the specification's own mechanism, and a curator chose to write it. It is not the same as a column the document does not have, which comes back as None. Do not collapse the two:

>>> doc.value("characteristics[disease]")[0]              # stated
'not applicable'
>>> print(doc.value("characteristics[nonesuch]")[0])      # no such column
None

Pooling

pool() merges documents into one analysis table. Columns are the union of every document's, ordered by SDRF's own block structure (sample metadata, then data-file metadata, then factor values), and a name that repeats is carried at the highest multiplicity any single document used, so nothing is dropped. A cell a document did not have is filled with "not available".

Give your documents names

The labels you give come back on every row, in comment[source document]. The four rows returned here are the first document's:

>>> pooled.source_documents()
['malaria', 'malaria', 'malaria', 'malaria']

Pass a plain list instead and mzLib falls back to containing-folder/file-stem, which depends on where the files happen to sit, so the same two documents pooled from a different directory produce a different table. caveats says so when that happened. Label every document or none; a partial set is refused, because a provenance column that means a chosen name on some rows and a local path on others cannot be undone downstream.

The pooled table is for analysis, not deposit

source name + assay name + comment[label] is unique within one document, but two experiments may both have a "Sample 1". A pooled table will therefore usually violate SDRF's uniqueness rule. Use comment[source document] as part of any key:

Not run: needs pandas, which pyMzLib does not depend on
import pandas as pd
df = pd.DataFrame(pooled.records)
df.groupby(["comment[source document]", "characteristics[organism part]"]).size()

limit and offset shape what comes back over the wire, never the table itself. The pool above asked for four rows and says so:

>>> pooled.returned_count, pooled.truncated
(4, True)

Writing it back out gives a well-formed SDRF file, and limit never shortens what lands on disk:

Not run: writes a file; the pool above was recorded without out=
pymzlib.sdrf.pool(docs, out="merged.sdrf.tsv", limit=5)   # 5 rows returned, all 24 written

Validate a deposit

Before you upload an SDRF to PRIDE, or trust one you downloaded, check its structure:

>>> v = pymzlib.sdrf.validate("sdrf_skeleton.sdrf.tsv")
>>> v.is_valid, v.error_count, v.warning_count
(False, 7, 2)
>>> for m in v.errors[:2]:
...     print(m.line_number, m.rule, m.column_name)
None RequiredColumn technology type
None RequiredColumn comment[instrument]

A file can be valid and still carry warnings:

>>> ok = pymzlib.sdrf.validate("sdrf_cohort.sdrf.tsv")
>>> ok.is_valid, ok.error_count, ok.warning_count
(True, 0, 2)
>>> [(m.line_number, m.rule) for m in ok.warnings]
[(10, 'ReservedWordCase'), (11, 'ReservedWordCase')]

This is mzLib's SdrfValidator, checking structure against SDRF-Proteomics v1.1.0: the required and recommended columns, column order and casing, malformed names (characteristics [organism] with a space), rows whose width differs from the header, replicate and fraction columns that are not positive integers, reserved-word casing, and the one hard row rule: source name + assay name + comment[label] must be unique.

Errors and warnings mean different things, and is_valid looks only at errors. An error means the file cannot be reliably consumed: a ragged row reads every later cell under the wrong column, a missing comment[data file] leaves nothing to join on. A warning is worth fixing and never worth refusing. mzLib calibrated every severity against the curated files of bigbio/sdrf-annotated-datasets: a rule that fired on most curated files was judged a wrong rule, not a corpus of wrong files.

Where a finding points. line_number is the line in the file, counting the header as line 1, so you can jump straight to it in an editor. row_index is the same place as a 0-based index into read(...).rows. Both are None for a finding about the whole document, such as a missing column or an empty header, never 0, which would name a real row. column_name is None when the finding is not about one column.

The findings are a table, so the usual summaries are one line:

>>> from collections import Counter
>>> Counter(zip(v.columns["severity"], v.columns["rule"]))
Counter({('Error', 'RequiredColumn'): 7, ('Warning', 'RecommendedColumn'): 2})

What a clean result does not mean

Controlled-vocabulary terms are not resolved. AC=MS:1003028;NT=Q Exactive is well-formed and passes, even though MS:1003028 is the Orbitrap Exploris 480. And a file that says nothing, with every sample cell not available, validates cleanly. assess() is the check for that.

Decide whether an SDRF is worth using

A file generated from a list of data files, with reserved words in every sample column, one replicate number and no factor, is valid SDRF. For grouping results by biology it is the same as having no SDRF at all. assess() is the gate:

>>> a = pymzlib.sdrf.assess("PXD000070.sdrf.tsv")
>>> a.verdict
'Partial'
>>> a.factor_value_varies, a.sample_is_described, a.biological_replicate_varies
(False, True, True)

mzLib's SdrfSampleInformativeness asks three questions of the sample half of the file:

Check Passes when
factor_value_varies some factor value[...] column holds two or more different real answers: the file says what the study varied
sample_is_described some characteristics[...] column other than organism and biological replicate holds a real answer. Organism is left out because a search can fill it without a human ever describing the samples
biological_replicate_varies characteristics[biological replicate] holds two or more different real answers
Verdict Means
Informative all three pass: the file can drive an experimental design
Partial some pass. Often legitimate: a single-condition study has nothing to vary, so whether it is good enough is your call. The three booleans say which check failed
Skeleton none pass

"A real answer" means neither empty nor a reserved word, and different answers are compared ignoring case and surrounding space, so liver and Liver are one value. The evidence is in columns: one row per column each check read, with rows, filled, absent, distinct_values and fill_rate (a fraction, 0 to 1). Here is why a skeleton is a skeleton:

>>> sk = pymzlib.sdrf.assess("sdrf_skeleton.sdrf.tsv")
>>> sk.verdict
'Skeleton'
>>> for row in sk.records:
...     print(row["column_name"], row["filled"], "of", row["rows"], "filled")
characteristics[disease] 0 of 2 filled
characteristics[organism part] 0 of 2 filled
characteristics[biological replicate] 2 of 2 filled

To gate a corpus, assess it in one call and keep what passes:

>>> import os
>>> batch = pymzlib.sdrf.assess_many(
...     ["sdrf_cohort.sdrf.tsv", "sdrf_skeleton.sdrf.tsv", "PXD000070.sdrf.tsv"]
... )
>>> batch.verdict_counts
{'informative': 1, 'partial': 1, 'skeleton': 1}
>>> [os.path.basename(p) for p in batch.paths_with("Informative", "Partial")]
['sdrf_cohort.sdrf.tsv', 'PXD000070.sdrf.tsv']

paths_with() refuses a verdict it does not know: "informative" in lower case would otherwise select nothing, silently.

Lint a pooled set

Validity is a property of one file; comparability is a property of the relationship between files. Before you pool() experiments and group by a column, check that they wrote it the same way:

>>> drift = pymzlib.sdrf.lint(
...     {"sdrf_cohort.sdrf.tsv": "cohort", "sdrf_cohort_partner.sdrf.tsv": "partner"}
... )
>>> drift.finding_count
4
>>> for row in drift.records:
...     print(row["finding_index"], row["kind"], row["value"], row["documents"])
0 AccessionNameConflict Exploris 480 ['partner']
0 AccessionNameConflict Orbitrap Exploris 480 ['cohort']
1 MixedTermAndFreeText controlled vocabulary term ['cohort']
1 MixedTermAndFreeText free text ['partner']
2 ColumnNameVariant Characteristics[sex] ['partner']
2 ColumnNameVariant characteristics[sex] ['cohort']
3 ValueCaseVariant Homo sapiens ['cohort']
3 ValueCaseVariant homo sapiens ['partner']

The table is one row per finding × variant: finding 3 is one concept written two ways, so it is two rows sharing a finding_index, the majority spelling first (variant_rank 0). documents lists the labels of the files that used each spelling, which is why you should label them. drift.findings() gives the same rows grouped.

kind What was inconsistent Why it matters
NameAccessionConflict one name, several accessions the serious one: a join on the accession splits the concept, a join on the name merges two
AccessionNameConflict one accession, several names harmless to a join on the accession; the earliest sign two files were annotated by different hands
MixedTermAndFreeText a CV term in some files, free text in others the two never compare equal, so half the corpus drops out of any query
ColumnNameVariant column names differing only by case or spacing they are different columns to every consumer. column_name is None: the finding is about a name
ValueCaseVariant free-text values differing only by case or spacing equality grouping treats Homo sapiens and homo sapiens as two populations

The majority is a description, not advice

variant_rank 0 is what most of these files did. In the curated corpus, characteristics[organism] is free text in most documents and a CV term in the rest, so following the majority would throw the accession away. Where variants are merely spellings of one thing, match the majority; where they differ in kind, keep the richer one.

Columns whose values are unique per row are never compared: source name, assay name, comment[data file], comment[searched data file] (mzLib#1335), comment[file uri] and comment[source document]. Two studies naming different raw files is data, not drift. Reserved words and empty cells are skipped too, so a set of files that say nothing lints clean.

One row per sample

An SDRF has one row per data file; a sample measured in eight fractions is eight rows. samples() turns that back into samples, with mzLib's SdrfSampleBlock, keyed on source name:

>>> s = pymzlib.sdrf.samples("sdrf_cohort.sdrf.tsv")
>>> s.sample_count, s.conflicts()
(6, [('S6', 'characteristics[disease]')])

The table is long, one row per sample × column × position, so repeated columns and ragged files cost nothing:

column meaning
source_name the sample, as its first row spelled it. Matched ignoring case and surrounding space, as mzLib keys it
sample_row_count how many rows (runs, fractions) carry the sample
column_name the header's own spelling: characteristics[Age] stays itself
column_kind source_name, characteristic or factor_value
position 0, 1, … for the first, second, … occurrence of a repeated column
status agreed, or conflicting
value the cell, verbatim, reserved words included
age_* on characteristics[age] rows only: the parsed age

A conflicting column is withheld, not guessed. When a sample's rows disagree (S6's first fraction says COVID-19 and its second says normal), copying either through would be a "first row wins" rule, which is how a sample silently acquires the wrong disease. mzLib withholds the column and names it; it appears here once, with status == "conflicting" and value None. That is not the same as a column the file lacks, which has no row at all.

Across files a source name is not unique, since two studies may both have Sample 1, so key samples on (source_index, source_name) in a samples_many() result.

Parse ages safely

characteristics[age] is free text in practice. mzLib's SdrfAge reads it into years and, more importantly, refuses what it cannot read without guessing:

>>> ages = pymzlib.sdrf.parse_ages(["58Y", "30Y6M", "40Y-85Y", "40Y-40Y", ">=90Y", "<1Y",
...                                 "6-8 weeks", "63", "not available", "", "about forty"])
>>> for row in ages.records:
...     years = None if row["years"] is None else round(row["years"], 2)
...     print(f"{row['cell']!r:15} {years!s:6} {row['precision']!s:11} {row['refusal']}")
'58Y'           58     Exact       None
'30Y6M'         30.5   Exact       None
'40Y-85Y'       62.5   Range       None
'40Y-40Y'       40     Exact       None
'>=90Y'         90     LowerBound  None
'<1Y'           1      UpperBound  None
'6-8 weeks'     0.13   Range       None
'63'            None   None        no_unit
'not available' None   None        reserved_word
''              None   None        empty
'about forty'   None   None        unreadable

Units are years: a month is 1/12 year, a week 7/365.25, a day 1/365.25, an hour 1/(365.25 × 24). years is the single figure for an age axis (the age, a range's midpoint, or a bound), so for a wide cohort range it is a poor stand-in for any one sample. Filter on precision before you plot:

precision Example years min_years max_years
Exact 58Y, 30Y6M, and 40Y-40Y (a degenerate range, mzLib#1333) the age the age the age
Range 40Y-85Y, 6-8 weeks the midpoint the low end the high end
LowerBound >=90Y, the usual de-identification cap the bound the bound None = unbounded
UpperBound <1Y the bound 0 the bound

max_years is None for two different reasons, and precision tells them apart. For a lower bound, mzLib's maximum is +infinity, which JSON cannot carry; for a refused cell there is no age at all. follows_specification is False for unambiguous words (3 year, 6-8 weeks): read, but not the specification's nYnMnD grammar.

A refused cell has no age. Never fill it with a guess. The reason is in refusal:

refusal The cell What to do
no_unit a bare number, 63; about one real age cell in ten ask the curator: 63 years and 63 days are both plausible in one study
reserved_word not available, anonymized, … in any case the experiment stated there is no age; respect it
empty blank, or None in your list the cell is missing
unreadable free text, about forty ask the curator

Worked example: ages across a corpus

samples_many() parses every characteristics[age] cell for you, on the same row as the sample it belongs to. From there it is ordinary pandas:

Not run: needs pandas and a corpus of SDRF files
import pandas as pd
import pymzlib

batch = pymzlib.sdrf.samples_many(paths, threads=-1, on_error="skip")
df = pd.DataFrame(batch.columns)

ages = df[(df.column_name.str.lower() == "characteristics[age]") & (df.status == "agreed")]
point = ages[ages.age_precision == "Exact"]            # one age per sample: safe to plot
capped = ages[ages.age_precision == "LowerBound"]      # ">=90Y": at least 90, no upper bound
refused = ages[ages.age_refusal.notna()]               # no age: report, do not impute

print(point.groupby("source_path").age_years.describe())
print(refused.groupby("age_refusal").size())

pandas turns None into NaN

In a float column pandas stores None as NaN, so age_max_years cannot tell you why it is missing once it is in a DataFrame. Always decide with age_precision and age_refusal, which are strings and keep their meaning; never with age_max_years.isna().

For ages that did not come from samples(), such as a column of read() or PRIDE project metadata, pass the cells to parse_ages() directly. It takes any number of cells in one bridge call, keeps blank and None cells in place, and returns row i for cell i:

Not run: the parse_ages recording above is for a different list of cells
doc = pymzlib.sdrf.read("PXD026824.sdrf.tsv")
ages = pymzlib.sdrf.parse_ages(doc.value("characteristics[age]"))

Turn an SDRF into a quantification design

The problem. FlashLFQ and MetaMorpheus need an experimental design: which runs are the same condition, which are biological replicates of one another, which are fractions of one sample. The SDRF already says all of that, but a hand-written ExperimentalDesign.tsv is how most people still give it, and an invalid one is worse than none: MetaMorpheus skips quantification with a single warning and no error.

design() reads the design out of the SDRF, or refuses and says why. It calls mzLib's SdrfLabelFreeDesign, which runs every check MetaMorpheus's own design validator runs and reports all of the failures at once, so you fix the file once rather than once per failure:

You want Call mzLib
The design, or every reason there is none design(path, condition_columns=...) SdrfLabelFreeDesign.Read
It restricted to the runs one search read design(..., searched_files=[...]) SdrfLabelFreeDesignOptions.SearchedFiles
It as FlashLFQ's spectra= list result.spectra() SdrfLabelFreeDesign.Files
It as median_polish(design=...) result.run_design() SdrfLabelFreeDesign.Files
MetaMorpheus's ExperimentalDesign.tsv design(..., out="ExperimentalDesign.tsv") WriteExperimentalDesignTsv

The examples in this section use mzLib's own design files: PXD067622 (a TurboID proximity-labelling study, two genotypes by four treatments, three biological replicates each) and PXD049018 (two pulldowns, ten SDS-PAGE bands each).

A valid design

PXD067622 has two factor columns. Say which ones make up the condition; their values are joined with _, in the order you give:

>>> both = ["factor value[genotype]", "factor value[treatment]"]
>>> d = pymzlib.sdrf.design("PXD067622.sdrf.tsv", condition_columns=both)
>>> d.is_valid, d.file_count, len(set(d.columns["condition"]))
(True, 24, 8)
>>> d.file_key_column
'comment[data file]'
>>> for run in d.files[:3]:
...     print(run.file_name, "|", run.condition, "|", run.biological_replicate)
20240830_HF_LC3_MAA_RK_12032_CA_DMSO4 | SPRTN-TurboID CA_DMSO (vehicle) | 0
20240830_HF_LC3_MAA_RK_12032_CA_DMSO5 | SPRTN-TurboID CA_DMSO (vehicle) | 1
20240830_HF_LC3_MAA_RK_12032_CA_DMSO6 | SPRTN-TurboID CA_DMSO (vehicle) | 2

The design is 0-based; ExperimentalDesign.tsv is 1-based

biological_replicate, technical_replicate and fraction are mzLib's SpectraFileInfo coordinates, which start at 0: exactly what pymzlib.flashlfq.quantify and median_polish take. out= writes MetaMorpheus's file, which starts at 1; mzLib adds the one when it writes, and nowhere else. Never add it yourself.

Hand the design straight to FlashLFQ. spectra() gives one mapping per run in the shape quantify(psms, spectra=...) takes:

>>> d.spectra()[0]  # doctest: +NORMALIZE_WHITESPACE
{'path': '20240830_HF_LC3_MAA_RK_12032_CA_DMSO4.raw', 'condition': 'SPRTN-TurboID CA_DMSO (vehicle)',
 'biological_replicate': 0, 'technical_replicate': 0, 'fraction': 0}

The SDRF names files without a directory. When your spectra live elsewhere, or the search read only some of them, pass searched_files= with the paths: each must be named exactly (case and extension) by one row, rows for other files are dropped and listed in notes, and your paths become full_path. To get MetaMorpheus's file instead, pass out="ExperimentalDesign.tsv" and put it beside the spectra, which is the only place MetaMorpheus looks. It must end in .tsv, and a refused design writes nothing.

A refusal lists every reason

PXD049018's depositors did not say which pulldown was treated: its factor value[treatment] is not available on every row. A condition built from an unknown factor would pair samples nobody said were alike, so mzLib refuses, once per row, naming each file:

>>> refused = pymzlib.sdrf.design("PXD049018.sdrf.tsv", condition_columns=both)
>>> refused.is_valid, len(refused.refusals), refused.file_count
(False, 20, 0)
>>> print(refused.refusals[0])  # doctest: +NORMALIZE_WHITESPACE
Line 2 (MSB67868ABand_01.raw): 'factor value[treatment]' is 'not available'. A condition
cannot be built from an unknown factor; fill it in, or leave the column out of the
declared condition columns to pool these rows.
>>> refused.spectra()  # doctest: +ELLIPSIS
Traceback (most recent call last):
...
pymzlib._bridge.UsageError: The design was refused, so there is no spectra list to give. mzLib's reasons:
...

A refusal is a result, not an exception. design() returns normally with is_valid = False, the table empty, and refusals filled; only spectra() and run_design() raise, so a refused design cannot reach FlashLFQ by accident. Print report for mzLib's whole account.

Without condition_columns, mzLib uses the document's only factor column. PXD067622 has two, so leaving them undeclared is itself a refusal: joining every factor could split a condition on a nuisance factor, so mzLib makes you choose.

Replicate numbers are ranked within each condition

A drafted SDRF often copies the study-wide run index out of the file names: WT_DMSO1-3, CA_DMSO4-6, ... CA_FA22-24. MetaMorpheus wants replicates numbered within each condition, so mzLib ranks them back to 1..3, and records every renumbering in notes, the only place it is kept:

>>> ranked = pymzlib.sdrf.design("PXD067622_studywide.sdrf.tsv", condition_columns=both)
>>> ranked.is_valid, len(ranked.notes)
(True, 7)
>>> print(ranked.notes[1])  # doctest: +NORMALIZE_WHITESPACE
Condition 'SPRTN-TurboID CA_formaldehyde 1 mM, 1 h': biological replicates renumbered
22 -> 1, 23 -> 2, 24 -> 3.
>>> ranked.columns == d.columns
True

The ranked design is identical to the hand-numbered one. (PXD067622_studywide.sdrf.tsv is PXD067622 with each biological replicate replaced by the number in its file name.)

Label-free only. An isobaric SDRF needs a channel design; design() does not build one. For the channels themselves, see Isobaric kits.

Many files in one call

validate_many(), assess_many() and samples_many() read a list of files in one bridge process:

>>> batch = pymzlib.sdrf.validate_many(
...     ["sdrf_skeleton.sdrf.tsv", "sdrf_cohort.sdrf.tsv", "missing.sdrf.tsv"],
...     on_error="skip",
... )
>>> batch.file_count, batch.read_count, batch.valid_count
(3, 2, 1)
>>> [f.is_valid for f in batch.files]
[False, True, None]
>>> [(os.path.basename(f.path), f.error.kind) for f in batch.files if f.error]
[('missing.sdrf.tsv', 'usage')]
  • Why not a loop? Each bridge call costs about 120 ms before mzLib does anything, so a corpus of a thousand files spends two minutes just starting processes. One call spends it once, and the parallelism happens inside .NET, where no Python thread pool is needed.
  • threads is how many files are read at once: 1 by default, because each worker holds a whole document in memory; -1 for every core. The result is the same at any value: rows are in input order, then the file's own order, which is tested byte for byte.
  • on_error="fail" (the default) raises on the first unreadable file in input order, so the same bad batch fails with the same message at any thread count. "skip" records the failure on that file's entry in files, as a FileError with kind usage (missing) or correctness (mzLib could not read it), and carries on. Every input keeps its entry either way.
  • The table gains source_index (the file's 0-based position in your list) and source_path as its first two columns.

lint() and pool() take many files by nature and use the {path: label} form instead.

comment[searched data file]

mzLib's SDRF builder (mzLib#1327) can write comment[searched data file] beside comment[data file]: the file a search actually read (an .mzML) next to the one the instrument acquired (a .raw). pyMzLib has no builder, but reads and pools such files like any other; the column is carried verbatim by read() and pool(). When it is present, join search results on it rather than on comment[data file]: it names the file whose scans the identifications refer to. lint() treats it as unique per row, so differing file names are never reported as drift.

What this does not do

  • Build an SDRF. mzLib's SdrfBuilder (with the searched data file, mzLib#1327, a bare comment[label], mzLib#1307, and atomic writing, mzLib#1324) takes mzLib-native types (Tolerance, DigestionAgent, Modification, DissociationType) that have no representation on a language-neutral wire yet. Designing one is a contract change, not a projection, so it is not offered. pool(out=...) writes an SDRF, and gets mzLib#1324's atomic write for free.
  • Draft an SDRF for a deposit that has none. mzLib's drafter, improver and publication-evidence readers are internal to mzLib, so no binding can call them.
  • Resolve controlled-vocabulary terms. Neither validate() nor lint() checks that an accession is real or names what it claims.
  • Audit quantification design. mzLib's SdrfQuantAuditor (is this TMT experiment channel-level or kit-only, which files are on disk) is public but not projected yet: it is a separate question with its own report, and needs its own verb.
  • Repair anything. Every function here reports; none rewrites a cell. Fixing a curated file is a curation decision, and a reader that made it silently would make it irreversibly.

Cite

If this guide's results go into a paper, cite mzLib (see Citing) and the SDRF-Proteomics format:

Further sources: