Benchmark method

Principles

One build. Every number on the results page comes from the same release artifact. Results from different versions are never combined.
Exact output first. A timing counts only if Fastmash’s output matches the expected output for that job.
Same conditions. GNU datamash and Fastmash run on the same host, with the same input files and locale, and alternating order. Each invocation’s output is captured in files on the same filesystem and checked for equality. Input is redirected from regular files; piped input can perform differently.
Repeated sessions. Each job runs in two separate sessions, each with one warm-up and six measured repetitions, alternating the programs. We report medians and keep the minimum and maximum. Files are in the page cache (warm-cache measurements).
Resources as well as time. CPU time and peak charged memory (from cgroups) are recorded for every job.

Representative jobs

Jobs are grouped into job families, each representing a kind of work:

Job family	What it measures	Example job
Startup	Fixed per-process cost	A tiny input
Decimal accumulation	Parsing and summing many numbers	Sum and mean of a million decimals
Retained numerical summaries	Operations that keep every value	Quantiles by group
Grouped numerical analysis	Grouping with numerical operations	Exon statistics by gene
Grouped text and counts	Grouping with text operations	Unique values by key

Datasets include public real-world data (RefGene genome annotations, the UCI Wine Quality data) and generated data with controlled shapes. The benchmark kit runs eight of the core jobs with their exact arguments, downloads the public data and regenerates the synthetic inputs from the same fixed seed.

Acceptance rules

The rules are fixed before a release candidate is measured:

Rule	Threshold
A job counts as a win	At least 20% lower median elapsed time and at least 5 ms saved per run, in both sessions and in at least five of six paired rounds
Fastmash is “faster” overall	On each host: wins in at least three of the four non-startup job families, including real-data wins in at least two; no unresolved material slowdowns; small commands within their limits
Material slowdown	Median elapsed time more than 10% and more than 5 ms longer
Small commands	At most 5 ms slower than GNU datamash, and at most 20 ms in total
CPU slowdown	More than 20% and more than 5 ms extra CPU time
Memory review	Peak memory more than twice GNU datamash’s and 64 MiB more

A job that fails, times out or gives a wrong answer counts as a failure, not a result. Each candidate is also compared with the previous accepted Fastmash build on every existing job (the predecessor guards); a slowdown beyond these limits needs an explicit, published justification.

Hosts

Host	CPU	OS	glibc	System `sort`
Laptop	Intel Core i7-8550U	CachyOS, native Linux 7.2.9	2.44	GNU coreutils 9.12
Desktop	AMD Ryzen 7 9800X3D	CachyOS, native Linux 7.2.9	2.44	GNU coreutils 9.12

GNU datamash’s -s jobs sort with the system sort, so its speed is part of their times.

Each complete command and its children run with a 512 MiB cgroup memory limit, no swap, at most 64 tasks (processes and threads), a 30-second timeout and a 512 MiB per-file output limit. Temporary files are on disk-backed Linux storage. Both programs get the same limits; Fastmash’s sort memory override is unset. These conditions matter for large sorts: the memory available to a command affects when it writes runs to disk. Separate, untimed observations confirm that the disk-spill and spill-large jobs write temporary runs on both hosts under these limits and still produce the expected output.

Full results

Mean of the two sessions’ median elapsed times, for every job and setting in the 0.1.0 measurement (release binary 1f1fec94…, 5 and 6 October 2026). Every job gave the expected output on both hosts. Sets: core and supplement are the representative jobs; locale-C, locale-en and locale-de repeat locale-sensitive jobs under C, en_US.UTF-8 and de_DE.UTF-8; guards are small edge-case jobs; the spill sets exercise larger sorts and their boundaries. CPU time and peak memory are in the measurement record. The displayed means average the retained session medians, already rounded to 0.1 ms, then round that mean to 0.1 ms. Ratios use the displayed values. Precise unrounded samples remain in the measurement record.

The elapsed verdict preserves each comparison’s two-session result. A regression crosses the material slowdown limits in both sessions, with Fastmash slower in at least five of six paired rounds in each. A threshold or direction disagreement is inconclusive; it does not establish a win or a competitive pass. Within margins means both sessions stay within the material regression limits. The named limitations retain the practical costs and uncertainty.

Intel laptop, native Linux

Set	Job	GNU datamash 1.9 (ms)	Fastmash 0.1.0 (ms)	GNU / Fastmash	Elapsed verdict
core	`decimal-1000`	1.6	1.6	1.00×	Within margins
core	`decimal-100000`	36.8	14.6	2.52×	Within margins
core	`decimal-1000000`	324.7	122.5	2.65×	Within margins
core	`disk-spill`	572.4	694.1	0.82×	Regression
core	`dominant-group`	21.9	11.9	1.84×	Within margins
core	`gene-many`	160.4	105.0	1.53×	Within margins
core	`gene-tiny`	2.6	1.4	1.86×	Within margins
core	`genes-example`	7.3	3.8	1.92×	Within margins
core	`grouped-decimal-100000`	69.9	38.5	1.82×	Within margins
core	`refgene-exons`	159.4	83.1	1.92×	Within margins
core	`refgene-quantiles`	112.8	26.6	4.24×	Within margins
core	`refgene-transcripts`	112.0	61.0	1.84×	Within margins
core	`scores-by-major`	2.6	1.4	1.86×	Within margins
core	`tiny-count`	1.2	1.5	0.80×	Within margins
core	`tiny-decimal`	1.2	1.5	0.80×	Within margins
core	`wine-by-quality`	6.4	3.2	2.00×	Within margins
core	`wine-red-summary`	1.7	1.6	1.06×	Within margins
core	`wine-white-summary`	4.6	2.8	1.64×	Within margins
disk-spill	`disk-spill`	771.3	773.5	1.00×	Inconclusive
guards	`distinct-20000-geomean`	4.8	5.9	0.81×	Within margins
guards	`distinct-200000-geomean`	37.4	44.8	0.83×	Regression
guards	`extreme-unique-128`	1.1	1.4	0.79×	Within margins
guards	`extreme-unique-20000`	4.9	7.2	0.68×	Within margins
guards	`missing-pairs`	3.0	1.8	1.67×	Within margins
guards	`moments-all-100000`	69.9	29.6	2.36×	Within margins
guards	`moments-alternating-100000`	69.8	52.8	1.32×	Within margins
guards	`paired-all-100000`	110.2	68.4	1.61×	Within margins
guards	`subnormal-unique-128`	1.1	1.7	0.65×	Within margins
guards	`subnormal-unique-20000`	10.6	8.8	1.20×	Within margins
locale-C	`locale-prepared`	22.1	24.9	0.89×	Within margins
locale-C	`locale-scientific`	41.8	30.6	1.37×	Within margins
locale-de	`locale-prepared`	22.8	25.0	0.91×	Within margins
locale-de	`locale-scientific`	104.1	30.7	3.39×	Within margins
locale-en	`locale-prepared`	22.6	25.0	0.90×	Within margins
locale-en	`locale-scientific`	104.2	30.5	3.42×	Within margins
many-keys	`many-keys`	780.3	965.1	0.81×	Regression
spill-guards	`alternating-original`	304.5	27.0	11.28×	Within margins
spill-guards	`alternating-packed`	305.8	31.9	9.59×	Within margins
spill-guards	`gene-many`	160.6	103.8	1.55×	Within margins
spill-guards	`refgene-exons`	158.8	82.7	1.92×	Within margins
spill-guards	`tiny-count`	1.1	1.3	0.85×	Within margins
spill-guards	`wide-original`	440.9	61.2	7.20×	Within margins
spill-guards	`wide-packed`	442.4	219.8	2.01×	Within margins
spill-half	`spill-half`	297.6	245.9	1.21×	Within margins
spill-large	`spill-large`	1418.0	1079.1	1.31×	Inconclusive
supplement	`base64-1000`	1.6	1.6	1.00×	Within margins
supplement	`base64-100000`	51.4	34.0	1.51×	Within margins
supplement	`cross-dense`	40.7	36.2	1.12×	Within margins
supplement	`cross-small`	2.5	1.4	1.79×	Within margins
supplement	`cross-sparse`	5.2	3.8	1.37×	Within margins
supplement	`extract-1000`	2.0	1.8	1.11×	Within margins
supplement	`extract-100000`	101.0	53.6	1.88×	Within margins
supplement	`hash-1000`	2.0	2.1	0.95×	Within margins
supplement	`hash-100000`	93.3	82.7	1.13×	Within margins
supplement	`mixed-1000`	1.8	2.0	0.90×	Within margins
supplement	`mixed-100000`	79.3	62.2	1.27×	Within margins
supplement	`path-1000`	1.5	1.7	0.88×	Within margins
supplement	`path-100000`	43.0	33.8	1.27×	Within margins
supplement	`rounding-1000`	3.1	2.8	1.11×	Within margins
supplement	`rounding-100000`	194.3	130.9	1.48×	Within margins
supplement	`table-small`	1.1	1.4	0.79×	Within margins
supplement	`table-tall`	7.3	5.0	1.46×	Within margins
supplement	`table-wide`	5.7	4.7	1.21×	Within margins
supplement	`wine-group-complete`	7.2	4.2	1.71×	Within margins
supplement	`wine-group-prepared`	4.2	4.0	1.05×	Within margins
supplement	`wine-means`	3.5	2.8	1.25×	Within margins
supplement	`wine-moments`	4.0	3.0	1.33×	Within margins
supplement	`wine-paired`	4.2	3.8	1.11×	Within margins
supplement	`wine10-means`	25.7	14.6	1.76×	Within margins
supplement	`wine10-moments`	31.0	17.2	1.80×	Within margins
supplement	`wine10-paired`	32.5	25.0	1.30×	Within margins

AMD desktop, native Linux

Set	Job	GNU datamash 1.9 (ms)	Fastmash 0.1.0 (ms)	GNU / Fastmash	Elapsed verdict
core	`decimal-1000`	0.6	0.6	1.00×	Within margins
core	`decimal-100000`	10.9	5.3	2.06×	Within margins
core	`decimal-1000000`	105.9	46.8	2.26×	Within margins
core	`disk-spill`	180.0	309.7	0.58×	Regression
core	`dominant-group`	9.0	4.5	2.00×	Within margins
core	`gene-many`	55.0	38.7	1.42×	Within margins
core	`gene-tiny`	1.1	0.6	1.83×	Within margins
core	`genes-example`	3.0	1.6	1.88×	Within margins
core	`grouped-decimal-100000`	25.6	14.8	1.73×	Within margins
core	`refgene-exons`	62.2	36.5	1.70×	Within margins
core	`refgene-quantiles`	50.3	13.1	3.84×	Within margins
core	`refgene-transcripts`	45.6	26.5	1.72×	Within margins
core	`scores-by-major`	1.0	0.5	2.00×	Within margins
core	`tiny-count`	0.5	0.6	0.83×	Within margins
core	`tiny-decimal`	0.5	0.5	1.00×	Within margins
core	`wine-by-quality`	2.8	1.4	2.00×	Within margins
core	`wine-red-summary`	0.6	0.6	1.00×	Within margins
core	`wine-white-summary`	1.7	1.2	1.42×	Within margins
disk-spill	`disk-spill`	176.3	311.6	0.57×	Regression
guards	`distinct-20000-geomean`	2.1	2.4	0.88×	Within margins
guards	`distinct-200000-geomean`	17.4	18.4	0.95×	Within margins
guards	`extreme-unique-128`	0.4	0.6	0.67×	Within margins
guards	`extreme-unique-20000`	2.2	3.1	0.71×	Within margins
guards	`missing-pairs`	1.4	0.8	1.75×	Within margins
guards	`moments-all-100000`	22.4	11.1	2.02×	Within margins
guards	`moments-alternating-100000`	24.2	20.1	1.20×	Within margins
guards	`paired-all-100000`	36.2	33.4	1.08×	Within margins
guards	`subnormal-unique-128`	0.4	0.8	0.50×	Within margins
guards	`subnormal-unique-20000`	1.6	3.7	0.43×	Within margins
locale-C	`locale-prepared`	7.3	10.5	0.70×	Within margins
locale-C	`locale-scientific`	15.5	12.6	1.23×	Within margins
locale-de	`locale-prepared`	7.6	10.5	0.72×	Within margins
locale-de	`locale-scientific`	43.1	12.6	3.42×	Within margins
locale-en	`locale-prepared`	7.5	10.5	0.71×	Within margins
locale-en	`locale-scientific`	43.0	12.7	3.39×	Within margins
many-keys	`many-keys`	314.8	405.9	0.78×	Inconclusive
spill-guards	`alternating-original`	97.7	8.5	11.49×	Within margins
spill-guards	`alternating-packed`	97.6	9.1	10.73×	Within margins
spill-guards	`gene-many`	55.1	38.8	1.42×	Within margins
spill-guards	`refgene-exons`	62.5	36.5	1.71×	Within margins
spill-guards	`tiny-count`	0.5	0.6	0.83×	Within margins
spill-guards	`wide-original`	140.3	23.8	5.89×	Within margins
spill-guards	`wide-packed`	141.9	96.3	1.47×	Within margins
spill-half	`spill-half`	95.3	100.8	0.95×	Within margins
spill-large	`spill-large`	263.1	467.6	0.56×	Regression
supplement	`base64-1000`	0.6	0.6	1.00×	Within margins
supplement	`base64-100000`	17.6	13.3	1.32×	Within margins
supplement	`cross-dense`	16.6	14.4	1.15×	Within margins
supplement	`cross-small`	1.1	0.6	1.83×	Within margins
supplement	`cross-sparse`	2.2	1.6	1.38×	Within margins
supplement	`extract-1000`	0.8	0.7	1.14×	Within margins
supplement	`extract-100000`	35.3	23.2	1.52×	Within margins
supplement	`hash-1000`	0.8	0.6	1.33×	Within margins
supplement	`hash-100000`	38.9	15.1	2.58×	Within margins
supplement	`mixed-1000`	0.6	0.8	0.75×	Within margins
supplement	`mixed-100000`	26.9	25.5	1.05×	Within margins
supplement	`path-1000`	0.5	0.6	0.83×	Within margins
supplement	`path-100000`	15.2	12.6	1.21×	Within margins
supplement	`rounding-1000`	1.3	1.1	1.18×	Within margins
supplement	`rounding-100000`	85.8	68.3	1.26×	Within margins
supplement	`table-small`	0.5	0.6	0.83×	Within margins
supplement	`table-tall`	2.3	2.1	1.10×	Within margins
supplement	`table-wide`	1.9	1.9	1.00×	Within margins
supplement	`wine-group-complete`	3.0	1.9	1.58×	Within margins
supplement	`wine-group-prepared`	1.6	1.7	0.94×	Within margins
supplement	`wine-means`	1.3	1.1	1.18×	Within margins
supplement	`wine-moments`	1.4	1.3	1.08×	Within margins
supplement	`wine-paired`	1.5	1.6	0.94×	Within margins
supplement	`wine10-means`	9.1	5.8	1.57×	Within margins
supplement	`wine10-moments`	10.1	7.6	1.33×	Within margins
supplement	`wine10-paired`	10.9	11.0	0.99×	Within margins

Keyboard shortcuts

Fastmash