Benchmarks

Speed is Fastmash’s reason to exist, so every release is measured against GNU datamash 1.9 on the same machines, with the same data, and the results are published in full: wins and losses.

Fastmash 0.1.0

Mean of the two sessions’ median elapsed times, in milliseconds; lower is better. Measured with the 0.1.0 release binaries themselves (fastmash sha256 1f1fec94…), the same bytes you download. The chart is generated from core-jobs.tsv by scripts/benchmark_chart.py.

The animated demo replays the Intel laptop’s RefGene quartile job below. Playback is slowed 20 times so that both runs are visible; its clocks show the measured times.

Across 71 combinations of jobs and settings, 28 on the Intel laptop and 19 on the native AMD desktop meet the full speed-win rule: at least 20% and 5 ms saved in both sessions, with consistent paired runs. Both hosts have wins in all four non-startup workload families. Every job gives the expected output. The named slower and inconclusive cases below retain their individual dispositions. Full tables and the rules are on the method page.

Where GNU datamash is still faster

These jobs cross the material elapsed slowdown rule in both sessions. Each cell shows the two session medians in milliseconds:

These individual costs are accepted for 0.1.0. The AMD spill medians reach 1.83 times GNU’s time on these invocations. Lower CPU work and charged peak memory in the spill measurements are separate observations; the elapsed verdicts remain regressions. The geometric mean keeps the existing numerical precision and range checks.

Three further comparisons are inconclusive under the two-session rule:

These uncertain results are also accepted as named limitations. They establish neither a GNU competitiveness pass nor a win. All samples remain in the record, including an Intel dedicated-spill Fastmash run of 4.887 seconds. Session medians give no tail-latency guarantee or upper bound for other input sizes.

The decision retains the measured implementation: every predecessor elapsed and CPU guard passes, both hosts meet the workload-family and small-command rules, and all checked outputs match. The numbers concern the recorded warm-cache, disk-backed, 512 MiB command cgroup conditions on these two native Linux hosts. Untimed observations confirm actual spilling and correct output; they do not establish an I/O bottleneck.

Run them yourself

The benchmark kit runs eight of these jobs on your machine with both programs, checks that they agree, and prints a table you can share:

curl -fsSLO https://raw.githubusercontent.com/pederbe/fastmash/main/bench/fastmash-bench.py
python3 fastmash-bench.py

The best benchmark is your own job. Compare both programs on your data:

export LC_ALL=C.UTF-8
time datamash -s -g 1 median 2 < your-data.tsv > /dev/null
time fastmash -s -g 1 median 2 < your-data.tsv > /dev/null

For stable numbers, repeat each command several times, alternate the two programs, and use a quiet machine. hyperfine does this for you:

hyperfine --warmup 2 \
  'datamash -s -g 1 median 2 < your-data.tsv' \
  'fastmash -s -g 1 median 2 < your-data.tsv'

If Fastmash is slower on a job you care about, please tell us: slow real-world jobs are what we optimize next.