Fastmash
Fast command-line statistics for delimited text.
Fastmash summarizes delimited text and quoted CSV, with grouping and statistics for each category. It also reshapes ordinary delimited text. It accepts the command language of GNU datamash, so existing datamash workflows can move over with few or no changes.
It also reads quoted CSV directly, gives your result columns stable names, calculates weighted means and selects the highest or lowest complete records. Inspect a table’s health or compare summaries from two exports with the same field selectors and portable arithmetic.
$ cat readings.tsv
site reading
west 2
east 4
west 6
west 7
$ fastmash -H -s -g site count reading median reading < readings.tsv
Grouped summary
────────────────────────────────
site count(reading) median(reading)
east 1 4
west 3 6
✔ Completed
Install it with one command, or see other ways:
curl -fsSL https://fastmash.io/install.sh | sh
Fastmash is faster than GNU datamash*
About five times faster on RefGene quartiles, with faster grouped summaries and decimal totals too.
* Large sorted summaries can be slower on Linux. All results and the method.
And better in other ways
| Feature | GNU datamash 1.9 | Fastmash |
|---|---|---|
| Results | Depend on the CPU and the C math library | Repeatable across supported machines with the same version, input and settings |
| Locales | Need locale data installed. Silently fall back to C without it | Built in: numbers in 318 locales, sorting in 194 |
perc:100 | Can read past the end of the values | Returns the largest value |
rmdup on several key fields | Crashes on an assertion | A clear refusal |
Malformed parameters such as strbin:2.0 | Misleading errors or undefined behavior | An accurate error message |
crosstab labels over 511 bytes | Truncated | Kept in full |
Fastmash supports over 70 operations and modes with familiar field selectors, grouping and headers. Most scripts can switch by changing one word. Check the documented differences when migrating a workflow. Migrating from datamash · All differences
Get started
- Install Fastmash on Linux, macOS or Windows
- Quick start: a five-minute tour
- Operations: every statistic and transformation
- Quoted CSV and result names: structured input and useful report labels
- Highest and lowest records: keep the records behind the numbers
- Table health and dataset comparison: inspect data and understand changes
Fastmash is open source under the MIT or Apache-2.0 license. Source on GitHub.