vaqum: compress, encrypt, diff and shred files from the command line

I kept reaching for five different tools to do one job: zip or tar to archive, gpg to encrypt, diff to compare, fdupes to find duplicates, shred to delete. They all have different flags, and not all of them exist on every OS.

vaqum puts them in one command that works the same on macOS, Linux and Windows. It's written in Rust, ships as a single binary, and at default settings it's about 5x faster than zip/gzip with output about 12% smaller.

What it does

Install

npm i -g @marcdomain/vaqum                    # macOS, Linux, Windows
brew install marcdomain/vaqum/vaqum           # Homebrew
curl -fsSL https://raw.githubusercontent.com/marcdomain/vaqum/main/install.sh | sh   # macOS, Linux
irm https://raw.githubusercontent.com/marcdomain/vaqum/main/install.ps1 | iex        # Windows PowerShell

The npm package pulls in a prebuilt binary for your platform, so there's nothing to compile. Check it worked:

vaqum --version

Compress and decompress

Compress a file. -v shows the stats:

vaqum compress report.txt -v

✔ Compressed 'report.txt' -> 'report.txt.vaqum'
  algorithm:    zstd
  original:     12.61 kB
  compressed:   516 B (24.44x)
  output size:  599 B (21.05x)

By default vaqum uses zstd at level 9. Pick a level from 1 (fastest) to 22 (smallest) with -l, or use --max to switch to xz (LZMA), which is slower and is usually smaller on larger inputs. On a tiny file like this one, zstd actually wins, so it's worth checking with --dry-run first:

vaqum compress report.txt --dry-run          # estimate without writing anything
vaqum compress report.txt -l 19              # higher zstd level
vaqum compress report.txt --max -o report-max.vaqum

Decompress, and check the result matches the original's SHA-256 checksum with --verify:

vaqum decompress report.txt.vaqum -o restored --verify

✔ verified checksum (48a077133b4b4e0541da861082275b948089ef3d6f3c33085a858095d34819c2)
✔ Decompressed -> restored

Folders and bundles

Compress a folder with -r, or several paths into one archive with -o:

vaqum compress project -r
vaqum compress report.txt notes-v1.txt notes-v2.txt -o bundle.vaqum

Skip paths with --exclude (repeatable). End a pattern with / for a folder; a name without / matches at any depth:

vaqum compress webapp -r --exclude node_modules/ --exclude "*.log" -o webapp.vaqum

Or put the patterns in a .vaqumignore file at the folder's root, one per line, like a .gitignore:

node_modules/
dist/
*.log

Deduplication

If a folder has repeated files, --dedup stores each unique file once:

vaqum compress photos -r --dedup -v

  new img1.txt  (12.61 kB)
  dup img2.txt  (12.61 kB)
  new img3.txt  (26 B)
✔ Compressed 'photos' -> 'photos.vaqum'
  algorithm:    zstd + dedup

Compression uses every CPU core by default; limit it with -t <threads>.

Encrypt

Add -e to encrypt with a password, or --key-file to use a file's bytes as the key:

vaqum compress secret.txt -e -o secret.vaqum
vaqum compress secret.txt --key-file key.bin -o secret.vaqum

✔ Compressed and encrypted 'secret.txt' -> 'secret.vaqum'
  encryption:   ChaCha20-Poly1305, Argon2id KDF

The data is encrypted with ChaCha20-Poly1305. The key is derived from your password with Argon2id, and only the salt is stored in the archive, never the key. decompress detects encryption by itself and asks for the password (or takes --key-file).

Open zip and tar.gz files too

decompress and info also read .zip and .tar.gz/.tgz, detected by content rather than file extension, so you don't need another tool to open something a teammate sent:

vaqum info shared.zip
vaqum decompress shared.tar.gz -o shared-restored

This is one-way: vaqum only writes its own format.

Protection from decompression bombs

A small archive can be crafted to expand into terabytes. decompress refuses archives that claim an expansion ratio above 1000:1, or that wouldn't fit on the disk. Change the limit with --max-ratio, or skip the check with --force when you trust the source.

Inspect

info works on archives without fully decompressing them, and on plain files and folders:

vaqum info report.txt.vaqum

report.txt.vaqum
  type:         single file
  name:         report.txt
  algorithm:    zstd
  original:     12.61 kB
  compressed:   516 B (24.44x)
  on disk:      599 B
  checksum:     sha256:48a077133b4b4e0541da861082275b948089ef3d6f3c33085a858095d34819c2
  encrypted:    no

Diff files, folders and archives

diff compares any mix of files, folders and .vaqum archives. That makes it easy to check a backup against the live copy without unpacking it first.

vaqum diff project-backup project -v

comparing 'project-backup' vs 'project':
  1 unchanged, 1 added, 0 removed, 1 modified
  + src/util.rs  (19 B)
  ~ src/main.rs
--- a/src/main.rs
+++ b/src/main.rs
@@ -1,3 +1,3 @@
 fn main() {
-    println!("v1");
+    println!("v2");
 }

Three other ways to view a diff:

vaqum diff a.txt b.txt --html diff.html   # self-contained, offline, side-by-side HTML report
vaqum diff a.txt b.txt --open             # same report, opened in your browser
vaqum diff old/ new/ -e                   # VS Code's split view (code --diff), one tab per changed file

Set VAQUM_DIFF_EDITOR to use another editor. When one side is a .vaqum archive, it's unpacked to a scratch copy first, so edits there don't change the archive.

Find duplicates

dedupe hashes every file in a folder and groups identical ones:

vaqum dedupe photos -v

photos
  files scanned:     3 (25.24 kB)
  duplicate groups:  1 (1 redundant file)
  reclaimable:       12.61 kB

  sha256:48a07713…819c2 (12.61 kB, 2 copies)
    img1.txt
    img2.txt

Add --link to replace duplicates with hardlinks to the first copy. That frees the space without deleting any path. Preview it first with --link --dry-run.

Search

Search file names and contents at once, recursively:

vaqum search TODO app

name     app/TODO-cleanup
content  app/main.rs:2: // TODO: wire up config
content  app/util.rs:1: // TODO: add tests

3 matches (1 file by name, 2 lines of content)

Use -i for case-insensitive, -E for a regular expression, and --names-only or --content-only to narrow it down:

vaqum search 'TODO: \w+' app -E --content-only

Shred

shred overwrites a file before deleting it, so it can't be recovered from the disk. It asks you to confirm first (skip that with -y in scripts):

vaqum shred secret.txt
vaqum shred old-keys/ -r -p 7      # a folder, with 7 overwrite passes
vaqum shred old-keys/ -r --dry-run # list what would be destroyed

⚠  This will permanently and irrecoverably destroy:
    secret.txt (28 B)
✔ Shredded (3 passes, DoD 5220.22-M-style overwrite)
  note: on SSDs, overwrite passes are best-effort (wear-leveling/TRIM) — not forensic-grade.

As the note says, SSDs move data around internally, so overwriting is best-effort there. For data that must never be recoverable on an SSD, use full-disk encryption from the start.

Save your defaults

vaqum config init writes a starter file at ~/.config/vaqum/config.toml (%APPDATA%\vaqum\config.toml on Windows):

[defaults]           # every compress
exclude = ["*.log"]

[compress]           # overrides [defaults] for compress

[profile.release]    # used with --profile release
max = true

vaqum compress dist -r --profile release -o release.vaqum
vaqum config show    # what's in effect
vaqum config path    # where the file is

Flags on the command line always win over the file.

Shell completions

vaqum completions zsh --install    # also bash, fish, powershell, elvish

--install writes the script to your shell's standard completions folder and tells you if one line needs adding to your shell config. It never edits your config itself.

Scripts and CI

Progress bars show on interactive terminals and hide themselves when output is piped. -q hides them explicitly. Combined with -y on shred and --key-file for encryption, every command runs without prompts:

vaqum compress build/ -r -q --key-file "$KEY_FILE" -o "build-$GITHUB_SHA.vaqum"

Links

• Source on GitHub
• npm package
• Run vaqum <command> --help for every flag