Pre-1.0 hoardDB is pre-1.0. Expect breaking changes.

Backup and Restore

hoardDB dump captures every database and bucket except blob payloads, which no dump contains — see Blobs are not restored at all; both dump and restore print a warning naming each blob bucket affected, on every run. Everything else is written to a directory of plain BSON files — the same shape mongodump uses — and hoardDB restore reads it back into a running server. No second binary, no proprietary archive format: the .bson files this produces are ordinary BSON, readable by bsondump and any other BSON-aware tool, on the day you take the dump and after.

This page is the how-to drill, the guarantees the commands actually make, and the caveats worth knowing before you rely on this for something that matters. Every transcript below is copied from a real session: a server started with no configuration, real inserts, a real dump, real destruction of the data, and a real restore.

The Drill

Start a server the normal way (see Getting Started) and connect the CLI. Create a database with a hash bucket and a fifo bucket so the drill covers both an engine that dumps in native storage order and one where order is the data:

[h]oardDB> create database shop
OK  created database shop
Next: use shop
Then: create bucket <name> {type: hash | btree | fifo | lifo | heap | blob}   (hash is the default)
[h]oardDB> use shop
Switched to database 'shop'
[h]oardDB [shop]> create bucket Customers { type: hash, indices: ["Email"] }
OK  created bucket shop.Customers (type hash)
Indexes: [Email]
[h]oardDB [shop]> db.Customers.insert({_id_: "c1", Name: "Ada Lovelace", Email: "ada@example.com"})
OK  id=c1
[h]oardDB [shop]> db.Customers.insert({_id_: "c2", Name: "Grace Hopper", Email: "grace@example.com"})
OK  id=c2
[h]oardDB [shop]> create bucket Orders { type: fifo }
OK  created bucket shop.Orders (type fifo)
[h]oardDB [shop]> db.Orders.push({item: "widget", qty: 3})
OK  id=668a4412-09e0-4f80-965e-91cbaa795914
[h]oardDB [shop]> db.Orders.push({item: "gadget", qty: 1})
OK  id=1e6d644a-5d95-43f2-beef-eeaf13fc76df
[h]oardDB [shop]> db.Orders.push({item: "gizmo", qty: 7})
OK  id=7f8c1e3b-80a1-4c18-a251-29be5e5811b0

dump — the standalone form

dump and restore are verbs of the same hoardDB binary, not a separate tool — dump and restore are also REPL statements sharing the same implementation (see below), because a database admin tool that splits “connect and query” from “back up and restore” into two programs is asking you to remember which binary does what:

hoardDB dump -address 127.0.0.1:17433 -user admin -password ****** -insecure --out ./dump

dumping shop/Customers...
  shop/Customers: 2 documents, 138 bytes, sha256:e6e192eaed90562ab4a7dbd1421902a85ab332bd7964fff001157ba6c8204d31
dumping shop/Orders...
  shop/Orders: 3 documents, 245 bytes, sha256:16fe3a1e5ca88e9030f70d4219762c07861175ef3e2c174cd881332d53aa5ad5
dump: 1 database(s), 2 bucket(s), 5 document(s), 383 byte(s)

-insecure is for a local development server, as in Getting Started — against anything else, pass the server’s certificate fingerprint instead.

The output directory:

dump/
  manifest.json
  shop/
    Customers.bson
    Customers.metadata.json
    Orders.bson
    Orders.metadata.json

manifest.json:

{
  "format_version": 1,
  "hoarddb_version": "devel",
  "created_at": "2026-09-19T09:59:35Z",
  "source_node": "dalek-dev",
  "databases": ["shop"],
  "buckets": 2,
  "documents": 5,
  "bytes": 383,
  "blobs_included": false,
  "credentials_included": false,
  "globally_consistent": false,
  "incomplete": false
}

shop/Customers.metadata.json — one of these per bucket, carrying the store type, indexes and a digest of the .bson file next to it:

{
  "format_version": 1,
  "hoarddb_version": "devel",
  "protocol_major": 1,
  "created_at": "2026-09-19T09:59:35Z",
  "source_node": "dalek-dev",
  "database": "shop",
  "bucket": "Customers",
  "store_type": "hash",
  "indexes": ["Email"],
  "documents": 2,
  "bytes": 138,
  "digest": "sha256:e6e192eaed90562ab4a7dbd1421902a85ab332bd7964fff001157ba6c8204d31",
  "blobs_included": false
}

shop/Customers.bson is a plain concatenation of BSON documents — no length prefix, no wrapper — which is what lets bsondump shop/Customers.bson and other BSON tooling read it unmodified. Note there is no users.json: see Account recreation is not implemented below.

The dump is plaintext — protecting it is your part

There is no password on these files and no --encrypt flag, deliberately: a dump is the artifact you carry to another machine, another version, another host, and sealing it under this server’s keys would make it restorable only where those keys exist, which is the opposite of a backup. That means the dump directory is exactly as sensitive as the database it came from — and a little more so, because it is one directory you can copy in a single command.

Protect it with tools you already trust: write it onto an encrypted volume, pipe the directory through gpg or age before it leaves the host, or give it the same permissions, retention and off-site policy as any other copy of the data. manifest.json holds no credentials (see Account recreation) but it does list every database and bucket, and the .bson files beside it are your documents, verbatim.

Destroy it

[h]oardDB [shop]> drop bucket Customers
OK  bucket "Customers" dropped from "shop" (data removed from disk)
[h]oardDB [shop]> drop bucket Orders
OK  bucket "Orders" dropped from "shop" (data removed from disk)
[h]oardDB [shop]> show buckets
BUCKETS in shop
---
(no buckets)

restore

hoardDB restore -address 127.0.0.1:17433 -user admin -password ****** -insecure --dir ./dump

restoring shop/Customers (2 document(s))...
  shop/Customers: 2 applied, 0 skipped
restoring shop/Orders (3 document(s))...
  shop/Orders: 3 applied, 0 skipped
restore: 1 database(s), 2 bucket(s), 5 document(s) applied, 0 skipped

Verify — counts and contents

[h]oardDB [shop]> show buckets
NAME       TYPE  ENTRIES
Customers  hash  2
Orders     fifo  3
[h]oardDB [shop]> db.Customers.get("c1")
{"Email": "ada@example.com", "Name": "Ada Lovelace", "_id_": "c1"}
[h]oardDB [shop]> db.Customers.get("c2")
{"Email": "grace@example.com", "Name": "Grace Hopper", "_id_": "c2"}
[h]oardDB [shop]> db.Orders.length()
3
[h]oardDB [shop]> db.Orders.pop()
{"_id_": "668a4412-09e0-4f80-965e-91cbaa795914", "item": "widget", "qty": 3}
[h]oardDB [shop]> db.Orders.pop()
{"_id_": "1e6d644a-5d95-43f2-beef-eeaf13fc76df", "item": "gadget", "qty": 1}
[h]oardDB [shop]> db.Orders.pop()
{"_id_": "7f8c1e3b-80a1-4c18-a251-29be5e5811b0", "item": "gizmo", "qty": 7}

Values round-tripped exactly, and the queue popped back out in the same order it was pushed in (widget, gadget, gizmo) — order is the data for a FIFO bucket, and the dump preserves it.

The REPL form

The exact same grammar works as a statement inside an interactive session, so you can dump one bucket without leaving the shell. Flags come before the positional database bucket arguments, same as the standalone form:

[h]oardDB [shop]> dump --out ./dump-repl shop Customers
dumping shop/Customers...
  shop/Customers: 2 documents, 138 bytes, sha256:4892e5983f3f67eb93462c22a2cf2c9de4223f83cc0fb02af93677f4e4273920
dump: 1 database(s), 1 bucket(s), 2 document(s), 138 byte(s)

Scripting it

--json prints exactly one JSON object to stdout — progress, byte counts and rate go to stderr — so a pipeline can parse the result without the human-readable noise mixed in:

hoardDB restore -address 127.0.0.1:17433 -user admin -password ****** -insecure \
  --dir ./dump --drop --json 2>restore.log

{"ok":true,"databases":["shop"],"buckets":1,"documents_applied":2,"documents_skipped":0,"users_recreated":0,"users_awaiting_password":0,"warnings":[],"errors":null}

Exit codes: 0 success, 1 usage error, 2 refused before anything was applied, 3 a verification failure (partial success — check errors), 4 interrupted. restore --drop replaces each target bucket instead of merging into it; without --drop, restore merges — and whether a re-run is safe depends on the bucket’s store type, because the underlying write is different per engine:

• hash and btree restore through a keyed insert (OpPut). Re-running the same restore twice converges: the second run overwrites the same keys rather than adding to them.
• FIFO, LIFO and heap restore through a push (OpFIFOPush/OpHeapPush, or a buffered reverse-push for LIFO — see server/restore.go’s applyRestoreDocument). A queue or heap element has no key to converge on, so re-running the same restore against one of these buckets delivers every element again, on top of what the first run already added. This is the specified, intentional behavior for a push-based engine, not a bug — but it means a cron job or retry loop that blindly re-runs restore must not merge into a FIFO/LIFO/heap bucket that already received a previous run; use --drop to replace the bucket outright instead of merging into it a second time.
• Blob buckets restore nothing at all, on the first run or any subsequent one — see Blobs are not restored at all below.

hoardDB dump --out /backups/$(date +%F) is the whole cron line: it needs no prompt, no TTY, and no flag to confirm when the destination is a fresh directory.

What the dump promises about consistency

Per bucket, a dump is a consistent point in time. Across buckets, it is not. This is the same thing mongodump documents about itself, for the same reason, and it is worth stating as plainly: a dump of shop reads Customers as of one instant and Orders as of a slightly later instant, because each bucket is read independently while the rest of the server keeps serving writes. manifest.json carries "globally_consistent": false so nothing downstream has to guess. If you need every bucket frozen at exactly the same instant, that is not what a logical dump gives you — stop writes for the duration, or dump during a quiet window.

Within one bucket, though, the guarantee is real: a hash/btree/heap bucket’s dump is read from a single consistent snapshot, and writers are not blocked while it runs. A FIFO/LIFO bucket’s dump is the queue as of the moment it started capturing, and the queue keeps taking pushes and pops throughout — see the next section for what that means for elements popped mid-dump.

A whole-installation dump only covers the node you connect to

On a multi-node cluster, hoardDB dump with no database/bucket argument does not automatically reach every node. OpListDBs/OpListBuckets answer “list what this node holds”, by design, and this release has no way to reach the other members and union their bucket lists for you: a node’s own address, as the rest of the cluster knows it, is the internode listen address, not a port a client can dial (confirmed while building this feature). Connect to node A and run a scopeless dump, and you get node A’s buckets only — silently, aside from a warning on stderr when the server you’re connected to knows it has peers:

warning: this server is part of a 3-node cluster; dump enumerates only the buckets the
connected node holds locally (dump-restore-spec.md §10.1) — connect to each node in
turn (--address) for full coverage

A script that only checks the exit code will not see this warning — read stderr, or parse warnings from --json output, the same as the accounts warning above.

To back up an entire cluster today, run dump once per node, pointing -address at each member in turn:

hoardDB dump -address node-a:17433 -user admin -insecure --out ./dump/node-a
hoardDB dump -address node-b:17433 -user admin -insecure --out ./dump/node-b
hoardDB dump -address node-c:17433 -user admin -insecure --out ./dump/node-c

Each node’s directory is that node’s own local buckets, not a deduplicated, cluster-wide snapshot — a bucket held by more than one replica is captured once per replica that has it. A --nodes flag (or equivalent) that fans one dump invocation out across every member automatically is not built in this release; this page will be updated when it is. Restore is unaffected by any of this: it always goes through a single connection to a single server (-address names it), the same as any other write.

Queues: what you get back may be a superset

Dumping a FIFO or LIFO bucket does not pause it. Pushes during the dump simply land after the captured range and are not included — nothing surprising there. Pops are the case worth knowing: a pop advances the queue’s head, but the bytes the dump already captured stay readable until the dump finishes with that bucket, so an element popped during the dump can still end up in the dump file.

The practical effect: a restore can re-deliver an element your application already consumed. For an at-least-once consumer this is exactly the cost of being at-least-once and is harmless. For an exactly-once consumer it is data — and this release does not tell you when it happened: the design this feature was specified against calls for a "concurrent_mutations_detected" manifest flag, but no such field is written by dump or read anywhere in this release (verified: cli/dump.go’s topLevelManifest has no such field, and neither concurrent_mutation nor ConcurrentMutations appears anywhere in the codebase outside the specification document). There is currently no flag to check. If you dump a FIFO or LIFO bucket that is being actively popped from, assume the restored queue may contain elements your application already processed — there is no way to confirm this after the fact from the dump’s own output. The way to get an exact, no-redelivery snapshot of a queue today is to dump it from a period with no concurrent pops.

Elements that were deleted (tombstoned) before the dump captured them are never emitted — those do not come back. LIFO is dumped in pop order (newest-first), so a restore reproduces the stack correctly rather than reversing it.

The compaction-pin ceiling on FIFO/LIFO dumps

Dumping a queue holds a pin that blocks that bucket’s background compaction for as long as the dump is reading it — pushes and pops are unaffected, but disk space used by consumed entries cannot be reclaimed until the pin releases. The pin is released the moment the dump finishes the bucket, the connection closes, or a ceiling is hit — whichever comes first.

The ceiling defaults to 5 minutes and is configurable:

A FIFO or LIFO bucket under continuous, heavy pop traffic can outrun this ceiling before its dump finishes. If that happens to you, dump that bucket during a maintenance window, or raise the ceiling for the duration of the dump and lower it again afterward — a pin held indefinitely against a high-throughput queue is a disk-exhaustion risk, which is exactly what the default keeps bounded.

The 16 MiB portability caveat

bsondump and most third-party BSON tooling assume every document is at most 16 MiB — the conventional MongoDB document-size cap. hoardDB’s own limits are different: a FIFO/LIFO entry may be up to the segment size (64 MiB by default), so a .bson file this tool produces from a legitimate bucket can exceed that 16 MiB convention. That is not a bug in the dump — it is a faithful copy of data that was already legally in the bucket — but it means a .bson file from a queue-heavy hoardDB installation is not guaranteed to open in tooling that hard-codes the 16 MiB assumption. hoardDB’s own restore has no such limit; the caveat is specifically about handing the file to something else.

Oversized documents are refused, not truncated

There is a real ceiling on what dump will write for you, and it is lower than the 64 MiB segment size above: ~31.94 MiB (33,488,896 bytes exactly — the 32 MiB wire-frame limit, minus headroom reserved for framing overhead). A document that cannot fit inside one dump chunk is refused outright: dump names the bucket and the key (or “queue element” for a FIFO/LIFO entry with no key) and points at blob buckets as the place for a payload that large. The bucket’s dump fails; the rest of the run continues, and the failure is recorded in manifest.json.

If you store payloads anywhere near this size, use a blob bucket instead of hash/btree/heap/fifo/lifo — blob payloads are designed for large binary content, and no dump chunk ceiling applies to them.

That trade is not free, and it is the opposite of a backup guarantee: a payload moved into a blob bucket leaves the dump entirely. A document that is too large for dump fails loudly and is recorded in manifest.json; the same bytes in a blob bucket are simply not captured, and restore brings the bucket back empty (next section). Until that changes, back blob payloads up out of band — dump and restore both warn on every run and name each blob bucket, so a scripted backup can fail on the warning rather than discover it during a recovery.

Blobs are not restored at all

Blob payloads are excluded from every dump, deliberately — manifest.json and each blob bucket’s metadata.json carry "blobs_included": false. The dump side does capture each blob’s metadata (key, filename, content type, size, checksum, creation time) into that bucket’s .bson file — a dump of a blob bucket is not empty on disk, and this is a deliberate default, not a current limitation: a dump that silently balloons into a full copy of every payload you store is the kind of thing that gets run once and never trusted again.

restore, however, does not write any of that back. In this release, every document in a blob bucket’s dump — metadata included — is counted into the restore’s skipped total and otherwise discarded (verified: server/restore.go’s applyRestoreDocument, the BucketTypeBlob case, increments a skip counter and returns with no write of any kind). A restored blob bucket in this release ends up completely empty — not “metadata present, payload missing.” restore does report the count of blob documents it skipped, so this is not silent, but do not expect a restored blob bucket to hold anything: no key, no filename, no size, no timestamp comes back. If you need a blob bucket’s metadata after a restore, read it from the dump directory’s .bson/.metadata.json files directly — dump captured it — rather than from the restored bucket, which restore does not populate.

Account recreation is not implemented in this release

hoardDB dumps and restores data and bucket definitions. It does not dump or restore user accounts.

Concretely: dump never writes a users.json file, and restore never reads one. If a dump directory happens to contain a users.json — hand- written, copied in from a different tool, or left over from a future version — restore notices it, prints a warning, and does not act on it:

warning: users.json present but account restore is not yet implemented; accounts were not recreated

{"warnings":["users.json present but account restore is not yet implemented; accounts were not recreated"]}

The warning appears in both the text summary and --json output, never only as a log line, so a script that only checks the exit code still has somewhere to see it if it parses warnings.

This is a real limitation of this release, not a partially-built feature: the server has no user-account CRUD (create/alter/drop user are not implemented — see Getting Started), so there is nothing for a dump to capture and nothing for a restore to recreate. A future release that adds persistent, mutable user accounts is expected to add account recreation to dump/restore at the same time; until then, back up your credential configuration (HOARDB_ROOT_USER/HOARDB_ROOT_PASSWORD or your config file) the way you already do for any other secret.

Summary of flags

dump [database [bucket]] and restore (source is always --dir) both work as REPL statements with the same flags, minus the connection ones — the REPL is already connected.

Source: docs/user/backup-restore.md in the repository.