Pre-1.0 hoardDB is pre-1.0. Expect breaking changes.

Indexes: unique, compound and TTL

A secondary index lets search find documents by a field’s value without reading the whole bucket. In this version an index can be declared only on a hash bucket, when it is created, and cannot be changed afterwards; the other store types refuse an index, because nothing would maintain it.

Every example below runs in the shell (hoardDB -insecure against a local server, after use app).

Declaring an index

create bucket Users { type: hash, indices: ["Email"] }

indices is a list. Each entry is a field name or an object:

A key the server does not know is refused and the refusal names the keys it accepts, so a misspelled uniqe: true never creates a plain index. Also refused, each with the reason: a unique TTL index, a TTL index over more than one field, desc: true (a descending component only matters to sorting, which indexes do not serve), a field named twice in one index, two indexes on the same fields, and two indexes with the same name. type: "range" is accepted with a warning and ignored: every index is the same kind.

db.Users.stats() (or describe bucket Users) lists the indexes:

Indexes:  Email_1 (unique), CustomerID_1_CreatedAt_1, CreatedAt_1 (ttl 3600s)

Unique indexes

create bucket Users { type: hash, indices: [{field: "Email", unique: true}] }
db.Users.insert({_id_: "u1", Email: "dalek@example.com"})
db.Users.insert({_id_: "u2", Email: "dalek@example.com"})

The second insert is refused, and nothing of it is written:

Duplicate value for unique index "Email_1" on bucket "Users": Email = "dalek@example.com"
is already used by document "u1". Change the value, update that document instead, or
delete it first.

A client receives this refusal with its own error code, DUPLICATE_KEY (0x0F), so “insert, and on a duplicate update instead” can branch on the code rather than on the text.

The same bucket, declared with a field line instead of the indices option:

create bucket Users { type: hash, Email string unique }

unique is the one field keyword that does anything; other keywords on a field line are accepted and noted, not enforced. alter bucket cannot add a unique index to an existing bucket — declare it when you create the bucket.

What unique means, exactly:

• One bucket. Uniqueness is checked within the bucket that declares the index — never across buckets, and never across databases. Two buckets can each hold dalek@example.com.
• Numbers are values, not spellings. 30, 30.0 and int32(30) are one value, and so are 0 and -0.0.
• A missing field is null. One document may lack the field (or hold null); a second one is refused as Email = null, the same as any other value.
• Updating a document keeps its own value. Rewriting the holder, or updating its other fields, is not a duplicate.
• A batch is all or nothing. If any document of a batchInsert — including two documents of the same batch carrying one value — would break the rule, none of the batch is written, and the refusal names the document and its position so you can re-send the batch without it.
• Update operators are checked too. $set, $inc and the rest reach the same check; a multi-update that moves one document off a value and another onto it, in that order, is accepted.

On a cluster the bucket’s owner refuses a duplicate before it is logged, so a refused write costs one lookup and is never shipped to the replicas.

Compound indexes

create bucket Orders { type: hash, indices: [{fields: ["CustomerID", "CreatedAt"]}] }
db.Orders.search({CustomerID: "c1"})
db.Orders.search({CustomerID: "c1", CreatedAt: ISODate("2026-09-29T10:00:00Z")})

A compound index serves a search on a prefix of its fields, in declaration order: (CustomerID) and (CustomerID, CreatedAt) above. A search on (CreatedAt) alone does not use it. A compound index can be unique: then the combination of values is unique, not each value.

What search does, and how it says so

search takes the same filter as find. When an index’s first fields are fixed by equality in the filter, that index serves the search; otherwise the search scans the bucket, as find does. Either way every predicate of the filter is checked against each document, so the answer is the one find gives. The response’s index_used field says which it was — the index’s name, such as "CustomerID_1_CreatedAt_1", or "none — full scan". A client program can read it; the shell prints the results only.

When two indexes fit equally well, the one covering more of the filter wins, then a unique one, then the one declared first — the choice is the same every time. find always reports "none — full scan": it has no index path.

TTL indexes

create bucket Sessions { type: hash, indices: [{field: "CreatedAt", expire_after_seconds: 3600}] }
db.Sessions.insert({_id_: "s1", CreatedAt: now()})

A document expires expire_after_seconds after the time its field holds. The field may hold a date, or a number of seconds since 1970 (Unix time). A document whose field is missing, null, a string, a boolean or anything else never expires.

Expiry is background work. A sweeper on each node runs every HOARDDB_TTL_SWEEP_SECONDS (default 60; 0 turns it off) for up to HOARDDB_TTL_SWEEP_BUDGET_SECONDS (default 5), so:

• An expired document can still be read until the next sweep removes it — up to one interval, plus however long a backlog takes to drain.
• Refreshing a document keeps it. Each expiry is checked again when it is applied, so a session whose CreatedAt you updated after the sweeper looked at it is left alone.
• An expired document takes its store fields with it, and the files of its blob fields, exactly as a delete does. A document whose store fields hold too many elements to delete in one step is not expired: it stays, whole, and the sweeper tries it again every pass. The server log names the document and the command that empties its fields, and hoarddb_ttl_sweep_documents_total{outcome="skipped"} counts each attempt.
• On a cluster, only the bucket’s owner sweeps, and each expiry is an ordinary replicated delete; a replica that is down does not slow the sweep.
• A bucket that expires documents faster than the sweeper deletes them grows. One node deleted about 50,000 expired documents a second in our measurement, and the owner of a three-node cluster about 23,000; plan a TTL bucket’s write rate well under that.

hoarddb_ttl_sweep_documents_total, hoarddb_ttl_sweep_budget_exhausted_total and hoarddb_ttl_backlog_estimate show the sweeper at work.

When an index is not ready

describe shows an index that cannot serve search, and search refuses it with the reason; find, count and dump are never affected:

• building — the node is rebuilding the index from the documents, once, after an upgrade from a version whose indexes did not record missing fields. It happens while the node starts. Retry, or use find.
• conflicted — a unique index found one value in two documents (a dump restored from elsewhere, a copy moved between nodes). Nothing is deleted or chosen for you: the refusal names both documents. Change or delete one of them, then restart the node; the index is rebuilt and becomes ready. Writes that would add a third document with that value are still refused.
• unreadable — the index was declared by a newer version of hoardDB, and this version cannot keep its rules. It is kept unchanged. A bucket with an unreadable unique index refuses writes until the newer version runs again.

hoarddb_indexes{state} counts the node’s indexes in each state.

What an index costs

An index opens no file and starts no thread. Each document costs one extra key per index — about the length of the indexed value plus the document’s key plus 13 bytes — and each write costs one lookup more per unique index whose value changes. A bucket with twelve compound indexes writes twelve keys per document and carries roughly 2 KB of index declarations in memory while it is open: declare many indexes on a bucket you will create a million of only if you mean it.

Dump and restore

dump records each index’s declaration — its fields, name and whether it is unique or TTL — and restore declares the new bucket from it, so a restored unique index is still unique. A dump written by an older version carries only field names and restores as plain single-field indexes, which is what it recorded.

Source: docs/user/indexes.md in the repository.