Pre-1.0 hoardDB is pre-1.0. Expect breaking changes.

Store Types

Each bucket is backed by exactly one store type. The store type determines the data structure and access semantics.

Quick Reference

FIFO (First In, First Out)

Append-only queue. Items are pushed to the tail and popped from the head. O(1) push and pop.

When to Use

• Message queues
• Event streams and log ingestion
• Comment threads (chronological)
• Time-series data
• Job queues (FIFO processing order)

Operations

-- Push: append to tail
db.Comments.push({Author: "Bob", Text: "Nice photo!"});

-- Pop: remove and return from head (oldest first)
db.Comments.pop();
-- Returns: {Author: "Bob", Text: "Nice photo!"}

-- Peek: look at head without removing
db.Comments.peek();

-- Peek N: look at next N without removing
db.Comments.peek(5);

-- Length: number of items
db.Comments.length();

How It Works

Data is stored in fixed-size segment files (default 64 MB) on disk. Each entry is length-prefixed with an 8-byte header followed by raw BSON bytes:

Push → [seg_001.fifo] → [seg_002.fifo] → [seg_003.fifo] ← Pop (head)

• O(1) push: Append to end of current segment (sequential I/O)
• O(1) pop: Read 8-byte header, read that many bytes forward
• O(1) cleanup: When a segment is fully consumed, unlink it

On-Disk Layout

/data/MyApp/Comments/
├── state.bson           # head/tail pointers, segment count
├── seg_00001.fifo       # segment files, fixed size
├── seg_00002.fifo
└── seg_00003.fifo

LIFO (Last In, First Out)

Stack. Items are pushed and popped from the same end. O(1) push and pop. Same segment file architecture as FIFO but reads backward from the tail.

When to Use

• Undo stacks
• Session state (latest-first access)
• Recent items (most recent first)
• Browser history
• Photo feeds (newest first)

Operations

-- Push: append to tail (same as FIFO)
db.Sessions.push({IP: "192.168.1.1", UA: "Chrome"});

-- Pop: remove and return from tail (newest first)
db.Sessions.pop();
-- Returns: {IP: "192.168.1.1", UA: "Chrome"}

-- Peek: look at tail without removing
db.Sessions.peek();

-- Length
db.Sessions.length();

FIFO vs LIFO

The API is identical — the store type determines behavior internally.

Hash (Key-Value)

Key-value hash table backed by BadgerDB. O(1) average-case lookup, insert, and delete.

When to Use

• User profiles and accounts
• Configuration storage
• Session data
• Any exact key-value lookup
• Metadata storage

Operations

-- Insert
db.User.insert({_id_: "jane_doe", Name: "Jane", Email: "jane@example.com"});

-- Find by key
db.User.find({_id_: "jane_doe"});
-- Returns: {Name: "Jane", Email: "jane@example.com"}

-- Find with filter
db.User.find({Email: "jane@example.com"});

-- Update
db.User.update({_id_: "jane_doe"}, {$set: {Name: "Janet"}});

-- Delete
db.User.delete({_id_: "jane_doe"});

-- Count
db.User.count();

How It Works

Each entry is stored in BadgerDB (LSM-tree) with a namespaced key format:

hash:{db}:{bucket}:{key}

Values are BSON-encoded documents. BadgerDB’s write-ahead log provides durability.

On-Disk Layout

/data/MyApp/User/
└── badger/
    ├── 000001.ldb    # BadgerDB LSM files
    ├── 000002.ldb
    └── ...

B+Tree (Sorted Key-Value)

Sorted key-value store backed by BadgerDB. Keys are stored in sorted order, enabling efficient range queries and ordered iteration.

When to Use

• Leaderboards and rankings
• Alphabetical or numerical listings
• Time-range queries on indexed fields
• Any data requiring ordered iteration

Operations

-- Insert with sorted key
db.Leaderboard.insert({Score: 100, User: "jane"});
db.Leaderboard.insert({Score: 85, User: "bob"});
db.Leaderboard.insert({Score: 92, User: "alice"});

-- Get by exact key
db.Leaderboard.get({Score: 92});

-- Range query
db.Leaderboard.range({Score: {$gte: 90}});
-- Returns all entries with Score >= 90

-- First (lowest key)
db.Leaderboard.first();

-- Last (highest key)
db.Leaderboard.last();

-- Length
db.Leaderboard.length();

How It Works

Keys are encoded to preserve sort order:

• String keys: Raw bytes (lexicographic sort)
• uint64: 8 bytes, big-endian
• int64: Flip sign bit, then 8 bytes big-endian
• float64: math.Float64bits, then big-endian

Key format: btree:{db}:{bucket}:{encoded_key}

On-Disk Layout

Same as Hash — BadgerDB instance per bucket:

/data/MyApp/Leaderboard/
└── badger/
    └── ...

Heap (Priority Queue)

Priority queue backed by BadgerDB with an in-memory index for O(1) peek. Highest-priority item is always accessible without removal.

When to Use

• Task queues (most important first)
• Alert prioritization
• Job scheduling
• Any “most important first” access pattern

Operations

-- Push with priority
db.Alerts.push({priority: 10, Msg: "Disk full", Host: "node-1"});
db.Alerts.push({priority: 5, Msg: "High CPU", Host: "node-2"});
db.Alerts.push({priority: 8, Msg: "Network latency", Host: "node-3"});

-- Pop: remove and return highest priority
db.Alerts.pop();
-- Returns: {priority: 10, Msg: "Disk full", Host: "node-1"}

-- Peek: see highest priority without removing
db.Alerts.peek();
-- Returns: {priority: 8, Msg: "Network latency", Host: "node-3"}

-- Length
db.Alerts.length();

How It Works

Keys encode inverted priority so the highest-priority entry sorts first in BadgerDB:

heap:{db}:{bucket}:{max_uint64 - priority}:{entry_id}

An in-memory max-heap provides O(1) peek. Rebuilt from BadgerDB on startup.

Complexity

Blob (Binary Storage)

Raw binary data storage. Blobs are stored as files on the filesystem within the L1 bucket’s directory.

When to Use

• File storage (images, PDFs, audio, video)
• Binary assets alongside metadata
• Any data where raw bytes are the primary content

Operations

-- Put: store binary data
db.Photos.put({data: <bytes>, name: "sunset.jpg", type: "image/jpeg"});
-- Returns: "0192a3b4-c5d6-7890-abcd-ef12345678a1"

-- Get: retrieve binary data + metadata
db.Photos.get({_id_: "0192a3b4-c5d6-7890-abcd-ef12345678a1"});
-- Returns: {data: <bytes>, name: "sunset.jpg", type: "image/jpeg", sha256: "..."}

-- Info: metadata only (no data transfer)
db.Photos.info({_id_: "0192a3b4-c5d6-7890-abcd-ef12345678a1"});
-- Returns: {name: "sunset.jpg", type: "image/jpeg", size: 2048576, sha256: "..."}

-- Delete
db.Photos.delete({_id_: "0192a3b4-c5d6-7890-abcd-ef12345678a1"});

-- Stream: get data as io.Reader (for large files)
db.Photos.stream({_id_: "0192a3b4-c5d6-7890-abcd-ef12345678a1"});

How It Works

• Blob files are raw binary on disk — no encoding, no transformation
• SHA-256 checksums in _index.bson catch corruption
• Streaming API (PutStream, GetStream) avoids loading large files into memory
• References in BSON documents store the DbID (UUID v7), not the blob data

On-Disk Layout

/data/MyApp/Photos/
├── _index.bson                              # DbID → metadata mapping
├── 0192a3b4-c5d6-7890-abcd-ef12345678a1     # raw binary blob
├── 0192a3b4-c5d6-7890-abcd-ef12345678a2
└── ...

Referencing Blobs

BSON documents store blob DbIDs, not the data itself:

{
  "_id_": "0192a3b4-c5d6-7890-abcd-ef1234567895",
  "Caption": "Sunset over the hills",
  "Photos": [
    "0192a3b4-c5d6-7890-abcd-ef12345678a1",
    "0192a3b4-c5d6-7890-abcd-ef12345678a2"
  ]
}

To retrieve the actual image, call Get(DbID) on the blob store.

L1 vs L2 Bucket Decision Guide

Buckets support hierarchical nesting. L1 is top-level, L2 is a sub-bucket inside an L1.

Levels

• L1 — Top-level bucket. Like a SQL table or MongoDB collection. Independently distributed on the hash ring.
• L2 — Sub-bucket inside an L1. Like an embedded document. Lives inside the parent’s directory.
• LN — Deeper nesting. Keep shallow (L1–L3 covers most use cases).

Decision Rules

The Critical Rule

Anything that can grow independently and unpredictably should be L1.

If a child collection could grow larger than its parent or break hash ring balance, prefer L1. L2 is safe for data that is naturally bounded by its parent.

L1  User                          ← top-level bucket
    └── L2  User.jane.friends     ← sub-bucket (bounded by user)

L1  Post                          ← independent top-level bucket
    └── L2  Post.comments         ← sub-bucket (bounded by post)

L2 Overflow Protection

L2 buckets have configurable hard limits:

bucket_limits:
  l2_soft_limit: 10000    # warning logged
  l2_hard_limit: 100000   # write rejected with error

When the hard limit is hit:

ERROR: bucket "Post.Comments" exceeded hard limit (100,000 entries).
Suggestions: archive old entries, split into multiple L1 buckets,
or increase the limit in config.

L2 → L1 Migration

When an L2 bucket approaches its limit, the application can migrate:

1. Create a new L1 bucket (e.g., Post_Comments_{postId})
2. Pop all entries from the L2 and push to the new L1
3. Update the parent’s reference to point to the new L1
4. Delete the empty L2

Example: PhotoJournal Schema

create database PhotoJournal;

create bucket User {
    _id_      uuid auto,
    FirstName string required,
    LastName  string required,
    Email     string unique,
    Posts     []uuid
};

create bucket Post {
    _id_      uuid auto,
    Caption   string
};

create bucket UserFriends {
    type: heap,
    indices: []
};

create bucket PostPhotos {
    type: blob,
    indices: []
};

create bucket PostComments {
    type: fifo,
    indices: []
};

Referential design: Cross-bucket references (e.g., User.Posts referencing Post entries, or a user’s Friends collection) are stored as arrays of UUIDs and resolved at query time via hash ring lookup. Store type selection is per-bucket, not per-field: create a separate bucket when you need a different store type (for example, maintain a dedicated heap or blob bucket for related data).

On-disk layout:

/data/PhotoJournal/
├── User/                          ← hash bucket
│   ├── badger/                    ← entry documents (fields)
│   └── _index.bson                ← Email index
│
├── Post/                          ← hash bucket
│   ├── badger/                    ← entry documents
│   └── _index.bson
│
├── UserFriends/                   ← heap bucket (separate)
│   └── badger/
│
├── PostPhotos/                    ← blob bucket (separate)
│   ├── _index.bson
│   └── 0192a3b4-...
│
└── PostComments/                  ← fifo bucket (separate)
    ├── state.bson
    └── seg_00001.fifo

Source: docs/user/store-types.md in the repository.