Introduction

fluent31 is an embedded key-value database engine in Rust whose query surface is WebAssembly. You install code into the database, and the engine runs it as reads, as transactions, and as triggers — against an LSM store built to move as few bytes as possible.

Your code runs inside the database

This is the feature everything else is arranged around. A module is a WASM binary stored in the database like any other value, invoked by name, executing next to the data against a kernel-style syscall ABI: get, get_for_update, put, delete, batched scans, fuel and memory limits. Its exports decide what it can be — and one binary can be several at once:

• A query runs read-only at one pinned snapshot. An aggregate over a million keys returns its five numbers instead of a million values, and the whole answer comes from one consistent state.
• An executor runs in a fresh optimistic transaction: exit 0 commits, anything else aborts, and a commit conflict re-runs the attempt against a fresh snapshot. Invariants that need read-then-write — uniqueness, non-negative balances, dense id allocation — hold under concurrency without a lock anywhere.
• A trigger consumer is invoked by the engine after every committed write into a key range you bind it to. Indexes, materialized views, live aggregates, changefeeds and cascades maintain themselves, whoever did the writing, with events captured durably alongside the write and effects applied exactly once.
• A descriptor turns the module into API. Export describe and installing it adds a typed, documented GraphQL field to the running server's schema.

Because module bytes are stored as ordinary versioned keys, your code is durable with the data, recovered with it, copied into every fork of it, and time-travelled alongside it — query_at runs the code as it was at a past sequence number against the data as it was then. And because the same bytes can run without being installed, a migration is a one-shot executor: one atomic transaction, no deployment, no trace but its writes.

Modules are sandboxed: fuel-metered, memory-capped, no WASI, no clock, no randomness, no imports but the host's own. What that buys is bounded, deterministic execution — the engine stays predictable no matter what the module does. Extending with WASM is the section on all of it: the roles, the two API surfaces, and what to build with them.

Built to move as few bytes as possible

The storage engine is an LSM tree with WiscKey-style key-value separation: values at or above value_threshold live in an append-only log and the tree holds pointers. Compaction therefore relocates pointers rather than payloads, so write amplification is governed by key volume, not value volume, and the index stays small enough to keep resident — every fragment's bloom filter and index in memory, for the whole dataset.

• Group commit. Under the default SyncMode::Always, concurrent writers share one value-log fsync and one WAL fsync per cycle. An ack still means fsynced; the cost is amortized across everyone committing at that moment, and stats reports how many fsyncs were saved.
• io_uring on Linux, probed at open with an automatic fallback to portable IO elsewhere.
• Batched value resolution. A scan over large values resolves value-log pointers in windows — one batched read per file — so a range read costs one IO round per group rather than one per entry.
• Lazy leveling. Tiered merges on the upper levels, one leveled run at the bottom: fewer rewrites for write-heavy ranges, a compact bottom for reads.
• No round trips. The fastest query is the one that never crosses a network boundary. Modules put the loop next to the data; triggers move the work off the read path entirely, so a report can be a single get of a number a trigger already folded.

A typed API you did not write

Run the server and the store gets a GraphQL plane: get, scan, put, writeBatch, module invocation, trigger and fork administration, engine stats — and GraphiQL to explore it. The part worth noticing is that the schema is not fixed. Every installed module that describes itself contributes its own root field with real argument and output types, and the schema is rebuilt and hot-swapped on install and uninstall, so shipping a module ships an API.

mutation { placeOrder(customer: "you", amountCents: "4200") { id customerTotalCents } }
query    { topCustomers(limit: 3) { customer orders totalCents avgCents } }
subscription { orderFeed { seqno event { id record } query { snapshotSeqno } } }

Subscriptions stream committed changes, raw or typed, and every item carries the whole query root pinned at the exact state in which that change became visible — so a consumer can read consistent context for an event without racing the writer. Forks are addressable too: each one is a full instance at /graphql/<instanceId> with its own modules, triggers and schema.

The pieces

• Engine — LSM storage with key-value separation, MVCC snapshots, optimistic transactions, io_uring on Linux.
• Modules — WASM installed in the database, run as queries, executors or trigger consumers.
• Triggers — a module bound to a key range, invoked after every committed write into it. Events are durable with the write and effects are exactly-once.
• Forks — fork("name") publishes a complete, consistent, hard-linked copy of the database at a cost proportional to the file count, not the data. Open it for a writable copy-on-write clone; pins make a point fork-able later.
• Journal — opt-in and off the commit path. An independent mutation log from which a fresh database is rebuilt when the store directory is lost.
• Server — one process, one store, two planes: GraphQL for typed and admin operations with live subscriptions, and a replication join point for full replicas and key-range edge caches.

The surfaces

What it is not

• Not a query language. There is no parser, no columns, no joins — key layout and module code take their place. Translation guide covers the relational vocabulary if you want the comparison.
• Not point-in-time recovery. Forks are named cuts, not continuous log archiving.
• Not a public-facing sandbox. The WASM limits protect reliability and integrity. Authentication and authorization are a layer you put in front.
• Single-node today. A store is one directory, locked by one process. Replicas are read-only followers.

The documents

This site is the usage documentation and its source. The same pages are generated as one markdown file each, under docs/p/, indexed by llms.txt — that is the copy to point an agent at, because a fetcher cannot address a page here. The specs below go beneath what this site describes; when a detail matters they win, and when the docs and the code disagree, the code wins.

Install

Stable Rust, one optional wasm target, two cargo features.

You need stable Rust (2021 edition). For modules, add the wasm target:

rustup target add wasm32-unknown-unknown

Build and test the workspace:

cargo build --workspace --release
cargo test --workspace

The example modules live in a separate workspace under guests/ and only build for wasm32:

cargo build --manifest-path guests/Cargo.toml --target wasm32-unknown-unknown --release
# artifacts: guests/target/wasm32-unknown-unknown/release/<name>.wasm

Cargo features

Platforms

Linux, where io_uring is probed automatically with a fallback to portable IO, and macOS with portable IO. Docker's default seccomp profile blocks io_uring, so run with --security-opt seccomp=unconfined or set io_backend = Std.

Tutorial · Step 1 of 6

Your first store

One command, no code: a running database you can write to and read back.

$ cargo run -p fluent-cli -- ./data
fluent31> put hello world
OK  (3.02 ms)
fluent31> get hello
"world"  (28.7 µs)

The directory did not have to exist. Opening it created the store, took an exclusive lock on the directory, recovered anything a previous run had left, and started the background threads that flush, compact and commit. Every command prints its own wall-clock latency.

Write a few records and read the range

fluent31> put user/ada engineer
fluent31> put user/grace admiral
fluent31> put user/katherine mathematician
fluent31> scan user/ user0
   1) "user/ada" => "engineer"
   2) "user/grace" => "admiral"
   3) "user/katherine" => "mathematician"

Two things happened there that are worth naming, because everything else builds on them.

Keys sort bytewise, which is why the three came back in that order and why a range read is the primitive rather than a special operation. A scan takes [lo, hi) — inclusive low, exclusive high — so scanning a prefix means scanning to that prefix with its last byte incremented: user/ ends at user0, because 0 is the byte after /. A - stands for an open end, so scan - - walks the whole store.

Enough of the shell to be useful

fluent31> count user/ user0            # how many, without printing them
fluent31> scan user/ user0 --rev --limit 2
fluent31> del hello                    # the scratch key from a moment ago
fluent31> begin                        # a transaction; the prompt shows (txn)
fluent31> tlock counter                # read + conflict-check at commit
fluent31> tput counter 1
fluent31> commit
fluent31> stats                        # levels, cache, group-commit amortization
fluent31> help

Byte arguments are plain UTF-8 or hex:DEADBEEF, and output prints printable bytes quoted and everything else as hex:.

What is on disk

./data now holds LOCK, CURRENT, a manifest, and a write-ahead log; table and value-log files appear as data is flushed out of memory. The lock is the part to remember: one process at a time owns a store directory. That is why the next step starts by leaving this one.

fluent31> exit

Tutorial · Step 2 of 6

Serve it over GraphQL

The same store on a network surface, with a schema, an explorer and live subscriptions — no code yet.

Make sure the shell from the previous step has exited: it holds the store's lock, and the server needs it.

$ cargo run -p fluent-server -- ./data --store-name prod
INFO db{dir=./data store=prod instance=6065…}: fluent31::db: store opened backend="io_uring" seqno=0 …
INFO fluent_server: serving graphql: /graphql (GraphiQL at /, …) listen=127.0.0.1:8317
INFO fluent_server: serving replication: … listen=127.0.0.1:8428 store=prod instance=6065…

The log is stderr; RUST_LOG sets the level (Operations).

--store-name is persisted in the store on first use, so it is passed once and then omitted. It fixes the store's identity, and it is what opens the replication join point; without a name the GraphQL plane still serves and that port stays closed.

Open http://127.0.0.1:8317/ for GraphiQL, which has the whole schema and its documentation built in. Everything below works there or over curl.

Reading

curl -s http://127.0.0.1:8317/graphql -H 'content-type: application/json' \
  -d '{"query":"{ get(key:{text:\"user/grace\"}) { text } }"}'
# {"data":{"get":{"text":"admiral"}}}

Keys and values are raw bytes, so an input takes exactly one of {text}, {base64} or {hex}, and an output offers text (null when the bytes are not UTF-8), base64, hex and len. Sequence numbers and byte counts travel as strings, because they are 64-bit.

Writing and scanning

mutation { put(key: {text: "user/alan"}, value: {text: "logician"}) }

query {
  scan(prefix: {text: "user/"}, limit: 10) {
    pairs { key { text } value { text } }
    hasMore
    nextAfter { text }
  }
}

scan takes either a prefix or an explicit lo/hi pair, and pages by cursor: pass nextAfter back as after to continue. There is no offset. Every read field in one query operation runs at a single pinned snapshot, so a multi-field query cannot see a write land halfway through.

Watching it change

subscription {
  changes(lo: {text: "user/"}, hi: {text: "user0"}) {
    kind key { text } value { text }
  }
}

The stream opens with one ATTACHED marker — the boundary — and then delivers every committed change into the range, in order. Run a put from another window and watch it arrive.

Administration is in the same schema

stats, modules, triggers, forks and pins are query fields; fork, pin, installModule, createTrigger, flush, compactAll, gcVlog and syncWal are mutations. There is no separate admin channel to learn.

Every field used so far is built in. The next step adds one that is not.

Tutorial · Step 3 of 6

Your first module

Install code into the database and call it by name. This is the feature the rest of the engine is arranged around.

Modules build for WebAssembly, so add the target once:

rustup target add wasm32-unknown-unknown

The crate

Guest modules live in their own workspace under guests/, because they only build for wasm32. Create guests/count/Cargo.toml and add "count" to the members list in guests/Cargo.toml:

[package]
name = "count"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
fluent-guest = { path = "../../crates/fluent-guest" }

use fluent_guest::Fail;

#[fluent_guest::query]
fn count(prefix: Vec<u8>) -> Result<String, Fail> {
    if prefix.is_empty() {
        return Err(Fail::new(2, "empty prefix"));
    }
    let n = fluent_guest::scan_prefix(&prefix)
        .map_err(|_| Fail::new(3, "scan failed"))?
        .count();
    Ok(n.to_string())
}

The attribute is what makes this a module: it exports an entry named query, and the export is the role. A query runs read-only against one pinned snapshot — a write from inside it returns EROFS. The function itself may not be called query, since that is the name the macro generates. Distinct Fail codes are the convention: the caller can tell one failure from another.

Build and install

cargo build --manifest-path guests/Cargo.toml --target wasm32-unknown-unknown --release
# guests/target/wasm32-unknown-unknown/release/count.wasm

$ cargo run -p fluent-cli -- ./data
fluent31> install count guests/target/wasm32-unknown-unknown/release/count.wasm
fluent31> query count user/
"4"

The scan ran inside the database. Nothing but the answer crossed a boundary, and the whole count came from one consistent state — which is what makes it different from looping over a range from the outside.

Make it an API

A module that describes itself becomes a typed GraphQL field. Add the descriptor and rebuild:

fluent_guest::fluent_describe!(r#"{
  "kind": "query",
  "description": "Number of keys under a prefix.",
  "output": "String!"
}"#);

Install it through the server this time — installModule runs describe and rejects a descriptor that does not hold up:

mutation Install($w: BytesInput!) {
  installModule(name: "countKeys", wasm: $w) { typed schemaError }
}
# variables: {"w": {"base64": "<base64 of count.wasm>"}}
# -> { "typed": true, "schemaError": null }

The schema was rebuilt and hot-swapped while the server kept running. Reload GraphiQL and the field is there, documented:

query { countKeys(input: {text: "user/"}) }

The field is named after the name you installed under, not the crate — countKeys, not count. A descriptor that declares args gets typed arguments instead of the raw input, and one that declares an output object gets a selectable result type.

One last thing worth knowing before moving on: module bytes are stored in the database as ordinary versioned keys. Your code is written durably with the data, recovered with it, copied into every fork of it, and readable at a past sequence number alongside it.

Tutorial · Step 4 of 6

Your first trigger

Bind a module to a key range and the engine invokes it after every committed write into that range. Derived data stops being the writer's problem.

This step uses a module the repository ships, guests/customer_index, which maintains a secondary index over order records:

orders/<id, 8 digits>         the record: JSON with a "customer" field
idx/customer/<name>/<id>      the index entry
idx/order/<id>                a back-pointer: what this record was last indexed as

Install and bind

fluent31> install customer_index guests/target/wasm32-unknown-unknown/release/customer_index.wasm
fluent31> mktrig customerIndex customer_index orders/ orders0
fluent31> triggers
   customerIndex  customer_index  [orders/, orders0)  keys  pending 0

The mode was not chosen — it was detected. The module exports on_touch, so the trigger runs in keys mode: it will be handed the keys that were touched and asked to reconcile them.

Write an order

As an ordinary put. Nothing about this write knows an index exists:

fluent31> put orders/00000001 {"customer":"acme"}
fluent31> scan idx/ idx0
   1) "idx/customer/acme/00000001" => ""
   2) "idx/order/00000001" => "acme"

Looking up one customer's orders is now a prefix scan of idx/customer/acme/. The index is ordinary keys — scannable, forkable and replicable like everything else.

Change it

fluent31> put orders/00000001 {"customer":"globex"}
fluent31> scan idx/ idx0
   1) "idx/customer/globex/00000001" => ""
   2) "idx/order/00000001" => "globex"

The stale acme entry is gone. A keys-mode event carries the key and nothing else — no old value — so the module found what to remove through its own back-pointer. That is the shape every keys-mode consumer takes: read current state, make the derived state match it, and converge no matter how the events arrive. Deleting the order removes both keys the same way.

Three things to carry forward

• It is asynchronous. Derived state trails the base data by the backlog. If a scan looks stale, the drain has not happened yet; triggers reports pending and the last error per trigger.
• Registration does not backfill. Keys already in the range when the trigger was created fire no events. Existing data is indexed by scanning it deliberately, or by re-writing the range.
• Trigger writes never fire triggers. The index entries above generated no events of their own, so cascades cannot loop.

What the engine guarantees in exchange is worth stating plainly: the event was committed in the same atomic batch as the write that caused it, and the module's writes commit together with consuming that event. Derived state built this way cannot drift, even across a crash.

Tutorial · Step 5 of 6

Embed it in Rust

The shell and the server are wrappers around a library. This is the library.

use fluent31::{Db, Options, WriteBatch};

let db = Db::open("./data", Options::default())?;

db.put("user/1", "ada")?;
assert_eq!(db.get(b"user/1")?.as_deref(), Some(&b"ada"[..]));
db.delete(b"user/0")?;                       // fine whether or not it existed

let mut b = WriteBatch::new();                 // atomic: one contiguous seqno range
b.put("user/2", "grace");
b.delete("user/3");
db.write(b)?;

for kv in db.iter(Some(b"user/"), Some(b"user0"), false)? {
    let (key, value) = kv?;
}

Db is Send + Sync, so one handle is shared across threads as Arc<Db> rather than opened twice — a second open of the same directory fails on the lock. Dropping it stops and joins the background threads.

Transactions

loop {
    let mut txn = db.begin();
    let n = txn.get_for_update(b"counter")?.map(decode).unwrap_or(0);
    txn.put("counter", encode(n + 1))?;
    match txn.commit() {
        Ok(()) => break,
        Err(fluent31::Error::Conflict) => continue,   // nothing written; run it all again
        Err(e) => return Err(e),
    }
}

Transactions are optimistic: no locks are taken, readers never block, and get_for_update is what puts a key in the conflict set. A commit that loses the race returns Conflict having written nothing, and the fix is always to re-run the whole read-modify-write — which is also why an executor module must be a pure function of its input and the data.

The rest of the surface

db.install_module("count", &wasm)?;      db.query("count", b"user/")?;
db.create_trigger("idx", "customer_index", Some(b"orders/"), Some(b"orders0"))?;
db.snapshot();                              // a consistent read point
db.fork("pre-migration")?;                   // a whole-database branch
db.subscribe(b"orders/", Some(b"orders0"))?;  // the change stream
db.stats();

Everything done through the shell and the server in the previous steps is available here, because that is what both of them call.

Next

Each step so far has shown one surface on its own. The last one puts them in a single program — one that installs, binds, contends and forks, and checks its own answers.

Tutorial · Step 6 of 6

The whole thing at once

One program that installs modules, binds a trigger, drives an executor under contention, waits for derived state and rehearses a destructive change on a fork — asserting every step.

The previous steps each showed one surface. This is the shape a real driver has, and it is a file in the repository rather than prose: crates/fluent31/examples/walkthrough.rs. The test suite compiles it and it asserts its own results, so it cannot quietly stop being true.

Run it

cargo run -p fluent31 --example walkthrough

Modules are ordinary crates compiled to wasm32-unknown-unknown, and the example builds them on first run. In a project of your own that step is yours: cargo build --manifest-path guests/Cargo.toml --target wasm32-unknown-unknown --release, then install the .wasm bytes it produces.

What it does

It uses two modules the repository already ships. place_order is an executor: one transaction allocates an order id from a counter key, writes the record and folds the amount into the customer's running total. customer_index is a trigger consumer that keeps a secondary index over those records. Neither knows about the other.

db.install_module("place_order", &guest_wasm("place_order"))?;
db.install_module("customer_index", &guest_wasm("customer_index"))?;

let mode = db.create_trigger("customerIndex", "customer_index",
                            Some(b"orders/"), Some(b"orders0"))?;

The mode is not an argument. customer_index exports on_touch, so the engine registers it in keys mode; a module exporting on_apply would get changes mode instead. The call returns what was detected.

The two loops you have to write

Everything else in the file is ordinary Rust. These two are the ones that are easy to leave out and painful to leave out.

Waiting for derived state. A trigger runs after the write that caused it has already committed, so a read taken the moment a write returns sees the state before the trigger ran. There is no synchronous "run the triggers now":

let deadline = Instant::now() + Duration::from_secs(30);
loop {
    let all = db.list_triggers()?;
    let mine: Vec<_> = all.iter().filter(|t| names.contains(&t.name.as_str())).collect();
    if let Some(t) = mine.iter().find(|t| t.last_error.is_some()) {
        panic!("trigger {} is stuck: {:?}", t.name, t.last_error);   // never transient
    }
    if mine.iter().all(|t| t.pending == 0) { break; }
    assert!(Instant::now() < deadline, "triggers did not drain in 30s");
    std::thread::sleep(Duration::from_millis(5));
}

last_error is fatal rather than transient: a module that fails holds its batch instead of dropping it, so a queue that stops moving does not start again on its own.

Retrying a conflict. execute runs the module inside a transaction and re-runs it on a commit conflict — but only execute_retries times, three by default. Under real contention those get spent and Conflict reaches the caller, having written nothing:

loop {
    match db.execute("place_order", input.as_bytes()) {
        Ok(out) => return serde_json::from_slice(&out)?,
        Err(Error::Conflict) => continue,        // nothing was written; run it again
        Err(e) => return Err(e),
    }
}

Four threads placing twenty-four orders against a single counter key spend the engine's three attempts routinely — a typical run reports around half the calls arriving here. Without this loop that is not a slow path, it is lost work.

What it proves

Every line of output is an assertion that held:

== install the modules
   modules: customer_index, place_order
   place_order describes itself as kind="execute" output="PlacedOrder!"

== bind the trigger over [orders/, orders0)
   mode detected from the module's exports: keys

== one order, through the executor
   {"amountCents":1250,"customer":"acme","customerOrders":1,"customerTotalCents":1250,"id":1}
   index entry: idx/customer/acme/00000001

== 4 threads x 6 orders, all retrying on conflict
   24 orders placed, 24 distinct ids, 14 reached the caller as Conflict

== the derived state agrees with the records
   acme: 9 orders, 3850 cents
   globex: 8 orders, 2800 cents
   initech: 8 orders, 3000 cents

== rehearse a destructive change on a fork
   forked at seqno 228 -> /tmp/.tmpAOxum2/archive/rehearsal
   fork: 1 keys left, parent: 26 — untouched

done: installed, bound, drained, contended and rehearsed.

Two of those are worth naming. Every order gets a distinct id even though every order increments the same counter, because the id is allocated inside the transaction — a losing attempt is discarded whole rather than leaving a gap or a duplicate. And the executor's own per-customer count, written by the module, matches the number of index entries written by an unrelated trigger.

Rehearsing on a fork

The last section deletes every order — on a copy. fork publishes a complete database directory by hard-linking what is already immutable, so the copy costs file count rather than data size, and opening it gives a writable clone:

let fork = db.fork("rehearsal")?;
let rehearsal = Db::open(&fork.path, Options::default())?;
// ... run the change here, against real data, with nothing at stake

This is what makes a migration checkable before it is real: run it on the fork, look at the result, and throw the fork away if it is wrong. Forks in practice covers the rest of the shape.

Where to go next

• Concepts explains why the engine behaves the way these steps showed — snapshots and the watermark, the consistency contract, what triggers guarantee, what a fork is.
• Extending with WASM is the whole of the module story: what each role means, the catalogue of what to build, the guest SDK and the host ABI, and how a module becomes a typed GraphQL field.
• Reference is the exact surface of each entry point: every Options field, the GraphQL schema, the shell commands, server configuration, replication, operations.
• Recipes are the worked shapes — aggregates, invariants, indexes, feeds, migrations, forks — each with the reference module that implements it.

The repository carries four more walkthroughs of the same kind, each asserting its own results:

cargo run -p fluent31 --example live_stats       # a live aggregate, checked against a full recount
cargo run -p fluent31 --example dynamic_index    # indexes created at runtime from a spec key
cargo run -p fluent31 --example cascade_delete
cargo run -p fluent31 --example claim            # N concurrent claimers, exactly one winner
scripts/demo-orders.sh                           # typed modules against a running server

Snapshots and seqnos

Every write gets a sequence number; every read happens at one. That single idea is the engine's whole notion of time.

Seqnos

Every write — a put, a WriteBatch, a transaction commit or an executor's writes — gets a contiguous range of sequence numbers (seqnos) and becomes visible all at once. db.seqno() is the latest committed seqno, the address of "now". A Snapshot reads at one seqno; a GraphQL query operation pins one for all its fields; a module invocation runs at one.

Readers never see a partial batch and never block writers: a reader at seqno s sees every version at or below s and nothing above it, however much has been committed since.

The watermark

A snapshot is a hold. The GC watermark is the seqno of the oldest live snapshot, and compaction keeps every version above it — for every key in the store, not only the keys that snapshot reads. Pins and subscriptions register the same way.

That is the one cost worth internalising: holding a snapshot open across a long job stalls version GC and value-log reclamation store-wide. Take one, read, drop it. The rules that follow from this are on The consistency contract.

The engine's own keyspace

Keys beginning with byte 0x00 belong to the engine: installed module bytes, trigger definitions and trigger queues live there. Reads and writes of those keys are rejected and scans clamp to the user keyspace — but because they are ordinary versioned keys underneath, your modules and triggers are written durably with the data, recovered with it, and copied into every fork of it.

Time travel

Any seqno still above the watermark is readable: db.snapshot_at(s) reopens that state, db.query_at(module, input, &snap) runs a module against it, and db.fork_at(name, s) cuts a whole database there. Because module bytes are versioned too, query_at travels code and data together. Below the watermark those addresses are gone — which is what a pin exists to prevent.

The consistency contract

MVCC is how the engine gives you consistent reads and optimistic transactions. It is not an application-level version store, and the rules below follow from that.

The rules

• Snapshots are operation-scoped. Take one, read, drop it. Its cost is store-wide: the GC watermark is the seqno of the oldest live snapshot, and nothing at or above it is reclaimed, for any key, not just the ones the snapshot reads.
• Seqnos are addresses, not ids. db.seqno() names the current state and stays resolvable only until GC passes it. Don't store seqnos in application data. A journal rebuild renumbers them wholesale, and a fork or restore mints a new store identity.
• Pins and forks are coarse, named cuts. Use them for a handful of deliberate points: before a migration, a staging clone, a rollback anchor. A pin is a durable store-wide GC hold; a fork is a whole database directory. Neither is priced per document, let alone per write.
• There is no retention policy. Old versions survive until the GC watermark passes them. There is no "keep N versions of this key".

If you need history, make it data

Bind a changes-mode trigger to the range. Every committed change (kind, key, seqno, and the value up to trigger_inline_value; larger values arrive key-only and you read them back) is delivered durably, in order, with exactly-once effect. Write it under keys you own:

doc/42                     current value       (what the app writes)
history/doc/42/<seqno>     one entry per write (what the trigger writes)

History is then scannable, replicable and live-tailable as a subscription, and you prune it yourself with a scan and a delete batch, on whatever schedule suits the data. Modules have no clock, so if entries need timestamps the writer puts them in the value. guests/order_feed is the reference shape.

Transactions

Optimistic, snapshot isolation, first committer wins.

commit() checks every key read with get_for_update and every key in the write set against the transaction's snapshot; a committed delete conflicts too. The loser gets Error::Conflict with nothing written, and the fix is to run the whole read-modify-write again. A plain get inside a transaction is a consistent read but not a conflict check, so use get_for_update for every key you base a write on.

The retry loop

loop {
    let mut txn = db.begin();
    let n = txn.get_for_update(b"seq")?.map(decode).unwrap_or(0);
    txn.put("seq", encode(n + 1))?;
    match txn.commit() {
        Ok(()) => break,
        Err(fluent31::Error::Conflict) => continue,
        Err(e) => return Err(e),
    }
}

Under SyncMode::Always, concurrent commits share fsyncs with plain writers through group commit. Validation and application happen as one atomic step against every other writer, including plain db.put.

Branching the database

fork("name") publishes a complete, consistent copy of the whole database — at a cost proportional to the number of files, not the amount of data.

Tables and sealed value-log files are immutable, so a fork hard-links them and copies only the growing head. Shared bytes exist once on disk, and divergence accrues only as parent and child compact away from the shared base. What you get is not an export or a dump: it is a database directory. Open it and you have a live, writable, copy-on-write clone — its own modules, its own triggers, its own data, its own identity — while the parent keeps serving, untouched.

What that makes cheap

• Rehearsal. Run the risky change against a fork first. Under the server every fork is a full instance at /graphql/<instanceId>, so "try it on production without trying it on production" is one mutation.
• Rollback anchors. Cut before the change; if it goes wrong, the rollback is a directory swap rather than a restore.
• Second environments. A staging clone with real data that cannot touch the original.
• Backups. A consistent snapshot on the same filesystem, instantly; copy the directory elsewhere at your leisure.

Pins: a cut you can take later

pin(name) durably marks the current seqno as still-materializable, so fork_at(name, seqno) can cut exactly there afterwards. A pin is cheap to take and costs retention while it is held — it holds the GC watermark for the whole store, like any snapshot.

Branches, not timelines

A fork contains exactly the history up to its cut and nothing the parent commits afterwards; forks branch, they do not follow. Each copy mints its own instance identity, which is how a replica notices that its master was replaced and re-attaches from scratch. And because a fork is a whole database, it is priced for a handful of deliberate cuts — not for per-record versioning, which is something you make out of data.

Reactive derived state

Bind a module to a key range and the engine invokes it after every committed write into that range. Derived data maintains itself, whoever did the writing.

Indexes, materialized views, running aggregates, changefeeds, referential cleanup: all of them are the same shape — data derived from other data, which goes stale the moment someone writes without updating it. A trigger removes that "without". Plain puts, batches, transactions, executors and every network surface all fire it; no writer has to know the derived state exists.

What the trigger writes is ordinary keys under a prefix you choose, so derived data is scannable, replicable and subscribable exactly like everything else. There is no separate index structure to learn.

Two contracts

The mode is chosen by which export the module carries, and it decides what your derived state is a function of.

• Keys mode reconciles. An event means "this key was touched — reconcile it", and re-touches of one key coalesce while a backlog exists. Right when the derived state is a function of current state, as an index is: read the key, upsert or remove the entry, converge.
• Changes mode folds. Every committed op arrives once, in commit order, carrying its kind and its value. Right when you need op kinds, ordering, or per-op deltas — feeds, exact aggregates, cascades — where coalescing would destroy information.

Why the derived state can be trusted

Derived data is only worth maintaining if it cannot silently diverge from what it is derived from, and that is a property of where the work happens rather than of how carefully the module is written. An event is captured inside the commit that caused it, so a write that survives a crash has an event waiting for it afterwards and a write that does not leaves nothing behind. Consuming that event happens inside the module's own transaction, together with whatever the module writes, so the two cannot come apart: an aggregate folded this way is exact rather than approximately right, however many times the attempt is retried.

Two consequences follow. Writes made by a trigger generate no events of their own, so no chain of triggers can loop or amplify — a cascade runs once. And the price of all of it is that the work is asynchronous: derived state trails the base data by whatever is queued, and nothing waits for it. A failure holds the queue rather than dropping it, which makes a broken module a visible backlog rather than a silent hole.

Code in the database

The query surface is WebAssembly. You install modules into the database and call them by name — as reads, as transactions, or as the consumers behind a trigger.

A module is a WASM binary stored in the database like any other value. Its exports are its roles — a read-only query, a transactional execute, an on_touch or on_apply trigger consumer, and an optional describe that turns the module into API — and one binary may carry several of them at once.

Why code lives here

• The computation runs next to the data. A report over a million keys returns its five numbers, not a million values, and it runs at one pinned snapshot, so the answer is internally consistent.
• The invariant belongs to the database, not to a client. An installed executor is the same logic on every surface — Rust, the shell, GraphQL — so a rule like "stock never goes negative" holds wherever the write came from.
• A module that describes itself becomes API. Export describe and installing the module adds its own typed field to the GraphQL schema, hot-swapped at install time.
• Module bytes are data. They live in the engine's own keyspace as ordinary versioned keys, so they are recovered with the store, copied into forks, and time-travelled by query_at — the store can answer "what code ran here".

The bargain

Modules are sandboxed. Fuel-metered and memory-capped, with no WASI, no clock and no randomness, they can import only the fluent host functions: get, get_for_update, put, delete, batched scans, output and log. Entropy and time are inputs, never ambient. That is the price of letting arbitrary code run in the write path, and it is what keeps the engine's own guarantees independent of any module. The limits protect reliability and integrity; authentication and authorization are a layer you put in front.

Roles and lifecycles

A module's exports decide what it is. Each role has its own input, its own execution context, and its own rules about what happens when it fails.

Required exports

"memory"     the guest's linear memory                  // required
"query"      () -> i32   read-only entry
"execute"    () -> i32   transactional entry
"on_touch"   () -> i32   keys-mode trigger consumer
"on_apply"   () -> i32   changes-mode trigger consumer
"describe"   () -> i32   typed GraphQL descriptor    // optional

Install is rejected unless the module exports memory and at least one role entry. Entries take no parameters and return an i32 exit code; everything they receive and everything they return travels through the host calls. One binary may carry any combination — a module that maintains an index and also answers questions about it is one artifact, not two.

The five roles

The query lifecycle

One snapshot is registered before the entry runs and released after it returns, so every read inside the invocation — however many scans and lookups it makes — sees one state. That is what lets a computed report be internally consistent without the caller coordinating anything. A query never writes: put, delete and get_for_update all return EROFS.

query_at pins a snapshot you choose instead of the current one. Because module bytes are themselves versioned keys, that travels the code as well as the data: a query run at an old sequence number is the module as it existed then.

The executor lifecycle

This is the role with real rules, because the engine may run the entry more than once per call.

• Each attempt begins a fresh transaction and gets fresh linear memory, fresh fuel and a fresh output buffer. Nothing survives a previous attempt except what is in the database.
• Exit 0 commits the transaction. Any other exit aborts it and surfaces as Error::GuestFailed { code, output }, with nothing written.
• A commit conflict discards the attempt and re-runs it against a fresh snapshot, up to execute_retries attempts (3 by default, the first included). When they are spent the call returns Conflict and re-running becomes the caller's job.
• So the entry has to be a pure function of its input and the database state: no side channels, no "have I already run" flag anywhere but the data, and no assumption that an earlier attempt's writes happened.
• Call get_for_update on every key a write depends on. That is what puts the key in the conflict set, and it is what makes the invariant hold under concurrency.
• Use checked arithmetic, and treat present-but-malformed state as corruption: fail loudly with a distinct code rather than defaulting. An executor that silently defaults or overflows corrupts durable state.
• EIO from any host call means the engine itself failed. The invocation fails host-side even if the guest swallows the errno and exits 0.

The trigger lifecycles

Both trigger roles run as executors, with the engine supplying the input and owning the transaction. What differs is what arrives and what it means.

on_touch is asked to reconcile. The input is a set of keys that were written, with repeated touches of one key coalesced into one entry while a backlog exists — no values, no op kinds, no order. The contract is to read each key's current state and make the derived state match: present means upsert, absent means remove. Written that way the module converges however the events were batched or replayed. Because the event carries no previous value, anything that needs to undo earlier work keeps its own back-pointer.

on_apply is told what happened. The input is every committed op in commit order, each with its kind, key, sequence number and — up to trigger_inline_value — its value. Nothing is coalesced: a key written three times produces three entries. Values above the inline limit arrive elided, and the module reads the key instead, knowing that read is current state and may be newer than the change it is holding. Derive output keys from the sequence number and a replay overwrites instead of duplicating.

A trigger's own writes generate no events, for any trigger, so consumers cannot chain or loop. Registration is by key range and is covered in Triggers.

Exit codes

Zero means success. The convention for everything else is one distinct non-zero code per failure class, with a human-readable message written to the output buffer. Callers see both — over GraphQL as guestExitCode and guestOutputText — so a client can tell "insufficient funds" from "malformed input" from "corrupt record" without parsing prose. The modules in guests/ use 2 through 7.

What to build with it

The shapes a module takes, grouped by the role that carries them — and an honest account of what a module cannot do.

Reads that belong next to the data

Anything whose inputs are much larger than its answer. The range never leaves the process, and the whole computation happens at one snapshot.

Writes that have to hold an invariant

Anything where reading and writing must be one indivisible step, or where several keys have to move together.

Derived data that maintains itself

Bound to a key range, a module stops being something callers invoke and becomes something the engine invokes for them. The mode picks itself from what the derived state is a function of.

Work that runs once and is never installed

The same executor contract, invoked on bytes that are stored nowhere: format migrations, backfills after a shape change, data repair, bulk re-encoding, and one-off reports. Nothing is listed, cached or replicated, and the committed writes are the only trace — so the script in your repository is the audit trail. Triggers still fire on those writes, which is what keeps indexes and feeds correct straight through a migration. Migrations & one-shots works the shape end to end.

What a module cannot do

• No clock. There is no time source at all. Anything time-shaped — timestamps on records, deadlines, rate limits, scheduled expiry — takes the instant from the caller and stores it as data.
• No randomness. Ids, tokens and nonces are either derived from data the module can read, or passed in.
• No network, files or environment. There is no WASI: the only imports are the host's own database calls. A module cannot call out, and nothing can call in except the engine.
• No state between invocations. Linear memory is fresh every time, and an executor's memory is fresh on every retry. The database is the only memory a module has.
• No vetoing a write. Triggers run after the commit, so a trigger can compensate but cannot reject. Validation that must refuse belongs in an executor that owns the write path.
• No reaching outside its store. A module sees the store it runs in, and inside it only the user keyspace — keys starting with 0x00 are the engine's own.
• No unbounded work. Fuel, memory, input, output, log volume and open scan handles are all capped per invocation, and a transaction's write set is capped too.

Within that, it is ordinary Rust. Any crate that compiles for wasm32-unknown-unknown without operating-system access works, which covers serialization, parsing, compression and arithmetic; the reference modules use serde_json. The limits exist to protect the engine's reliability and integrity — authentication and authorization are a layer you put in front.

The modules in the repository

Ten working modules under guests/, one per shape, each with a demo script that exercises it.

Choosing the shape works the same ground from the other direction — starting at the kind of database work rather than the role — and the recipes from Queries in the database onward carry the code.

The guest SDK

fluent-guest is the Rust crate you write modules against. It wraps the host ABI in safe functions, and its macros generate the exports.

Crate setup

[package]
name = "my_module"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
fluent-guest = { path = "../../crates/fluent-guest" }
serde_json = "1"      # optional; works on wasm32-unknown-unknown

cargo build --release --target wasm32-unknown-unknown
# → target/wasm32-unknown-unknown/release/my_module.wasm

There is no WASI, so nothing that needs an operating system links: no std::time, no std::env, no rand, no sockets or files. Entropy and time are inputs.

Entry points

Put one attribute per role on a function of the shape fn(T: FromInput) -> Result<O: IntoOutput, Fail>. The macro generates the export, decodes the input and encodes the result: Ok becomes exit 0 with the encoded output, and Err(Fail { code, message }) becomes a non-zero exit with the message in the output buffer.

use fluent_guest::{Change, Fail};

#[fluent_guest::query]     fn view(input: Vec<u8>)        -> Result<String, Fail> { .. }
#[fluent_guest::execute]   fn write(input: String)        -> Result<Vec<u8>, Fail> { .. }
#[fluent_guest::on_touch]  fn index(keys: Vec<Vec<u8>>)   -> Result<(), Fail>     { .. }
#[fluent_guest::on_apply]  fn feed(changes: Vec<Change>)  -> Result<(), Fail>     { .. }
fluent_guest::fluent_describe!(r#"{ ... }"#);   // optional typed surface

IntoOutput is implemented for Vec<u8>, String and (). Fail converts from String and &str with code 1, so ? works on string errors, and Fail::new(code, message) sets the code deliberately. The annotated function must not be named after the export it generates — a #[query] function called query is a duplicate definition.

The fluent_query!, fluent_execute!, fluent_on_touch! and fluent_on_apply! macros are the declarative form of the same thing, for a module that would rather export a block than annotate a function. fluent_describe! has no attribute form: the descriptor is a static string.

Data access

fluent_guest::get(&[u8]) -> Option<Vec<u8>>
fluent_guest::get_for_update(&[u8]) -> Result<Option<Vec<u8>>, i32>   // Err = errno
fluent_guest::put(&[u8], &[u8]) -> Result<(), i32>
fluent_guest::delete(&[u8]) -> Result<(), i32>

fluent_guest::scan(lo: Option<&[u8]>, hi: Option<&[u8]>) -> Result<Scan, i32>
fluent_guest::scan_rev(lo: Option<&[u8]>, hi: Option<&[u8]>) -> Result<Scan, i32>
fluent_guest::scan_prefix(prefix: &[u8]) -> Result<Scan, i32>

Reads see the invocation's snapshot, and in an executor the transaction's own buffered writes overlaid on top. None bounds are unbounded; the range is half-open, [lo, hi). get fetches whole values — to read a value larger than guest memory, drop to the raw ABI, where get returns the full length and copies from an offset.

Scan is an Iterator<Item = (Vec<u8>, Vec<u8>)> that batches under the hood. One entry per iteration; a scan that hits an entry too large for its buffer stops, and scan.skip_pending() -> bool drops that entry so iteration can continue.

Input, output and logging

fluent_guest::input() -> Vec<u8>    // the whole input blob
fluent_guest::output(&[u8])         // APPENDS to the output; call repeatedly to stream
fluent_guest::log(&str)             // a debug event under target fluent31::wasm::guest

The entry macros call input() and output() for you; reach for them directly when the entry is written by hand or when the output is built in pieces. Logs are for debugging only — they are rate-capped and invisible unless the host asks for them, so results never travel that way.

Trigger input

enum Change {
    Put    { seqno: u64, key: Vec<u8>, value: Option<Vec<u8>> },   // None = elided
    Delete { seqno: u64, key: Vec<u8> },
}
impl Change { fn seqno(&self) -> u64;  fn key(&self) -> &[u8]; }

fluent_guest::trigger_keys() -> Option<Vec<Vec<u8>>>   // keys mode, from input()
fluent_guest::changes()     -> Option<Vec<Change>>     // changes mode, from input()
fluent_guest::parse_trigger_keys(&[u8]) -> Option<Vec<Vec<u8>>>
fluent_guest::parse_changes(&[u8])      -> Option<Vec<Change>>

value: None means the value was above trigger_inline_value and was elided, not that it was empty — read the key if you need it, remembering that the read is current state and may be newer than the change in hand. None from the parsers means the input was not that shape, which is a programming error: the entry is bound to the wrong mode.

Errnos

Every fallible host call returns one of these as a negative integer. fluent_guest::errno exports them as constants.

Authoring checklist

1. Pick the role or roles, and export the matching entries.
2. Define the keyspace. Validate anything from the input that becomes a key segment: non-empty, bounded length, no separator character.
3. get_for_update on every read-modify-write key.
4. Distinct exit codes per failure class, with the message in the output. Malformed state fails loudly.
5. Checked arithmetic everywhere a number is stored.
6. A static descriptor with prefixed type names, if the module should be API.
7. Build with --release, install, and confirm typed: true, schemaError: null.
8. Test the happy path, each failure exit, concurrency for executors, and a restart — the typed field must come back.

The host ABI

The raw interface between the engine and a guest: thirteen imported functions and a handful of exports. The guest SDK wraps all of it — this page is what you need to write a module in another language, to hand-write WAT, or to reason about a limit precisely.

Conventions

• All pointers and lengths are u32 passed as wasm i32. Out-of-range memory access traps, and the invocation fails with Error::Wasm; semantic misuse returns an errno instead.
• Errnos are negative return values, in i32 or i64: NOT_FOUND -1, EROFS -2, EINVAL -3, ENOSPC -4, EBADF -5, ELIMIT -6, EIO -8.
• Keys beginning with byte 0x00 are the engine's reserved keyspace. Reads and writes there return EINVAL; scans are silently clamped to the user keyspace. Empty keys are EINVAL.
• Entries take no parameters and return an i32 exit code. The guest must export memory so the host can read and write the buffers it is given.

Input and output

input_len  : () -> i32
input_read : (dst: i32, cap: i32, off: i32) -> i32

The invocation's input blob. input_read copies up to cap bytes starting at input offset off into guest memory at dst, and returns the number of bytes copied. Large inputs are read in as many passes as the guest has room for.

output_write : (ptr: i32, len: i32) -> i32

Appends len bytes to the invocation's output. Returns 0, or ENOSPC once the total would exceed max_wasm_output. Check the return value wherever truncated output would be a correctness bug rather than a cosmetic one.

log : (level: i32, ptr: i32, len: i32) -> i32

Debug logging, capped at max_wasm_log total bytes and then ENOSPC. The host emits each line as a debug event under the fluent31::wasm::guest target (RUST_LOG=fluent31::wasm::guest=debug to see them). Never use logs to communicate results.

Point access

get            : (kptr, klen, off, vbuf, vcap: i32) -> i64
get_for_update : (kptr, klen, off, vbuf, vcap: i32) -> i64

A point lookup at this invocation's snapshot, with an executor's own buffered writes overlaid. Both return the full value length as a non-negative i64 and copy min(vcap, len - off) bytes from value offset off into vbuf — so a value larger than guest memory is read by calling again with a larger buffer or an advancing off. NOT_FOUND if the key is absent.

get_for_update additionally adds the key to the transaction's read set, which is what makes first-committer-wins apply to it; use it for every read-modify-write. In a read-only query it returns EROFS.

put    : (kptr, klen, vptr, vlen: i32) -> i32
delete : (kptr, klen: i32) -> i32

Buffer a write in the transaction. EROFS in query mode. EINVAL for a reserved, empty or oversized key (max_key_size, 16 KiB) or an oversized value (max_value_size, 256 MiB). ENOSPC once the transaction's write set would exceed max_txn_write_bytes. Deleting an absent key succeeds.

Scans

scan_open : (lo_ptr, lo_len, hi_ptr, hi_len, flags: i32) -> i32

Opens an iterator over [lo, hi) at the snapshot. A zero-length lo or hi means unbounded on that side. flags bit 0 selects reverse order; every other bit is EINVAL. Returns a handle (≥ 0), or ELIMIT past max_wasm_scans concurrently open handles. Handles are per-invocation and never survive the entry returning.

scan_next : (h: i32, buf: i32, cap: i32) -> i32

Fills buf with as many whole entries as fit in cap, subject to a host-side batch ceiling of 16 MiB. Each entry is packed as:

[klen uvarint][vlen uvarint][key bytes][value bytes]

Returns the number of bytes written; 0 means the range is exhausted. ENOSPC means the next single entry does not fit in cap — grow the buffer, or ask how big it is and decide:

scan_entry_hint : (h: i32) -> i64   // packed size of the next entry; 0 at the end
scan_skip       : (h: i32) -> i32   // drop the next entry; 1 if skipped, 0 at the end
scan_close      : (h: i32) -> i32   // free the handle

EBADF is a handle that is not open; EIO is an engine error. The SDK's Scan iterator is exactly this loop, with skip_pending() exposing scan_skip.

Limits

Every one of these is an engine Options field, so they are the operator's to set — the defaults are what an unconfigured store uses. These are the per-invocation budgets, reset for every call and, for an executor, for every retry of it:

Two more bound any single write rather than the invocation, and apply to every writer on every surface: a key is at most max_key_size (16 KiB) and a value at most max_value_size (256 MiB). put returns EINVAL past either.

Typed GraphQL fields

A module that exports describe becomes its own root field the moment it is installed. No schema file, no resolver, no server restart.

kind: "query" lands the field on Query, kind: "execute" on Mutation, and a feed declaration on Subscription. The schema is rebuilt and hot-swapped on every install and uninstall, at server start, and on mutation { reloadSchema } — so the API changes with the code that backs it.

The descriptor

fluent_guest::fluent_describe!(r#"{
  "kind": "execute",
  "description": "docs for the root field",
  "args": [{"name": "customer", "type": "String!"},
           {"name": "amountCents", "type": "U64!"},
           {"name": "note", "type": "String"}],
  "types": [{"name": "PlacedOrder", "fields": [
    {"name": "id", "type": "U64!"},
    {"name": "customerTotalCents", "type": "U64!"}]}],
  "output": "PlacedOrder!",
  "feed": {"prefix": "feed/", "event": "OrderFeedEntry!"}
}"#);

It is a static string, evaluated by running the export with an empty input, so it cannot depend on the data. Read one back with db.describe_module("name"), or with db.describe_wasm(&bytes) to inspect a module before installing it; both return None when the module does not export describe.

The type grammar

• Scalars are String, Int (32-bit), Float, Boolean, U64 (a string on the wire, with numbers accepted on input) and Json (opaque). At most one list level.
• args reference scalars only. output and the fields of types may also reference types declared in the same descriptor.
• ! marks non-null, on args, on fields and on the output.
• Limits: 32 types, 64 fields per type, 16 args, and a 64 KiB descriptor.

Shape rules

• At least one of kind and feed must be present.
• output is required with kind and rejected without it; args require kind.
• A trigger-only module declares just feed plus types — no kind, no output.
• The feed's event type must be one of the declared types.
• Every declaration must be backed by its export: "query" by query, "execute" by execute, feed by on_apply. Otherwise the install is rejected.

How arguments reach the module

With args, the entry receives one JSON object holding every declared argument, with omitted optional ones as null and U64 as a number. Without args, the field takes an optional input: BytesInput and the entry receives raw bytes.

Only the GraphQL layer builds that object. Through the shell or db.execute, a typed module receives whatever bytes you hand it — so call it with the same JSON object yourself:

exec placeOrder {"customer":"acme","amountCents":1250,"note":null}

How output is validated

The output is parsed as JSON and checked against output. Undeclared keys are dropped. Missing declared fields become null, which is an error if the field is !. A violation surfaces as OUTPUT_SCHEMA_VIOLATION, and for an executor it carries committed: true — the transaction has already committed and only the response failed to typecheck. Never blind-retry that one.

Naming

The field is named exactly what you install the module as; the crate name is irrelevant. The reference modules are installed under camelCase names — scripts/demo-orders.sh installs place_order.wasm as placeOrder and top_customers.wasm as topCustomers, and those are the field names.

The name must be a valid GraphQL name and must not shadow a built-in root field. Type names must not be reserved and must not collide with another module's, so prefix yours: PlacedOrder, not Order.

Install and confirm

mutation($w: BytesInput!) {
  installModule(name: "placeOrder", wasm: $w) { name typed schemaError }
}

The server's installModule runs describe and rejects a descriptor that does not hold up. The engine's install_module — Rust and the shell — checks only the exports, so a module installed that way with a bad descriptor ends up degraded: still callable through the generic byte fields, but with no typed field. modules { name typed schemaError } says why, and reloadSchema is the resync after an out-of-band install.

Invoking and debugging

Every surface reaches the same modules, installed or not, with the same limits and the same failures.

Installed modules

Both return the guest's output bytes. A trigger consumer is never invoked directly — it is bound to a range and the engine calls it.

Managing them

list_modules returns a ModuleInfo per module: its name, its size in bytes, and a content fingerprint that lets a caller skip re-processing bytes it has already seen. module_entries and wasm_entries return the role exports the engine found, which is how you check what a binary actually is before or after installing it. GraphQL's installModule also accepts WAT text (wasm: {text: "(module ...)"}).

Installing over an existing name replaces the bytes. Invocations already running finish on the bytes they started with. Module bytes are ordinary versioned keys in the engine's own keyspace, so they recover with the store, copy into forks, and are visible to query_at at an old sequence number — the store can answer "what code ran here".

Running bytes that are never installed

Same ABI, SDK, limits and retry loop, except that the code is pinned across all attempts. Nothing is listed, cached or replicated; an executor's committed writes are the only trace. Triggers still fire on those writes. describe is ignored, so there is no typed field, and a trigger can only ever bind to an installed module.

What failure looks like

Over GraphQL the guest's own failures arrive as guestExitCode and guestOutputText rather than as transport errors, so a client can branch on the code. In the shell a guest failure prints guest exited with code N, output …, and a conflict prints CONFLICT (first committer wins).

A trigger consumer's failure is not returned to anyone — the write that caused it has already committed. It surfaces as a stalled queue: triggers shows a growing pending and a lastError, and the runner keeps retrying with backoff until the module is fixed or replaced. Triggers covers the drain loop.

Seeing inside a module

Run with RUST_LOG=fluent31::wasm::guest=debug and the host emits fluent_guest::log output as log lines; otherwise the calls are cheap and silent. Logs are capped at max_wasm_log per invocation, so they are a debugging channel and never a results channel. Beyond that the tools are the ordinary ones: exercise the module against a fork of real data, and pin the invariant with a test that runs the executor concurrently — Testing has the harness.

Embedded API

The fluent31 crate, embedded in a Rust process: open, read, write, scan, snapshot, transact, maintain.

Opening

let db = Db::open(path, Options {
    sync: SyncMode::Periodic { every: Duration::from_millis(50) },
    ..Options::default()
})?;

Db::open creates the directory when create_if_missing is set (the default), takes an exclusive flock so that a second open of the same directory fails, recovers from the WAL, and then starts the flush, compaction, commit and trigger threads. Recovery time is proportional to the unflushed WAL.

Options

Every field, with its default:

Keys and values

A user key is non-empty, does not start with byte 0x00, and is at most max_key_size (16 KiB); values go up to max_value_size (256 MiB). Keys starting with 0x00 are the engine's own (modules, trigger definitions and queues): reading or writing one is InvalidArgument from the API and EINVAL inside a module, and scans clamp to the user keyspace.

Key layout is your schema. The examples use <entity>/<id> for records, zero-padded numeric ids so they sort (orders/00000042), and derived data under its own prefix (idx/customer/acme/00000042).

Point operations and batches

db.put(key, value)?;           // key/value: impl Into<Vec<u8>>
db.delete(key)?;               // succeeds whether or not the key exists
let v: Option<Vec<u8>> = db.get(b"key")?;

let mut b = WriteBatch::new();
b.put("a", "1"); b.delete("b");
b.len(); b.is_empty(); b.byte_size();
db.write(b)?;                  // atomic, one contiguous seqno range

Point writes and batches carry no read set. put, delete and write are never conflict-checked and can never return Error::Conflict — a batch is atomic, not isolated. If what you write depends on what you read, that is a transaction, not a batch.

Scans

// [lo, hi); None = open end; reverse = descending
let it: DbIterator = db.iter(Some(b"user/"), Some(b"user0"), false)?;
for kv in it {
    let (key, value): (Vec<u8>, Vec<u8>) = kv?;   // Item = Result<(Vec<u8>, Vec<u8>)>
}

The iterator resolves value-log pointers in batches (a prefetch window of 32 entries or 256 KiB, one batched read per value-log file), so a scan over large values costs one IO round per group rather than one per entry. An error ends iteration. Bounds are byte-exact and there is no prefix argument at this layer: a prefix scan is [prefix, prefix+1), where prefix+1 is the prefix with its last byte incremented, so user/ scans to user0. GraphQL's scan(prefix:) and the SDK's scan_prefix compute that for you. To resume after a key k when paging, start the next scan at k ++ 0x00, the smallest key greater than k. Pages that belong to one logical read should share a snapshot (iter_at).

Snapshots

let snap: Snapshot = db.snapshot();          // registers a GC hold
snap.seqno();
db.get_at(b"k", &snap)?;
db.iter_at(lo, hi, reverse, &snap)?;
db.query_at("module", input, &snap)?;        // module bytes AND data at the snapshot
drop(snap);                                  // releases the hold

let s: SeqNo = db.seqno();                   // "now", without a hold
let snap = db.snapshot_at(s)?;               // Err(InvalidArgument) once GC passed s

Transactions

let mut txn: Txn = db.begin();
txn.snapshot_seqno();
let cur = txn.get(b"k")?;                    // consistent read, no conflict check
let cur = txn.get_for_update(b"k")?;         // read + conflict check at commit
txn.put("k", "v")?; txn.delete("j")?;        // buffered until commit
txn.write_set_len();
for kv in txn.iter(lo, hi, reverse)? { }     // snapshot merged with this txn's writes
txn.commit()?;                               // or txn.rollback(), or drop to discard

Semantics and the retry loop are on Transactions.

Durability and maintenance

db.sync_wal()?;      // barrier: everything acked before this is durable on return
db.flush()?;         // freeze the memtable and wait until it is in tables
db.compact_all()?;   // compact until no trigger fires
db.gc_vlog()?;       // one value-log GC pass; Ok(Some(file_id)) if a file was retired
db.log_stats();      // the stats() snapshot as one info log line (the heartbeat calls this)
let s: DbStats = db.stats();

DbStats has backend ("io_uring" or "std"), visible_seqno, memtable_bytes, immutable_memtables, levels as a Vec<(runs, files, bytes)>, vlog_files, vlog_retired (retired files waiting on the deletion gates), discard_bytes (value-log bytes known to be dead), cache_hits, cache_misses, commit_groups, commit_batches (the difference from commit_groups is how many fsyncs group commit saved), wal_syncs, subscriptions (live stream subscriptions, each buffering up to sub_queue_bytes) and snapshots (registered snapshots — explicit, transactions, pins and subscription holds — every one a GC hold).

Compaction and value-log GC run on their own on background threads. The manual calls exist for tests, benchmarks and "reclaim now".

Modules and triggers

db.install_module("name", &wasm_bytes)?;   // validates: exports memory + a role entry
db.uninstall_module("name")?;
db.list_modules()?;                        // Vec<ModuleInfo>: name, size, content_hash

let out: Vec<u8> = db.query("name", input)?;     // requires the `query` export
let out: Vec<u8> = db.execute("name", input)?;   // requires `execute`; OCC-retried
db.query_wasm(&wasm, input)?;              // one-shot: bytes never installed
db.execute_wasm(&wasm, input)?;

db.describe_module("name")?;                // Option<Vec<u8>>: the `describe` output, None if not exported
db.describe_wasm(&wasm)?;                  // the same, on bytes you have not installed

db.create_trigger("name", "module", Some(b"orders/"), Some(b"orders0"))?;
db.delete_trigger("name")?;
db.list_triggers()?;                       // Vec<TriggerInfo> { name: String, module: String,
                                           //   lo: Vec<u8>, hi: Vec<u8> (empty = open), mode: TriggerMode::{Keys, Changes},
                                           //   pending: u64, last_error: Option<String> }

The full contracts are on Roles and lifecycles and Triggers.

Change stream

let mut sub: Subscription = db.subscribe(b"orders/", Some(b"orders0"))?;
sub.start_seqno();                                   // everything strictly above flows
loop {
    match sub.recv_timeout(Duration::from_secs(1))? {
        None => continue,                            // timeout
        Some(StreamEvent::Batch(entries)) => for e in entries {
            // StreamEntry { key, seqno, commit_seqno, kind: Put|Delete, value: Option<Vec<u8>> }
        },
        Some(StreamEvent::Lagged) => break,          // queue cap exceeded; re-subscribe
    }
}

Delivery is post-commit, seqno-ascending and gap-free past start_seqno, and values arrive resolved. seqno is the op's own. commit_seqno is the last seqno of the atomic commit the op belonged to, which is the one state in which the op became visible; snapshot_at(commit_seqno) reads it. Value-log GC relocations re-put a live value through the write path, so they show up as Put entries carrying the unchanged value, which is harmless for any consumer. Drop subscriptions you stop consuming, since an undropped one holds a GC pin. The journal, GraphQL subscriptions and replication are all built on this stream.

Identity

if let Some(id) = db.identity() {
    id.name; id.instance_id; id.instance_hex();
    id.parent;   // Option<(InstanceId, cut_seqno)>
}

Errors

Triggers

Bind a module to a key range and the engine invokes it after every committed write into the range: indexes, views, feeds, cascades.

Registering

db.create_trigger("name", "module", Some(b"orders/"), Some(b"orders0"))?;   // Rust

mktrig NAME MODULE [LO|-] [HI|-]            # shell; - = open end

mutation { createTrigger(name: "idx", module: "customer_index",
                         lo: {text: "orders/"}, hi: {text: "orders0"}) }
query { triggers { name module lo { text } hi { text } mode pending lastError } }
mutation { deleteTrigger(name: "idx") }     # discards pending events

None bounds mean an open end. The module must already be installed. Names follow the module-name rules and must be unique. lo >= hi, or a bound longer than max_key_size, is rejected. One module may back many triggers.

The mode is detected from the module's exports at registration and fixed for the trigger's life: on_apply present means changes mode; otherwise on_touch means keys mode; neither is rejected. Replacing the module's bytes later does not change the mode. A changes-mode trigger whose module lost on_apply fails its drains loudly (lastError) and holds its events.

Each trigger has its own queue, and one runner thread drains them independently. There is no ordering between triggers, and overlapping ranges each get their own copy of an event.

Keys mode (on_touch)

The input is the touched keys, up to trigger_batch per invocation. No values, no op kind, no order. Re-touches of one key coalesce into one pending event while a backlog exists.

The contract: an event means "reconcile this key". Read the key at your snapshot. If it is present, upsert your derived state; if it is absent, remove it. Written this way the module converges under replay, coalescing and reordering. Updates and deletes need your own back-pointer (say idx/order/<id> pointing at the customer) to find the stale entry, because the event carries no old value. guests/customer_index is the reference.

Changes mode (on_apply)

The input is the ordered list of committed changes, one per op, up to trigger_batch per invocation. Each carries seqno (the op's own seqno, assigned at commit, unique and strictly increasing across the feed), the kind (put, delete, or put with the value elided), the key, and the value inline up to trigger_inline_value (64 KiB). Above that, value is None, and you read the key knowing the read is current state, possibly newer than the change.

The contract: one event per op, in commit order, never coalesced. A key written three times yields three changes. Filter in code; the range is only the coarse cut. Derive output keys from the seqno (feed/<seqno zero-padded>) so that replays overwrite instead of duplicating. Old values are still your job. A hot key grows the backlog where keys mode would coalesce it. The references are guests/order_feed, guests/live_stats, guests/dynamic_index and guests/cascade_delete.

Delivery guarantees

• Durable capture. Events commit in the same atomic batch as the write that caused them. A write that survives a crash fires its trigger after recovery; one that doesn't, doesn't.
• At-least-once invocation, exactly-once effects. Consumed events are deleted inside the module's own transaction. A crash or a conflict re-runs the whole attempt, and your writes and the events' consumption are inseparable.
• No stacking. Writes made by a trigger invocation never generate events, for any trigger. No chains, no loops.
• Asynchronous. Derived state trails the base data by the backlog. Nothing waits for a trigger. Watch pending to see how far behind it is.
• Failure holds, never drops. A failing module (a guest error, a missing module, conflict exhaustion) leaves the batch queued. The runner backs off per trigger, starting at 100 ms and doubling up to a 6.4 s ceiling. list_triggers and triggers { pending lastError } show the depth and the reason. Fix the module by reinstalling it and the backlog drains.
• Batch bounds. A drain hands the module at most trigger_batch events and never more than max_wasm_input bytes. Inlined values are clamped so that every event fits.
• Trigger definitions and queues live in the reserved keyspace, so they are versioned, recovered and forked with everything else. A store rebuilt from the journal has neither, so recreate the triggers.

Every writer fires triggers: plain puts, batches, transactions, executors, one-shot executors, and every network surface. Trigger invocations themselves, value-log GC relocations, and writes made while wasm_enabled = false do not.

Waiting for a drain

There is no synchronous "run the triggers now". Poll list_triggers() until pending == 0 for the triggers you care about and last_error is None. This is the reference loop (crates/fluent31/examples/util/mod.rs::drain):

let deadline = Instant::now() + Duration::from_secs(30);
loop {
    let triggers = db.list_triggers()?;
    if let Some(err) = triggers.iter().find_map(|t| t.last_error.clone()) {
        panic!("trigger failed: {err}");          // a failing module never clears itself
    }
    if triggers.iter().all(|t| t.pending == 0) { break; }
    assert!(Instant::now() < deadline, "triggers did not drain in 30s");
    std::thread::sleep(Duration::from_millis(10));
}

Forks, pins, clones

A fork is a named, consistent branch of the whole database, published as a complete database directory.

What a fork is

Forks land under <dir>/archive/<name>/. Tables and sealed value-log files are immutable, so the fork hard-links them. Creation cost is proportional to the number of files, plus one bounded copy of the still-growing value-log head (at most vlog_file_size). Shared bytes exist once on disk. du on the archive re-counts shared inodes, so the apparent size is not the added size. Real divergence accrues only as parent and child compact away from the shared base.

A fork exists completely or not at all. It is built in a temporary directory, fsynced and published by a single rename, and a crashed build is swept at the next open.

Cutting

fork_at needs a point that is still materializable: the head, a seqno captured moments ago with db.seqno(), or one held by pin(name). A pin is a durable, store-wide GC hold recorded in the manifest. It survives restarts and costs retention until unpin. Seqnos below the watermark are refused.

Live readers and writers keep running during a fork. What the store pays is one memtable flush, a brief hold of the manifest lock (structural installs pause, traffic does not), and, because the cut is a registered snapshot for the build's duration, GC held at the cut and value-log deletions deferred until the build finishes.

The API

let f: ForkInfo = db.fork("before-migration")?;
// ForkInfo { name, instance_id, created_unix_ms, last_seqno, path }
let clone = Db::open(&f.path, Options::default())?;      // live CoW clone

let p: PinInfo = db.pin("pre-import")?;                  // durable store-wide GC hold
let f = db.fork_at("rollback", p.seqno)?;                // cut exactly there
db.unpin("pre-import")?;
db.pins();                                               // Vec<PinInfo>, oldest first

let s = db.seqno();                                      // capture "now"
let a = db.fork_at("replica-a", s)?;
let b = db.fork_at("replica-b", s)?;                     // identical cuts

db.list_forks()?;  db.delete_fork("name")?;              // refused while the fork is open
fluent31::list_forks_at(Path::new("./data"))?;           // lock-free, works on a live store
fluent31::restore_to(&archive_path, &dest, Some("copy-name"))?;

Fork and pin names use [A-Za-z0-9._-], at most 64 characters, with no leading dot. restore_to refuses an existing dest and an archive that has already been opened read-write (fork that live copy instead).

Using a fork

• Open equals activate. Db::open(fork.path, ..) gives you a live, writable, copy-on-write clone. New writes land in its own files and its compactions unlink only its own links. The parent is untouched.
• restore_to(archive, dest, new_name) hard-links the archive into a fresh directory, or copies it when dest is on another filesystem, so the archived cut stays pristine. new_name is required for forks of a named store, since each copy mints its own identity.
• delete_fork(name) refuses while the fork is open as a database.
• list_forks_at(dir) reads archive/*/fork.meta without taking a lock, so it works on a store another process has open.
• Under the server, every fork is an instance at /graphql/<instanceId> with the same full surface, including its own forks.

Rolling back the primary

There is no in-place restore. A rollback swaps directories.

1. Before the risky change, fork("pre-migration"), or pin now and fork_at later. Rehearse the change on the fork's own instance.
2. To roll back, stop the process. Then either open the fork directly as the new primary (its first read-write open fixes its identity under the fork's name), or keep the archive pristine with restore_to(archive, "<new-dir>", Some("prod-2")) and start on <new-dir>. Pass no store_name on later opens, since the name is persisted.
3. The rolled-back primary has a new instance id. Every replica notices on its next connection and re-attaches from scratch; nothing else needs telling. Any stored seqnos are meaningless across the swap.
4. Delete the abandoned directory when you are sure.

The shell and GraphQL expose fork and pin but not restore. Step 2 is a filesystem operation or the Rust call.

Durability & recovery

What an ack means, what survives a crash, and the journal for the day the store directory itself is lost.

Sync modes

Options::sync decides when writes reach stable storage. The server flags spell these always, periodic:<ms> and never.

What survives a crash

Under SyncMode::Always, every acked write survives. A value-log payload is synced before the WAL record that points at it, so a durable pointer never precedes its data. Under Periodic, everything up to the last timer tick or sync_wal survives. Under Never, whatever the OS had flushed.

In every mode the store reopens consistent. The WAL's torn tail is truncated, tables are self-describing and synced before the manifest references them, and the manifest flips atomically. Corruption in a sealed file is a hard Corruption error, never silent.

The test suite proves this with a SIGKILLed child process (crash_recovery), a fault-injecting IO backend (fault_injection, which shows a failed fsync is never a false ack) and a byte-mutation sweep (corruption_fuzz, which shows no on-disk byte can panic the reader).

Degraded state

A hard IO failure in the write path or a background thread failure sets a store-wide error. After that, writes, flush, sync_wal, pin and subscriptions return Error::Background, while get, iter and snapshots keep serving what is there. Reopen the store; recovery brings it back to the last durable state.

The journal

The store's own WAL and manifest are its durability. The journal is for the day that is not enough: a bad disk block, a truncated file, a lost directory. It is off unless you attach it, and it never sits on the commit path.

At attach it writes a base snapshot of the user keyspace, then trails the change stream on a background thread, appending each mutation to journal-*.log segments that rotate at rotate_bytes. Once the delta bytes written since the last base exceed compact_when_deltas_exceed times that base's size, and also exceed compact_min_bytes, it writes a fresh base and prunes the superseded segments, so disk stays near the live set plus one window of recent deltas. If the consumer ever lags past sub_queue_bytes, it heals by writing a new base. The log header records the source instance id, and a different store's journal in the same directory is refused.

use fluent31::{Journal, JournalConfig, journal};

let db = Arc::new(Db::open(dir, opts)?);
let j = Journal::attach(db.clone(), "./journal")?;       // base snapshot now, deltas trail
let j = Journal::attach_with_config(db.clone(), dir, JournalConfig {
    rotate_bytes: 128 << 20,
    compact_when_deltas_exceed: Some(1.0),               // None = manual only
    compact_min_bytes: 64 << 20,
})?;
j.stats();               // deltas_written, base_records_written, last_seqno,
                         // rebaselines, compactions, files_pruned, last_error
j.request_checkpoint();  // compact now
drop(j);                 // joins the drainer, final flush

How to attach it on each surface:

Observing the journal

An embedder that mirrors the journal somewhere else — another volume, an object store — attaches it with an observer instead of polling the directory. The observer hears every fact of the log's life, in the order it became true, and every fact is already durable on disk when it is reported: bytes of a file below a reported length are fsynced and will never change (the log is append-only; the one truncation it performs, a torn tail after a crash, happens before attached reports the file). So a mirror copies exactly the reported bytes, deletes exactly the reported files, and never re-reads or lists the source.

use fluent31::{Journal, JournalConfig, JournalObserver, journal};

struct Ship;
impl JournalObserver for Ship {
    fn attached(&self, dir: &Path, files: &[(u64, u64)]) {}       // (id, durable length) of every file present
    fn appended(&self, file: u64, durable_len: u64) {}              // file is fsynced through durable_len
    fn rotated(&self, sealed: u64, sealed_len: u64, next: u64) {}   // sealed is final; next is active
    fn pruned(&self, anchor: u64) {}                                // every id below anchor is deleted
    fn stopped(&self, error: Option<&str>) {}                       // None on a clean detach
}
let j = Journal::attach_observed(db.clone(), dir, JournalConfig::default(), Arc::new(Ship))?;
journal::log_file_name(14);          // "journal-000014.log" — the naming contract
journal::log_file_id("journal-000014.log"); // Some(14)

attached arrives on the attaching thread before attach_observed returns; everything after it on the journal's own thread. A compaction reports the anchor file's base as durable (appended) before it reports the superseded files gone (pruned), so a mirror that applies facts in order never holds only superseded files. Observers return promptly and do their I/O elsewhere — the drainer waits for them.

Rebuilding from the journal

fluent-cli journal-rebuild <journal-dir> <dest-dir>
# prints: source instance, base keys, deltas applied, last seqno

Or fluent31::journal::rebuild(journal_dir, dest, opts), where opts are the rebuilt store's Options (give it a store_name for a fresh root identity). dest must be absent or an empty directory; a directory that already holds anything is refused (InvalidArgument), never merged into. The rebuilt store holds all user data as of the journal's last durable record, as a new lineage: seqnos are renumbered, the instance id is fresh, and modules, triggers, pins and forks are not restored, so redeploy them. A missing middle segment is refused (JournalGap), never rebuilt around.

The tail is approximate in both directions. The journal's last few unsynced records can be lost. And under Periodic or Never, the journal, which is fed from the in-memory commit stream, can hold writes the crashed store lost, so a rebuild is slightly ahead of what the store would have recovered. Both are acceptable. You reach for the journal only when the store itself is gone, and the rebuild replaces it.

Backups

• For a consistent snapshot on the same filesystem, fork(name). It is cheap, instant, consistent and hard-linked.
• To copy elsewhere, copy archive/<name>/, which is a plain directory tree (the hard links copy as full files), or restore_to into a mount point.
• For continuous off-box protection, mirror the journal through a JournalObserver (above): it reports every durable byte and every deletion, so the mirror needs no polling. A reassembled journal is verified for contiguity at rebuild.

GraphQL API

POST /graphql for the primary, POST /graphql/<instanceId> for a fork. A GET serves GraphiQL; a WebSocket upgrade with the graphql-ws subprotocol serves subscriptions.

Encoding

Keys and values are raw bytes. Inputs take exactly one of {text}, {base64} or {hex} (BytesInput, a @oneOf input). Outputs expose text (null if not UTF-8), base64, hex and len (Bytes).

U64 is a string-encoded 64-bit unsigned scalar used for seqnos, timestamps and byte totals. Inputs also accept numbers. Json is opaque passthrough, used by typed modules only.

Query

Every read field of one query operation runs at one pinned snapshot. stats, modules, forks, pins, triggers and seqno report current state.

Mutation

Mutation fields run serially in document order, each as an independent atomic write, and executor fields each run their own transaction. A document is never one transaction.

Subscription

subscription {                      # raw plane: no module needed
  changes(lo: {text: "orders/"}, hi: {text: "orders0"}) {
    kind seqno commitSeqno key { text } value { text }
    query { snapshotSeqno get(key: {text: "orders/count"}) { text } }
  }
}
subscription {                      # typed plane: a module with a `feed` descriptor
  orderFeed { kind seqno commitSeqno key { text } event { seqno op id record elided } }
}

• kind is ATTACHED, PUT or DELETE. The stream opens with one ATTACHED marker with no key, value or event. Its seqno is the attach boundary: everything at or below it is readable through the marker's query, and everything above arrives on the stream. Gap-free, with no overlap.
• Every item carries query: Query!, the full Query root pinned at the item's commitSeqno, which is the exact state in which the op became visible. The ops of one atomic commit share a commitSeqno.
• Typed feeds deliver puts only, so feed GC deletes are invisible. event is the written value validated against the declared event type.
• A consumer that falls behind sub_queue_bytes is cut off with a "lagged" error. Re-subscribe and re-scan from the new boundary. Items hold snapshots, so consume promptly. A server restart ends every subscription; nothing about them is persisted.
• The idiom: history is a scan of the feed range, the latest value is a get, and live is a subscription. A disconnected client misses nothing durable as long as the module materializes its feed.

Errors

Engine failures map to errors[].extensions.code: IO, CORRUPTION, INVALID_ARGUMENT, CONFLICT (retries exhausted), CLOSED, BACKGROUND, WASM, GUEST_FAILED (with guestExitCode, guestOutputBase64, and guestOutputText when the output is UTF-8), PROVENANCE_MISMATCH, GONE, JOURNAL_GAP and OUTPUT_SCHEMA_VIOLATION (a typed output mismatch, carrying committed: true for executors). Documents are capped at depth 32 and complexity 5000.

Root fields are always outer-nullable, so a failure yields field: null plus an errors entry rather than a spec-invalid response.

Instances

fork(name:) { instanceId } returns the address of the new branch, and /graphql/<instanceId> serves it with the same full surface: its own modules, triggers, schema and forks. Instances open lazily on the first request and close when idle (fork-idle-ttl-secs) or when evicted past fork-max-open. Forks nest up to 8 deep under one primary. An unknown id is a 404. The id is routing, not authorization.

Demo

cargo run -p fluent-server -- ./data
scripts/demo-orders.sh [endpoint]     # builds the guests, installs placeOrder + topCustomers, seeds, ranks

mutation { placeOrder(customer: "you", amountCents: "4200") { id customerTotalCents } }
query    { topCustomers(limit: 3) { customer orders totalCents avgCents } }

The shell

An interactive prompt over one store, and the journal rebuild tool.

fluent-cli <db-dir> [--std|--uring] [--nosync] [--sync-every <ms>]
fluent-cli journal-rebuild <journal-dir> <dest-dir>

--std and --uring force the IO backend. --nosync is SyncMode::Never and --sync-every is Periodic. Every command prints its wall-clock latency. Byte arguments are plain UTF-8 or hex:DEADBEEF. Output shows printable bytes quoted and everything else as hex:.

Commands

count shares scan's parser, so it takes the same - bounds, --rev and --limit; unlike scan it has no default limit, so it counts the whole range. quit is the same as exit. Values longer than 160 bytes print truncated with their length. A guest failure prints guest exited with code N, output …. A transaction conflict prints a CONFLICT (first committer wins) line; the transaction is rolled back.

Server mode

One process, one Db, two planes.

fluent-server <db-dir> [--config FILE] [--store-name NAME]
              [--graphql ADDR:PORT] [--replication ADDR:PORT]
              [--sync always|never|periodic:<ms>] [--max-body-bytes N]
              [--journal DIR]
fluent-server --print-schema                   # the base SDL (built-ins only)

The store directory is flocked, so the planes cannot be split across processes. Server mode is how they share one handle. --store-name is persisted in the store, so pass it once. Without a name, graphql serves and the join point stays closed; the log says so.

On the first SIGINT or SIGTERM the server stops accepting and drains in-flight GraphQL requests, then the process exits and open replication connections drop (the WAL keeps the store consistent). A second signal exits immediately. The log (stderr; RUST_LOG sets the level — Operations) reports each bound address as it comes up and, for a named store, its name and instance id; every flush, compaction, fork and journal event follows at info, with a stats heartbeat every 60 s. If the engine degrades (Error::Background), GraphQL answers BACKGROUND and replication answers ERR; restart the process.

Config file

--config server.toml. The top-level keys, [listen] and [graphql].max-body-bytes mirror the flags, and an explicit flag wins. The rest is file-only. Unknown keys are an error. Every key, with its default:

dir = "./data"
store-name = "prod"
sync = "always"               # always | never | periodic:<ms>

[listen]
graphql = "127.0.0.1:8317"
replication = "127.0.0.1:8428"

[graphql]
max-body-bytes = 33554432     # 32 MiB request body cap
fork-max-open = 8             # open fork instances beyond the primary (LRU past this)
fork-idle-ttl-secs = 300      # idle instances close after this

[replication]
max-frame-bytes = 1048576
ping-every-ms = 2000

[journal]                     # present = attached; absent = off
dir = "./journal"             # required once the section exists
rotate-bytes = 134217728
compact-when-deltas-exceed = 1.0
compact-min-bytes = 67108864

[log]
stats-every-secs = 60         # stats heartbeat per open store; 0 = off

[engine]                      # every fluent31::Options tunable, kebab-case
create-if-missing = true
wasm-enabled = true
io-backend = "auto"           # auto | uring | std
compression = "none"          # none | lz4
memtable-size = 8388608
# … every Options field from the Embedded API page, kebab-case

Embedding the server

use fluent_server::{Server, ServerConfig};

let db = Arc::new(Db::open(&dir, opts.clone())?);
let server = Server::start(db.clone(), &dir, opts, ServerConfig::default()).await?;
server.graphql_addr; server.replication_addr;   // replication_addr: None when unnamed
server.db();
server.shutdown().await;

ServerConfig holds graphql_addr, replication_addr, max_body_bytes, registry: RegistryConfig { max_open, idle_ttl }, replication: ReplServerConfig { max_frame, ping_every } and stats_every (the heartbeat period; 60 s, zero = off). Nothing is served unless every bind succeeds; failures come back as StartError::{Engine, Bind}. The TOML loader is public too (FileConfig::load, overlay, server_config, engine_options, parse_sync).

For the GraphQL plane alone: SchemaManager::new(db), then InstanceRegistry::new(..), then fluent_graphql::router(registry, max_body), which is an axum::Router. Call registry.evict_idle() periodically; the server ticks every 60 seconds. The stats heartbeat is fluent_graphql::stats_heartbeat(registry, every), a future to spawn. For replication: ReplServer::new(db, cfg)? fails with InvalidArgument on an unnamed store; then .serve(listener).

Replication

Read-only replicas that attach to a running master's join point and hold the slice of the tree overlapping their key scope.

The scope is unbounded for a full replica and narrow for an edge cache. The overlapping index fragments are copied locally, values are fetched lazily and cached, and committed in-scope writes stream in. The replica is a library component: the process that needs the scoped reads embeds an EdgeReplica and reads through its store (get and scan, clamped to the scope). REPLICATION.md is the spec.

# master: fluent-server on a named store opens the join point (:8428)
fluent-server ./data --store-name prod [--replication 127.0.0.1:8428]

How it behaves

• Named master only. An unnamed store cannot open a join point: fluent-server leaves the port closed and ReplServer::new returns InvalidArgument.
• Provenance. Every connection compares the master's instance id. With the same id, every cached byte stays valid across disconnects and lag. With a different id (the master was restored, forked or replaced) the edge wipes and re-attaches from scratch. Stale history is never served.
• Gap-free attach. The edge subscribes first and then pulls the slice, so the union covers everything. Overlap is harmless because entries carry seqnos.
• Ephemeral. The edge directory is a cache, wiped on attach, and the master keeps no per-edge state beyond the subscription. A stale file reference answers GONE and the edge re-pulls. Only committed user-key data is readable: no modules, no triggers, no queries or executors.
• Lag. A slow edge is cut off (LAGGED) rather than stalling the master. It re-syncs and keeps its caches.
• Scope. An out-of-scope get is refused (InvalidArgument), scans clamp to the scope, and the reserved keyspace is never copied or streamed.

The limits are deliberate: one contiguous scope per replica, read-only, embedded (a replica serves no network protocol of its own), a memory-only stream overlay (a restart re-attaches), and no WASM at the edge.

Embedding a replica

let mut cfg = EdgeReplicaConfig::new("127.0.0.1:8428", "/tmp/edge",
                                     b"user/".to_vec(), Some(b"user0".to_vec()));
// fields: master_addr, dir, scope_lo, scope_hi, refresh_every (300 s; None = only on re-sync),
//         value_cache_bytes (256 MiB), block_cache_size (32 MiB)
cfg.refresh_every = Some(Duration::from_secs(60));
let replica = EdgeReplica::start(cfg)?;      // returns once a complete scoped view is available
replica.store().get(b"user/1")?;
replica.store().stats();                     // EdgeStats
replica.master();                            // StoreIdentity

The library surface is fluent_replication::{ReplServer, ReplServerConfig, ReplClient, EdgeReplica, EdgeReplicaConfig, MasterInfo} and, on the engine side, fluent31::edge::{EdgeStore, EdgeConfig, EdgeStats, ValueFetcher}. Lower level: ReplClient::connect(addr) gives (client, MasterInfo { name, instance_id, visible_seqno }), with snapshot, fetch_table_chunk and fetch_value.

A replica logs its attach, every slice pull and every re-sync at info; a lag cut, a broken stream and a changed master identity are warn. The master logs each stream it serves and why it ended.

Store identity

A store can carry an operator-chosen name. From the name the engine mints a deterministic 128-bit instance id, and forks and restores mint new ones. Replication verifies the id on every connection, so a replaced master invalidates every replica at once. Under the server, every fork is an instance addressed at /graphql/<instanceId>. The id is an address, not a credential.

Operations

The directory on disk, the knobs that matter, what to watch, and the hard limits.

Directory layout

<dir>/
  LOCK               exclusive flock for the process lifetime
  CURRENT            names the live MANIFEST
  MANIFEST-<gen>     full metadata snapshot
  wal-<id>.log       write-ahead logs, one per memtable generation
  sst-<id>.tbl       immutable table fragments
  vlog-<id>.vlog     value-log files; one active head, the rest sealed
  archive/<name>/    forks, each a complete database directory

One process per directory. Everything is CRC32C-checked. Don't hand-edit anything, and don't delete WALs or manifests.

Sizing

Writers stall rather than fail when frozen memtables exceed max_immutable_memtables or L0 exceeds l0_stall_trigger. A deep merge does not hold L0 back: compaction works in slices of compaction_slice_bytes, and between slices a level that crossed its trigger takes over from the deeper job, which resumes afterwards. Sustained stalls therefore mean compaction as a whole cannot keep up, not that one large job is in the way. Each stall episode is logged: warn as it begins, with its cause; info as it ends, with its duration.

Monitoring

• stats (engine, shell, GraphQL) reports the seqno, the memtable and level shape, value-log live, retired and discardable bytes, the cache hit rate, group commit amortization, and the live subscription and snapshot counts. An edge replica reports through EdgeStats instead.
• triggers reports pending (the backlog depth) and lastError per trigger.
• Journal::stats(): last_seqno against db.seqno() is the journal lag; last_error is the last failure.

Logging

The engine emits tracing events. The binaries write them to stderr and read RUST_LOG for the level (default info; fluent-cli defaults to warn so the shell stays quiet). An embedding process installs its own subscriber; without one the events cost nothing. Every engine line names the store it is about (db{dir=… store=… instance=…}), so a server holding forks stays legible.

GraphQL requests are not logged.

The stats heartbeat is the stats snapshot as one info line per open store (the primary and every fork the server holds open) plus the fork registry's occupancy, every 60 s by default — [log] stats-every-secs in the server config, 0 turns it off; an embedder gets the same line from Db::log_stats(). When memory grows, the heartbeat says which it was: imms climbing (flush not keeping up — a stall follows), subscriptions, snapshots pinning history, or fork instances.

Guest log output is a debug event under its own target, enabled alone with RUST_LOG=fluent31::wasm::guest=debug.

Limits

Known limits (v1, deliberate): no block compression by default (LZ4 is opt-in); value-log discard statistics lag, since dead pointers are only discovered when compaction reaches them; GC relocations bump seqnos, so a hot large-value key can cost a transaction a retry; a fixed level count; and bottom-level merges rewrite the whole bottom level.

Compatibility

Every Options field except store_name may change between opens of the same store, and compression affects only newly written tables. On-disk formats are versioned. An unnamed store stays on manifest format 1, a named one writes format 2, and pins bump it to 3; older binaries read only the formats they know. The replication protocol advertises its version in HELLO.

Platform notes

IoBackend::Auto probes io_uring at open and falls back to portable IO; stats.backend tells you which one is active. Docker's default seccomp profile blocks io_uring, so use --security-opt seccomp=unconfined or io-backend = "std". macOS uses portable IO throughout.

Testing

One workspace suite, plus fault injection, endurance and benches.

cargo test --workspace                              # engine model tests, group commit, wasm, graphql,
                                                    # server e2e, replication e2e, durability suites
cargo test -p fluent31 --features fault-injection   # fsync failure / ENOSPC / read-fault paths
cargo test --test backup_and_soak -- --ignored      # endurance soak
cargo check -p fluent31 --no-default-features       # the engine without the WASM layer
cargo run --release -p fluent31 --example bench     # throughput probe
cargo run --release -p fluent31 --example gc_bench -- [threads] [always|never] [ops-per-thread] [txn]

Suites worth knowing by name

engine (a randomized model test against a BTreeMap with interleaved flush, compaction, GC and reopen), crash_recovery (a SIGKILLed child), fault_injection, corruption_fuzz, journal_rebuild, durability_modes, group_commit, fork_stress (forks under concurrent writers, flush, compaction and GC), trigger_changes, trigger_robustness, wasm and wasm_sandbox. fluent-graphql/tests/graphql.rs has WAT fixtures for modules, including describe; fluent-server/tests/server.rs and fluent-replication/tests/replication.rs are the end-to-end suites.

To test your own modules, look at the GraphQL suite's WAT fixtures for minimal modules. For executors, spawn N concurrent calls and assert no lost updates. Restart the server and assert the typed field reappears.

Under Docker

docker run --security-opt seccomp=unconfined -v $PWD:/src -w /src rust:1 \
  sh -c "rustup target add wasm32-unknown-unknown && cargo test --workspace"

Architecture

The engine as implemented: the write path, the storage layout, the concurrency-control machinery, recovery, and the subsystems built on top of them. The behaviour documented elsewhere on this site follows from the mechanisms described here.

Write path

A batch is placed before it is logged. Values at or above value_threshold are appended to the value-log head and the corresponding tree entry becomes a pointer; smaller values remain inline. The batch is then written to the write-ahead log, inserted into the memtable, and published by advancing the visible sequence number, so that no reader observes a partial batch.

Under SyncMode::Always a dedicated commit thread drains everything queued in each cycle and applies it in size-bounded chunks. Each chunk costs one value-log fsync and one write-ahead-log fsync, which the participating writers share; the steady-state group size therefore approaches the number of concurrent writers. Transactions are validated and applied inside the same critical section as plain writes, and are revalidated against batches applied earlier in the same group.

Storage layout

The tree is laid out for lazy leveling: upper levels are merged tierwise, where a full level merges into a single run at the front of the level below, and the bottom level holds one leveled run. Runs are divided into key-bounded fragments of approximately target_file_size, each carrying its own bloom filter and index, sized so that the indexes of an entire dataset can remain resident in memory. Bloom filters share the block cache with data blocks rather than being pinned, so a fragment nobody queries costs no memory for its filter.

Key-value separation keeps the tree small: compaction relocates pointers rather than payloads, so the cost of a merge is governed by key volume rather than value volume. The value log is reclaimed by a separate collector, which rewrites a file's live records through the ordinary write path, retires the file, and unlinks it only once no registered snapshot can still reach the superseded versions and the relocations are present in fsynced tables. The vlog_retired statistic counts the files held between those two conditions.

Multi-version concurrency control

The garbage-collection watermark is the sequence number of the oldest registered snapshot; pins and stream subscriptions register in the same way. Compaction retains every version above the watermark, together with the newest version at or below it, and discards the remainder. Two consequences follow directly: a snapshot held for any key holds every version of every key, and a sequence number remains addressable only while it is above the watermark. Per-key retention policies therefore cannot exist.

Commit validation reads the newest committed version of each key read under get_for_update and each key in the write set, tombstones included, within the same critical section used by every other writer. The transaction's own snapshot bounds the watermark for its duration, so the evidence validation depends upon cannot be compacted away mid-transaction.

The versions the engine retains exist to serve in-flight readers and to validate commits, and they are discarded as soon as neither purpose requires them. Superseded versions are therefore not a record's history: they are unaddressable once the watermark passes them, they are renumbered wholesale by a journal rebuild, and a fork or restore mints a new identity for them. A history that must survive those events is written as data — by a changes-mode trigger, under keys chosen for the purpose — and is then subject to the same retention rules as any other data.

Recovery

The manifest is a complete metadata snapshot, rewritten on each structural change and made current by an atomic rename of CURRENT. Table files are fsynced before any manifest references them, so a referenced file is always readable.

Recovery replays every write-ahead log at or above the manifest's floor, validates value-log pointers against the scanned prefix of each file, and truncates a torn tail on the newest log. The replayed memtable is flushed synchronously, so that a crash during recovery results only in a repeated replay. A fresh value-log head is opened rather than appended to, because the engine never resumes writing to a file that predates a crash. Orphaned files and partially built forks are swept at the same time.

Module execution

Modules are compiled and run by wasmtime with fuel metering, a memory limit, NaN canonicalization, deterministic SIMD and no WASI imports. A query executes against a snapshot registered for the duration of the invocation. An executor runs inside a transaction; a commit conflict discards the instance and re-runs the invocation against a fresh snapshot with fresh memory, fuel and output.

Compiled modules are cached by content hash, while one-shot bytes are compiled without being cached, so that a stream of one-shot invocations cannot evict installed modules. Module bytes are stored at \x00wasm\x00<name> as ordinary versioned keys, which is what allows query_at to travel code and data together.

Trigger capture and drain

Capture occurs inside the commit critical section. The keys of each committed batch are matched against the trigger registry and the resulting event records are appended to that same batch, so an event shares one write-ahead-log record and one sequence-number range with the write that caused it.

Keys-mode queues are addressed at \x00trgq\x00<trigger>\x00<key>, where the touched key is itself the queue entry, which is why repeated touches coalesce. Changes-mode queues are addressed at \x00trgq\x00<trigger>\x00<seqno> with the change as the value, which is why events remain ordered and are never coalesced.

A runner thread drains each backlog in chunks, as a system transaction pre-seeded with deletions of the consumed entries. System transactions are exempt from capture, which is the mechanism behind the no-stacking rule. Because the consumed queue keys are in the drain transaction's write set, a touch landing after the drain's snapshot conflicts the commit and the drain re-runs against fresh state; ordinary optimistic concurrency control closes the race.

Fork construction

A fork at the head flushes the memtable, registers the cut under a brief hold of the manifest lock, hard-links every table and sealed value-log file, copies the value-log head up to its synced length, writes a fresh manifest and fork metadata, fsyncs, and publishes the result by a single rename. A fork at an earlier sequence number additionally rewrites the tables to that cut with one merge, retaining the newest version at or below the cut for each key, while values remain hard-linked. Pins are manifest records that re-register a snapshot at every open, before the background threads start.

Change stream

A subscription taps the apply path immediately after the visible sequence number is advanced, so delivery is ordered and gap-free from the point of installation. Entries carry unresolved value-log pointers, which the consumer resolves off the write path under an advancing snapshot pin, ensuring that value-log collection cannot unlink a file still in flight. A subscriber exceeding sub_queue_bytes is dropped rather than allowed to stall writers. GraphQL subscriptions, the journal and replication are all consumers of this single stream.

Instance identity and replication

The instance identifier is derived as a hash of the store name for a root store, and of the parent identifier, cut and fork name for a fork. Derivation is deterministic, so a crash between minting and persisting produces the same identifier again. File identifiers and offsets are unique only within one store lifetime; the instance identifier is the outer qualifier that every replica verifies on connection.

An edge replica copies the index fragments overlapping its scope, cross-checking bounds and sizes and verifying block checksums, applies the change stream into an overlay memtable, and resolves values inline first, then from its local cache, and finally by fetching from the master. Reads traverse the same merge and MVCC iterator stack as the engine's own.

Threads and lock order

Each store runs the user's writer threads, one flush thread, one compaction thread that also performs value-log collection, the commit thread and the trigger runner. An attached journal adds a drainer thread, and the GraphQL plane adds one forwarder thread per active subscription. A background failure degrades the store rather than leaving waiters blocked. The lock order is strict: write, then manifest, then state, then snapshots.

Glossary

The words the docs lean on, in one place.

Choosing the shape

Six shapes cover the work. Start from the category of database work you have, not from the tool.

Module or app code?

If your process is the only writer and the logic fits there, the embedded API is enough — db.begin() gives you the same OCC transaction an executor gets. Write a module when one of three things is true:

• The computation must run near the data. An aggregate over a million keys should not ship a million values to the client for five numbers.
• The invariant must hold for every writer. An installed executor is the same logic on every surface — Rust, shell, GraphQL — so no caller can skip the constraint.
• The logic must react to writes you don't control. Only a trigger sees every commit into a range, whoever made it.

Installed or one-shot?

Install what is part of the system: called repeatedly, backing a trigger, or exposed as a typed GraphQL field. Installed bytes are versioned in the store, recovered, forked and time-travelable, so the store can answer "what code ran here". One-shot what is an event: migrations, backfills, repairs. A one-shot leaves no record in the database — the script in your repo and its git history are the audit trail. Install the big audited migrations; one-shot the rest.

Keys mode or changes mode?

The two trigger modes are two different contracts, and the choice is about what your derived state is a function of.

• Keys mode reconciles. An event means "this key was touched — reconcile it". Re-touches coalesce. Right when derived state is a function of current state, as an index is: read the key, upsert or remove the entry, converge.
• Changes mode folds. Every committed op arrives once, in order, with its value. Right when you need op kinds, ordering, or per-op deltas — feeds, exact aggregates, cascades — where coalescing would destroy information.

Rule of thumb: if a replay of the same event must be harmless by re-reading, use keys mode; if it must be harmless by overwriting the same derived keys, use changes mode.

Queries in the database

The category: report-shaped reads — count, sum, rank, filter, limit — where the answer is small and the data it summarizes is not.

Doing this from the client means scanning the whole range over the API to compute a few numbers. A query module moves the loop into the database: it runs at one pinned snapshot (so the report is internally consistent), it can only read (writes return EROFS), and it is callable from every surface.

When to reach for it

• The result is much smaller than the data scanned.
• The whole answer must come from one consistent snapshot.
• You want the same report callable from Rust, the shell and GraphQL.

When not: point reads and plain range reads. get and scan are already optimal — wrapping them in a module adds a WASM invocation and removes nothing.

The shape

guests/agg is the minimal reference — count, sum, min and max over a prefix, as one fold over a scan:

#[fluent_guest::query]
fn agg(prefix: Vec<u8>) -> Result<Vec<u8>, Fail> {
    if prefix.is_empty() {
        return Err(Fail::new(2, "empty prefix not allowed"));
    }
    let scan = fluent_guest::scan_prefix(&prefix).map_err(|_| Fail::new(3, "scan failed"))?;
    let (mut count, mut sum, ..) = ..;
    for (_key, value) in scan {
        count += 1;
        // fold the value into the aggregates
    }
    Ok(out)                          // 40 bytes out, however many keys in
}

The category's contract, visible even in the trimmed loop: validate the input (distinct Fail code per failure class), scan once, fold, return the answer — never the rows.

The typed variant

guests/top_customers is the same category with a describe descriptor, so installing it (as topCustomers — the install name is the field name) creates a real GraphQL field — customers ranked by lifetime spend, floored and limited, computed inside the database at the operation's snapshot:

query { topCustomers(limit: 3, minTotalCents: "1000") { customer orders totalCents avgCents } }

Two category-level details worth copying from it: clamp caller-supplied limits in the module (it caps limit at 100 — the module is the last line of defense, whatever the surface), and decide the corrupt-record policy per role: a read-only report may skip a malformed record and log it, because it damages nothing; a writer never may (see Invariants & procedures).

Run it

db.query("agg", b"accounts/")?          // Rust
query agg accounts/                      # shell
{ wasm(module: "agg", input: {text: "accounts/"}) { hex } }   # GraphQL, untyped

Invariants & procedures

The category: writes whose correctness depends on what was read — transfers, unique claims, id allocation, multi-record updates that must land together.

The shape is the guarded write, and it is an execute module: each call runs in a fresh optimistic transaction, exit 0 commits, and a conflicting concurrent write re-runs the whole attempt against a fresh snapshot. The engine's OCC loop is what turns "read, decide, write" into an invariant that holds under concurrency.

When to reach for it

• More than one key must change together, or not at all.
• A constraint must survive concurrent writers: uniqueness, non-negative balances, monotonic ids.
• Every surface must go through the same logic — an installed executor cannot be bypassed by a caller doing raw puts if callers write through it.

When not: independent blind writes — a WriteBatch is already atomic. And if your Rust process is the only writer, an embedded Txn with get_for_update is the same machinery without the module.

The shape

guests/claim is a uniqueness invariant in one match statement — concurrent claimers race through OCC and exactly one wins:

let key = format!("uname/{}", input.username);
let already = match fluent_guest::get_for_update(key.as_bytes()) {
    Ok(Some(holder)) if holder == input.owner.as_bytes() => true,  // idempotent re-claim
    Ok(Some(holder)) => return Err(Fail::new(1, format!("taken by {..}"))),
    Ok(None) => {
        fluent_guest::put(key.as_bytes(), input.owner.as_bytes())
            .map_err(|_| Fail::new(3, "claim write failed"))?;
        false
    }
    Err(_) => return Err(Fail::new(3, "claim read failed")),
};

Everything the category demands is in those lines:

• get_for_update on every key a write depends on. That puts the key in the conflict set: two concurrent claims of one name cannot both commit — the loser re-runs, sees the winner, and fails cleanly.
• Idempotent under re-execution. The module is a pure function of (input, snapshot). A re-claim by the current owner is success ("already": true), not an error, so client retries and OCC re-runs are harmless.
• Distinct exit codes per failure class, message in the output — the caller can tell "taken" from "bad input" from "engine trouble".

Scaling the shape up

guests/transfer is the two-account balance move: both balances locked with get_for_update, insufficient funds as its own clean exit code, checked arithmetic throughout — an executor that overflows corrupts durable state.

guests/place_order is the full multi-write procedure: allocate a monotonic id from a counter key, write the order record, fold the amount into the customer's running stats — three coordinated writes in one transaction, which is exactly the point of an executor over plain put. Its strictest rule is the one to internalize: present-but-malformed state is corruption, not a default. An unparseable counter fails loudly with its own code — "reset to 1" would silently overwrite existing orders.

The caller's side

execute_retries bounds the engine's loop — 3 attempts by default. Under real contention (twenty writers on one key) those attempts get spent and Error::Conflict reaches the caller with nothing written. That is not a broken invariant; it is the caller's turn. Either raise execute_retries or retry from outside:

let out: Vec<u8> = loop {
    match db.execute("claim", &input) {
        Ok(out) => break out,
        Err(fluent31::Error::Conflict) => continue,   // retries spent; the whole call is safe to re-run
        Err(e) => return Err(e.into()),
    }
};

Over GraphQL the same outcome is CONFLICT in errors[].extensions.code, and the client retries the mutation. Because the executor is a pure function of its input and the data, re-running the whole call is always safe — which is why that rule matters.

Run it

cargo run -p fluent31 --example claim    # N concurrent claimers, exactly one winner, asserted
db.execute("claim", br#"{"username":"ada","owner":"a-1"}"#)?     // a typed module takes the same JSON here
mutation { placeOrder(customer: "you", amountCents: "4200") { id } }   # typed field: the module is installed as "placeOrder"

Secondary indexes

The category: finding records by something other than their key — a trigger plus a key convention.

An index in fluent31 is not a special structure — it is ordinary keys under their own prefix, maintained by a trigger so it stays current no matter who writes:

orders/00000042                   the record            (what the app writes)
idx/customer/acme/00000042  ""    the index entry       (what the trigger writes)
idx/order/00000042          acme  the back-pointer      (how updates find the stale entry)

The lookup is a prefix scan of idx/customer/acme/. No writer cooperates: plain puts, batches, transactions and other executors all keep the index current, because the trigger fires on every commit into the range.

The keys-mode shape

guests/customer_index is the reference. Keys mode fits indexing exactly because an index is a function of current state — the event only says "this key was touched", and the module reconciles:

let old = fluent_guest::get(&back_key);          // what the index says now
let cur = /* read the record; its customer, or None if deleted */;
if old == cur { continue; }                      // replay or no-op touch
if let Some(o) = &old {
    fluent_guest::delete(&stale_entry(o))?;      // unindex via the back-pointer
}
match &cur {
    Some(c) => {
        fluent_guest::put(&entry(c), b"")?;      // index current state
        fluent_guest::put(&back_key, c)?;
    }
    None => fluent_guest::delete(&back_key)?,   // gone: drop the back-pointer too
}

The two category rules this encodes:

• Reconcile, don't apply. Read current state and make the index match it. Written this way the module converges under replay, coalescing and reordering — all of which keys mode will do to you.
• Keep a back-pointer, and delete it with the record. The event carries no old value, so updates and deletes need the module's own record of what it indexed last time. Dropping the back-pointer on delete is not optional: if it lingered, a delete followed by a re-create would compare equal (old == cur) and the record would never be re-indexed.

Indexes created at runtime

guests/dynamic_index pushes the category further: index definitions are themselves ordinary keys. Writing idxspec/<name> = {"field": "city"} creates a fully backfilled index over that field; updating the spec swaps generations; deleting it tears the index down. One module backs two changes-mode triggers — one on the data range, one on the spec range — and the backfill runs inside the same transaction that consumes the spec event, so the index appears atomically, already complete.

It is also the answer to a sharp edge: trigger registration does not backfill. Keys already in the range fire no events. Either scan on demand as dynamic_index does, or re-put the range with a one-shot executor.

Run it

mktrig customerIndex customer_index orders/ orders0      # shell
cargo run -p fluent31 --example dynamic_index            # spec write → backfilled index, asserted

Views, feeds, cascades

The category: derived data that must be exact and ordered — live aggregate tables, audit logs and event streams, referential cleanup. All changes mode.

Where an index is a function of current state (reconcile — keys mode), everything on this page needs the actual sequence of committed operations: op kinds, order, values, one event per op. That is changes mode, and the engine's delivery guarantees — durable capture, exactly-once effects, no stacking — are what make these patterns correct rather than approximately correct.

Live aggregates

guests/live_stats keeps per-customer totals that are never recomputed. Every committed change adjusts the group's totals by exactly its delta:

let new = /* what this record is NOW: the event's inline value */;
let old = fluent_guest::get(&fold_key);   // what it contributed BEFORE
if old == new { continue; }
if let Some((customer, cents)) = &old { adjust(customer, -1, -cents)?; }
if let Some((customer, cents)) = &new { adjust(customer, 1, cents)?; }
// then record the new contribution under fold_key

Why this is exact and not merely close: the fold commits atomically with the events' consumption, so effects are exactly-once — totals cannot drift under retries, crashes or concurrency. Updates move a record between groups; deletes subtract it; the fold/ back-pointer is what makes both subtractable. The demo (cargo run -p fluent31 --example live_stats) proves it: after a concurrent write storm, the folded stats equal a full recount.

Changefeeds and audit logs

guests/order_feed materializes an ordered, durable changefeed — the CDC / audit-log / event-sourcing shape. One JSON entry per committed op, written under feed/<seqno, zero-padded>. Deriving the feed key from the seqno is the category's core trick: a replay after a crash or conflict overwrites the same entries instead of duplicating them.

Because it also declares a feed in its descriptor, installing it gives a typed live subscription. The idiom that falls out is the whole point:

history:  { scan(prefix: {text: "feed/"}) { ... } }        # replayable, replicable
live:     subscription { orderFeed { event { seqno op id record } } }

A disconnected consumer misses nothing durable — history is a scan, live is the tail of the same range. This is also how you build per-record history: the same shape, keyed history/<id>/<seqno>.

Cascades

guests/cascade_delete is referential cleanup: when a parent doc/<id> is deleted, scan and delete its doc/<id>/… subtree. Two contract details carry the pattern:

• Op kinds matter. Only Delete events of parent keys act; puts and descendant traffic are filtered out in code. Keys mode would have to read every touched key just to ask "was this a delete?".
• No stacking, by construction. The sweep deletes keys inside its own watched range, yet trigger writes never generate events — cascades cannot loop or amplify. One event, one sweep, done.

Run them

cargo run -p fluent31 --example live_stats       # folded stats == full recount, asserted
cargo run -p fluent31 --example cascade_delete
mktrig orderFeed order_feed orders/ orders0      # mode auto-detected from on_apply

Migrations & one-shots

The category: changes you make once — format migrations, backfills, data repair, ad-hoc admin jobs. An ordinary executor, invoked without being installed.

One-shot invocation (execonce in the shell, wasmExecuteOnce in GraphQL, db.execute_wasm in Rust) runs module bytes that are never stored: nothing is listed, cached or replicated, and the committed writes are the only trace. Same ABI, same SDK, same limits, same OCC retry loop. Triggers still fire on its writes, so trigger-maintained indexes and feeds stay correct straight through a migration.

The recipe

Walk a prefix, skip already-migrated records, rewrite the rest:

#[fluent_guest::execute]
fn user_v2(_input: Vec<u8>) -> Result<String, Fail> {
    let scan = fluent_guest::scan_prefix(b"user/").map_err(|_| Fail::new(2, "scan"))?;
    let mut migrated = 0u64;
    for (key, value) in scan {
        let old: serde_json::Value = serde_json::from_slice(&value)
            .map_err(|_| Fail::new(3, "corrupt record"))?;   // fail loudly, never default
        if old.get("v").is_some() {
            continue;              // already v2: idempotent re-run
        }
        let new = serde_json::json!({ "v": 2, "name": old });
        fluent_guest::put(&key, new.to_string().as_bytes())
            .map_err(|_| Fail::new(4, "put"))?;
        migrated += 1;
    }
    Ok(format!("migrated {migrated}"))
}

$ fluent-cli ./db
> execonce guests/target/wasm32-unknown-unknown/release/user_v2.wasm
migrated 41283

The whole migration is one transaction: atomic even across conflict retries, invisible until commit, serialized against concurrent writers by OCC. That is also its bound — the write set must fit max_txn_write_bytes (256 MiB) and the work must fit wasm_fuel. For a bigger keyspace, shard by cursor: take a start key as input, migrate up to N records, return the next start key, and drive the loop from the caller. Each chunk is then its own atomic, retry-tolerant transaction.

The category's rules

1. Idempotent by inspection. Detect an already-migrated record in the data itself and skip it. Never track "did it run" anywhere else — a retried attempt or a re-run must always be safe.
2. Fail loudly on the unexpected. A record that parses wrong is corruption, with its own exit code. A non-zero exit aborts the whole transaction: nothing half-migrated ever survives.
3. Rehearse on a fork. Fork the instance, run the one-shot against the fork (its own /graphql/<instanceId> endpoint under the server), inspect the result, then run it on the primary. See Forks in practice.
4. The repo is the audit trail. A one-shot leaves no record in the store, so the script and its git history are the record of what ran. Install the big audited migrations; one-shot the rest.

Forks in practice

The category: everything you'd want a copy of production for — rehearsal, staging, rollback anchors, backups — priced at hard-link cost.

fork("name") publishes a complete, consistent database directory under archive/<name>/, built from hard links: cost proportional to the file count, not the data. Opening it gives a live, writable, copy-on-write clone with its own identity — modules, triggers and data included. The playbooks below are the category; Forks, pins, clones has the mechanics.

The pre-change anchor

Before anything risky — a migration, a bulk import, a new trigger over live data:

db.fork("pre-migration")?;               // anchor now
// or: pin now, decide later whether to materialize the cut
let p = db.pin("pre-import")?;
let f = db.fork_at("rollback", p.seqno)?;

If the change goes wrong, rollback is a directory swap: stop the process, open the fork (or restore_to a pristine copy) as the new primary. Replicas notice the new identity and re-attach on their own. If the change goes right, delete_fork and move on. A pin is the lighter anchor — a durable GC hold you can still fork_at later — but it costs store-wide retention until unpin.

The rehearsal

Under the server, every fork is a full instance at /graphql/<instanceId> — same schema, same modules, its own data. That makes "try it on prod without trying it on prod" one mutation:

mutation { fork(name: "rehearsal") { instanceId } }
# run the migration one-shot against /graphql/<instanceId>, inspect the result,
# then run the same bytes against the primary — or delete the fork and rethink

This is the standing advice from Migrations & one-shots: rehearse every migration on a fork first, with the exact bytes you will run for real.

The staging clone

Open a fork and you have a second environment with real data that cannot touch the first: new writes land in the clone's own files, its compactions unlink only its own links, the parent is untouched. Forks branch, they do not follow — the clone sees nothing the parent commits after the cut, which is exactly what a staging environment wants.

The backup

A fork is a consistent snapshot backup on the same filesystem, instantly. For an off-box copy, copy archive/<name>/ — a plain directory tree — or restore_to onto a mount point; the hard links copy out as full files. For continuous off-box protection, that is the journal's job, not a fork cadence.

Identical cuts

Seeding two environments that must start byte-identical: capture one seqno, cut twice.

let s = db.seqno();
let a = db.fork_at("replica-a", s)?;
let b = db.fork_at("replica-b", s)?;      // same cut, same contents

What forks are not for

Per-document versioning and point-in-time recovery. Forks and pins are coarse, named, and few — a handful of deliberate cuts, each holding GC for the whole store while it matters. If you need history per record, materialize it with a changes-mode trigger; that is the consistency contract.

Coming from SQL

A complete translation of the relational vocabulary into fluent31: what each construct becomes, what it costs, and what has no equivalent at all.

Three things replace one language. Key layout takes the place of the schema: a record is bytes under a key you chose, and the order keys sort in is the only order the engine knows. Modules take the place of the query language: code installed in the database, invoked by name, running next to the data. Triggers take the place of the machinery that keeps derived data current: indexes, materialized views and cascades maintain themselves after every commit.

What is missing is the planner. Nothing chooses an access path for you — the access path is the range you scan and the order you scan it in, so the modelling decisions below are load-bearing in a way they are not in SQL.

Data modelling

The one modelling trap worth stating outright: bytewise order is not numeric order. orders/10 sorts before orders/9. Zero-pad to a fixed width (orders/00000009) and the two agree.

Reading

let one = db.get(b"orders/00000042")?;                          // by primary key
let page = db.iter(Some(b"orders/"), Some(b"orders0"), false)?;   // prefix, ascending
let newest = db.iter(Some(b"orders/"), Some(b"orders0"), true)?     // ORDER BY pk DESC LIMIT 10
    .take(10);
let mut next = last_key.to_vec(); next.push(0);                  // resume after last_key
let more = db.iter(Some(&next), Some(b"orders0"), false)?;

Writing

Transactions and locking

Constraints

One difference matters more than the table can show. In SQL a constraint is enforced by the engine against every writer. Here a constraint holds only because every writer goes through the executor that checks it: a raw put to the same key bypasses it, and a trigger cannot help, because triggers run after the commit and cannot veto a write. Treat the executor as the write API for constrained data, and keep raw puts for the ranges that carry no invariant.

Indexes

Views and aggregates

Procedures, triggers, notifications

Schema changes and migrations

Operations

What has no equivalent

Stated plainly, so the gaps are found here rather than late:

• Ad-hoc joins chosen by a planner. Every access path is written by hand, in a module or at the call site.
• Engine-enforced constraints. Foreign keys, CHECK, NOT NULL and column types hold only as far as the executor that checks them.
• Write vetoes. There is no BEFORE trigger and no rule system; nothing can reject a write after it has been made.
• Point-in-time recovery. Recovery targets a named cut, not an arbitrary instant.
• Multi-node writes. One process owns the store; replicas are read-only followers, and there are no distributed transactions.
• OFFSET paging, and ordering by a value you have not indexed. Both require an index you maintain, or a full scan.
• The SQL wire protocol. No psql, no JDBC or ODBC drivers; access is the embedded API, the shell, or GraphQL.
• Built-in full-text, geospatial and window-function libraries. They are code you write, or data you maintain.