KDB

What it is

KDB is a hybrid: a database and a version control system in one engine, holding three kinds of thing under a single commit.

Those are not three subsystems bolted together. A single transaction writes the bytes, the metadata document and the commit atomically, so a file and the record describing it can never drift apart, and both are versioned by the same history.

The engine is not trying to be a general-purpose SQL database — SQL is an index and query interface over data whose storage model is documents, not the storage model itself. And there is no central authoritative node: every peer is equal.

The database half

Declare a schema when you want one and it becomes a typed, indexed lens over your documents. It never constrains what a document may contain — a document with fields the schema has never heard of is stored intact, and those fields stay reachable. SQL addresses data through the schema; the _doc column always gives access to the whole document, so one query can filter on typed columns and return raw JSON.

On top of that sits an ordinary database surface:

• KDB-SQL — joins, group-by, DML and DDL, views, and GRANT/REVOKE for access control.
• A JDBC driver — the highest-priority integration surface, because it is what makes the engine usable from Java ORMs, SQL IDEs and BI tools without any of them knowing that the thing underneath is a versioned document store. There is a database/sql driver on the Go side and a Hibernate integration on the JVM.
• Indexes — hash, B-tree, composite, full-text, and vector similarity as an optional index type.
• Transactions — idempotent, keyed by transaction id, with compare-and-set preconditions, unique-key enforcement and leases carrying monotonic fence tokens.
• Stored procedures — described below.

Stored procedures

Named, versioned scripts that run inside the backend process, close to storage, and are called like a SQL statement — so logic can be pushed to the data instead of round-tripping whole documents to a client.

They are written in a restricted subset of JavaScript: no filesystem, no network, no process, no native modules, and no require of anything but an injected kdb binding. The runtime is GraalVM Polyglot, whose sandboxing primitives and resource limits are what make a restricted subset enforceable rather than merely documented.

A script never talks to storage directly. Its host API proxies into the same authorized entry points that ordinary SQL and document requests already use, and every call re-enters the authorizer against the calling principal — so a procedure adds no new privileged path into storage, and does not run with elevated owner rights.

Binary files, with metadata you can query

A file in KDB is three separate things that a single commit keeps consistent:

Every logical file has a stable fileId that never changes across versions, so "this file, over time" is a first-class thing to ask about. Files may be stored individually or grouped into a bundle under a shared bundleId — one archive blob, or a manifest plus member blobs — and a bundle member can be retrieved on its own without unpacking the rest.

Because blobs are content-addressed, a file that has not changed between two commits is stored once no matter how many versions point at it. That is the same property that makes document history cheap, applied to bytes: keeping fifty versions of a document set with a large unchanged attachment costs one copy of the attachment.

kdb file put  <namespace> [--id UUID] [--zip] <local-path>
kdb file put  <namespace> --bundle <UUID> [--zip] <paths...>
kdb file get  <namespace> --id <UUID> [-o path]
kdb file get  <namespace> --bundle <UUID> --member <fileId> [-o path]
kdb file meta <namespace> --id <UUID> | --bundle <UUID>

Metadata fields can be declared in the schema and indexed, which means "every PDF over 10 MB added to this namespace last month" is a SQL query, not a directory walk.

The version control half

Every namespace carries a full commit DAG — branches, tags, checkout, diff. The CLI is modelled on git deliberately, because the mental model transfers. Seven concepts carry the whole model:

Because the tree is a persistent Merkle structure, committing costs O(changed documents) rather than O(namespace size), and two versions that share content share memory and disk. The schema is versioned alongside the data, so a migration is a commit like any other and can be branched, diffed and reverted like any other.

Peers, not replicas

Synchronisation is source control rather than replication. Peers are equal and any peer can sync with any other directly. Divergence is normal; merging is explicit and application-controlled. Conflicts are detected by comparing content hashes at the base and target trees — so two clients writing different documents do not conflict, and two writing identical content do not conflict either. When a real conflict exists it is surfaced to the application; KDB never silently resolves one.

Clients pick one of three modes: pure stream, write-back stream, or full peer.

Two implementations, one specification

The Kotlin Multiplatform tree is the original implementation: the entire engine compiles to the browser (Kotlin/JS), the JVM, and native targets from one codebase. A Go port mirrors the same layered architecture for native servers, CLI, database/sql and browser WASM.

Both follow the same specification and are kept in step by cross-language wire golden tests — a frame produced by one must be byte-identical to the frame produced by the other. Digests are RFC 6234 SHA-256 in both trees.

The split is deliberate: Go is the deployment target because removing the JVM from a deployment is decisive on small instances, and Kotlin is the multiplatform reference because it reaches the browser and the JVM from one source. The golden tests are what make having two implementations safe rather than expensive.

Underneath

The engine core — transaction engine, commit DAG, query engine, schema engine, index layer — is the same code whether it runs inside a CLI process or behind a server. A server adds sessions, admission control and listeners; it does not change the semantics of a commit.

Storage is two durable stores, which is the design's most consequential choice:

• a delta log holding commits, and
• an LSM store (write-ahead log, memtable, SSTables) holding blobs.

The commit log alone can rebuild a namespace, so backup, verify and restore all operate on one simple append-only artefact. Segments are named by zero-padded sequence, making file order equal commit order for any object store without a separate index — but replay still applies commits topologically rather than trusting that order, so an ordering bug degrades to a slower open instead of an unopenable namespace. A torn tail on the newest segment is tolerated as the expected shape of an unclean shutdown; corruption anywhere else is treated as real and never silently truncated.

Writes are serialised per namespace by a bounded, deadline-aware write gate rather than a mutex, because a mutex cannot express "queue full" or "your deadline passed". Durability is decoupled from serialisation — log position is fixed under the gate and the fsync happens after releasing it — so concurrent commits share one physical sync. Storage is tiered hot → warm → cold → ice archive, and every failure carries a typed error code with retry semantics, because a client must decide "retry / retry later / never retry / resubmit smaller" without parsing prose.

Quick start

Go

cd go && go test ./...
make build-go

./go/bin/kdb --data-dir /tmp/kdb-data init myapp/users
OUT=$(./go/bin/kdb --data-dir /tmp/kdb-data put myapp/users '{"name":"Ada"}')
# {"docId":"<uuid>","docIdShort":"<8-hex>","commit":"<64-hex>"}
./go/bin/kdb --data-dir /tmp/kdb-data get myapp/users "$(echo "$OUT" | jq -r .docId)"

Kotlin

./gradlew build
./gradlew :kdb-cli:runCli --args="init myapp/users"
./gradlew :kdb-jdbc:test

Embedding

// JVM — JDBC, in-memory
Class.forName("dev.kdb.jdbc.KdbDriver");
Connection c = DriverManager.getConnection("jdbc:kdb:memory:///demo/users");

// Go — database/sql
import _ "github.com/limidus/kdb/go/kdb/driver"
db, _ := sql.Open("kdb", "kdb://memory:///demo/users?unique=true")

The Go module declares itself as github.com/limidus/kdb/go, but the repository currently lives at github.com/dbeasty/kdb — so go get on that path does not resolve yet. Until the two agree, depend on it from a local checkout with go mod edit -replace github.com/limidus/kdb/go=../kdb/go.

File persistence uses the server storage engine with delta replay and an exclusive .kdb.lock while open — jdbc:kdb:file://… and --data-dir on the JVM, --data-dir and kdb://file://… in Go. Releases ship Go binaries, Kotlin jars and an embeddable Go source zip carrying only the packages a downstream project actually reaches; every binary reports the version and the exact commit it was built from, and a build from a dirty tree is marked as such.

Where it actually stands

KDB is an early implementation and says so. This is what is real today rather than what is planned:

APIs and behaviour are subject to change. Known limitations are listed exhaustively in the low-level design rather than summarised away.

Read further

• High-level architecture — the container view, key decisions and their rationale, quality attributes and risks. Start here.
• User guide — running the CLI or server, embedding, and operating it: backup, verify, restore, governance.
• File attachments spec — metadata, bundles, compression and blob reachability.
• Stored procedure spec — the sandbox, the host API and the authorization path.
• Low-level design — implementation reference in seven parts: components, flows, concurrency, storage, KDB-SQL, protocol and operations.
• Architecture specification — the full system design, protocols and layer specs.

Licensed under the GNU Affero General Public License v3.0. Copyright © 2026 Limidus Corp.