An SQL database for Go that stores everything in S3. Written by agents, trusted with simulation testing.
s4ql is an embedded database engine in the spirit of SQLite. Your Go application talks to the bucket directly, and there is no server to run. Every line of it is written by AI agents, and no human reviews the code. Trust has to come from verification.
- Experiment
- Storage built
- SQL not built
- Not ready for use
Why it exists
Two experiments in one repository. One is a database that needs nothing but object storage. The other is whether software that nobody reads can still be trusted.
- Nothing to operate
- Your Go application talks to S3 directly through the module. There is no server, no cluster, and no replica to look after. The bucket is the only infrastructure and the source of truth.
- A database per user
- Each database is a key prefix, so one bucket holds as many databases as you like. One database per user costs nothing extra, and write-heavy work spreads across them.
- SQLite's SQL, strictly
- s4ql speaks a strict subset of SQLite's SQL. Whatever it accepts behaves exactly as in SQLite. Everything else is rejected, never reinterpreted.
- Trust without reading
- AI agents write and own the code. Markus sets the goals and takes part in design and architecture; he does not review code. The bet is that a database can earn trust when its correctness is checked, not read.
What it does
A database in S3 is a log. Each commit is an immutable object named by the SHA-256 of its content and linked to its parent, as in git. One small file, manifest.json, points at the newest commit. Swapping that pointer is the commit.
manifest.json
{
"version": 0,
"branches": {
"main": {
"head": "d39febcf6aba…"
}
},
"lease": {
"holder": "5b0e3a4f9c1d77c3",
"duration": "30s",
"renewed": "2026-11-12T12:00:30.123Z",
"salt": "c41d07a9e2b35f68"
}
}
A compare-and-swap replaces this file. The transaction is committed when the swap succeeds, and not before.
Illustration. Each bar code is the 256 bits of a real SHA-256, taken over sample content. A real commit object records the rows it changed, with the SQL kept beside them as a note.
- Transactions
- ACID within one database. No transaction spans databases.
- Writes
- One write transaction at a time per database. A second writer waits until the first has committed or rolled back.
- Reads
- In parallel, from a local SQLite cache, without contacting S3. Each read sees one consistent state, even while a write is in progress.
- Cache
- Holds the same tables and rows as the database, is never ahead of S3, and can be rebuilt from the log at any time.
- Lease
- One instance at a time holds the database's lease, recorded in the manifest. It lasts 30 seconds and is renewed every 20. A spare instance waits and takes over when the lease times out.
- Unknown outcome
- If the swap ends without an answer, the instance asks S3 until it knows. The transaction is committed if and only if its commit object is in the log.
- S3 must offer
- Strong consistency, compare-and-swap, and create-if-absent.
- Interface
- Go's
database/sql. Agreed, not built yet.
How it is built
Agents write the code and nobody reviews it, so the tests have to show that it works. The production code runs unchanged inside a simulation where S3, time, and randomness are driven by one seed. The same seed always replays the same run, failures included.
- Deterministic simulation
- S3, clocks, and randomness are reached only through interfaces. One seed controls a whole run, so any failure the simulation finds can be replayed exactly.
- Buggify
- Marked points in the production code do something legal but unusual in simulation: return an error, add a delay, take a rare branch. Each run switches on a different subset.
- SQLite is the oracle
- s4ql writes its own parser and engine, and checks them against SQLite. Results and errors are compared with SQLite's, and the parser is run on SQLite's own test statements.
- Real S3
- A conformance suite and integration tests run against a real S3-compatible bucket on Tigris, to check that the simulated store behaves like the real one.
- Tests that test the tests
- Property-based tests, fuzzing, and mutation testing. Bugs are planted on purpose to count how many seeds the simulation needs to find each one.
- Everything written down
- The spec, the research, the builders' diaries, and the conversations behind each decision are in the repository.
Status
As of 16 November 2026. s4ql is an experiment. It is not ready for use, and anything in it may change.
| Built |
|
|---|---|
| Not built | The pass after the parser, the engine, and the rest of the SQL. There is no engine and no public API yet, so the database cannot be used. |
| Next | The first slice of SQL: |
Roadmap
SQL grows in small slices. Nothing is supported until it is listed, all the way down to select 1, and each slice goes through every layer and is checked against SQLite before the next one starts.
- Built
- Agreed, not built
- Open
-
Built
Storage core
The log of commit objects, the manifest swap, the lease, crash recovery, and the local cache.
-
Built
S3 client
Uploads are signed with the commit object's SHA-256, so the store refuses content that does not match it.
-
Built
Tokenizer
The first piece of the SQL front end.
-
Built
Parser
s4ql's own parser for all of SQLite's grammar. It agrees with SQLite on which statements parse, on the message of each syntax error, and on the shape of the syntax tree.
-
Agreed, not built
The pass after the parser
It rejects whatever the roadmap does not list, and it raises the errors that SQLite's grammar actions raise while they parse.
-
Agreed, not built
Slice 1: one integer column
create table,insert,select *, anddrop tableon a strict table with oneint primary key. The answers must survive a restart with an empty cache. -
Agreed, not built
The driver
A
database/sqldriver as the main interface. -
Open
More SQL, slice by slice
Each slice is agreed before it is built. Every table is strict and without rowid, and foreign keys are always enforced.
-
Open
Storage format version 1
The formats are drafts until then. From version 1 on, every version of s4ql reads what any older version wrote.
Inspired by
s4ql builds on ideas from other databases and storage engines. How each idea works in its own project is written up in docs/inspiration.md.
- SQLite
- The embedded database s4ql is modelled on. s4ql speaks a subset of its SQL, checks its own answers against it, and uses it for the local cache.
- FoundationDB
- A distributed, transactional key-value store, tested in deterministic simulation. s4ql's simulation and its buggify points follow its example, and so does the rule that an application checks its own data after an unknown commit outcome.
- Graft
- A storage engine in Rust that replicates SQLite pages to object storage as an append-only log of commits, with no server in between.
- SlateDB
- An embedded key-value store in Rust that keeps all its files in object storage, with a manifest that lists the current state of the database.
- Turbopuffer
- A search service where object storage is the only source of truth and each namespace is a prefix in the bucket, with caches in memory and on disk in front of it.
- Litestream
- A tool in Go that replicates a SQLite database to object storage from its write-ahead log, and a SQLite plugin that queries the copy in object storage directly.
Follow along
The code, the spec, the open questions, and every conversation behind the decisions are public. Watch the repository to see whether a database written by agents can earn trust.
maragudk/s4ql on GitHub