General Context Labs
10Software Engineering15 of 44 runs pass

RocksDB Wide-Column Blobs

Add wide-column entity support to RocksDB blob files with direct write, including the compaction-filter interface changes it requires.

Results

Each square represents one run. Score is the percentage of runs that pass: 0, 25, 50, 75, or 100 for four runs.

#ModelRolloutsPass@1ScoreTime
1GPT-6 Astra xhigh4/41001.9h
2GPT-6.1 Sol max4/41003.8h
3Claude Sonnet 5.5 max4/41006.6h
4Grok 4.7 xhigh2/450.01.2h
5GLM 5.3 Flash max1/425.07.6h
6Claude Opus 5.5 max0/404.5h
7GPT-5.6 Sol max0/403.6h
8Gemini 3.8 Flash high0/402.3h
9Qwen3.8 Max xhigh0/405.8h
10Muse Spark 1.3 max0/404.4h
11Kimi K3 max0/406.0h

Background

RocksDB is an embedded key-value store built on a log-structured merge tree (LSM). Wide-column entities are always stored inline, despite the tree's v2 layout supporting per-column blob references. Upstream #14386 made entities first-class BlobDB citizens; the workspace starts from its parent commit. The feature cuts across the engine: flush and compaction must separate columns into blob files, every read path must hide the separation, garbage collection must relocate entity columns, and the compaction filter API must grow a lazy interface without breaking existing filters.

Task

The agent makes each entity column eligible for blob storage under the same options and thresholds as plain values, keeping the old encoding for fully-inline entities so older releases can read them. Every read API, iterator, merge, snapshot and read-only instance must return what it would for an inline entity, and entity-referenced blob files follow the same lifetime and garbage-collection rules as any other. The interface is pinned: a column resolver class, a FilterV4 compaction-filter method that lets existing FilterV3 filters keep seeing resolved values, and a partition selector for blob direct write. The on-disk format must use the documented layout so other builds can read it. Only committed work is graded.

Workspace

The offline sandbox (4 CPUs, 8 GB, 8 hours) holds a RocksDB checkout with full history at the parent of the upstream landing, the library and main test binaries already built. The hidden feature tests, scenario driver and any build of the finished feature are held out.

Verification

The committed tree is rebuilt from scratch in a separate environment. Shared test infrastructure must be untouched, and build-file changes are limited to listing new files. The verifier compiles its own copies of the existing tests in 32 binaries, and all must pass, along with upstream's adapted feature tests. 94 seeded scenarios in nine families run on both the submission and the upstream implementation, comparing column placement after flush and compaction, read transparency, garbage collection, filter behavior, blob direct write, merges, snapshots and cross-build interoperability. A pass requires every family to pass.

Environment

Toolchain
C++
Compute
4 CPUs · 8 GB RAM
Network
Offline
Time limit
8 hours

References

  • RocksDB (GPL-2.0 or Apache-2.0): The store the agent extends; #14386 and a follow-up fix form the upstream implementation the scenarios are compared against.
  • RocksDB wiki: BlobDB: Describes integrated BlobDB, its options, garbage collection and the key-value separation the task extends to entities.
All tasks