RocksDB Wide-Column Blobs
Add wide-column entity support to RocksDB blob files with direct write, including the compaction-filter interface changes it requires.
Results
Each square represents one run. Score is the percentage of runs that pass: 0, 25, 50, 75, or 100 for four runs.
| # | Model | Rollouts | Pass@1 | Score | Time |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra xhigh | 4/4 | 1.9h | ||
| 2 | GPT-6.1 Sol max | 4/4 | 3.8h | ||
| 3 | Claude Sonnet 5.5 max | 4/4 | 6.6h | ||
| 4 | Grok 4.7 xhigh | 2/4 | 1.2h | ||
| 5 | GLM 5.3 Flash max | 1/4 | 7.6h | ||
| 6 | Claude Opus 5.5 max | 0/4 | 4.5h | ||
| 7 | GPT-5.6 Sol max | 0/4 | 3.6h | ||
| 8 | Gemini 3.8 Flash high | 0/4 | 2.3h | ||
| 9 | Qwen3.8 Max xhigh | 0/4 | 5.8h | ||
| 10 | Muse Spark 1.3 max | 0/4 | 4.4h | ||
| 11 | Kimi K3 max | 0/4 | 6.0h |
Background
RocksDB is an embedded key-value store built on a log-structured merge tree (LSM). Wide-column entities are always stored inline, despite the tree's v2 layout supporting per-column blob references. Upstream #14386 made entities first-class BlobDB citizens; the workspace starts from its parent commit. The feature cuts across the engine: flush and compaction must separate columns into blob files, every read path must hide the separation, garbage collection must relocate entity columns, and the compaction filter API must grow a lazy interface without breaking existing filters.
Task
The agent makes each entity column eligible for blob storage under the same options and thresholds as plain values, keeping the old encoding for fully-inline entities so older releases can read them. Every read API, iterator, merge, snapshot and read-only instance must return what it would for an inline entity, and entity-referenced blob files follow the same lifetime and garbage-collection rules as any other. The interface is pinned: a column resolver class, a FilterV4 compaction-filter method that lets existing FilterV3 filters keep seeing resolved values, and a partition selector for blob direct write. The on-disk format must use the documented layout so other builds can read it. Only committed work is graded.
Workspace
The offline sandbox (4 CPUs, 8 GB, 8 hours) holds a RocksDB checkout with full history at the parent of the upstream landing, the library and main test binaries already built. The hidden feature tests, scenario driver and any build of the finished feature are held out.
Verification
The committed tree is rebuilt from scratch in a separate environment. Shared test infrastructure must be untouched, and build-file changes are limited to listing new files. The verifier compiles its own copies of the existing tests in 32 binaries, and all must pass, along with upstream's adapted feature tests. 94 seeded scenarios in nine families run on both the submission and the upstream implementation, comparing column placement after flush and compaction, read transparency, garbage collection, filter behavior, blob direct write, merges, snapshots and cross-build interoperability. A pass requires every family to pass.
Environment
- Toolchain
- C++
- Compute
- 4 CPUs · 8 GB RAM
- Network
- Offline
- Time limit
- 8 hours
References
- RocksDB (GPL-2.0 or Apache-2.0): The store the agent extends; #14386 and a follow-up fix form the upstream implementation the scenarios are compared against.
- RocksDB wiki: BlobDB: Describes integrated BlobDB, its options, garbage collection and the key-value separation the task extends to entities.