egglog Container Proofs
Extend egglog proof mode to container sorts so equalities over sets, vectors and maps carry explicit proofs through the term and proof encodings.
Results
Each square represents one run. Score is the percentage of runs that pass: 0, 25, 50, 75, or 100 for four runs.
| # | Model | Rollouts | Pass@1 | Score | Time |
|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 max | 2/4 | 6.7h | ||
| 2 | Claude Opus 5.5 max | 1/4 | 4.2h | ||
| 3 | Qwen3.8 Max xhigh | 1/4 | 6.8h | ||
| 4 | GPT-6 Astra xhigh | 0/4 | 5.6h | ||
| 5 | GPT-5.6 Sol max | 0/4 | 3.3h | ||
| 6 | Gemini 3.8 Flash high | 0/4 | 3.4h | ||
| 7 | GLM 5.3 Flash max | 0/4 | 7.9h | ||
| 8 | Muse Spark 1.3 max | 0/4 | 6.8h | ||
| 9 | Kimi K3 max | 0/4 | 7.2h | ||
| 10 | Grok 4.7 xhigh | 0/4 | 3.4h | ||
| 11 | GPT-6.1 Sol max | 0/4 | 6.6h |
Background
egglog is a Rust engine combining equality saturation with Datalog. Programs declare terms, rules and rewrites; the engine grows an e-graph until it can answer queries or extract the best equivalent term. Its proof mode gives every derived fact a proof an independent checker can replay, but until upstream #927 it rejected container sorts (Vec, Set, Map, MultiSet, Pair). The e-graph never unions containers directly: it rebuilds them into normal form whenever their elements' equivalence classes merge. After a union of A and B, {A, B} as a set collapses to one element, and rows holding those containers become congruent. Proof mode must reproduce each consequence and justify it from terms alone.
Task
The agent makes all five container sorts, including nested ones, work under term encoding, proof encoding and proof testing with the same observable results as the default e-graph. Containers work wherever the language allows them, including rule bodies, and every fact from container rebuilding needs a proof the existing checker accepts. The proof format is pinned: containers appear as constructor applications in canonical structural order independent of value ids, maps use a flat map-of primitive, and a normalization step proves each rebuilt container equal to its sorted and deduplicated form. No existing test may be removed or weakened.
Workspace
The offline sandbox (4 CPUs, 16 GB, 8 hours) holds an egglog checkout at commit 4141a970, the pinned Rust 1.91.0 toolchain, cargo-nextest and every locked crate, a prebuilt release build and example programs. Upstream tests for the feature, the program generator and reference build are held out.
Verification
The committed tree is rebuilt from a clean checkout with existing integration tests and snapshots restored. The whole suite must pass, and hidden upstream tests must pass. Secret-seeded programs cover nine families, from each container sort and nesting to multi-round rules and containers in rule bodies. Each runs in default, term-encoding, proof and proof-testing modes, with results compared against the upstream implementation and every proof replayed by an independent checker. A pass requires every family to reach 90%.
Environment
- Toolchain
- Rust
- Compute
- 4 CPUs · 16 GB RAM
- Network
- Offline
- Time limit
- 8 hours
References
- egglog: The engine the agent extends, starting at the parent of #927, the upstream change that added container proofs.
- Zhang et al.: Better Together: Unifying Datalog and Equality Saturation (PLDI 2023): The paper introducing egglog's design.