PyTensor Matrix Assumptions
Add a matrix assumption system to PyTensor that infers and accepts structural facts about matrices and lets the linear-algebra rewrites use them.
Results
Each square represents one run. Score is the percentage of runs that pass: 0, 25, 50, 75, or 100 for four runs.
| # | Model | Rollouts | Pass@1 | Score | Time |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra xhigh | 4/4 | 6.3h | ||
| 2 | GPT-6.1 Sol max | 3/4 | 7.4h | ||
| 3 | Claude Opus 5.5 max | 1/4 | 5.8h | ||
| 4 | GPT-5.6 Sol max | 0/4 | 4.1h | ||
| 5 | Gemini 3.8 Flash high | 0/4 | 1.0h | ||
| 6 | Qwen3.8 Max xhigh | 0/4 | 6.1h | ||
| 7 | GLM 5.3 Flash max | 0/4 | 7.6h | ||
| 8 | Muse Spark 1.3 max | 0/4 | 4.7h | ||
| 9 | Kimi K3 max | 0/4 | 7.3h | ||
| 10 | Grok 4.7 xhigh | 0/4 | 2.5h | ||
| 11 | Claude Sonnet 5.5 max | 0/4 | 6.7h |
Background
PyTensor is the tensor library behind PyMC. Linear algebra gets much faster when code knows a matrix is diagonal, triangular, symmetric, positive definite or orthogonal, but PyTensor has no general way to track those facts across a computation graph.
Task
The agent builds a matrix assumption system: six structural properties with three-valued answers (true, false or unknown), implications between them, and lazy inference through constructors, products, shape operations, indexing, elementwise arithmetic, batched operations and decompositions. User assertions must survive compilation, and a graph feature keeps cached facts correct while rewrites replace variables. Linear-algebra rewrites then use the facts. Unknown is always a safe answer; a true or false answer that is not justified is a bug, because rewrites act on it.
Workspace
The offline sandbox (4 CPUs, 8 GB, 8 hours) holds the PyTensor repository at the starting commit.
Verification
The committed HEAD is rebuilt in a clean checkout with the trusted starter tests and configuration overlaid. Adapted hidden upstream feature tests run in 15 families, and a secret-seeded generator draws stratified cases that run on both the submission and the reference: every inferred fact must be sound against the numeric truth, inference must be as complete as the sound reference, and rewrites must fire and give correct values. The starter suite must still pass, and every family must reach 90%.
Environment
- Toolchain
- Python
- Compute
- 4 CPUs · 8 GB RAM
- Network
- Offline
- Time limit
- 8 hours