15Scientific Computing8 of 44 runs pass

PyTensor Matrix Assumptions

Add a matrix assumption system to PyTensor that infers and accepts structural facts about matrices and lets the linear-algebra rewrites use them.

Results

Each square represents one run. Score is the percentage of runs that pass: 0, 25, 50, 75, or 100 for four runs.

#ModelRolloutsPass@1ScoreTime
1GPT-6 Astra xhigh4/41006.3h
2GPT-6.1 Sol max3/475.07.4h
3Claude Opus 5.5 max1/425.05.8h
4GPT-5.6 Sol max0/404.1h
5Gemini 3.8 Flash high0/401.0h
6Qwen3.8 Max xhigh0/406.1h
7GLM 5.3 Flash max0/407.6h
8Muse Spark 1.3 max0/404.7h
9Kimi K3 max0/407.3h
10Grok 4.7 xhigh0/402.5h
11Claude Sonnet 5.5 max0/406.7h

Background

PyTensor is the tensor library behind PyMC. Linear algebra gets much faster when code knows a matrix is diagonal, triangular, symmetric, positive definite or orthogonal, but PyTensor has no general way to track those facts across a computation graph.

Task

The agent builds a matrix assumption system: six structural properties with three-valued answers (true, false or unknown), implications between them, and lazy inference through constructors, products, shape operations, indexing, elementwise arithmetic, batched operations and decompositions. User assertions must survive compilation, and a graph feature keeps cached facts correct while rewrites replace variables. Linear-algebra rewrites then use the facts. Unknown is always a safe answer; a true or false answer that is not justified is a bug, because rewrites act on it.

Workspace

The offline sandbox (4 CPUs, 8 GB, 8 hours) holds the PyTensor repository at the starting commit.

Verification

The committed HEAD is rebuilt in a clean checkout with the trusted starter tests and configuration overlaid. Adapted hidden upstream feature tests run in 15 families, and a secret-seeded generator draws stratified cases that run on both the submission and the reference: every inferred fact must be sound against the numeric truth, inference must be as complete as the sound reference, and rewrites must fire and give correct values. The starter suite must still pass, and every family must reach 90%.

Environment

Toolchain
Python
Compute
4 CPUs · 8 GB RAM
Network
Offline
Time limit
8 hours
All tasks