Groundwater Nitrate Legacy Lag
Predict groundwater nitrate response to proposed fertilizer programs from monitoring data, accounting for the multi-decade legacy lag in the unsaturated zone and aquifer.
Results
Each square represents one run. Score is the percentage of runs that pass: 0, 25, 50, 75, or 100 for four runs.
| # | Model | Rollouts | Pass@1 | Score | Time |
|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol max | 2/4 | 5.9h | ||
| 2 | GPT-6 Astra xhigh | 1/4 | 3.8h | ||
| 3 | Claude Opus 5.5 max | 0/4 | 6.3h | ||
| 4 | GPT-5.6 Sol max | 0/4 | 1.6h | ||
| 5 | Gemini 3.8 Flash high | 0/4 | 0.3h | ||
| 6 | Qwen3.8 Max xhigh | 0/4 | 4.7h | ||
| 7 | GLM 5.3 Flash max | 0/4 | 5.5h | ||
| 8 | Muse Spark 1.3 max | 0/4 | 1.1h | ||
| 9 | Kimi K3 max | 0/4 | 4.4h | ||
| 10 | Grok 4.7 xhigh | 0/4 | 0.9h | ||
| 11 | Claude Sonnet 5.5 max | 0/4 | 5.1h |
Background
Midwestern states set nitrate load targets for rivers draining tile-drained farmland, choosing between fertilizer cuts, cover crops and restored wetlands. The data are what a state agency holds: 85 years of county nitrogen budgets, continuous nitrate sensors at four nested gauges with gaps and drift, older monthly grab samples, and tracer ages from a dozen wells. None measures the lag directly. Fast tile flow and slow groundwater stores hold nitrogen on different timescales; the residence times control whether a 20-year forecast is useful.
Task
The agent delivers a Python model whose predict(scenario) returns the annual flow-weighted nitrate-N concentration at each of four gauges for every year 2025 to 2049. The scenario provides future county inputs, wetland fractions, weather and discharge. Three programs are graded, each ramping in over five years: a 30% commercial fertilizer cut, cover crops on 40% of acreage, and wetlands treating 45% of tile drainage in two of the four areas. The cover-crop effect and wetland removal model are stated. Only NumPy, SciPy and pandas may be used, with 300 seconds wall, 600 seconds CPU and 4 GB per call.
Workspace
The offline sandbox (8 CPUs, 16 GB, 8 hours) holds daily discharge and sensor nitrate at four gauges for 2010 to 2024, from sensors with staggered start dates and inter-visit drift. Additional records include grab samples back to 1978, county nitrogen budgets back to 1940, daily weather, basin geography and tritium/CFC-12 ages from 12 wells. An example scenario and checker are provided. The basin's true response is held out.
Verification
The model runs once per program in a separate offline environment, workspace read-only. Three programs at four gauges make 12 cases, each compared with the basin's true response. A case passes when the predicted 2045-to-2049 mean is within 15% of truth and the predicted milestone year within 3 years. The milestone is the last year of the first five-year window whose mean drops to 75% of the 2020-to-2024 level; a matching "not by 2049" counts. At least 10 of 12 must pass, including all four fertilizer-cut cases.
Environment
- Toolchain
- Python
- Compute
- 8 CPUs · 16 GB RAM
- Network
- Offline
- Time limit
- 8 hours
References
- Iowa Nutrient Reduction Strategy: A real state strategy of the kind the task's committee serves, weighing fertilizer management, cover crops and wetlands against a nitrate load target.