Isomorphic Perturbation Testing
🔍
1
Evaluate rule hypotheses for genuine reasoning vs shortcuts
Do reasoning LLMs actually reason — or learn to game the test? IPT allows for detecting reward hacking in inductive programming tasks (SLR-Bench).
Evaluate rule hypotheses for genuine reasoning vs shortcuts
Reward shortcut behavior in LLMs via IPT