@david_chisnall@infosec.exchange @futurebird@sauropods.win You don't have to anthropomorphise to understand that these models really love reward hacking. Text is important, because code IS text, and you can actuate a lot of the physical world through code.
Anthropomorphising is also easier for conversation, because the alternative would be to say:
"The sampled token sequence is consistent with the model having traversed a low-loss region of the policy manifold in which the KL-regularised objective, as shaped by RLHF reward model gradients, assigns high probability mass to trajectories that an external observer applying the intentional stance would parsimoniously compress as 'wanting Y'"
... each time you wanted to refer to a reasoning LLM's action trajectory...
