$/M tokens is stupid

[u/lordmairtis]

https://medium.com/@zdengineering/anthropic-is-misleading-us-639b61fe01f0

Maybe I'm late to the party, or idk, but truly, Opus 5.5 has double the $/MT price as Sonnet 5.5, almost identical scores on the test, costs almost the same to do the test. GPT 6.1 Sol maybe (?) has barely worse results, same $/MT, tasks cost multitudes less. So great, we now need to measure this for ourselves too!

And don't even get me started on simple tasks. It is "intelligent", but if you use the larger model accidentally, simple prompts will eat your limits for lunch. We need proxies, routers and all that just to make sure Opus doesn't get easy questions, but then again, we need to check whether the model used is actually cheaper for the task, or just by the published numbers....

</rant>

Feels like the only way forward is really just running some simpler task for all new models and check if a new model does better or the one we are using already, and for how much... does not guarantee better and/or cheaper results on the actual work still.