The price is wrong: AI cost calculation has to consider task completion rates, not just token costs

When it comes to AI services, you don't necessarily get what you pay for. It turns out that AI models with expensive tokens may cost less than models with cheap tokens for particular tasks. And the tooling attached to those models can have a significant effect on cost and output quality. Databricks, which sells data analytics software and services, recently devised an internal coding benchmark to assess the tradeoff between price and performance using various AI models. Matei Zaharia, CTO of Dat