Paul Graham Argues AI Companies Are Misusing Token Counts as a Metric for Inference
Y Combinator co-founder Paul Graham has criticized the current method of measuring and pricing AI inference based on token counts. He argues that this metric is becoming outdated, as more advanced models can solve increasingly complex problems with fewer tokens. Instead, he suggests defining a new standard focused on problem-solving capability rather than just counting generated tokens. Graham proposed using a series of progressively difficult tasks to evaluate an AI's performance, where each task could be solved by one model iteration. This approach would better reflect the true value and efficiency improvements in newer models. His comments align with recent findings from AlphaSense that higher-priced frontier AI models can deliver superior results at lower overall costs for complex financial analysis. The discussion comes amid rising concerns about inference costs, infrastructure expenses, and potential interest rate hikes impacting the AI investment trade. Chinese AI model usage has also surged among U.S. firms, raising questions about global competition in the space.
Topics: Artificial Intelligence Inference Metrics Token Pricing Y Combinator