SigmaCore.ai Benchmark Study Finds Standard LLMs Achieve Only 63.5% Accuracy in Financial Math
SigmaCore.ai has released a benchmark study titled "Can We Trust the AI to Do the Math?", which evaluates the reliability of Large Language Models (LLMs) in commercial financial applications. The research, involving 3,400 test cases, revealed that standard generative LLMs achieve only a 63.5% precision rate in direct mathematical computations. Even when augmented with code execution, accuracy was inconsistent due to logic vulnerabilities and incorrect formula selection. In contrast, SigmaCore's proprietary deterministic engine, ElizaZH (known as Eliza), achieved a 100% exact precision rate across all tested financial scenarios, including loan amortization and three-year projections. The company is currently in advanced discussions with multiple financial institutions to deploy its Eliza platform for automated business planning and SBA documentation workflows.
Topics: AI Precision Deterministic AI Financial Technology Large Language Models