Pusan National University Researchers Propose New Frameworks for Financial AI Evaluation
Researchers from Pusan National University and international collaborators have introduced a new approach to financial artificial intelligence, emphasizing decision quality over mere prediction accuracy. The team developed the Signature-Informed Transformer (SIT), an AI model that considers how market prices evolve and assets interact, leading to better risk-adjusted investment decisions in U.S. and Chinese equity markets. In parallel, they evaluated 164 studies on financial LLMs from 2023–2025 and identified biases such as the use of future data, survivor bias, unrealistic testing environments, and ignored transaction costs that may inflate AI performance claims. To address these issues, the researchers proposed a Structural Validity Framework to assess whether AI systems are evaluated under realistic conditions. Together, the studies advocate for training financial AI with real-world decision-making in mind and suggest future applications like market
