DeepSeek's Latest Update Scores Lower on Accounting Work as Open Models Trail on Consistency, DualEntry Benchmark Finds.

The recent DualEntry Accounting AI Benchmark results indicate that while open models are approaching closed models in overall accuracy for accounting tasks, they still demonstrate less consistency. DeepSeek's latest update, V4.1 Flash 1017, scored lower than its previous version, illustrating that model iterations do not always lead to linear improvements across all metrics.

The benchmark highlights a performance gap between open and closed models, particularly at the top tier. Zhipu AI's GLM-5.3 Flash leads the open models with 88.1% accuracy, but this is still three percentage points behind Google's Gemini 4 Argon, a closed model at 91.1%. Closed models also hold the top three scores on 'hard' tasks, indicating that for more complex accounting challenges, proprietary systems currently offer an advantage.

These results imply that developers of open models face ongoing work to match the reliability and advanced capabilities of closed counterparts, especially in specialized domains like accounting. While accuracy figures might be close, the consistency and performance on difficult tasks are areas where open models are still trailing. For businesses considering AI tools in finance, this distinction between accuracy and consistency could be a key factor in deployment decisions.