Terence Tao: LLM's Hardest Challenge Isn't Math

Terence Tao stated that the hardest challenge for LLMs is not mathematics. While training primarily uses linear algebra and some calculus, why models perform remarkably on some tasks and fail on others remains unpredictable. Model construction is understood, but their capabilities are still being explored.