Value Functions, Emotions, and the Human Advantage in Learning
Two short articles emphasize the role of value functions and emotions in efficient learning and decision-making, pointing out that current AI systems lack similar mechanisms. Value functions can transform sparse terminal rewards into immediate feedback, enhancing the learning efficiency of long-term tasks. The emotional system, as a biological value function, provides decision-making motivation and leads to a decline in decision-making ability after damage, indicating that emotions are an indispensable component of decision-making. Both articles argue that the current reinforcement learning paradigm lacks the dense, intrinsic value-guiding signals found in humans.
Key insights
Value Functions Significantly Enhance Learning Efficiency in Long-Term Tasks
The articles point out that humans rely on an intrinsic value function in long-term tasks, transforming sparse terminal rewards into immediate, dense learning signals (for example, instantly recognizing the loss of an important piece while playing chess). This mechanism can overcome the issue of sparse signals in reinforcement learning for long-term tasks, providing a pathway for more efficient learning.
Emotions as an Evolved Value Mechanism Supporting Human Decision-Making and Learning
Based on neuroscience cases, the articles state that emotions are not irrational attachments but rather mechanisms similar to value functions: when the brain regions that process emotions are damaged, individuals lose effective decision-making ability despite retaining logic, demonstrating that emotions provide real-time motivation and guidance for decision-making.