TLDR: A research paper by Brandon Kaplowitz demonstrates that reinforcement learning can explain two puzzling empirical patterns in household consumption behavior: higher marginal propensities to consume (MPCs) out of stimulus transfers for low-asset unemployed households, and a “scarring” effect where past unemployment leads to persistently lower consumption. The model, which uses Q-learning with neural network approximation, departs from standard rational expectations and replicates both findings simultaneously through value function approximation errors that evolve with experience, offering a unifying framework for understanding how past experiences shape current consumption.
A new research paper titled “Reinforcement Learning and Consumption-Savings Behavior” by Brandon Kaplowitz explores how a sophisticated learning mechanism, known as reinforcement learning, can shed light on two puzzling patterns observed in how households manage their money, particularly during tough economic times.
The paper tackles two key empirical findings that challenge traditional economic models. First, during the COVID-19 recession, studies showed that unemployed households with low liquid assets spent a significantly larger portion of their stimulus checks (a marginal propensity to consume, or MPC, of 0.50) compared to those with high assets (0.34). This was surprising because even with large transfers, neither group was facing immediate borrowing constraints. Second, research has documented a “scarring” effect: households with more past unemployment experiences tend to maintain persistently lower consumption levels, even after accounting for their current financial situation. This suggests that past hardships leave a lasting mark on spending habits.
Standard economic models, which often assume people have perfect information and can solve complex financial equations, struggle to explain these two phenomena simultaneously. Existing explanations often rely on people updating their beliefs about future income risks or on pre-existing differences between households, but these don’t fully capture both effects.
Kaplowitz’s model introduces a different approach. Instead of assuming perfect knowledge, agents in this model use Q-learning, a type of reinforcement learning, with neural networks to make their consumption and savings decisions under uncertain income. Essentially, these agents learn over time from their experiences, forming estimates of the future value of their choices rather than solving a fixed, complex problem every period. They update these estimates based on “surprises” – the difference between what they expected from a decision and what they actually experienced.
The crucial insight is that this reinforcement learning mechanism, through errors in how agents approximate future value that evolve with experience, can generate both higher MPCs and lower consumption levels at the same time. For instance, an agent who has experienced more unemployment might become more cautious, leading to lower overall consumption, but also more inclined to spend a sudden windfall due to their learned value function. The simulation results from this model closely match the empirical estimates, providing a compelling explanation for these observed behaviors.
Also Read:
- Simulating the Market: How AI Agents Uncover Consumer Secrets
- Unlocking the Free Energy Principle: A New Neural Architecture for Self-Organizing Intelligence
This work suggests that adaptive learning, where past experiences directly shape current financial decisions, offers a powerful and unifying framework for understanding consumer behavior beyond what current economic conditions alone would predict. It highlights that how we learn and adapt to our financial environment plays a significant role in our spending and saving habits. You can read the full research paper here: Reinforcement Learning and Consumption-Savings Behavior.
