TLDR: A research paper investigates whether large language models (LLMs) can generalize humor across different types. The study found that LLMs are capable of some humor transfer, with performance varying by humor type and model. Dad Jokes were surprisingly effective for enabling transfer to other humor types but difficult to generalize to, while simpler forms like headlines and one-liners were easier to learn but less useful as training sources. Increased diversity in training data generally improved generalization, and models maintained strong performance even with reduced in-domain data, suggesting diversity is key for future humor AI.
Humor, a complex and diverse form of human communication, has long posed a significant challenge for artificial intelligence. While humans effortlessly grasp various forms of jokes, sarcasm, and wit, machines have traditionally struggled, often excelling only in very specific humor tasks. A recent research paper titled ‘One Joke to Rule them All? On the (Im)possibility of Generalizing Humor’ by Mor Turgeman, Chen Shani, and Dafna Shahaf delves into a crucial question: can large language models (LLMs) learn to generalize humor across different types, or is the fragmentation of computational humor an unavoidable reality?
The researchers from The Hebrew University of Jerusalem and Stanford University set out to understand if competence in one or more specific humor tasks could enable an LLM to transfer that understanding to novel, unseen types of humor. This inquiry is particularly relevant today, as new forms of humor, like memes and AI fails, constantly emerge online. For LLMs to keep pace, they need to capture deeper, transferable mechanisms of humor rather than just memorizing specific joke structures.
Investigating Humor Transfer
To explore this, the team conducted a series of transfer learning experiments using two prominent LLMs: LLaMA-2-7B and Mistral-7B. They utilized four distinct humor datasets, each representing a different style:
- Amazon Questions: User-submitted questions about products, often sarcastic or ironic.
- One Liners: Brief, standalone jokes, frequently involving puns or wordplay.
- Sarcasm Headlines: News headlines, including satirical ones from The Onion.
- Reddit Dad Jokes: Multi-sentence narratives, puns, and cultural references from the r/dadjokes subreddit.
The experiments were structured in three ways: training models on a single dataset, on pairs of datasets, and on three datasets, always testing their ability to understand a humor type not included in their training. This allowed the researchers to observe how well the models could generalize.
Key Findings: A Hierarchy of Humor
The study revealed that LLMs are indeed capable of some humor transfer, with one model achieving up to 75% accuracy on unseen datasets. However, the success of this transfer varied significantly depending on the humor type and the model used. Mistral-7B consistently demonstrated stronger transfer capabilities than LLaMA-2-7B.
A fascinating insight emerged regarding the relationships between different humor types, suggesting a hierarchy of complexity:
- Dad Jokes: Surprisingly, Dad Jokes proved to be the best enabler of transfer, meaning training on them helped models understand other humor types. However, they were also the most challenging type for models to generalize *to* from other training data. This suggests their unique structure, often involving multi-sentence narratives and complex wordplay, makes them a rich source for learning but difficult to master without direct exposure.
- Amazon Questions: This dataset occupied a middle ground. Its diverse, user-generated content allowed it to transfer reasonably well to simpler styles and also benefited from training on Dad Jokes.
- Headlines and One Liners: These were the most receptive to transfer from other humor styles, meaning models could easily generalize to them. However, they offered comparatively less utility when used as source material for teaching models other humor types, likely due to their more constrained and homogeneous structure.
The Power of Diversity
The research also highlighted the significant impact of data diversity on humor transfer learning. Training models on a more diverse range of humor types generally improved their ability to generalize to unseen styles. The largest gains were observed when moving from single-dataset training to multi-dataset training, with diminishing returns as more datasets were added. This suggests that moderate diversity might be nearly as effective as maximal diversity for cross-domain humor transfer.
Interestingly, even when the amount of in-domain training data was significantly reduced (to 33%), the models retained strong performance on that specific humor type. This implies that for future humor applications, prioritizing data diversity over sheer data size could be a more effective strategy.
Also Read:
- New Research Uncovers How Quantization Affects Different Types of Knowledge in Large Language Models
- Unmasking Model Weaknesses: How Out-of-Distribution Tests Fall Short in AI Evaluation
Looking Ahead
The findings of this paper suggest that while humor is complex, it is not entirely opaque to transfer-based learning for LLMs. The asymmetric nature of transfer between humor types reflects underlying structural differences, aligning with cognitive theories that distinguish between surface cues and deeper mechanisms of humor. This work opens up exciting avenues for future research, including expanding to more humor types and modalities (like memes or videos), exploring multilingual and cross-cultural humor, and further aligning computational findings with cognitive and neuroscience theories of humor. You can read the full research paper here: One Joke to Rule them All? On the (Im)possibility of Generalizing Humor.
