TLDR: WANDER is a new framework that uses novelty search and Large Language Models (LLMs) to generate highly diverse sets of images from a single text prompt. It employs “emitters” to guide prompt evolution and CLIP embeddings to measure novelty, significantly outperforming existing methods in image diversity and token efficiency for creative applications.
Text-to-image diffusion models have become incredibly powerful, creating stunning and realistic images from simple text prompts. However, a common challenge with these advanced models is their tendency to produce similar outputs when given the same prompt repeatedly. This lack of diversity can be a significant hurdle for creative tasks like ideation or exploratory design, where generating a wide range of novel ideas is crucial.
Traditional methods for optimizing prompts often focus on improving aesthetic quality or performance for specific tasks. These approaches aren’t well-suited for open-ended creative exploration, which prioritizes novelty and a broad range of outputs rather than converging on a single “best” image.
To tackle this limitation, researchers from University College London have introduced a new framework called WANDER. This innovative approach uses a novelty search mechanism to generate diverse sets of images from just one initial text prompt. WANDER operates directly on natural language prompts, leveraging the power of Large Language Models (LLMs) to semantically evolve prompts and create a wide array of visual concepts.
How WANDER Works
At its core, WANDER employs an evolutionary cycle that repeats over multiple generations, consisting of three main steps: Emitter Selection, Prompt Evolution, and Pool Update.
First, WANDER starts with an initial pool of prompt-image pairs. In each generation, it performs mutations on these prompts. The “Emitter Selection” step involves choosing predefined mutation strategies, called emitters. These emitters are like specific instructions for the LLM, such as “change the composition,” “adjust the lighting,” or “add elements,” guiding the evolution into distinct areas of the prompt space.
Next, in “Prompt Evolution,” the framework uses an LLM (specifically GPT-4o-mini in their experiments) to transform prompts. This can happen in two ways: mutation, where a single prompt is modified based on the chosen emitter, or crossover, where elements from two existing prompts are combined to create a new variation. This process ensures that the new prompts maintain semantic coherence while fostering diversity.
Finally, the “Pool Update” step is where novelty is measured and the image pool is refined. For each newly evolved prompt, an image is generated. The novelty of this image is then quantified using CLIP embeddings, which measure the average distance between an image’s embedding and its nearest neighbors in the current pool. If a newly generated image is more novel than the least novel image currently in the pool, it replaces it. This iterative process continuously improves the diversity of the generated image set.
Also Read:
- Visual Prompts for Balancing Safety and Responsiveness in Multimodal AI
- PreferThinker: A New AI System for Understanding Your Unique Image Preferences
Key Findings and Impact
Empirical evaluations showed that WANDER significantly outperforms existing evolutionary prompt optimization methods in diversity metrics. It achieved higher Vendi and LPIPS scores, which are indicators of diversity, while also being remarkably token-efficient compared to some baselines. The research also confirmed that the use of human-designed mutation strategies, or emitters, plays a crucial role in enhancing the diversity of generated images.
Interestingly, the study found that more capable LLMs, such as OpenAI’s GPT-4o, were more effective at mutating prompts, leading to even more diverse image pools, although they typically consumed more computational tokens. Visualizations of image embeddings also demonstrated a clear trend of increasing diversity across generations, confirming the effectiveness of the evolutionary process in exploring the latent image space.
While WANDER represents a significant step forward, the authors acknowledge some limitations, such as the potential for “relevance drift,” where images might occasionally diverge from the initial prompt’s core concept. The emitters also require manual specification, which could introduce bias. Future work aims to explore generating emitters using LLMs themselves and extending WANDER to other modalities like text and audio.
This research introduces a powerful, adaptive strategy for generating varied image sets, opening new possibilities for creative exploration with diffusion models. You can read the full research paper here: Evolve to Inspire: Novelty Search for Diverse Image Generation.
