TLDR: The “WithAnyone” research introduces a new model and dataset to address the “copy-paste” problem in identity-consistent image generation. By creating the large-scale MultiID-2M dataset and the MultiID-Bench evaluation benchmark, along with a novel contrastive identity loss training paradigm, WithAnyone generates high-quality, identity-preserving images with improved controllability over pose and expression, significantly reducing unwanted direct replication from reference images.
A new research paper introduces “WithAnyone,” a novel approach to generating images that are highly consistent with a person’s identity while also offering greater control over elements like pose and expression. This work addresses a common challenge in text-to-image generation where models often “copy-paste” reference faces, limiting natural variation and controllability.
Current identity-consistent image generation models frequently struggle due to a lack of large-scale datasets containing multiple images of the same individual in diverse scenarios. This forces many models to rely on reconstruction-based training, which can lead to the “copy-paste” artifact. This means the model directly replicates the reference face instead of preserving the identity across natural changes in pose, expression, or lighting, making the generated images less flexible and controllable.
To overcome these limitations, the researchers behind WithAnyone have made several key contributions. First, they constructed a massive new dataset called MultiID-2M. This dataset is specifically designed for multi-person scenarios and provides diverse reference images for each identity, including 500,000 group photos with 1-5 recognizable celebrities, each with hundreds of individual reference images, plus 1.5 million additional unpaired group photos.
Second, they introduced MultiID-Bench, a comprehensive benchmark to evaluate both the “copy-paste” artifacts and the balance between identity fidelity and variation. This benchmark uses a new metric, Copy-Paste (CP), which quantifies the bias of a generated image towards the reference versus the ground truth, helping to identify models that simply copy rather than truly synthesize an identity.
Third, they proposed a new training method that includes a contrastive identity loss. This loss leverages the paired data in MultiID-2M to strike a better balance between maintaining identity fidelity and allowing for diversity in generated images. The model, WithAnyone, is built on the FLUX architecture and effectively reduces copy-paste artifacts while maintaining high identity similarity.
WithAnyone’s training pipeline involves four phases: initial reconstruction pre-training with fixed prompts, followed by reconstruction pre-training with full captions, then a crucial “paired tuning” phase using the MultiID-2M dataset to discourage direct copying, and finally, a quality tuning phase to enhance perceptual fidelity and style robustness.
Also Read:
- Seamless Multi-View Image Editing Without Extensive Training
- Achieving Consistent Multi-View Customization with MVCustom
Extensive experiments show that WithAnyone significantly reduces copy-paste artifacts, improves controllability over pose and expression, and maintains strong visual quality. User studies further confirm that the method achieves high identity fidelity while enabling expressive and controllable generation. This research marks a significant step towards more faithful, controllable, and robust identity customization in image generation. You can find the full research paper here: WithAnyone: Towards Controllable and ID Consistent Image Generation.
