TLDR: oboro: is a new text-to-image generation model developed by AiHUB Inc. in Japan, specifically designed to create high-quality images from limited, copyright-cleared datasets. It addresses ethical concerns of unlicensed data by using a verified dataset and features a custom Diffusion Transformer with Multi-Multi-Head Attention and a T5-XXL text encoder. The model achieves performance comparable to larger models like Stable Diffusion 1.5 and SDXL with significantly less training data and computational resources, making it a viable solution for commercial applications, particularly in the anime industry. The foundation model is open-source under the Apache License 2.0.
A new text-to-image generation model named “oboro:” has been developed by AiHUB Inc. in Japan, addressing critical challenges in the generative AI landscape, particularly for the anime production industry. This project, supported by Japan’s Ministry of Economy, Trade and Industry (METI) and the New Energy and Industrial Technology Development Organization (NEDO) under the GENIAC initiative, focuses on creating a high-quality image generation model from scratch, using only copyright-cleared images for training.
The development of “oboro:” is a direct response to issues like labor shortages in Japan’s anime industry and the ethical concerns surrounding large-scale datasets used by many existing models. Unlike models trained on vast, often unlicensed datasets like LAION, “oboro:” prioritizes legal and ethical compliance by exclusively using data with verified rights. This approach is crucial for commercial adoption, especially in industries like anime where audience perception and legal frameworks around copyright are paramount.
At its core, “oboro:” is a text-conditioned image generation diffusion model. It incorporates several advanced architectural features to achieve high performance even with limited training data. The model utilizes the T5 V1.1 XXL text encoder, known for its deep understanding of grammar, syntax, and context, which helps in accurately interpreting complex text prompts. For image inference, it employs a custom Diffusion Transformer (DiT) architecture, a scalable backbone that has shown success in next-generation models like Stable Diffusion 3.
A standout feature of “oboro:” is its innovative Multi-Multi-Head Attention mechanism. This design assigns a varying number of attention heads to different layers within the DiT blocks. Early blocks use fewer heads (8 or 16) to capture global, low-frequency features like overall layout, while later blocks use more heads (24 or 48) to specialize in refining local, high-frequency details such as textures and contours. This hierarchical processing, inspired by U-Net architectures, enhances learning efficiency and image quality.
The model also leverages the FLUX VAE, a 16-channel Variational Autoencoder, which is critical for minimizing information loss during image compression and reconstruction. This allows “oboro:” to faithfully render fine details like facial expressions, textures, and small text, addressing common artifacts seen in models with lower-channel VAEs.
For its training, “oboro:” was built upon the Megalith-10m dataset, a collection of images with clear copyright status (CC0-equivalent). To ensure data quality and prevent overfitting, the dataset underwent rigorous deduplication using CLIP image embeddings, custom captioning with Florence-2 (chosen for its understanding of anime-style expressions), and quality scoring using Aesthetic Predictor V2.5. The training process itself involved a staged learning approach, starting with smaller 256×256 images and progressing to 512×512-area images.
Despite being trained on significantly less data—approximately 1/50th the image count and 1/10th the computational time compared to models like SD1.5 and SDXL—“oboro:” achieved comparable evaluation metrics. This demonstrates its remarkable efficiency in learning from limited, copyright-cleared datasets, validating the project’s primary objective. Qualitatively, the model excels at rendering light, shadow, color, and contrast, producing natural and responsive images to prompts.
While “oboro:” shows great promise, it does have some limitations. It may sometimes over-render details, leading to a grainy texture, and is not yet proficient at generating human figures with stable quality. Its genre capabilities are also somewhat constrained by the dataset’s composition. However, these limitations are viewed as less critical for a foundation model, as the intended workflow involves specialized fine-tuning by anime studios using their proprietary datasets, which can address specific character generation and stylistic needs.
Also Read:
- Enhancing Safety in Text-to-Image Models: A New Approach for Unlearned Systems
- Condition Preference Optimization: A New Approach for Precise Image Generation Control
The foundation model, “oboro:base,” along with its inference code, is publicly available under the Apache License, Version 2.0. Users can access the resources and learn more about its setup and execution at the project’s Hugging Face repository. You can read the full research paper here.
