Personalizing Text-to-Image Generation to Individual Taste
| Authors | Anne-Sofie Maerten et al. |
| Year | 2026 |
| HF Upvotes | 4 |
| arXiv | 2604.07427 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Modern text-to-image (T2I) models generate high-fidelity visuals but remain indifferent to individual user preferences. While existing reward models optimize for "average" human appeal, they fail to capture the inherent subjectivity of aesthetic judgment. In this work, we introduce a novel dataset and predictive framework, called PAMELA, designed to model personalized image evaluations. Our dataset comprises 70,000 ratings across 5,000 diverse images generated by state-of-the-art models (Flux 2 and Nano Banana). Each image is evaluated by 15 unique users, providing a rich distribution of subjective preferences across domains such as art, design, fashion, and cinematic photography. Leveraging this data, we propose a personalized reward model trained jointly on our high-quality annotations and existing aesthetic assessment subsets. We demonstrate that our model predicts individual liking with higher accuracy than the majority of current state-of-the-art methods predict population-level preferences. Using our personalized predictor, we demonstrate how simple prompt optimization methods can be used to steer generations towards individual user preferences. Our results highlight the importance of data quality and personalization to handle the subjectivity of user preferences. We release our dataset and model to facilitate standardized research in personalized T2I alignment and subjective visual quality assessment.
Engineering Breakdown
Plain English
This paper tackles the problem that modern text-to-image models like Flux and Stable Diffusion generate high-quality images but ignore individual user preferences—they optimize for 'average' human taste rather than personalized aesthetics. The authors created PAMELA, a dataset of 70,000 image ratings across 5,000 generated images, each rated by 15 different users to capture subjective preference variation across art, design, fashion, and photography domains. They then built a personalized reward model trained on these annotations to predict what specific users will like, rather than what the general population prefers. This moves beyond existing one-size-fits-all reward models by directly modeling the inherent subjectivity of aesthetic judgment.
Core Technical Contribution
The core innovation is a large-scale personalized preference dataset (PAMELA) paired with a reward modeling framework that captures individual taste rather than averaging preferences. Instead of training a single reward model optimized for broad appeal, the authors propose jointly training a model that conditions on user identity or user-specific embeddings to predict personalized image quality scores. This represents a fundamental shift from global optimization (what the average person likes) to local optimization (what specific users like), enabling downstream T2I systems to be fine-tuned toward individual aesthetic preferences. The dataset itself is a significant contribution—70,000 annotations with 15-person redundancy per image provides statistical power to model preference variance rather than just central tendency.
How It Works
The system operates in two stages: data collection and model training. In the collection phase, the authors generated 5,000 diverse images using Flux 2 and Nano Banana models (presumably covering different prompts and generation conditions), then recruited 15 independent raters per image to provide subjective quality scores, creating a 70,000-rating annotation matrix with rich preference variance. For the model training stage, they construct a personalized reward model that takes as input an image embedding (likely extracted from a vision backbone like CLIP or similar), paired with a user embedding that captures that person's aesthetic preferences. The model then jointly learns a mapping from (image, user) pairs to predicted preference scores, effectively learning both what makes an image objectively good and what makes it personally appealing to different users. During inference, given a text prompt and a target user, the system can generate images and rank or rank-and-reorder them using the personalized reward score, enabling user-specific optimization loops.
Production Impact
Adopting this approach enables several concrete improvements to T2I production systems: first, you can add a personalization layer to existing image generation pipelines by training a lightweight user-preference model on top of fixed image embeddings, allowing fast re-ranking of generated images without retraining the T2I base model. Second, this enables feedback loops where user ratings on generated images incrementally improve their personal preference model, creating a system that gets better at predicting each user's taste over time. Third, you could integrate this into ranking or retrieval systems (e.g., showing top-3 images per prompt ranked by user preference), improving user satisfaction without changing generation speed. The main trade-offs are: you need 15-sample-per-image labeling budgets to build robust user preference models (higher data cost than single-sample annotation), the reward model adds inference latency for re-ranking (typically 10-50ms per image depending on batch size), and you must manage user embeddings/preference models per person, increasing storage and serving complexity proportional to your active user base.
Limitations and When Not to Use This
The paper does not address how to handle new users with no rating history (cold-start problem)—you would need transfer learning or a hybrid approach combining global and personalized signals. The dataset appears limited to images generated by specific models (Flux 2, Nano Banana) in specific domains (art, design, fashion, cinematic), so the preference patterns learned may not generalize to other generation models, photorealistic content, or user preferences outside these domains. The paper assumes that 15 raters per image is sufficient to capture meaningful preference variance, but this may be insufficient for rare aesthetic preferences or edge-case user tastes, especially if raters come from a narrow demographic background. Finally, the abstract is truncated, so the actual training procedure, architecture details, and quantitative results are not visible—it's unclear if this framework actually outperforms simpler baselines like user-conditioned fine-tuning or whether the gains justify the added complexity in production.
Research Context
This work builds on the broader trend of moving reward modeling beyond average-case optimization (as seen in RLHF for language models and recent work on reward modeling for image generation) toward personalized or stratified preference modeling. It extends prior research on human preference annotation datasets (like HPS, Pick-a-Pic, and others) by adding the user-diversity dimension—instead of one gold-standard rating, you get 15 ratings reflecting real disagreement, which is a more honest representation of aesthetic subjectivity. The paper opens up the research direction of personalized diffusion model optimization, where you don't just train one global policy but rather user-specific policies or adaptive scoring mechanisms. This aligns with broader trends in recommendation systems and personalization ML where moving from one-size-fits-all to user-centric models significantly improves downstream satisfaction, and suggests that future T2I research should explicitly model preference heterogeneity rather than averaging it away.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::