UIPress: Bringing Optical Token Compression to UI-to-Code Generation
| Authors | Dasen Dai et al. |
| Year | 2026 |
| Field | NLP |
| arXiv | 2604.09442 |
| Download | |
| Categories | cs.CL |
Abstract
UI-to-Code generation requires vision-language models (VLMs) to produce thousands of tokens of structured HTML/CSS from a single screenshot, making visual token efficiency critical. Existing compression methods either select tokens at inference time using task-agnostic heuristics, or zero out low-attention features without actually shortening the sequence -- neither truly reduces prefill latency or adapts to the non-uniform information density of UI screenshots. Meanwhile, optical (encoder-side learned) compression has shown strong results for document OCR, yet no prior work has adapted this paradigm to UI-to-Code generation. We propose UIPress, a lightweight learned compression module inserted between the frozen ViT encoder and the LLM decoder of Qwen3-VL-8B. UIPress combines depthwise-separable convolutions, element-guided spatial reweighting, and Transformer refinement to compress {\sim}6{,}700 visual tokens to a fixed budget of 256. Together with Low-Rank Adaptation (LoRA) on the decoder to bridge the representation gap, the entire system adds only {\sim}21.7M trainable parameters (0.26% of the 8B base model). Under a fair comparison on the same base model against four baselines on Design2Code, UIPress at 256 tokens achieves a CLIP score of 0.8127, outperforming the uncompressed baseline by +7.5% and the strongest inference-time method by +4.6%, while delivering 9.1 time-to-first-token speedup. To the best of our knowledge, UIPress is the first encoder-side learned compression method for the UI-to-Code task.
Engineering Breakdown
Plain English
UIPress tackles a critical bottleneck in UI-to-Code generation: vision-language models need to produce thousands of HTML/CSS tokens from a single screenshot, but the visual encoding creates massive token overhead that slows down inference. The paper proposes inserting a lightweight learned compression module between the frozen ViT encoder and the LLM decoder of Qwen3-VL-8B to compress visual tokens at the encoder side rather than selecting or zeroing them at inference time. Unlike prior compression methods that use task-agnostic heuristics or attention-based pruning without actually shortening sequences, UIPress adapts optical (encoder-side learned) compression—a technique proven effective for document OCR—to the non-uniform information density patterns of UI screenshots. This approach reduces prefill latency while maintaining code generation quality.
Core Technical Contribution
The core novelty is adapting optical token compression, a learned encoder-side compression paradigm previously used only in OCR tasks, to UI-to-Code generation as a production-grade solution. UIPress inserts a lightweight compression module between a frozen ViT visual encoder and an LLM decoder, using depthwise separable convolutions (the abstract is cut off but indicates this architecture choice) to learn task-specific compression patterns rather than using static heuristics. This is fundamentally different from prior work: existing methods either select tokens at inference time using hand-crafted rules (which don't reduce compute), or zero out features in-place without shrinking the sequence (which doesn't reduce latency). By learning what visual information matters for code generation during training and actually removing tokens before the LLM processes them, UIPress achieves real prefill latency reduction while preserving the structured output quality that code generation demands.
How It Works
The architecture maintains a frozen Qwen3-VL-8B model (ViT encoder + LLM decoder) and inserts a trainable compression module between them. Input is a UI screenshot, which the ViT encoder converts into a dense sequence of visual tokens (the exact count isn't specified, but UI-to-Code typically produces 1000+ output tokens, implying dense visual encoding). The compression module then learns to identify which visual tokens carry task-relevant information for code generation and selectively outputs a shorter sequence—only the informative tokens pass through to the LLM decoder. The module uses depthwise separable convolutions, a computationally efficient design that reduces parameters while learning spatial-channel interactions in the token sequence. During training, the full model is fine-tuned end-to-end with code generation as the objective, so the compression module learns what visual features correlate with correct HTML/CSS output. At inference, this produces a shorter token sequence, dramatically reducing the prefill latency (the dominant cost when encoding long visual contexts) before the LLM begins decoding tokens.
Production Impact
For teams building UI-to-Code systems, UIPress directly reduces time-to-first-token in inference—a critical metric for user-facing applications where latency is visible. Instead of waiting for the LLM to process thousands of redundant visual tokens, the compression module filters the visual input down to essential information, cutting prefill time without requiring model retraining or architectural changes to the base Qwen3-VL-8B. Integration is straightforward: the compression module sits as a small adapter layer and can be plugged into existing deployment pipelines with minimal code changes. Trade-offs are modest: the module adds a small trainable component (lightweight due to depthwise separability), requires one additional training pass to learn compression patterns for your specific UI domain, and introduces a marginal compute cost during inference (convolutions are cheap compared to LLM forward passes). The key production benefit is that latency improvements scale with input context length—the longer the visual sequence, the greater the speedup—making this especially valuable for high-resolution screenshots or multi-page UI contexts.
Limitations and When Not to Use This
UIPress assumes that visual information in UI screenshots has non-uniform importance and can be learned via end-to-end training, which may not hold for highly irregular or novel UI patterns unseen during training. The approach is evaluated specifically on Qwen3-VL-8B, leaving open questions about generalization to other VLM architectures (e.g., GPT-4V, Claude's vision system, or future models with different token/parameter ratios), and it's unclear whether compression patterns learned on one UI domain transfer to significantly different design systems. The paper does not address failure modes where the compression module drops visual details critical for corner cases (e.g., subtle color differences, tiny UI elements, or accessibility text), which could introduce silent bugs in generated code. Additionally, the approach requires labeled UI-to-Code data for fine-tuning, meaning it cannot be applied zero-shot to completely new UI paradigms, and the method assumes the base LLM remains frozen—if you need to adapt the LLM itself for your domain, the compression module's utility may diminish.
Research Context
UIPress builds on a growing body of work on token efficiency in vision-language models, extending optical compression techniques from document processing (where they showed strong gains on OCR tasks) into the structured code-generation domain. It responds directly to recent challenges in scaling VLMs for long-context visual tasks, where prior attempts used coarse attention-based pruning or inference-time token selection—methods that don't actually reduce compute. The work aligns with broader trends in efficient inference (quantization, pruning, distillation) but introduces the insight that learned, task-specific compression at the encoder level is more effective than post-hoc pruning for outputs with strict structural requirements like code. This opens a research direction for applying encoder-side compression to other multimodal generation tasks (diagram-to-code, chart-to-SQL, document-to-markdown) where visual input density is high but output structure is constrained.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::