Latent Preference Modeling for Cross-Session Personalized Tool Calling
| Authors | Yejin Yoon et al. |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2604.17886 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Users often omit essential details in their requests to LLM-based agents, resulting in under-specified inputs for tool use. This poses a fundamental challenge for tool-augmented agents, as API execution typically requires complete arguments, highlighting the need for personalized tool calling. To study this problem, we introduce MPT, a benchmark comprising 265 multi-session dialogues that cover three challenges: Preference Recall, Preference Induction, and Preference Transfer. We also propose PRefine, a test-time memory-augmented method that represents user preferences as evolving hypotheses. Through a generate--verify--refine loop, it extracts reusable constraints from history and improves tool-calling accuracy while using only 1.24% of the tokens required by full-history prompting. These results indicate that robust personalization in agentic systems depends on memory that captures the reasons behind user choices, not just the choices themselves.
Engineering Breakdown
Plain English
This paper addresses a critical gap in LLM-based tool-calling agents: users often provide under-specified requests that omit required details, making it impossible to execute API calls without additional information. The authors introduce MPT, a benchmark of 265 multi-session dialogues covering three specific challenges (Preference Recall, Preference Induction, and Preference Transfer), and propose PRefine, a memory-augmented method that learns and applies user preferences across conversations. PRefine achieves better tool-calling accuracy while using only 1.24% of the tokens required by naive full-history prompting, making it dramatically more efficient than baseline approaches.
Core Technical Contribution
The core innovation is a test-time memory-augmented architecture that models user preferences as evolving hypotheses rather than static slots or features. PRefine uses a generate-verify-refine loop that extracts reusable constraints from conversation history and applies them to complete under-specified API arguments in subsequent tool calls. This approach is novel in treating preference modeling as an online learning problem that scales across multiple sessions, rather than pre-training or fine-tuning on user-specific data. The key technical insight is that preferences can be efficiently represented as extractable constraints that transfer across conversations, enabling dramatic token savings while maintaining or improving accuracy.
How It Works
PRefine operates in three phases executed at test time. First, when the model generates a tool call, it includes placeholder positions for potentially under-specified arguments. Second, the verify phase checks if previous conversations in the multi-session history contain relevant preference signals—either explicit preferences the user stated or implicit patterns in their past choices. Third, the refine phase uses these extracted constraints to fill in missing arguments, with the entire process tracked as an evolving set of user preference hypotheses. The system maintains a compact memory of discovered preferences (constraints on things like output formats, default parameters, value ranges) and applies them without requiring the full conversation history to be re-processed with each new request, which is why token efficiency is so dramatic compared to in-context learning baselines.
Production Impact
This directly solves a major operational problem: most production tool-calling agents either ask users repetitive clarification questions (degrading user experience) or fail when given under-specified inputs (causing wasted API calls and errors). Adopting PRefine would reduce token consumption for multi-turn personalized agent systems by 98% compared to full-history prompting while improving accuracy, which translates to lower latency and significantly reduced inference costs at scale. The 1.24% token ratio means a system handling millions of daily requests across many users could save substantial compute. The trade-offs are: you need to maintain per-user preference memory across sessions (adding a small database component), the method assumes preferences are somewhat stable within a user (may fail for adversarial or rapidly-changing preference patterns), and the approach requires evaluation on your specific domain since the MPT benchmark covers only the three named preference tasks.
Limitations and When Not to Use This
The paper's scope is limited to preference modeling within the bounds of tool arguments—it does not address cases where users fundamentally change what they want to accomplish or where preferences conflict across different application domains. The MPT benchmark, while valuable, contains only 265 multi-session dialogues, which may not cover the long tail of preference types in truly diverse user populations. The method assumes preferences are learnable from history and transferable; it may perform poorly for users with highly idiosyncratic, context-dependent, or contradictory preferences. Additionally, the paper does not deeply explore failure modes such as when extracted constraints lead to incorrect assumptions about future user intent, or how to handle preference decay and drift over very long timescales (weeks or months).
Research Context
This work builds on the growing problem of making LLM-based agents more practical and efficient, following prior research on in-context learning, tool use, and user modeling in dialogue systems. It extends beyond simple slot-filling personalization by treating preferences as dynamic, learnable structures that evolve across sessions. The MPT benchmark contributes a new evaluation resource for the community, filling a gap between single-turn tool-calling benchmarks and real-world multi-session scenarios. This research opens future directions in learning efficient user models, handling preference conflicts, and applying similar constraint-extraction ideas to other domains where under-specified user requests are common.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::