Predicting integers from continuous parameters
| Authors | Bas Maat & Peter Bloem |
| Year | 2026 |
| HF Upvotes | 3 |
| arXiv | 2602.10751 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
We study the problem of predicting numeric labels that are constrained to the integers or to a subrange of the integers. For example, the number of up-votes on social media posts, or the number of bicycles available at a public rental station. While it is possible to model these as continuous values, and to apply traditional regression, this approach changes the underlying distribution on the labels from discrete to continuous. Discrete distributions have certain benefits, which leads us to the question whether such integer labels can be modeled directly by a discrete distribution, whose parameters are predicted from the features of a given instance. Moreover, we focus on the use case of output distributions of neural networks, which adds the requirement that the parameters of the distribution be continuous so that backpropagation and gradient descent may be used to learn the weights of the network. We investigate several options for such distributions, some existing and some novel, and test them on a range of tasks, including tabular learning, sequential prediction and image generation. We find that overall the best performance comes from two distributions: Bitwise, which represents the target integer in bits and places a Bernoulli distribution on each, and a discrete analogue of the Laplace distribution, which uses a distribution with exponentially decaying tails around a continuous mean.
Engineering Breakdown
Plain English
This paper addresses the problem of predicting integer-valued outputs (like upvote counts or bike availability) by treating them as discrete distributions rather than continuous regression targets. The authors argue that standard regression approaches inappropriately convert discrete label distributions into continuous ones, losing important structural properties of the data. They propose a method where neural networks directly predict parameters of discrete distributions (like Poisson or discretized variants of continuous distributions), allowing the model to respect the integer constraint while leveraging the flexibility of neural network feature extraction. This approach is particularly relevant for real-world counting problems where the inherent discreteness matters.
Core Technical Contribution
The core innovation is framing integer prediction as discrete distribution parameter estimation rather than continuous regression. Instead of training a network to output a real-valued prediction that gets rounded, the authors design architectures where the network outputs parameters (e.g., λ for Poisson, or bin probabilities for discretized distributions) that fully specify a probability distribution over integers. This preserves the discrete nature of the output space and enables proper probabilistic inference. The technical contribution includes making discrete output distributions differentiable and compatible with standard backpropagation during neural network training.
How It Works
The system works by having a neural network encoder take continuous input features and produce parameters that define a discrete probability distribution over integers. For example, with a Poisson distribution, the network outputs λ (lambda), the rate parameter, which directly determines P(Y=k) for integer k. For bounded integer ranges, the authors describe using discretized versions of continuous distributions (like binning a normal or exponential distribution) or categorical distributions over the discrete support. During training, the network is optimized via maximum likelihood: for a given training instance with integer label y, the loss is -log P(y | parameters), where P is evaluated using the predicted distribution. At inference time, the network outputs the distribution parameters, allowing either point predictions (via the mode or mean) or full probabilistic predictions with uncertainty quantification.
Production Impact
This approach directly improves systems predicting counts or bounded integers: social media analytics (predicting engagement), ride-sharing platforms (predicting available bikes/scooters), inventory management, and demand forecasting. The main benefit is that models naturally capture discreteness and can output calibrated probability distributions rather than point estimates, enabling better decision-making under uncertainty. Engineers would replace standard MSE/MAE regression heads with discrete distribution heads—a straightforward architectural change with minimal overhead. Trade-offs include slightly increased model complexity (need to specify which distribution family to use), potential difficulty in choosing appropriate distributions for multimodal or long-tailed count data, and the requirement that training data actually fit the chosen distribution reasonably well.
Limitations and When Not to Use This
The paper assumes that integer outputs follow a known parametric family of discrete distributions (Poisson, negative binomial, etc.), which may not hold for complex, multimodal, or heavy-tailed count data common in practice. The approach requires careful distribution selection—choosing Poisson when the data has much higher variance (requiring negative binomial) will lead to poor calibration and performance. For very large integer ranges or when the distribution is highly irregular, the method may not scale well or require custom discretization schemes. The paper's focus on neural network output distributions means it may not address challenges in feature engineering or handling missing/sparse data that often dominate real counting prediction tasks.
Research Context
This work builds on decades of statistical literature on discrete distributions (Poisson regression, generalized linear models) and more recent work on distributional regression in deep learning (e.g., predicting quantiles or mixture distributions). It connects to neural network output distribution design, similar to how other papers have explored predicting variance, mixture components, or Gaussian process parameters. The paper advances the specific but underexplored case of integer outputs, filling a gap where most deep learning frameworks default to continuous regression. This opens research directions in distributional prediction for structured outputs and may inspire similar discrete-by-design approaches for other constrained output spaces (graphs, sequences with discrete tokens, etc.).
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::