QEIL v2: Heterogeneous Computing for Edge Intelligence via Roofline-Derived Pareto-Optimal Energy Modeling and Multi-Objective Orchestration
| Authors | Satyam Kumar & Saurabh Jha |
| Year | 2026 |
| HF Upvotes | 4 |
| arXiv | 2602.06057 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Deploying large language models (LLMs) on heterogeneous edge devices demands frameworks that jointly optimize energy efficiency, inference quality, and reliability. Our prior QEIL v1 (Kumar & Jha, 2026) achieved 4.82x IPW improvement but relied on static efficiency factors, greedy optimization, and unverified candidate selection. QEIL v2 replaces every static heuristic with physics-grounded, runtime-adaptive models. We introduce three device-workload metrics: DASI (roofline-derived compute utilization), CPQ (memory pressure from allocation theory), and Phi (thermal yield from CMOS leakage physics), forming a unified energy equation with every coefficient traceable to semiconductor physics. For optimization, PGSAM (Pareto-Guided Simulated Annealing with Momentum) simultaneously minimizes energy, latency, and device underutilization. At inference time, the EAC/ARDE selection cascade with CSVET early stopping provides progressive verification among repeated samples. Evaluated on WikiText-103, GSM8K, and ARC-Challenge across seven model families (125M-8B parameters, including one pre-quantized variant), QEIL v2 achieves 75.7% pass@k at 63.8W (IPW=0.9749), a 2.86x improvement over standard inference. When applied to a 4-bit Llama-3.1-8B, QEIL v2's physics-grounded routing achieves IPW=1.024 at 54.8W -- the first edge orchestration system to surpass the IPW=1.0 empirical reference mark, with the gain attributable entirely to QEIL v2's workload-adaptive device allocation on a model with reduced memory bandwidth requirements. Total energy drops 75.6% vs. standard with 38.3% latency reduction, zero thermal throttling, and 100% fault recovery across all benchmarks and model families.
Engineering Breakdown
Plain English
This paper presents QEIL v2, an improved framework for running large language models efficiently on resource-constrained edge devices (phones, IoT hardware, embedded systems). The authors replaced the static, heuristic-based optimization of their v1 system (which achieved 4.82x improvement in instructions-per-watt) with physics-grounded models that adapt at runtime based on actual device conditions. They introduce three new metrics—DASI (compute utilization), CPQ (memory pressure), and Phi (thermal efficiency)—that are mathematically derived from semiconductor physics rather than hand-tuned rules. Their new optimization algorithm, PGSAM, uses Pareto optimization and simulated annealing to balance energy, inference quality, and reliability simultaneously, with every coefficient traceable back to actual physics rather than empirical guessing.
Core Technical Contribution
The core novelty is replacing static efficiency heuristics with runtime-adaptive, physics-grounded models throughout the entire optimization pipeline. Rather than pre-computing fixed efficiency factors (as v1 did), QEIL v2 derives device-workload metrics directly from semiconductor physics: DASI comes from roofline model theory (a CPU/GPU bottleneck analysis framework), CPQ derives from memory allocation theory, and Phi emerges from CMOS leakage physics. The second major contribution is PGSAM, a new optimization algorithm that jointly minimizes energy, quality, and reliability by simultaneously exploring the Pareto frontier using simulated annealing with momentum—this replaces v1's greedy approach which couldn't guarantee good solutions when multiple objectives conflict. By making every design choice traceable to first principles rather than empirical tuning, the authors enable the system to generalize better across different device-workload combinations.
How It Works
The system takes an LLM, a target edge device, and a workload specification (input constraints, latency requirements) as inputs. First, it profiles the device using three physics-based metrics: DASI measures how effectively the hardware's compute units are being utilized relative to theoretical peak (based on roofline analysis of memory bandwidth vs. compute throughput), CPQ quantifies memory pressure by modeling how allocation pressure affects energy consumption, and Phi estimates thermal efficiency by applying CMOS leakage physics equations to predict how much power dissipates as heat at current temperatures. These three metrics feed into a unified energy equation where coefficients are derived from semiconductor theory, not tuned empirically. The PGSAM optimizer then explores candidate configurations (quantization levels, layer fusion, pruning strategies) by simultaneously optimizing multiple objectives: it uses simulated annealing to escape local minima, maintains momentum to avoid oscillation, and tracks the Pareto frontier so engineers can trade off quality loss against energy savings. The output is a deployment configuration that maximizes instructions-per-watt while keeping inference quality loss and thermal violations within acceptable bounds.
Production Impact
For teams deploying LLMs to phones, edge servers, or IoT devices, this directly addresses the core tension: you want fast, accurate inference but the devices are power-constrained. Instead of manually tuning quantization levels and layer configurations per device (a time-consuming, error-prone process), QEIL v2 automatically generates near-optimal configurations by understanding the physics of power, memory, and thermals. The previous 4.82x improvement in instructions-per-watt from v1 means your models run longer on battery or with smaller power budgets—critical for real deployments. The runtime adaptation is especially valuable because device conditions change: temperature, background load, and available voltage all shift during operation, and static configs can't adapt. However, the production cost is non-trivial: you need to profile each device type once (computational cost of running the physics models and PGSAM optimization, likely minutes to hours), and you must instrument your inference path to measure DASI, CPQ, and Phi at runtime. The system also requires that you have accurate semiconductor physics models for your target devices—generic models may not work for custom chips or exotic hardware. Integration complexity is moderate: you'd replace your current quantization/pruning pipeline with QEIL v2's optimizer, retrain or distill your models with the selected configurations, and add telemetry to measure the three metrics during inference.
Limitations and When Not to Use This
The paper assumes you have accurate semiconductor physics models and can measure roofline metrics (memory bandwidth, peak compute) for your devices—this breaks down for custom silicon or when OEMs don't publish detailed specs. PGSAM's simultaneous optimization of energy, quality, and reliability creates a three-way trade-off space that is expensive to explore; the paper doesn't specify runtime cost of the optimization or whether it must run for every new workload. The approach is empirically validated only on the v1 benchmark (4.82x IPW improvement), but we don't know how it performs on newer architectures like Llama 3, Mistral, or multimodal models, or whether the physics equations hold at extreme quantization levels (INT4 or lower). The physics-grounded models assume standard CMOS technology; emerging technologies like FinFET, GAA (gate-all-around), or analog in-memory compute may violate the leakage and thermal assumptions. Finally, the paper doesn't address how to handle mixed-device clusters (some devices with GPUs, some with specialized accelerators) or how to distribute inference across heterogeneous edge networks—it appears to optimize for single-device deployment only.
Research Context
QEIL v2 builds on the authors' own v1 framework (Kumar & Jha, 2026) and advances the broader field of efficient LLM inference on edge hardware. It relates to prior work on roofline modeling (Lakshminarasimhan et al.) for understanding hardware bottlenecks, memory allocation theory for modeling power efficiency, and CMOS device physics literature for thermal estimation. The work fits into the growing ecosystem of edge ML frameworks like TensorFlow Lite, ONNX Runtime, and Qualcomm's AI Engine, but differs by focusing on joint optimization of energy-quality-reliability rather than treating them separately. This opens a research direction toward `physics-grounded AI systems engineering'—the idea that infrastructure ML (deployment, optimization, resource scheduling) should be derived from first principles rather than empirical heuristics, enabling better generalization and provable guarantees. Future work likely includes extending this to multi-device clusters, dynamic voltage/frequency scaling (DVFS) integration, and applications to other LLM inference patterns (batched inference, speculative decoding).
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::