The Provenance Gap

Databricks open-sources MLflow. Artifact-level experiment tracking becomes standard. Records which dataset file and code commit produced which model. Does not record which rows were processed or how.

DVC adds Git-like versioning for datasets. DSLog tracks operator-level provenance for NumPy workloads. Both excellent at their stage - neither crosses the boundary into training or attribution.

Park et al. (MIT/ICML 2023) introduce TRAK - training data attribution via sparse JL projection. LDS 0.0290 on CIFAR-2 in 691 seconds on GPU. Excellent attribution quality. Operates entirely on post-preprocessing tensors with no source file knowledge.

EU Regulation 2024/1689 published. Article 26 requires audit trails for high-risk AI. Article 13 requires transparency. GDPR Article 17 erasure obligations already in force. The compliance clock starts.

First unified system connecting source files through preprocessing through training to predictions. Three layers: ProvenanceTensor lineage, JL-projected GradientStore attribution, provenance-guided approximate unlearning. Sub-1% overhead at 1M elements. pip install traceprop.

Logging obligations enter force for new high-risk AI systems. Credit scoring, HR decisions, educational AI, critical infrastructure must maintain audit trails. Systems deployed after this date need audit trail infrastructure from day one.

All deployed high-risk AI systems must be compliant. Systems deployed before August 2026 have until this date to retrofit audit trail infrastructure. The 18-month window is shorter than most ML infrastructure projects.