EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Embedding Fine-tuning - Contrastive Learning Visualization
Contrastive Training Pairs
Positive pair
"dog playing"pulled together ↔"puppy running"
Negative pair
"dog playing"pushed apart ←→"car engine"
InfoNCE Loss - Training Curve (batch=64, steps=30)
L = -log[exp(q·k⁺/τ) / Σᵢ exp(q·kᵢ/τ)]
LossSteps0.188
Embedding Space - Cluster Separation (progress: 100%)
Before training
DogsCatsCars
fine-tuning
→
t=1.00
After training
DogsCatsCars
Contrastive training pulls similar items together and pushes dissimilar items apart in embedding space. After training, clusters are tighter and better separated - improving retrieval precision.
Controls
Batch Size
Size64
Larger batch = more negatives = better InfoNCE
Loss Type
InfoNCE
triplet
contrastive
Training Steps
Steps30
InfoNCE with large batch sizes treats all other items in the batch as negatives - cheap and effective. Hard negatives (similar but different) create harder training signal.

Embedding Fine-tuning - Contrastive Learning Visualization - Interactive Visualization

Contrastive learning trains embedding models by pulling similar texts closer and pushing dissimilar texts apart in embedding space. InfoNCE loss uses all other items in the batch as negatives - so larger batch sizes give more negatives and better training signal. Hard negatives (items that are superficially similar but semantically different) create harder training signal that produces more discriminative embeddings.

  • InfoNCE: -log[exp(q·k⁺/τ) / Σᵢ exp(q·kᵢ/τ)] - all batch items are negatives
  • Triplet loss: max(0, d(anchor,pos) - d(anchor,neg) + margin)
  • Larger batch = more negatives = steeper gradient signal for InfoNCE
  • Hard negatives: semantically similar but different - e.g. "dog barking" vs "wolf howling"
  • After training: same-class embeddings cluster tightly, different-class clusters separate

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.