Audio → Language (Whisper) - Interactive Visualization

Whisper (OpenAI, 2022) converts audio into text by first transforming 30-second chunks into 80-bin log-mel spectrograms, then encoding them with a convolutional stem followed by transformer blocks, and finally decoding to text tokens using a standard language model decoder. The same model performs language detection, translation, and transcription. Music and noise cause high uncertainty in language detection - the model distributes probability mass across many languages.

• Step-through visualization: waveform → spectrogram → encoder layers → output tokens
• Real spectrogram rendering for speech, music, and noise clips
• Language detection bar chart: see confidence across English, Spanish, French, Chinese, Arabic
• Encoder layer breakdown: Conv stem, positional embedding, 4 transformer blocks, cross-attention
• Chunk size slider: see how 30s chunk size affects the input representation
• Architecture flow diagram showing the full encode-decode pipeline

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.