EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Flink Stream Processing
Flink Dataflow Graph
Source
Kafka / File
p=2
Filter
event != null
p=2
Map
extract fields
p=2
KeyBy
user_id
p=2
Window
TumblingWindow
p=2
Aggregate
count / sum
p=2
Sink
Redis / DB
p=2
Tumbling Window
Fixed 20s non-overlapping windows. Each event belongs to exactly one window. Simple and efficient - good for per-minute/per-hour aggregates.
Event Timeline - 20s window
W1W2W30s10s20s30s40s50s60swatermark7242889179856590587385202737439313118738554816684863239887100eventlate eventwatermark
Late events (red) arrived after the watermark. Flink can either drop them or route to a side output for separate handling.
Flink Config
Window Size20s
5s60s
Parallelism2
14
Watermark
Watermark = event_time - 3s lag. Marks how far behind real-time Flink is willing to wait for late arrivals.
KeyBy partitions the stream by key before windowing - each key's events go to the same task. Required for per-user aggregations.

Flink Stream Processing - Interactive Visualization

Apache Flink is the industry standard for stateful stream processing at scale. Unlike batch systems, Flink operates on unbounded event streams using a dataflow graph of operators. The hardest part of stream processing is handling time correctly: events arrive out of order, with delays, and you must decide when to close a window and emit results. Watermarks are Flink's mechanism for tracking event-time progress and tolerating late arrivals. This demo makes window types and watermark behavior interactive.

  • Tumbling windows: fixed, non-overlapping intervals (e.g., 1-minute buckets) - each event belongs to exactly one window
  • Sliding windows: overlapping intervals (e.g., 1-minute window every 30s) - events can appear in multiple windows
  • Session windows: gap-based, dynamically sized - close after a configurable inactivity period
  • Watermarks track event-time progress; Flink waits for watermark T before emitting windows closing before T
  • Late events arriving after the watermark can be routed to a side output rather than silently dropped
  • Flink dataflow graphs are DAGs of source → transform → sink operators, each running at configurable parallelism

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.