How JD.com Built a Lakestream with Apache Fluss and Apache Hudi
JD.com keeps fresh data in Fluss and tiers it into Hudi, cutting batch compute cost by more than 80% and storage cost by more than 50% in recommendation ranking.
Engineering, ideas, and stories from the streaming frontier.
JD.com keeps fresh data in Fluss and tiers it into Hudi, cutting batch compute cost by more than 80% and storage cost by more than 50% in recommendation ranking.
34 articles
Recover a side-effecting worker from Fluss event history, reconcile uncertain actions with downstream receipts, and query the trace with Flink SQL.
How Apache Doris 5.0 reads Fluss log and primary-key tables, combines Paimon history with recent data, and manages JVM memory during Union Read.
Shared tables for streaming, serving and analytics
Fluss 1.0 strengthens the real-time data foundation for AI with Fluss Gateway, native clients, and deeper streaming and lakehouse integration.
A production case study in columnar streaming, cold-data isolation, and lakehouse integration.
Apache Fluss graduates to a Top-Level Project, marking a milestone in community growth and the evolution of real-time Lakehouse infrastructure.
Run lake tiering in production: identify failure modes, avoid deployment pitfalls, and monitor the signals that reveal operational health.
Tune lake tiering with a practical guide to parallelism, table types, freshness, multi-table scheduling, and scaling out tiering jobs.
Follow a lake-tiering round from its timer to the committed snapshot, and understand the processes and table states that make it work.