Apache Fluss (Incubating) is an open-source, lakehouse-native streaming storage. It collapses the message broker, online KV store, stream-processing state backend, and lakehouse cold store into a single coherent foundation, making the Lakehouse truly real-time.
-- Register Apache Fluss as a Flink catalogCREATE CATALOG fluss_catalog WITH ('type' = 'fluss','bootstrap.servers' = 'coordinator-server:9123');USE CATALOG fluss_catalog;-- Create a primary-key tableCREATE TABLE pk_table (shop_id BIGINT,user_id BIGINT,num_orders INT,PRIMARY KEY (shop_id, user_id) NOT ENFORCED) WITH ('bucket.num' = '4');INSERT INTO pk_table VALUES (1234, 1234, 1);SELECT * FROM pk_table WHERE shop_id = 1234;
A conventional real-time AI stack stitches together a message broker, a stream processor, an online store, an offline store, and a synchronization layer. Every boundary is an integration point where data silently diverges. Apache Fluss collapses that stack into one substrate.
5 Systems · 4 Sync Boundaries · Continuous Engineering Tax
1 Substrate · 0 Sync Boundaries · Single Source of Truth
Each pillar is a direct consequence of a specific architectural mechanism. Together they collapse the fragmented real-time stack into a single coherent foundation.
One system for messaging, applications, analytics, and AI.
Replaces the message queue, key-value store, and OLAP engine with a single platform serving transport, lookups, and queries from the same data.
Architectural basisDual representation of PK Tables (Log Store & KV Store).
One copy of data across real-time and batch layers.
Hot and cold tiers share the same schema and are queryable as one substrate, so streaming and historical reads hit one source of truth.
Architectural basisTiering Service and Union Read across Iceberg, Paimon, and Lance.
Lean, elastic, stateless compute with fast recovery.
Stateless compute recovers in seconds and runs up to 85% cheaper than Kafka-based topologies. State lives on the Fluss leader, not Flink slots.
Architectural basisStateless compute model with leader-resident state and KV snapshots.
Pruning that compounds.
Server-side projection, predicate pushdown, and partition pruning on Arrow-format streams compound into order-of-magnitude I/O and network savings.
Architectural basisARROW log format and the compound pruning stack on the TabletServer.
Multi-modal data on one substrate, ready for ML and AI.
Row, columnar, and vector data on one store. Online features, RAG context, and analytics collapse into one PK Table accessed through different views.
Architectural basisUnified substrate spanning structured features and vector context.
Open formats. No vendor lock-in.
Readable by Flink, Spark, Trino, StarRocks, and DuckDB. Native hot tier plus Iceberg, Paimon, and Lance for the cold tier, open formats end to end.
Architectural basisOpen lake formats throughout, governed at the Apache Software Foundation.
Kafka is the streaming transport. Fluss is the streaming storage. If your need is large-scale stream processing with Flink, real-time analytics, AI/ML, or a sub-second lakehouse, Fluss is the shared streaming storage substrate behind all of them. Read the full breakdown to see which fits your stack.
Apache Fluss is developed openly by a global community of contributors. Join the discussion, file an issue, or send a patch.
View on GitHub