Skip to main content
Version: 0.7

Deploying with Docker

This guide will show you how to run a Fluss cluster using Docker. We will introduce the prerequisites of the Docker environment, and how to quickly create a Fluss cluster using docker run commands or a docker compose file.

Prerequisites​

Hardware

Recommended configuration: 4 cores, 16GB memory.

Software

Docker and the Docker Compose plugin. All commands were tested with Docker version 27.4.0 and Docker Compose version v2.30.3.

Deploy with Docker​

The following is a brief overview of how to quickly create a complete Fluss testing cluster using the docker run commands.

Create a shared tmpfs volume​

Create a shared tmpfs volume:

docker volume create shared-tmpfs

Create a Network​

Create an isolated bridge network in docker

docker network create fluss-demo

Start Zookeeper​

Start Zookeeper in daemon mode. This is a single node zookeeper setup. Zookeeper is the central metadata store for Fluss and should be set up with replication for production use. For more information, see Running zookeeper cluster.

docker run \
    --name zookeeper \
    --network=fluss-demo \
    --restart always \
    -p 2181:2181 \
    -d zookeeper:3.9.2

Start Fluss CoordinatorServer​

Start Fluss CoordinatorServer in daemon and connect to Zookeeper.

docker run \
    --name coordinator-server \
    --network=fluss-demo \
    --env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://coordinator-server:0, CLIENT://coordinator-server:9123
advertised.listeners: CLIENT://localhost:9123
internal.listener.name: INTERNAL
" \
    -p 9123:9123 \
    -d fluss/fluss:0.7.0 coordinatorServer

Start Fluss TabletServer​

You can start one or more tablet servers based on your needs. For a production environment, ensure that you have multiple tablet servers.

Start with One TabletServer​

If you just want to start a sample test, you can start only one TabletServer in daemon and connect to Zookeeper. The command is as follows:

docker run \
    --name tablet-server \
    --network=fluss-demo \
    --env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://tablet-server:0, CLIENT://tablet-server:9123
advertised.listeners: CLIENT://localhost:9124
internal.listener.name: INTERNAL
tablet-server.id: 0
kv.snapshot.interval: 0s
data.dir: /tmp/fluss/data
remote.data.dir: /tmp/fluss/remote-data" \
    -p 9124:9123 \
    --volume shared-tmpfs:/tmp/fluss \
    -d fluss/fluss:0.7.0 tabletServer

Start with Multiple TabletServer​

In a production environment, you need to start multiple Fluss TabletServer nodes. Here we start 3 Fluss TabletServer nodes in daemon and connect to Zookeeper. The command is as follows:

  1. Start tablet-server-0
docker run \
    --name tablet-server-0 \
    --network=fluss-demo \
    --env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://tablet-server-0:0, CLIENT://tablet-server-0:9123
advertised.listeners: CLIENT://localhost:9124
internal.listener.name: INTERNAL
tablet-server.id: 0
kv.snapshot.interval: 0s
data.dir: /tmp/fluss/data/tablet-server-0
remote.data.dir: /tmp/fluss/remote-data" \
    -p 9124:9123 \
    --volume shared-tmpfs:/tmp/fluss \
    -d fluss/fluss:0.7.0 tabletServer
  1. Start tablet-server-1
docker run \
    --name tablet-server-1 \
    --network=fluss-demo \
    --env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://tablet-server-1:0, CLIENT://tablet-server-1:9123
advertised.listeners: CLIENT://localhost:9125
internal.listener.name: INTERNAL
tablet-server.id: 1
kv.snapshot.interval: 0s
data.dir: /tmp/fluss/data/tablet-server-1
remote.data.dir: /tmp/fluss/remote-data" \
    -p 9125:9123 \
    --volume shared-tmpfs:/tmp/fluss \
    -d fluss/fluss:0.7.0 tabletServer
  1. Start tablet-server-2
docker run \
    --name tablet-server-2 \
    --network=fluss-demo \
    --env FLUSS_PROPERTIES="zookeeper.address: zookeeper:2181
bind.listeners: INTERNAL://tablet-server-2:0, CLIENT://tablet-server-2:9123
advertised.listeners: CLIENT://localhost:9126
internal.listener.name: INTERNAL
tablet-server.id: 2
kv.snapshot.interval: 0s
data.dir: /tmp/fluss/data/tablet-server-2
remote.data.dir: /tmp/fluss/remote-data" \
    -p 9126:9123 \
    --volume shared-tmpfs:/tmp/fluss \
    -d fluss/fluss:0.7.0 tabletServer

Now all the Fluss related components are running.

Run the below command to check the Fluss cluster status:

docker container ls -a

Deploy with Docker Compose​

The following is a brief overview of how to quickly create a complete Fluss testing cluster using the docker compose up -d commands in a detached mode.

Create docker-compose.yml file​

Compose file to start Fluss cluster with one TabletServer​

You can use the following docker-compose.yml file to start a Fluss cluster with one CoordinatorServer and one TabletServer.

services:
  coordinator-server:
    image: fluss/fluss:0.7.0
    command: coordinatorServer
    depends_on:
      - zookeeper
    environment:
      - |
        FLUSS_PROPERTIES=
        zookeeper.address: zookeeper:2181
        bind.listeners: INTERNAL://coordinator-server:0, CLIENT://coordinator-server:9123
        advertised.listeners: CLIENT://localhost:9123
        internal.listener.name: INTERNAL
        remote.data.dir: /tmp/fluss/remote-data
    ports:
      - "9123:9123"
  tablet-server:
    image: fluss/fluss:0.7.0
    command: tabletServer
    depends_on:
      - coordinator-server
    environment:
      - |
        FLUSS_PROPERTIES=
        zookeeper.address: zookeeper:2181
        bind.listeners: INTERNAL://tablet-server:0, CLIENT://tablet-server:9123
        advertised.listeners: CLIENT://localhost:9124
        internal.listener.name: INTERNAL
        tablet-server.id: 0
        kv.snapshot.interval: 0s
        data.dir: /tmp/fluss/data
        remote.data.dir: /tmp/fluss/remote-data
    ports:
        - "9124:9123"
    volumes:
      - shared-tmpfs:/tmp/fluss
  zookeeper:
    restart: always
    image: zookeeper:3.9.2

volumes:
  shared-tmpfs:
    driver: local
    driver_opts:
      type: "tmpfs"
      device: "tmpfs"

Compose file to start Fluss cluster with multi TabletServer​

You can use the following docker-compose.yml file to start a Fluss cluster with one CoordinatorServer and three TabletServers.

services:
  coordinator-server:
    image: fluss/fluss:0.7.0
    command: coordinatorServer
    depends_on:
      - zookeeper
    environment:
      - |
        FLUSS_PROPERTIES=
        zookeeper.address: zookeeper:2181
        bind.listeners: INTERNAL://coordinator-server:0, CLIENT://coordinator-server:9123
        advertised.listeners: CLIENT://localhost:9123
        internal.listener.name: INTERNAL
        remote.data.dir: /tmp/fluss/remote-data
    ports:
      - "9123:9123"
  tablet-server-0:
    image: fluss/fluss:0.7.0
    command: tabletServer
    depends_on:
      - coordinator-server
    environment:
      - |
        FLUSS_PROPERTIES=
        zookeeper.address: zookeeper:2181
        bind.listeners: INTERNAL://tablet-server-0:0, CLIENT://tablet-server-0:9123
        advertised.listeners: CLIENT://localhost:9124
        internal.listener.name: INTERNAL
        tablet-server.id: 0
        kv.snapshot.interval: 0s
        data.dir: /tmp/fluss/data/tablet-server-0
        remote.data.dir: /tmp/fluss/remote-data
    ports:
      - "9124:9123"
    volumes:
      - shared-tmpfs:/tmp/fluss
  tablet-server-1:
    image: fluss/fluss:0.7.0
    command: tabletServer
    depends_on:
      - coordinator-server
    environment:
      - |
        FLUSS_PROPERTIES=
        zookeeper.address: zookeeper:2181
        bind.listeners: INTERNAL://tablet-server-1:0, CLIENT://tablet-server-1:9123
        advertised.listeners: CLIENT://localhost:9125
        internal.listener.name: INTERNAL
        tablet-server.id: 1
        kv.snapshot.interval: 0s
        data.dir: /tmp/fluss/data/tablet-server-1
        remote.data.dir: /tmp/fluss/remote-data
    ports:
      - "9125:9123"
    volumes:
      - shared-tmpfs:/tmp/fluss
  tablet-server-2:
    image: fluss/fluss:0.7.0
    command: tabletServer
    depends_on:
      - coordinator-server
    environment:
      - |
        FLUSS_PROPERTIES=
        zookeeper.address: zookeeper:2181
        bind.listeners: INTERNAL://tablet-server-2:0, CLIENT://tablet-server-2:9123
        advertised.listeners: CLIENT://localhost:9126
        internal.listener.name: INTERNAL
        tablet-server.id: 2
        kv.snapshot.interval: 0s
        data.dir: /tmp/fluss/data/tablet-server-2
        remote.data.dir: /tmp/fluss/remote-data
    ports:
      - "9126:9123"
    volumes:
      - shared-tmpfs:/tmp/fluss
  zookeeper:
    restart: always
    image: zookeeper:3.9.2

volumes:
  shared-tmpfs:
    driver: local
    driver_opts:
      type: "tmpfs"
      device: "tmpfs"

Launch the components​

Save the docker-compose.yaml script and execute the docker compose up -d command in the same directory to create the cluster.

Run the below command to check the container status:

docker container ls -a

Interacting with Fluss​

After the Fluss cluster is started, you can use Fluss Client (e.g., Flink SQL Client) to interact with Fluss. The following subsections will show you how to use 'Docker' to build a Flink cluster and use Flink SQL Client to interact with Fluss.

You can start a Flink standalone cluster refer to Flink Environment Preparation

Note: Make sure the Fluss connector jar already has copied to the lib directory of your Flink home.

bin/start-cluster.sh

Enter into SQL-Client​

Use the following command to enter the Flink SQL CLI Container:

bin/sql-client.sh

Create Fluss Catalog​

Use the following SQL to create a Fluss catalog:

Flink SQL
CREATE CATALOG fluss_catalog WITH (
    'type' = 'fluss',
    'bootstrap.servers' = 'localhost:9123'
);
Flink SQL
USE CATALOG fluss_catalog;

Do more with Fluss​

After the catalog is created, you can use Flink SQL Client to do more with Fluss, for example, create a table, insert data, query data, etc. More details please refer to Flink Getting started