Amazon S3

Amazon Simple Storage Service (Amazon S3) is cloud object storage with industry-leading scalability, data availability, security, and performance.

Configurations setup

To enabled S3 as remote storage, there are some required configurations that must be added to Fluss' server.yaml:

# The dir that used to be as the remote storage of Fluss
remote.data.dir: s3://<your-bucket>/path/to/remote/storage
# access key
s3.access-key: <your-access-key>
# secret key
s3.secret-key: <your-secret-key>
# region
s3.region: <your-s3-region>

S3-Compatible Storage (RustFS, MinIO, etc.)

For S3-compatible storage services such as RustFS or MinIO, you need to configure a custom endpoint and enable path-style access:

remote.data.dir: s3://<your-bucket>/path/to/remote/storage
s3.endpoint: http://<your-s3-compatible-endpoint>:9000
s3.access-key: <your-access-key>
s3.secret-key: <your-secret-key>
s3.region: us-east-1
s3.path-style-access: true

AssumeRole STS Configuration

Some S3-compatible services (such as RustFS) require the use of AssumeRole instead of GetSessionToken to obtain temporary security credentials. This is necessary for features like KV snapshots that rely on delegation tokens.

To enable AssumeRole, add the following configurations alongside the base S3 settings above:

remote.data.dir: s3://<your-bucket>/path/to/remote/storage
s3.endpoint: http://<your-s3-compatible-endpoint>:9000
s3.access-key: <your-access-key>
s3.secret-key: <your-secret-key>
s3.region: us-east-1
s3.path-style-access: true
s3.assumed.role.arn: <your-role-arn>
s3.assumed.role.sts.endpoint: http://<your-s3-compatible-endpoint>:9000

Configuration	Description
s3.assumed.role.arn	The ARN of the IAM role to assume. When set, Fluss uses AssumeRole instead of GetSessionToken to obtain temporary credentials. The s3.access-key and s3.secret-key are still required — they authenticate the AssumeRole call itself.
s3.assumed.role.sts.endpoint	Custom STS endpoint URL. Required for S3-compatible services that host their own STS API. When not set, the default AWS STS endpoint is used.

note

Without s3.assumed.role.arn, Fluss falls back to GetSessionToken (the default AWS behavior). This is fully backward compatible — existing AWS users do not need to change their configuration.

Default AWS Credential Chain (IRSA, Instance Profiles)

When running Fluss on Kubernetes with IAM Roles for Service Accounts (IRSA) or on EC2 with instance profiles, you can omit s3.access-key and s3.secret-key. The server will authenticate using the default AWS credential chain.

In this mode, s3.assumed.role.arn is required — the server uses AssumeRole to generate temporary credentials for clients (Flink/Spark connectors) that read tiered data from S3.

remote.data.dir: s3://<your-bucket>/path/to/remote/storage
s3.region: <your-s3-region>
s3.assumed.role.arn: <your-delegation-role-arn>

The server's IAM role (e.g., the IRSA service account role) must have:

• Read/write permissions on the S3 bucket (for the server's own data access)
• sts:AssumeRole permission on the role specified in s3.assumed.role.arn

note

The server authenticates using its own credentials (static keys, IRSA, instance profile, or environment variables) for S3 data access. For delegation, the server calls AssumeRole with the configured s3.assumed.role.arn, so clients receive credentials for that role — which can have different permissions (e.g., read-only). Note that the server uses the same identity for both its own S3 access and the STS AssumeRole call. Further decoupling is planned.