8fad85aed7
* feat(s3): support WEED_S3_SSE_KEY env var for SSE-S3 KEK Add support for providing the SSE-S3 Key Encryption Key (KEK) via the WEED_S3_SSE_KEY environment variable (hex-encoded 256-bit key). This avoids storing the master key in plaintext on the filer at /etc/s3/sse_kek. Key source priority: 1. WEED_S3_SSE_KEY environment variable (recommended) 2. Existing filer KEK at /etc/s3/sse_kek (backward compatible) 3. Auto-generate and save to filer (deprecated for new deployments) Existing deployments with a filer-stored KEK continue to work unchanged. A deprecation warning is logged when auto-generating a new filer KEK. * refactor(s3): derive KEK from any string via HKDF instead of requiring hex Accept any secret string in WEED_S3_SSE_KEY and derive a 256-bit key using HKDF-SHA256 instead of requiring a hex-encoded key. This is simpler for users — no need to generate hex, just set a passphrase. * feat(s3): add WEED_S3_SSE_KEK and WEED_S3_SSE_KEY env vars for KEK Two env vars for providing the SSE-S3 Key Encryption Key: - WEED_S3_SSE_KEK: hex-encoded, same format as /etc/s3/sse_kek. If the filer file also exists, they must match. - WEED_S3_SSE_KEY: any string, 256-bit key derived via HKDF-SHA256. Refuses to start if /etc/s3/sse_kek exists (must delete first). Only one may be set. Existing filer-stored KEKs continue to work. Auto-generating and storing new KEKs on filer is deprecated. * fix(s3): stop auto-generating KEK, fail only when SSE-S3 is used Instead of auto-generating a KEK and storing it on the filer when no key source is configured, simply leave SSE-S3 disabled. Encrypt and decrypt operations return a clear error directing the user to set WEED_S3_SSE_KEK or WEED_S3_SSE_KEY. * refactor(s3): move SSE-S3 KEK config to security.toml Move KEK configuration from standalone env vars to security.toml's new [sse_s3] section, following the same pattern as JWT keys and TLS certs. [sse_s3] kek = "" # hex-encoded 256-bit key (same format as /etc/s3/sse_kek) key = "" # any string, HKDF-derived Viper's WEED_ prefix auto-mapping provides env var support: WEED_SSE_S3_KEK and WEED_SSE_S3_KEY. All existing behavior is preserved: filer KEK fallback, mismatch detection, and HKDF derivation. * refactor(s3): rename SSE-S3 config keys to s3.sse.kek / s3.sse.key Use [s3.sse] section in security.toml, matching the existing naming convention (e.g. [s3.*]). Env vars: WEED_S3_SSE_KEK, WEED_S3_SSE_KEY. * fix(s3): address code review findings for SSE-S3 KEK - Don't hold mutex during filer retry loop (up to 20s of sleep). Lock only to write filerClient and superKey. - Remove dead generateAndSaveSuperKeyToFiler and unused constants. - Return error from deriveKeyFromSecret instead of ignoring it. - Fix outdated doc comment on InitializeWithFiler. - Use t.Setenv in tests instead of manual os.Setenv/Unsetenv. * fix(s3): don't block startup on filer errors when KEK is configured - When s3.sse.kek is set, a temporarily unreachable filer no longer prevents startup. The filer consistency check becomes best-effort with a warning. - Same treatment for s3.sse.key: filer unreachable logs a warning instead of failing. - Rewrite error messages to suggest migration instead of file deletion, avoiding the risk of orphaning encrypted data. Finding 3 (restore auto-generation) intentionally skipped — auto-gen was removed by design to avoid storing plaintext KEK on filer. * fix(test): set WEED_S3_SSE_KEY in SSE integration test server startup SSE-S3 no longer auto-generates a KEK, so integration tests must provide one. Set WEED_S3_SSE_KEY=test-sse-s3-key in all weed mini invocations in the test Makefile.
198 lines
7.1 KiB
TOML
198 lines
7.1 KiB
TOML
# Put this file to one of the location, with descending priority
|
|
# ./security.toml
|
|
# $HOME/.seaweedfs/security.toml
|
|
# /etc/seaweedfs/security.toml
|
|
# this file is read by master, volume server, filer, and worker
|
|
|
|
# comma separated origins allowed to make requests to the filer and s3 gateway.
|
|
# enter in this format: https://domain.com, or http://localhost:port
|
|
[cors.allowed_origins]
|
|
values = "*"
|
|
|
|
# this jwt signing key is read by master and volume server, and it is used for write operations:
|
|
# - the Master server generates the JWT, which can be used to write a certain file on a volume server
|
|
# - the Volume server validates the JWT on writing
|
|
# the jwt defaults to expire after 10 seconds.
|
|
[jwt.signing]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
# by default, if the signing key above is set, the Volume UI over HTTP is disabled.
|
|
# by setting ui.access to true, you can re-enable the Volume UI. Despite
|
|
# some information leakage (as the UI is not authenticated), this should not
|
|
# pose a security risk.
|
|
[access]
|
|
ui = false
|
|
|
|
# by default the filer UI is enabled. This can be a security risk if the filer is exposed to the public
|
|
# and the JWT for reads is not set. If you don't want the public to have access to the objects in your
|
|
# storage, and you haven't set the JWT for reads it is wise to disable access to directory metadata.
|
|
# This disables access to the Filer UI, and will no longer return directory metadata in GET requests.
|
|
[filer.expose_directory_metadata]
|
|
enabled = true
|
|
|
|
# this jwt signing key is read by master and volume server, and it is used for read operations:
|
|
# - the Master server generates the JWT, which can be used to read a certain file on a volume server
|
|
# - the Volume server validates the JWT on reading
|
|
# NOTE: jwt for read is only supported with master+volume setup. Filer does not support this mode.
|
|
[jwt.signing.read]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
|
|
# If this JWT key is configured, Filer only accepts writes over HTTP if they are signed with this JWT:
|
|
# - f.e. the S3 API Shim generates the JWT
|
|
# - the Filer server validates the JWT on writing
|
|
# NOTE: This key is ALSO used as a fallback signing key for S3 STS if s3.iam.config does not specify a signingKey.
|
|
# the jwt defaults to expire after 10 seconds.
|
|
[jwt.filer_signing]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
# If this JWT key is configured, Filer only accepts reads over HTTP if they are signed with this JWT:
|
|
# - f.e. the S3 API Shim generates the JWT
|
|
# - the Filer server validates the JWT on reading
|
|
# the jwt defaults to expire after 10 seconds.
|
|
[jwt.filer_signing.read]
|
|
key = ""
|
|
expires_after_seconds = 10 # seconds
|
|
|
|
# gRPC mTLS configuration
|
|
# All gRPC TLS authentications are mutual (mTLS)
|
|
# The values for ca, cert, and key are paths to the certificate/key files
|
|
# The host name is not checked, so the certificate files can be shared
|
|
[grpc]
|
|
ca = ""
|
|
# Set wildcard domain for enable TLS authentication by common names
|
|
allowed_wildcard_domain = "" # .mycompany.com
|
|
|
|
# Volume server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to volume server
|
|
[grpc.volume]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# Master server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to master server
|
|
[grpc.master]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# Filer server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to filer server
|
|
[grpc.filer]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# S3 server gRPC options (server-side)
|
|
# Enables mTLS for incoming gRPC connections to S3 server
|
|
[grpc.s3]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.msg_broker]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.msg_agent]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.admin]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.worker]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
[grpc.mq]
|
|
cert = ""
|
|
key = ""
|
|
allowed_commonNames = "" # comma-separated SSL certificate common names
|
|
|
|
# gRPC client configuration for outgoing gRPC connections
|
|
# Used by clients (S3, mount, backup, benchmark, filer.copy, filer.replicate, upload, etc.)
|
|
# when connecting to any gRPC server (master, volume, filer)
|
|
[grpc.client]
|
|
cert = ""
|
|
key = ""
|
|
|
|
# HTTPS client configuration for outgoing HTTP connections
|
|
# Used by S3, mount, filer.copy, backup, and other clients when communicating with master/volume/filer
|
|
# Set enabled=true to use HTTPS instead of HTTP for data operations (separate from gRPC)
|
|
# If [https.filer] or [https.volume] are enabled on servers, clients must have [https.client] enabled=true
|
|
[https.client]
|
|
enabled = false # Set to true to enable HTTPS for all outgoing HTTP client connections
|
|
cert = "" # Client certificate for mTLS (optional if server doesn't require client cert)
|
|
key = "" # Client key for mTLS (optional if server doesn't require client cert)
|
|
ca = "" # CA certificate to verify server certificates (required when enabled=true)
|
|
insecure_skip_verify = false # Skip TLS certificate verification (NOT recommended for production)
|
|
|
|
# Volume server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to volume server
|
|
[https.volume]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
|
|
# Master server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to master server (web UI, HTTP API)
|
|
[https.master]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
|
|
# Filer server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to filer server (web UI, HTTP API)
|
|
[https.filer]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
# disable_tls_verify_client_cert = true|false (default: false)
|
|
|
|
# Admin server HTTPS options (server-side)
|
|
# Enables HTTPS for incoming HTTP connections to admin server
|
|
[https.admin]
|
|
cert = ""
|
|
key = ""
|
|
ca = ""
|
|
|
|
# Admin server authentication
|
|
# If password is set, users must login to access the admin interface
|
|
# These can be overridden by environment variables with WEED_ prefix:
|
|
# WEED_ADMIN_USER, WEED_ADMIN_PASSWORD
|
|
# WEED_ADMIN_READONLY_USER, WEED_ADMIN_READONLY_PASSWORD
|
|
[admin]
|
|
user = ""
|
|
password = ""
|
|
|
|
[admin.readonly]
|
|
user = ""
|
|
password = ""
|
|
|
|
# SSE-S3 server-side encryption key management
|
|
# These settings configure the Key Encryption Key (KEK) for S3 SSE-S3 encryption.
|
|
# Set exactly one of kek or key. If neither is set, SSE-S3 is disabled.
|
|
# Can also be set via env vars: WEED_S3_SSE_KEK, WEED_S3_SSE_KEY
|
|
[s3.sse]
|
|
# hex-encoded 256-bit key, same format as the legacy /etc/s3/sse_kek filer file.
|
|
# Use this to migrate from a filer-stored KEK: copy the value from /etc/s3/sse_kek.
|
|
# Generate a new one with: openssl rand -hex 32
|
|
kek = ""
|
|
# any secret string; a 256-bit key is derived automatically via HKDF-SHA256.
|
|
# Cannot be used while /etc/s3/sse_kek exists on the filer — delete it first.
|
|
key = ""
|
|
|
|
# white list. It's checking request ip address.
|
|
[guard]
|
|
white_list = ""
|